System, program, and the like

Abstract

To use an image obtained by a camera attached to a vehicle to identify the cause of an accident more effectively than before, and to utilize the image more effectively without being limited to the identification of the cause of the accident.SOLUTION: Processing means of a system has a function to record an image captured by a camera attached to a vehicle, perform image analysis of the images captured by the camera, and a function that executes processing corresponding to the analysis results on the basis of the analysis results.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to, for example, a system, a program, etc. for processing an image captured by a camera or the like attached to a vehicle or the like.

Background Art

[0002] The following Patent Document 1 discloses a system applied to a drive recorder or the like. This system repeats a cycle of recording video data in a recording area. When a predetermined cycle end condition is satisfied, the currently ongoing cycle is terminated, and a new cycle is started in which the video data recorded in the terminated cycle is overwritten with new video data in an old order. Information about the overwrite position is output to the output means based on at least one of the capacity of the video data recorded in the currently ongoing cycle and the capacity of the video data recorded in the immediately preceding cycle that has not been overwritten.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The images recorded by a drive recorder are effectively used to identify the cause of an accident or the like. It is desired to make the images acquired by a camera attached to a vehicle or the like more useful for identifying the cause of an accident than before, and not limited to identifying the cause of an accident, but to utilize them more effectively.

[0005] ​​​​ The object of the invention of the present application is not limited to this, and for a configuration aimed at obtaining the effects exhibited by the parts of the configuration disclosed in this specification, drawings, etc., the applicant also has the intention of obtaining rights through divisional applications, amendments, etc. For example, in this specification, the part described as "can be" is read as "is a problem", and such a problem is disclosed in this specification. The problems are described as independent of each other, and for the configuration for solving these problems, the applicant also has the intention of obtaining rights through separate divisional applications, amendments, etc. Even if the problem is implicitly understood from the description of the specification, the applicant has the intention of making part of the configuration described in this specification the scope of the patent claim through amendment or divisional application. Also, problems combining these independent problems are also disclosed.

Means for Solving the Problem

[0006] (1) A system having a processing means having a function of recording an image captured by a camera attached to a vehicle, and a function of performing image analysis on the image captured by the camera and executing a process corresponding to the analysis result based on the analysis result may be used.

[0007] Not only can an image captured by a camera attached to a vehicle be recorded, but the image can also be utilized more effectively.

[0008] Examples of vehicles include, for example, passenger cars, freight trucks, special vehicles, etc. The camera may, for example, image at least one of the inside and outside of the vehicle. Next, "inside the vehicle" and "outside the vehicle" may be interpreted as follows, for example.

[0009] ​​​​​​​​​​​​​​When the interior space and the exterior space are clearly separated by the vehicle body, the body interior may be referred to as "inside the vehicle", and the exterior of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it is advisable to refer to the continuous passenger space and cargo space as "inside the vehicle". Similarly, in the case of a sedan where the passenger space and the cargo space are partitioned, it is advisable to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered cargo vehicle with a sealed cargo bed automobile, it is advisable to refer to the interior of the passenger space and the sealed cargo bed as "inside the vehicle". In an open-bed cargo vehicle (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space. However, the space for accommodating the open cargo may also be referred to as "inside the vehicle". Additionally, the passenger space of an open-bed cargo vehicle may also be referred to as "inside the vehicle". In some special vehicles such as forklifts, both the passenger space and the cargo space are open and it may be difficult to clearly distinguish the passenger space and the cargo space from the surrounding space. Even in such cases, it is advisable to refer to both the passenger space and the cargo space as "inside the vehicle". For example, the space above the seat where the passenger sits and the space above the fork that holds and lifts the cargo may be referred to as "inside the vehicle". Thus, as the vehicle moves, the space that can be considered to move along with the vehicle is referred to as "inside the vehicle", and the surrounding space is referred to as "outside the vehicle".

[0010] As the "camera attached to the vehicle", a camera with a fixed relative position and orientation with respect to the vehicle in the normal usage state may be used. Also, as the "camera attached to the vehicle", a hemispherical or omnidirectional 360-degree camera may be used. A hemispherical or omnidirectional ​​​​When using a 360-degree camera of the type, a wider range of images can be acquired compared to using a normal camera. For example, with a single camera, it is possible to acquire images both inside and outside the vehicle. Note that instead of a 360-degree camera, a plurality of normal cameras may be attached to the vehicle.

[0011] The image captured by the camera is preferably made into a video. When the image captured by the camera is made into a video, the processing means can easily determine the movement of the subject, the presence or absence of the subject before and after a certain point in time, etc. by performing image analysis.

[0012] The function of recording the image captured by the camera may be realized by a server that communicates with the in-vehicle device of the vehicle on which the camera is mounted, but it is even better to perform it on the in-vehicle device. As the recording medium, it is advisable to use a removable medium such as an SD card. When using a removable medium as the recording medium, the removable medium on which the image is recorded can be removed from the in-vehicle device, and the image can be displayed on other terminals such as a personal computer.

[0013] The function of analyzing the image captured by the camera may be realized by a server that communicates with the in-vehicle device of the vehicle on which the camera is mounted, but it is even better to perform it on the in-vehicle device. When the in-vehicle device performs the analysis of the image, processing corresponding to the analysis result can be quickly executed in the vehicle.

[0014] The processing means may be composed of one device or a plurality of devices. For example, a drive recorder has a function of recording the image captured by the camera. The function of the processing means of this system is the microcomputer and memory built into this drive recorder. ​​​​​​It may be realized by the application program stored in []. Also, for example, the processing steps may be configured by two devices, namely an in-vehicle device and a server. The in-vehicle device records the image, and the server analyzes the image. It is advisable that the in-vehicle device records the image and the server analyzes the image.

[0015] In addition, a system may be provided with a processing means having a function of recording an image captured by a camera attached to an object other than a vehicle, and a function of performing image analysis on the image captured by the previous camera, and based on the analysis result, executing a process corresponding to the analysis result. However, it is particularly advisable to be as in (1). It is advisable to be as in (1).

[0016] (2) The processing means has a function of determining whether or not the movement of a person in the image corresponds to a specific movement registered in advance as the function of the image analysis. When the movement of the person corresponds to the specific movement, as a process corresponding to the analysis result, at least one of the processes of recording an image of the specific movement of the person, transmitting the image of the specific movement to the server, notifying that the person has made the specific movement, and issuing an alarm is executed, and the system according to claim 1 is adopted. It is advisable to be so.

[0017] For example, the recorded image or the image transmitted to the server can be used as evidence that the person has made a specific movement. For example, by notifying that the person has made a specific movement, it is possible to inform an external person that there is a person making the specific movement. For example, by the issued alarm, the person making the specific movement is motivated to stop the specific movement by noticing that an alarm has been issued to himself / herself.

[0018] ​​​​​​​​​​​As a person performing a specific movement, for example, it may be a driver of a vehicle. As the specific movement, for example, it may be an operation other than driving by the driver. For example, looking at a mobile terminal such as a smartphone or the like, an operation of eating food, an operation of looking around or looking sideways, dozing off, etc. may be used. For example, the presence or absence of the sideways glance operation may be determined based on the detection result of the position of the driver's eyes. The recorded image can be used for improving the driver's awareness, analyzing the cause of an accident, etc. When the processing means detects that the driver is operating a mobile terminal such as a smartphone, it is advisable to issue a warning to prompt the driver not to hold the mobile terminal while driving. If the driver notices the warning and stops the specific movement, the danger can be avoided in advance. As the server of the data transmission destination, it may be the server of the vehicle administrator or the server of the police. The vehicle administrator can evaluate the driving attitude of the driver based on the transmitted data. The police can evaluate the driving attitude of the driver, analyze the cause of an accident, etc. based on the transmitted image.

[0019] As a person performing a specific movement, for example, it may be a suspicious person outside the vehicle. As the specific movement, for example, an operation of attempting to illegally enter the vehicle, for example, an operation of attempting to break a glass window with a hammer or an operation of peeping into the vehicle may be used. By issuing a warning, it is possible to make the suspicious person who attempts to illegally enter the vehicle refrain from entering. The recorded image or the image transmitted to the server can be used as useful information for arresting a thief after a crime targeting a vehicle occurs. By notifying the vehicle owner or driver that a person has performed a specific movement, the owner or driver can immediately return to the vehicle and prevent damage caused by a crime targeting the vehicle. There may be cases where it can be done. Also, the notified driver or owner may report it to the police. In this way, this function can be used for crime prevention.

[0020] Regarding a person making specific movements, for example, as a suspicious person inside the vehicle, and regarding specific movements, various modus operandi on the vehicle, such as illegal entry into the vehicle while parked, suspicious movements of a person inside the parked vehicle, the operation of removing a device inside the vehicle, etc. would be appropriate. By issuing an alarm, when the thief panics and disperses, the effect of reducing the damage targeted at the vehicle can be obtained. The recorded image, or the image sent to the server, can be used as useful information for arresting the thief after a crime targeting the vehicle occurs. By notifying the owner or driver of the vehicle that a person has made specific movements, the owner or driver can immediately return to the vehicle or report it to the police, which can reduce the damage targeted at the vehicle. Examples of external notifications include sending an email with an attached image of the suspicious person. As the email recipient, it would be appropriate to set the vehicle administrator, driver, owner, police, etc. Regarding a person making specific movements, for example, as a passenger in a commercial vehicle where unspecified people board, and regarding specific movements, various modus operandi of robbery, such as an action of brandishing a knife, an action of holding a firearm, an action of using violence, etc. would be appropriate. By the processing means executing the process of issuing an alarm, if the robber stops the crime, the damage can be prevented. The recorded image, or the image sent to the server, can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made specific movements, the administrator can take measures against the robbery. Regarding a person making specific movements, for example, as a passenger in a commercial vehicle where unspecified people board, and regarding specific movements, various modus operandi of robbery, such as an action of brandishing a knife, an action of holding a firearm,

[0021] an action of using violence, etc. would be appropriate. By the processing means executing the process of issuing an alarm, if the robber stops the crime, the damage can be prevented. The recorded image, or the image sent to the server, can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made specific movements, the administrator can take measures against the robbery. the image sent to the server, can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made specific movements, the administrator can take measures against the robbery. If one notices the occurrence, they can report it to the police. Also, it is preferable that the processing means has a function of reporting to the police using a communication function. It is preferably to have a function of reporting to the police.

[0022] As a person making a specific movement, for example, an infant in the vehicle being ridden together, and as the specific movement, it is preferably, for example, the movement when waking up from sleep. When detecting the awakening of the infant by image analysis, the processing means may notify the guardian by e-mail or the like. When the infant falls asleep in the vehicle during driving and the driver wants to do a little shopping or finish some errands with the engine running, there may be such a case. At this time, since waking up the infant forcefully will make the infant grumpy, there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, they will be concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail will be sent to the guardian's mobile terminal, so the parent can immediately rush to the vehicle and deal with the awakening of the infant.

[0023] As a person making a specific movement, for example, a person in need of care or rescue outside the vehicle, and as the specific movement, it is preferably, for example, an action peculiar to a person in need of care or an action corresponding to a situation where rescue is considered necessary. For example, as an example of an action peculiar to a person in need of assistance, for example, walking while leaning on a cane, bending the waist, and staggering steps can be mentioned. As an example of an action corresponding to a situation where rescue is considered necessary, actions corresponding to a situation where a person has fallen and is foaming at the mouth and lying down can be mentioned.

[0024] The function of detecting a person making a specific movement includes a function of selecting at least one of the images in front of the vehicle, behind the vehicle, and inside the vehicle, and finding the corresponding person from the selected image. It is advisable to detect objects. The selection of at least one image may be made possible for the user to specify. This way, the processing means does not need to analyze unnecessary or less important images, eliminating the need to endow the processing means with excessive processing capabilities.

[0025] It is advisable to include a function for detecting the driver's drowsiness in the function for detecting a person making a specific movement. For example, the pulse may be measured from the variation in the redness of the driver's skin, and based on the variation in the pulse, it may be determined whether the driver is drowsy. It is advisable to enhance the detection accuracy of drowsiness by coordinating the measurement of body temperature using a thermal camera, the measurement of pulse using microwaves, etc. When the processing means of this system detects drowsiness, it may be advisable to lower the set temperature of the air conditioner or issue a warning by voice or sound.

[0026] (3) The processing means has a function of discriminating whether a person in the image has a predetermined characteristic as the function of the image analysis, and as a process corresponding to the analysis result, it is advisable to use a system that executes a process according to a specific characteristic of the person in the image.

[0027] Appropriate processing can be performed according to the characteristics of the person captured in the image.

[0028] It is advisable to include, for example, a face recognition function in the function of discriminating whether a person in the image has a specific characteristic. For example, perform face recognition on the person sitting in the driver's seat. If the person sitting in the driver's seat does not correspond to a pre-registered person, the processing means may prevent the engine from starting by accessing the vehicle. For example, it may be advisable to send a signal to prohibit the engine from starting to the engine control unit (ECU ) from the processing means. This way, the vehicle cannot be started without permission in advance. ​​​​​​Register capable individuals and prohibit unregistered individuals from driving the vehicle, which can enhance the security function of the vehicle. It is also effective as a measure against children's mischief. In addition to the process of prohibiting engine startup, operations such as disabling the accelerator, preventing the release of the side brake (parking brake), preventing the shift lever from entering the drive position, sounding an alarm,

[0029] and sending an email with an image to the registered email address may be performed. In addition, for example, perform face recognition of a person outside the vehicle. If the person outside the vehicle corresponds to a pre-registered person, the processing means may control the vehicle to unlock the doors. For example, when the vehicle is provided for a car-sharing service, it is advisable to pre-register the registered members of the car-sharing service. The registered members can move their faces within the camera's field of view of the vehicle when approaching the vehicle, and can use the vehicle even without possessing a card key or the like. This improves the usability of the vehicles in the car-sharing service.

[0030] If the vehicle is a commercial vehicle that allows unspecified individuals to board, the processing means may determine whether the passenger is a criminal being sought using the face recognition function. The face photo of the criminal being sought may be transmitted to the processing means of this system through communication via LTE, wireless LAN, etc. from the police database and pre-registered in the processing means. If the processing means determines that the passenger is a criminal being sought, notifications to the vehicle administrator, the police, etc. may be made through the communication device and communication network. Furthermore, the processing means may notify the driver in a way that is not noticed by the passenger. ​​It is good. Vehicle administrators, the police, drivers, etc. can recognize that a criminal on the wanted list has boarded the vehicle and take prompt and appropriate measures.

[0031] For example, a taxi becomes a closed space where only the crew and passengers can board, so there seems to be a high demand for this system in terms of ensuring the safety of the crew. When the processing means detects that a criminal has boarded the taxi, it is advisable to notify the taxi company and the police of the criminal's boarding. In addition, according to this system, it is possible to detect the boarding of a criminal when the criminal boards the vehicle, before the criminal actually commits a crime inside the vehicle. Therefore, it is also possible to prevent crimes.

[0032] For the function of determining whether a person in the image has specific characteristics, for example, compare the appearance of the person in the image with a specific pre-registered appearance, and determine whether the two match (hereinafter referred to as the appearance authentication function). It is advisable to register in advance specific appearances unique to loiterers, criminals, etc. As an example of the appearance unique to a criminal, for example, carrying a knife on the road, carrying dangerous goods, etc. can be cited. In addition, by combining images from infrared cameras, ultraviolet cameras, etc. with images from normal cameras, it is possible to detect persons carrying illegal drugs. When the processing means detects a person with a specific appearance in the image, it is advisable to notify the police and the vehicle administrator via the communication device and the communication network.

[0033] When wanting to find a specific missing person or loiterer, for example, register the appearance at the time of going missing as search information with the processing means individually. For example, by gender, height, clothing The type of clothing, the color of the clothing, whether a hat is worn, whether a cane is held, whether the waist is bent It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle. It may be good to register such individual search information. Furthermore, it may be good to register an area (search target area) where wandering is suspected. When a vehicle is located in the search target area, the processing means may execute a detection process for a person who matches the individual search information. Such individual search information may be transmitted from a database (server) such as a police station or a regional support center to the processing means of this system. The in-vehicle device may perform a function of determining whether a person in the image has specific features. When the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle.

[0034] For the function of determining whether a person in the image has specific features, for example, a function of determining whether a person sitting in the passenger seat or the rear seat is wearing a seat belt (hereinafter referred to as the appearance authentication function) may be included. When the processing means of this system detects a person not wearing a seat belt, it may issue a message prompting the person to wear the seat belt. For the function of determining whether a person in the image has specific features, for example, a function of determining whether a person sitting in the passenger seat or the rear seat is wearing a seat belt (hereinafter referred to as the appearance authentication function) may be included. When the processing means of this system detects a person not wearing a seat belt, it may issue a message prompting the person to wear the seat belt. For the function of determining whether a person in the image has specific features, for example, a function of determining whether a person sitting in the passenger seat or the rear seat is wearing a seat belt (hereinafter referred to as the appearance authentication function) may be included. When the processing means of this system detects a person not wearing a seat belt, it may issue a message prompting the person to wear the seat belt. For the function of determining whether a person in the image has specific features, for example, a function of determining whether a person sitting in the passenger seat or the rear seat is wearing a seat belt (hereinafter referred to as the appearance authentication function) may be included. When the processing means of this system detects a person not wearing a seat belt, it may issue a message prompting the person to wear the seat belt.

[0035] In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway. In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway. In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway. In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway. In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway. In particular, when a person in the rear seat approaches the entrance of the highway in a state of not wearing a seat belt, a message prompting the person to wear the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the highway becomes less than or equal to a reference distance, it may be determined that the vehicle is approaching the entrance of the highway. In addition, when a signal for checking whether an ETC card has been inserted is received, it may be determined that the vehicle is approaching the entrance of the highway. Also, when an image in front of the vehicle is analyzed and the entrance of the highway is detected in the image, it may be determined that the vehicle is approaching the entrance of the highway.

[0036] (4) The processing means has a function of counting the number of people in the image as a function of the image analysis and, as a process corresponding to the analysis result, executes a process of recording or transmitting the number-of-people data in the image It is preferable to use a system that performs such operations.

[0037] The number of people can be counted by this system without using a special device for counting the number of people. It is possible to do so.

[0038] For example, it is advisable to attach a camera to a bus and use an image inside the vehicle as the image for counting the number of people. The processing means may display the counting result to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle, if the counting result of the number of passengers does not become 0, the crew can know that there are people left in the vehicle. For example, it is possible to detect the presence of people who cannot express their own will, such as a baby, or people who cannot get off by themselves, such as a disabled person. It is possible to detect the presence of people who cannot get off by themselves, such as a disabled person.

[0039] For example, the vehicle to which the camera is attached is a type of vehicle in which the number of passengers is predetermined, such as a sightseeing bus or a long-distance express bus. It is advisable to register the predetermined number of passengers in the processing step of this system. The processing means compares the counting result of the number of people with the registered number of people. If there is a shortage or excess, it is advisable to warn the crew of the shortage or excess. This prevents the vehicle from departing with a shortage of passengers. Also, it is possible to easily grasp that a person who does not have the qualification to board has boarded and prompt them to get off. It is possible to easily grasp that a person who does not have the qualification to board has boarded and prompt them to get off.

[0040] For example, a camera (e.g., a hemispherical camera on the ceiling) may be installed inside a bus, and the processing means may perform image analysis to count the number of passengers, the number of standing passengers, and the number of sitting passengers. Furthermore, the processing means may acquire the current position information of the bus, the opening and closing signals of the bus doors, the bus disembarkation buzzer, the touch of the IC card when getting on and off the bus, and the signals of the fare box, and have the function of recording each of them as a trigger. Accumulate these information in a data lake, and it becomes possible to perform big data analysis on where the passengers got on and off and what the seating situation inside the bus was during that period. Such data can be valuable information for consulting companies that conduct urban planning and the like. Furthermore, it is advisable to acquire the information transmitted through an in-vehicle network such as CAN and the vital information of the driver together. These information can be used to identify the causal relationship when an abnormality occurs in the vehicle, such as whether the abnormality is due to a driving factor, a passenger factor, a driver's own factor, or a vehicle factor. For example, it is advisable to adopt an image outside the vehicle as the image for counting the number of people. The processing means of this system may record the counting result as number data.

[0041] Based on the recorded number data, it is possible to easily extract the locations where there were a large number of people. It is advisable to record the image used as the basis for counting using the counting result as a trigger. For example, when the number of people exceeds a predetermined threshold, the image may be recorded.

[0042] (5) As the function of the image analysis, the processing means has the function of detecting the display content of the vehicle display device or the operation status of the vehicle operation device from the image inside the vehicle, and performs processing corresponding to the analysis result. For the analysis result, The system may be configured to execute processing according to the display content of the display device or the operation status of the operation device. This system may be configured to execute processing according to the display content of the display device or the operation status of the operation device.

[0043] The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc. The system can acquire data that is the basis for the display content of the display device or data corresponding to the operation status of the operation device without retrofitting a device for acquiring such data. For example, these data can be acquired by this system without connecting a retrofitted device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting third-party products to the OBD connector, these data can be acquired by this system. The display device may include various instruments, in-vehicle warning lights (warning lamps), a head-up display that projects an image onto the windshield, etc.

[0044] As the display content of the display device to be detected, for example, the vehicle speed displayed on the speedometer may be used. The processing means may acquire the vehicle speed data by pattern recognition of the position of the needle of the analog speedometer and character recognition of the numbers on the dial. Also, the vehicle speed data may be acquired by character recognition of the numbers displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described. As the display content of the display device to be detected, for example, the vehicle speed displayed on the speedometer may be used. The processing means may acquire the vehicle speed data by pattern recognition of the position of the needle of the analog speedometer and character recognition of the numbers on the dial. Also, the vehicle speed data may be acquired by character recognition of the numbers displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described. As the display content of the display device to be detected, for example, the vehicle speed displayed on the speedometer may be used. The processing means may acquire the vehicle speed data by pattern recognition of the position of the needle of the analog speedometer and character recognition of the numbers on the dial. Also, the vehicle speed data may be acquired by character recognition of the numbers displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described. As the display content of the display device to be detected, for example, the vehicle speed displayed on the speedometer may be used. The processing means may acquire the vehicle speed data by pattern recognition of the position of the needle of the analog speedometer and character recognition of the numbers on the dial. Also, the vehicle speed data may be acquired by character recognition of the numbers displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described. As the display content of the display device to be detected, for example, the vehicle speed displayed on the speedometer may be used. The processing means may acquire the vehicle speed data by pattern recognition of the position of the needle of the analog speedometer and character recognition of the numbers on the dial. Also, the vehicle speed data may be acquired by character recognition of the numbers displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described.

[0045] The processing means may have a function of automatically detecting the speedometer by image recognition. By this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape of the speedometer of the vehicle on which the processing means is mounted in the image may be registered in advance in the processing means. Thereby, the processing means can easily detect the speedometer. The processing means may have a function of automatically detecting the speedometer by image recognition. By this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape of the speedometer of the vehicle on which the processing means is mounted in the image may be registered in advance in the processing means. Thereby, the processing means can easily detect the speedometer. The processing means may have a function of automatically detecting the speedometer by image recognition. By this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape of the speedometer of the vehicle on which the processing means is mounted in the image may be registered in advance in the processing means. Thereby, the processing means can easily detect the speedometer. The processing means may have a function of automatically detecting the speedometer by image recognition. By this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape of the speedometer of the vehicle on which the processing means is mounted in the image may be registered in advance in the processing means. Thereby, the processing means can easily detect the speedometer. The processing means may have a function of automatically detecting the speedometer by image recognition. By this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape of the speedometer of the vehicle on which the processing means is mounted in the image may be registered in advance in the processing means. Thereby, the processing means can easily detect the speedometer.

[0046] Standalone car navigation devices use location information obtained by GPS to acquire the current position. When GPS signals cannot be received, such as in a tunnel, the current position of the vehicle must be determined by other means. As an example, the current position of the vehicle on the map is moved at a constant speed of about 80% of the speed immediately before the GPS signal loss for 5 minutes. The mode for performing such processing is called the tunnel mode. The tunnel mode is effective not only when entering a tunnel but also when driving under an elevated road. However, if a traffic jam occurs while processing in the tunnel mode, the error in the current position becomes excessively large when assuming driving at a constant speed, and it becomes practically unusable. If an acceleration sensor is attached to the vehicle and the acceleration data from the acceleration sensor is used, the accuracy of the current position information in the tunnel mode can be improved. However, it is difficult and costly for the user to attach an acceleration sensor, so it is not practical. It is advisable to read the vehicle speed from the display content of the speedometer and update the current position information of the vehicle using the map matching function implemented in the car navigation system based on the read speed. This can improve the accuracy of the current position information of the vehicle. At locations where the road in the tunnel runs parallel to the ground road, and where the elevated road runs parallel to the ground road under the elevated road, it is advisable to select which road to match based on whether the tunnel mode is activated. For example, when the tunnel mode is activated, the vehicle position is

[0047]

[0048] ​​​​​​​​​​​​Map it to the road inside the tunnel or the ground road under the elevated road, and when the tunnel mode is not activated, map the position of the vehicle to the ground road or elevated road running parallel to the road inside the tunnel. When it is not, it may be mapped to the ground road or elevated road running parallel to the road inside the tunnel. It may be good.

[0049] As the display content of the display device to be detected, for example, it may be the remaining fuel amount displayed on the fuel gauge. The processing means may detect the remaining fuel amount by performing image recognition on the fuel gauge. When the processing means detects that the remaining fuel amount has fallen below a predetermined reference value, it may notify the driver of a message prompting refueling. For example, when it returns from the destination to near the vehicle storage location at home or at the company in conjunction with the current position information of the vehicle, if the remaining fuel amount is at or below the reference value, for example, 1 / 4 or less of a full tank, the processing means may issue a warning and prompt refueling. Also, when the car navigation device is linked with this system, the processing means may compare the distance to the destination with the remaining fuel amount and prompt refueling at the start or on the way of the route to the destination based on the comparison result. For example, near a gas station on the way of the route, it may output a message prompting refueling at a gas station near the current location.

[0050] The function of extracting the speedometer and fuel gauge from the image may be realized by the artificial intelligence (AI) installed in the processing means of this system. When it is difficult to extract the speedometer or fuel gauge from the image by artificial intelligence, the position and image information of the speedometer and fuel gauge in the image may be registered in the processing means in advance.

[0051] The processing means reads the fuel gauge of the vehicle provided by the car-sharing service, and when it detects that the remaining amount is less than a predetermined reference value, it performs refueling for the car-sharing service. It may be advisable to notify the driver that they can receive the benefits of the card. For example, issue a message such as "If you refuel now you can receive a discount equivalent to the amount for 15 minutes."

[0052] As the display content of the display device to be detected, for example, it may be the presence or absence of the lighting of the warning lamp in the instrument panel. For the warning lamp for determining the presence or absence of lighting, include the headlight lighting indicator, turn signal indicator, high beam indicator (high beam warning lamp), parking brake indicator, half door warning lamp, fuel warning lamp, etc. The processing means of this system can detect the state of the vehicle corresponding to the warning lamp by determining the presence or absence of the lighting of these warning lamps.

[0053] As the display content of the display device to be detected, for example, it may be the warning information displayed by the head-up display. The processing means captures the change in the image displayed by the head-up display to detect the warning content and executes a predetermined process according to the warning content. For example, it may be advisable to issue a warning by sound or voice. By changing the visual information-based warning to an auditory information-based warning, the effect that the driver can easily notice the warning can be obtained.

[0054] For example, in cooperation with the external brightness information and the current position information of the vehicle, when the processing means of this system detects that the headlight lighting indicator is not lit even though it is a situation where the headlights should be lit, it may be advisable to notify the driver to turn on the headlights. Situations where the headlights should be lit include, for example, when the outside of the vehicle is darker than a certain brightness ​​​​​​​​​​This includes dark situations, situations where the current position of the vehicle is inside a tunnel or a dark underpass, etc. Head Even though the situation where the headlight should be turned on has been resolved, if the headlight indicator remains lit, the processing means of this system may notify the driver to turn off the headlight. Since there are also tunnels that are bright inside due to lighting, if it is determined whether it is a situation where the headlight should be turned on only based on the brightness information outside the vehicle, a misjudgment may occur where it is wrongly determined that it is not a situation where the headlight should be turned on even though the vehicle is inside the tunnel. By determining whether the vehicle is inside the tunnel based on the current position information of the vehicle, such misjudgments can be avoided. Also, not only for the headlight, but also if it is detected that the indicator of the side marker light (small light) is not lit even though it should be lit in a situation where the side marker light should be turned on, it may be appropriate to notify the driver to turn on the side marker light. The processing means of this system determines the presence or absence of an oncoming vehicle from the image in front of the vehicle, and if it detects that the high beam indicator is lit even though there is an oncoming vehicle, it may be appropriate to warn the driver to switch to low beam. Since the high / low beam switching operation differs depending on the vehicle type, when the driver is driving a vehicle of a different vehicle type from the one they usually use, an event such as the intention to switch to low beam but remaining on high beam is likely to occur. In such a case, this warning function is particularly effective.

[0055] The processing means of this system determines the presence or absence of an oncoming vehicle from the image outside the vehicle when it is dark outside, and when it detects that the turn signal is blinking, it synchronizes with the blinking cycle of the turn signal and blinks the indicator inside the vehicle as a turn signal. Since the high / low beam switching operation differs depending on the vehicle type, when the driver is driving a vehicle of a different vehicle type from the one they usually use, an event such as the intention to switch to low beam but remaining on high beam is likely to occur. In such a case, this warning function is particularly effective.

[0056] The processing means of this system detects that the turn signal is blinking from the image outside the vehicle when it is dark outside, and synchronizes with the blinking cycle of the turn signal to blink the indicator inside the vehicle as a turn signal. It may be recognized as a turn signal indicator. By doing so, the processing means can easily find the turn signal indicator in the instrument panel.

[0057] Even though the processing means of this system has detected that there is no one in the vehicle from the image inside the vehicle, when it detects that the headlight and the side marker light are on, it may notify the driver outside the vehicle or the vehicle manager that it is forgotten to turn off the lights. This notification may be made, for example, by sending an email to a pre-registered email address. In addition, the processing means may notify by email or the like that the hazard lamp is forgotten to be turned off and the room light is forgotten to be turned off. The lighting state of the hazard lamp may be detected from the blinking state of the turn signal indicator. The lighting state of the room light may be detected from the brightness of the room light reflected in the image.

[0058] Generally, there are three types of settings for the room light: always-on mode, always-off mode, and door opening / closing state interlock mode (hereinafter simply referred to as interlock mode). In many vehicles, these three modes are circulated and updated each time the room light on / off switch is pressed once. There are vehicles that have a function where the room light turns on when the engine is stopped in a dark place (at night) and automatically turns off after a while (for example, after a few seconds have passed). There are also cases where you want to keep the room light on for a longer time to prepare for getting out of the car after stopping the engine. For example, after stopping the engine, during the period of organizing and taking out the purchased items and taking out the mobile terminal, etc., you want to keep the room light on. In such a case, when driving The hooligan makes the room light not automatically turn off when a certain period of time has elapsed since the engine stopped, so as to switch the setting of the room light to the interlock mode or the always-on mode. However, when the door is open, it is difficult to determine whether the current setting of the room light is in the always-on mode or the interlock mode. Even if the current setting of the room light is in the always-on mode, if one mistakenly thinks it is in the interlock mode and gets off the vehicle as it is, the room light will remain on. By using this system, the driver or administrator can notice that the room light has been forgotten to be turned off and turn off the room light, thereby preventing the vehicle's battery from running out due to the long-time lighting of the room light. Just by looking at the operation switch, it is impossible to know which mode the current setting of the room light is in. For example, in a vehicle with an operation switch structured such that the setting of the room light cyclically switches each time the push button is pressed, this function of this system is particularly beneficial. When the setting of the room light is in the interlock mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side is open. There are vehicles that have a function of keeping the room light on until a certain period of time (the turn-off delay time ) elapses after the door is closed when the setting of the room light is in the interlock mode. In such vehicles, the processing means of this system, after detecting that the door has been closed, will keep the room light on until a certain period of time (the turn-off delay time ) elapses.

[0059] When the setting of the room light is in the interlock mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side is open. When the setting of the room light is in the interlock mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side is open.

[0060] When the setting of the room light is in the interlock mode, there are vehicles that have a function of keeping the room light on until a certain period of time (the turn-off delay time ) elapses after the door is closed. In such vehicles, the processing means of this system, after detecting that the door has been closed, will keep the room light on until a certain period of time (the turn-off delay time Until the light-off delay time elapses, it is advisable not to determine whether the light-off has been forgotten. Light-off The delay time may be set by this system based on an input from the user.

[0061] Also, even if a certain period of time has elapsed after no one is left in the vehicle, if the processing means detects that the room light remains lit, it may transmit a signal to turn off the room light to the vehicle controller.

[0062] The processing means of this system may have a function of issuing an alarm to notify that any one of the headlights, side marker lights, hazard lights, room lights, etc. remains lit when the vehicle door is detected to be opened while in the lit or flashing state. This allows the driver to notice the forgotten light-off before getting out of the vehicle and prevent the forgotten light-off. .

[0063] Based on the lighting of the parking brake indicator, when the processing means of this system detects that the parking brake is engaged, even if it detects that the driver is sleeping from the driver's image, it is advisable not to issue an alarm. In addition, when the parking brake is engaged, even if it detects that the driver is performing an operation that is regarded as a dangerous operation during driving, such as looking at a mobile terminal, it is advisable not to issue an alarm.

[0064] When the processing means of this system detects that a warning light indicating that early inspection or repair is necessary remains lit for a certain period of time, it may have a function of notifying the vehicle dealer or the registered repairer of the vehicle's condition. For example, those interested in automobiles ​​​​​ If the warning light is on but the vehicle can still be driven, a user who is unaware may continue driving without noticing the warning light and may often neglect it. For example, when the warning light of the engine itself comes on, the vehicle usually becomes inoperable, but it can still be driven even when the warning light indicating an abnormality in the engine oil is on. Also, in the case of an O2 sensor abnormality, although the fuel consumption deteriorates, driving itself is still possible. If driving is possible and the vehicle is driven without inspection and repair, a minor fault may become more serious. For dealers and repair shops, the notification of the warning light being on is useful information for conducting inspection and repair business activities with the vehicle owner whose warning light is on. Also, vehicle owners can prevent the fault from becoming more serious by performing inspections and maintenance early.

[0065] As the display content of the display device to be detected, for example, it may be the temperature displayed in the temperature display area within the instrument panel that shows the temperature measurement value measured by a temperature sensor that measures the outside temperature. The processing means may have a function of reading the temperature by character recognition from the image displayed in the temperature display area of the instrument panel as a function of image analysis. As processing corresponding to the analysis result, the processing means may execute processing of transmitting temperature information to an external server via a communication device and a communication network.

[0066] For example, in order to obtain data related to pollen scattering, sensors are installed in the houses of general users, and by collecting measurement data from the sensors, a detailed pollen scattering situation is predicted, and a service that distributes pollen scattering information is provided by companies such as Weathernews Inc. Also, outdoor photos posted by general users or outdoor live cameras installed in the houses of ordinary families The collected images are incorporated into weather forecasts. In addition, services (such as Weathernews SOLiVE24) that distribute these images are provided. The system that provides these weather information services can utilize many vehicles as temperature measurement probes by collecting outside air temperature information measured by many vehicles. When many vehicles that transmit temperature information are distributed over a wide area, the temperature information collected from many vehicles can be utilized as spotty information. For example, it becomes possible to issue spotty heatstroke warnings that cannot be expressed by general weather information. For a company that provides such weather information services, the effect of being able to collect temperature information over a wide area without installing temperature sensors by itself can be obtained. The company that receives the provision of temperature information may provide free or discounted paid weather information services provided by the company as a gift to the owners of the vehicles that provided the temperature information. Not only the temperature information read from the temperature display area in the instrument panel, but also by recognizing the numbers displayed on the temperature display signs on the roadside that have moved into the image acquired by the camera, it is advisable to obtain outside air temperature information. The detection of the temperature display sign may be performed, for example, by recognizing characters such as "current temperature" in the sign. The obtained outside air temperature information may be transmitted by the processing means of this system to an external server in the same manner as the temperature information obtained from the temperature display area in the instrument panel. In the function of detecting the operation status of operating equipment by image analysis, the accelerator pedal and the brake

[0067]

[0068]

[0069] ​​​​​​​​​​​​​​It is advisable to include a function for detecting the position of the driver's foot relative to the pedal. As processing according to the operating state of the operating device, a process of notifying the driver should be executed so that the driver can easily recognize whether the driver's foot is at the position of the accelerator pedal or the brake pedal. For example, when the foot is at the position of the accelerator pedal and when it is at the position of the brake pedal, it may be possible to notify the driver by changing the emission color of the light-emitting element. Whether the foot is on the accelerator pedal or the brake pedal is detected, and the detection result is displayed on a display as visual information in a vehicle having a function of displaying it. The processing means of this system performs image analysis on the content displayed on this display and detects the position of the driver's foot relative to the accelerator pedal and the brake pedal. Furthermore, even if an obstacle is detected ahead by image analysis, when the foot is at the position of the accelerator pedal, an alarm may be issued by sound or the like. By the driver noticing the alarm, it is possible to suppress the occurrence of a mistake in stepping on the accelerator pedal and the brake pedal.

[0070] It is advisable to include a function of detecting the rotation angle of the steering wheel (handwheel) in the function of detecting the operating state of the operating device by image analysis. As processing according to the operating state of the operating device, a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed should be executed. The processing result may be given to, for example, a stand-alone car navigation device. The stand-alone car navigation device can receive GPS signals based on the information on the traveling direction and the traveling distance given from this system in a state where the GPS signals cannot be received.

[0071] ​​​​​​​​Even in such a situation, the driving route of the vehicle can be calculated.

[0072] The function for performing image analysis may include a function for detecting whether the vehicle is in autonomous driving (including autonomous driving levels 1 to 5) or manual driving (autonomous driving level 0). This detection may be performed, for example, by recognizing the display state of an indicator showing whether various functions of autonomous driving displayed on the instrument panel are operating from an image. As processing corresponding to the analysis result, for example, when it is detected that the autonomous driving function has been canceled, a process of notifying the driver that the autonomous driving function has been canceled may be executed. Also, if it is detected that the driver is not touching the steering wheel even though the autonomous driving function has been canceled, the processing means may issue a warning prompting the driver to operate the steering wheel. Furthermore, triggered by the cancellation of the autonomous driving function and the fact that the driver is not touching the steering wheel, the processing means may record the image captured by the camera. The recorded image can be used as evidence indicating that the driver's operation was inappropriate. In a vehicle equipped with a radar detector, when it is determined that the autonomous driving function is valid, the processing means may disable the function of the radar detector. This can eliminate the occurrence of unnecessary alarms from the radar detector. When the autonomous driving state is invalid, the processing means may enable the function of the radar detector. Whether the vehicle is in autonomous driving or manual driving, and what level (mode) of autonomous driving it is (mode) is

[0073] In a vehicle equipped with a radar detector, when it is determined that the autonomous driving function is valid, the processing means may disable the function of the radar detector. This can eliminate the occurrence of unnecessary alarms from the radar detector. When the autonomous driving state is invalid, the processing means may enable the function of the radar detector.

[0074] Whether the vehicle is in autonomous driving or manual driving, and what level (mode) of autonomous driving it is Determine whether it is the case based on indicators, the movement of the steering wheel, the position of the driver's hand, the direction in which the driver's head is facing, etc. It is advisable to make such a determination.

[0075] (6) As a function of the image analysis, the processing means has a function of recognizing characters displayed on a sticker attached to the vehicle from an image inside the vehicle, and performs processing corresponding to the analysis result and A system that executes a process of raising an alert according to the information indicated by the recognized characters may be used. It is advisable to do so.

[0076] By being alerted, the driver, administrator, or owner of the vehicle can direct their attention to the character information of the sticker attached to the vehicle and perform appropriate processing according to the character information. It is possible.

[0077] For example, it is advisable to include an inspection sticker (vehicle inspection sticker) in the sticker to be subjected to image analysis. On the inner side surface of the inspection sticker, the date of expiration of the vehicle inspection is displayed. The processing means reads the information of the date of expiration of the vehicle inspection by character recognition, and if the period from the current time to the expiration date of the vehicle inspection (the deadline of the vehicle inspection) becomes shorter than a predetermined period, It is advisable to alert the driver to undergo a vehicle inspection. For example, the processing means may alert by emitting a voice such as "The expiration date of the vehicle inspection is approaching." When the period until the expiration date of the vehicle inspection becomes even shorter, For example, the processing means may issue a warning in a voice such as "Please undergo a vehicle inspection."

[0078] For example, it is advisable to include a maintenance inspection completed sticker in the sticker to be subjected to image analysis. On the maintenance inspection completed sticker, the deadline for the next regular inspection is displayed in year, month, and day. The processing means The section reads, by character recognition, the information of the year, month, and day indicating the deadline of the regular inspection. If the period from the current time to the deadline of the regular inspection becomes shorter than a predetermined period, it is advisable to arouse the driver's attention to undergo the regular inspection. For example, the processing means may arouse the attention by emitting a voice such as "The deadline of the regular inspection is approaching." When the period until the deadline of the regular inspection becomes even shorter, for example, the processing means may issue a warning in a voice such as "Please undergo the regular inspection." It is advisable to include in the function of image analysis a function of determining whether an inspection mark is shown in the image captured by the camera. As a process corresponding to the analysis result, in the case where the inspection mark is not shown in the image, the processing means may execute a process of arousing the driver's attention to adjust the mounting position and posture of the camera so that the inspection mark is captured. For example, the processing means may emit a voice such as "The inspection mark is not shown. Please adjust the position and posture of the camera so that the inspection mark is captured." Thereby, the position and posture can be easily adjusted when the camera is installed. It is more advisable that the processing means has a function of reading the content displayed on the display devices in the vehicle, such as the speedometer and the fuel gauge, and has a function of arousing the driver's attention so that the display devices in the vehicle are captured in the image. It is advisable to include in the function of image analysis a function of determining whether stickers other than the stickers recognized by law are pasted on the windshield, the driver's side window, and the passenger's side window. As a process corresponding to the analysis result, when it is determined that stickers other than the stickers recognized by law are pasted, it is advisable to execute a process of notifying the driver or the like to peel off the stickers.

[0079]

[0080] ​​​​​​​​​​​​​​​​​

[0081] (7) As the image analysis function, the processing means includes a function of analyzing an image of an area where the passenger seat is reflected, and as processing corresponding to the analysis result, it is preferable to adopt a system that executes processing according to the situation of the passenger seat. For example, appropriate processing can be executed according to whether there is a person in the passenger seat or whether there is an object placed on the passenger seat. It is preferably mounted on a vehicle having a function of issuing a warning when a person sitting in the passenger seat is not wearing a seat belt. As the image analysis function, the processing means preferably has a function of determining whether there is a person sitting in the passenger seat from an image of an area where the passenger seat is reflected. As processing corresponding to the analysis result, when the processing means detects that there is no person sitting in the passenger seat, it is preferably to invalidate the function of issuing a warning for non-wearing of the seat belt of the passenger seat. As the image analysis function, the processing means preferably has a function of determining whether there is an object placed on the passenger seat. As processing corresponding to the analysis result, when there is an object placed on the passenger seat, the processing means preferably executes processing to invalidate the function of issuing a warning for non-wearing of the seat belt of the passenger seat.

[0082] For example, appropriate processing can be executed according to whether there is a person in the passenger seat or whether there is an object placed on the passenger seat. Appropriate processing can be executed accordingly.

[0083] The processing means is preferably mounted on a vehicle having a function of issuing a warning when a person sitting in the passenger seat is not wearing a seat belt. As the image analysis function, the processing means preferably has a function of determining whether there is a person sitting in the passenger seat from an image of an area where the passenger seat is reflected. As processing corresponding to the analysis result, when the processing means detects that there is no person sitting in the passenger seat, it is preferably to invalidate the function of issuing a warning for non-wearing of the seat belt of the passenger seat. The processing means preferably has a function of determining whether there is an object placed on the passenger seat from an image of an area where the passenger seat is reflected. As processing corresponding to the analysis result, when the processing means detects that there is an object placed on the passenger seat, it is preferably to execute processing to invalidate the function of issuing a warning for non-wearing of the seat belt of the passenger seat. The vehicle may misjudge that a person is in the passenger seat when an object equivalent to the weight of a human is placed on the passenger seat, and issue a warning for non-wearing of the seat belt. When this system is installed in the vehicle, when there is an object placed on the passenger seat, the vehicle will not issue a warning for non-wearing of the seat belt. For this reason, the driver will not be confused by the warning. In particular, when a heavy object is placed on the passenger seat, in a vehicle having a specification that the warning buzzer continuously sounds, this function For example, appropriate processing can be executed according to whether there is a person in the passenger seat or whether there is an object placed on the passenger seat. For example, appropriate processing can be executed according to whether there is a person in the passenger seat or whether there is an object placed on the passenger seat.

[0084] The vehicle may misjudge that a person is in the passenger seat when an object equivalent to the weight of a human is placed on the passenger seat, and issue a warning for non-wearing of the seat belt. When this system is installed in the vehicle, when there is an object placed on the passenger seat, the vehicle will not issue a warning for non-wearing of the seat belt. For this reason, the driver will not be confused by the warning. In particular, when a heavy object is placed on the passenger seat, in a vehicle having a specification that the warning buzzer continuously sounds, this function will prevent the driver from being confused by the warning. The energy is effective. To prevent the vehicle from issuing a warning for unfastened seat belts , for example, the processing means may send a signal to the vehicle controller through the in-vehicle network (CAN) to invalidate the issuance of the warning .

[0085] Regarding the rear seats as well, in a vehicle having a similar function, when it is detected that there is an object placed on the rear seat , the processing means may cancel the unfastened seat belt warning function

[0086] (8) As the function of the image analysis, the processing means has a function of comparing the current image inside the vehicle with the past image inside the vehicle , and as the processing corresponding to the analysis result, when the degree of difference between the current image inside the vehicle and the past image inside the vehicle exceeds a predetermined level , a system that executes a process of notifying is preferable .

[0087] By receiving the notification, the vehicle owner or administrator can know that the situation inside the vehicle has changed beyond a certain level . As the past image inside the vehicle, it is preferable to use an image in a state where no objects other than those originally attached to the vehicle have been brought into the vehicle . By comparing this image with the current image inside the vehicle , it is possible to determine whether there is something brought into the vehicle .

[0088] By comparing the images at a plurality of past times with the current image , it is preferable to calculate the time that the thing brought into the vehicle remains in the vehicle . When the processing means detects something that has been left in the vehicle for a long time

[0089] For example, in a state where no luggage is loaded on the vehicle and preferably only the driver is in the vehicle Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. The part within the image with an optical flow corresponding to the vehicle speed is recognized as a window. Other moving parts are recognized as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. The part within the image with an optical flow corresponding to the vehicle speed is recognized as a window. Other moving parts are recognized as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. The part within the image with an optical flow corresponding to the vehicle speed is recognized as a window. Other moving parts are recognized as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. The part within the image with an optical flow corresponding to the vehicle speed is recognized as a window. Other moving parts are recognized as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage. Press the initial setting button of this system and drive the vehicle. The processing means recognizes the part within the image with an optical flow corresponding to the vehicle speed as a window. It recognizes other moving parts as the driver. Parts other than those recognized as the window and the driver are recognized as areas where there is a possibility of placing objects, and are recorded as an image (video) in the unloaded state with no luggage loaded on the vehicle. The processing of the initial setting is completed by the processing up to this point. Thereafter, at the time of engine start, engine stop, etc., the difference between the unloaded state image and the current image is taken to identify the presence of objects brought into the vehicle. When there is no change in the position of the objects identified as being brought into the vehicle for a certain period, a warning is issued. Thereby, the driver and the like can be made aware of objects left in the vehicle for a long time. This function is particularly effective for people whose vehicle interiors quickly become full of garbage.

[0090] (9) As the function of the image analysis, the processing means has a function of determining whether a pre-registered object appears in the image inside the vehicle, and as the processing corresponding to the analysis result, it is preferable to be a system that executes the processing of issuing an alarm when a pre-registered object appears in the image inside the vehicle. As the function of the image analysis, the processing means has a function of determining whether a pre-registered object appears in the image inside the vehicle, and as the processing corresponding to the analysis result, it is preferable to be a system that executes the processing of issuing an alarm when a pre-registered object appears in the image inside the vehicle. As the function of the image analysis, the processing means has a function of determining whether a pre-registered object appears in the image inside the vehicle, and as the processing corresponding to the analysis result, it is preferable to be a system that executes the processing of issuing an alarm when a pre-registered object appears in the image inside the vehicle. As the function of the image analysis, the processing means has a function of determining whether a pre-registered object appears in the image inside the vehicle, and as the processing corresponding to the analysis result, it is preferable to be a system that executes the processing of issuing an alarm when a pre-registered object appears in the image inside the vehicle.

[0091] The driver and the like can be made aware of the presence of pre-registered objects inside the vehicle by the warning. For example, it is advisable to register an image of an object to be checked for forgotten items (a specific bag, wallet, etc.) with the processing means. When the processing means detects that a registered object appears in the image when there is no one inside the vehicle, it is advisable to issue an alarm or notify of a forgotten item. As an example of issuing an alarm, for example, it is advisable to sound a buzzer or the like attached inside the vehicle. The driver and the like can be made aware of the presence of pre-registered objects inside the vehicle by the warning. For example, it is advisable to register an image of an object to be checked for forgotten items (a specific bag, wallet, etc.) with the processing means. When the processing means detects that a registered object appears in the image when there is no one inside the vehicle, it is advisable to issue an alarm or notify of a forgotten item. As an example of issuing an alarm, for example, it is advisable to sound a buzzer or the like attached inside the vehicle. The driver and the like can be made aware of the presence of pre-registered objects inside the vehicle by the warning. For example, it is advisable to register an image of an object to be checked for forgotten items (a specific bag, wallet, etc.) with the processing means. When the processing means detects that a registered object appears in the image when there is no one inside the vehicle, it is advisable to issue an alarm or notify of a forgotten item. As an example of issuing an alarm, for example, it is advisable to sound a buzzer or the like attached inside the vehicle. The driver and the like can be made aware of the presence of pre-registered objects inside the vehicle by the warning. For example, it is advisable to register an image of an object to be checked for forgotten items (a specific bag, wallet, etc.) with the processing means. When the processing means detects that a registered object appears in the image when there is no one inside the vehicle, it is advisable to issue an alarm or notify of a forgotten item. As an example of issuing an alarm, for example, it is advisable to sound a buzzer or the like attached inside the vehicle. The driver and the like can be made aware of the presence of pre-registered objects inside the vehicle by the warning. For example, it is advisable to register an image of an object to be checked for forgotten items (a specific bag, wallet, etc.) with the processing means. When the processing means detects that a registered object appears in the image when there is no one inside the vehicle, it is advisable to issue an alarm or notify of a forgotten item. As an example of issuing an alarm, for example, it is advisable to sound a buzzer or the like attached inside the vehicle. As an example of knowledge, for example, it may be notified by e-mail or the like to a smartphone or a mobile phone of a driver or the like wirelessly. It may be notified.

[0092] When the processing means detects, for example, an item left in the vehicle and a movement to open the door, it may issue an alarm. By the alarm, the driver or the like can notice the item left before leaving the vehicle. When the processing means detects, for example, an item left in the vehicle and a movement to close the door, it may notify the driver or the like by e-mail or the like that there is an item left. The driver or the like can notice that there is an item left after leaving the vehicle. The detection of the movement of opening and closing the door may be performed, for example, by detecting an optical flow. It may be performed by detecting an optical flow, for example.

[0093] The processing means may have a function of detecting other movements in the vehicle by detecting an optical flow. For example, as a function of image analysis, the processing means may detect movements such as opening and closing of the door, opening and closing of the window, operation of the turn signal, operation of the steering wheel, raising and lowering operation of the sun visor, etc. This system can acquire these information without connecting to a connector of the in-vehicle network of the vehicle (for example, an OBD connector). When the sun visor is down, especially when imaging the driver side with a 360-degree camera attached to the windshield, the sun visor gets in the way and no image in that direction can be obtained at all. Therefore, even if the front of the vehicle is bright enough to determine that the sun visor is unnecessary, when the processing means detects that the sun visor is down, it may issue a warning.

[0094] When the sun visor is down, especially when imaging the driver side with a 360-degree camera attached to the windshield, the sun visor gets in the way and no image in that direction can be obtained at all. Therefore, even if the front of the vehicle is bright enough to determine that the sun visor is unnecessary, when the processing means detects that the sun visor is down, it may issue a warning. Whether the sun visor is down or not may be determined by the processing means based on the ratio of the area of the sun visor occupied in the image. It may issue a warning. Whether the sun visor is down or not may be determined by the processing means based on the ratio of the area of the sun visor occupied in the image. It may be determined.

[0095] (10) As the function of the said image analysis, the said processing means has the function of imaging an object brought close to the said camera by the user and performing image recognition, and as the processing corresponding to the analysis result, it is preferable to adopt a system that executes processing according to the information written on the object brought close to the said camera by the user. The user can execute processing according to the information written on the object by bringing the object he / she possesses close to the camera. For example, it is preferable that the object brought close to the camera is the driver's license of the driver. The processing means recognizes the expiration date written on the driver's license, and when the period from the current time to the expiration date is shorter than the predetermined period, it is preferable to notify the user that the renewal date of the driver's license is approaching.

[0096] For example, it is preferable that the object brought close to the camera is the receipt of a gas station. The processing means recognizes the date, fuel quantity, and amount displayed on the receipt, and it is preferable to record this information and create a database. For example, it is preferable that the object brought close to the camera is an identity certificate (for example, driver's license, employee ID card, resident card, etc.). The processing means recognizes the name etc. written on the identity certificate, and it is preferable to determine whether the person is a person registered in advance. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle.

[0097] For example, it is preferable that the object brought close to the camera is the driver's license of the driver. The processing means recognizes the expiration date written on the driver's license, and when the period from the current time to the expiration date is shorter than the predetermined period, it is preferable to notify the user that the renewal date of the driver's license is approaching. The processing means recognizes the expiration date written on the driver's license, and when the period from the current time to the expiration date is shorter than the predetermined period, it is preferable to notify the user that the renewal date of the driver's license is approaching. For example, it is preferable that the object brought close to the camera is the receipt of a gas station. The processing means recognizes the date, fuel quantity, and amount displayed on the receipt, and it is preferable to record this information and create a database. For example, it is preferable that the object brought close to the camera is the receipt of a gas station. The processing means recognizes the date, fuel quantity, and amount displayed on the receipt, and it is preferable to record this information and create a database.

[0098] For example, it is preferable that the object brought close to the camera is the receipt of a gas station. The processing means recognizes the date, fuel quantity, and amount displayed on the receipt, and it is preferable to record this information and create a database. The processing means recognizes the date, fuel quantity, and amount displayed on the receipt, and it is preferable to record this information and create a database. For example, it is preferable that the object brought close to the camera is an identity certificate (for example, driver's license, employee ID card, resident card, etc.). The processing means recognizes the name etc. written on the identity certificate, and it is preferable to determine whether the person is a person registered in advance. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle.

[0099] For example, it is preferable that the object brought close to the camera is an identity certificate (for example, driver's license, employee ID card, resident card, etc.). The processing means recognizes the name etc. written on the identity certificate, and it is preferable to determine whether the person is a person registered in advance. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. For example, it is preferable that the object brought close to the camera is an identity certificate (for example, driver's license, employee ID card, resident card, etc.). The processing means recognizes the name etc. written on the identity certificate, and it is preferable to determine whether the person is a person registered in advance. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. The processing means recognizes the name etc. written on the identity certificate, and it is preferable to determine whether the person is a person registered in advance. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. When the person of the identity certificate is not registered, it is preferable to perform processing to prohibit the start of the engine. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. Thereby, the person who can drive the vehicle can be limited to the person registered in advance. Further, the processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. The processing means preferably records the name etc. of the person of the identity certificate. The recorded information is used for the operation management of the vehicle. It can be used to specifically identify the driver.

[0100] As a function of image recognition, the processing means preferably has a function of recognizing an object that a user brings close to the camera outside the vehicle, and more preferably has a function of recognizing an object that a user brings close to the camera inside the vehicle. For example, when an object occupies a portion wider than a predetermined range within the viewing angle of the camera compared to a normal image, the processing means may recognize that an object has been brought close to the camera. In particular, when the processing means recognizes that an object occupying a portion wider than a predetermined range within the viewing angle of the camera is held by the user, it may be determined that the user has brought the object close to the camera. It is better to have a function of recognizing an object that a user brings close to the camera inside the vehicle. For example, when an object occupies a portion wider than a predetermined range within the viewing angle of the camera compared to a normal image, the processing means may recognize that an object has been brought close to the camera. When an object occupies a portion wider than a predetermined range within the viewing angle of the camera, the processing means may recognize that an object has been brought close to the camera. In particular, when the processing means recognizes that an object occupying a portion wider than a predetermined range within the viewing angle of the camera is held by the user, it may be determined that the user has brought the object close to the camera. When the processing means recognizes that an object occupying a portion wider than a predetermined range within the viewing angle of the camera is held by the user, it may be determined that the user has brought the object close to the camera. It is preferable to make such a determination.

[0101] (11) As a function of the image analysis, the processing means has a function of performing image analysis on an object captured in the front image, and as processing corresponding to the analysis result, it is preferable to use a system that executes processing for providing the driver with information that should be noted regarding the running of the vehicle. By prompting the driver to pay attention according to the situation ahead, it is possible to suppress the occurrence of accidents that may occur in the future when the vehicle moves forward. It is better to use a system that executes processing for providing the driver with information that should be noted regarding the running of the vehicle as processing corresponding to the analysis result.

[0102] By prompting the driver to pay attention according to the situation ahead, it is possible to suppress the occurrence of accidents that may occur in the future when the vehicle moves forward. It is possible to suppress the occurrence of accidents that may occur in the future when the vehicle moves forward.

[0103] The function of image analysis preferably includes a function of calculating at least one of the inter-vehicle distance and relative speed with the vehicle ahead. Based on the inter-vehicle distance or relative speed, a situation with a high risk of collision is detected, and it is preferable to notify the driver that the risk of collision is high by issuing an alarm or the like. The inter-vehicle distance may be calculated from information such as the size of the vehicle ahead in the image, for example, the size of the license plate, the size of the tread (tire interval), the width of the taillight, etc. The relative speed may be calculated from the inter-vehicle distance Based on the inter-vehicle distance or relative speed, a situation with a high risk of collision is detected, and it is preferable to notify the driver that the risk of collision is high by issuing an alarm or the like. Based on the inter-vehicle distance or relative speed, a situation with a high risk of collision is detected, and it is preferable to notify the driver that the risk of collision is high by issuing an alarm or the like. The inter-vehicle distance The inter-vehicle distance may be calculated from information such as the size of the vehicle ahead in the image, for example, the size of the license plate, the size of the tread (tire interval), the width of the taillight, etc. The inter-vehicle distance may be calculated from information such as the size of the vehicle ahead in the image, for example, the size of the license plate, the size of the tread (tire interval), the width of the taillight, etc. It may be calculated from the rate of change.

[0104] The image analysis function may include a function of detecting the color of a traffic signal ahead. The processing means may notify the driver to prompt a stop when the traffic signal is in the red light state.

[0105] The image analysis function may include a function of recognizing a road sign ahead. The processing means may prompt the driver according to the information indicated by the road sign. For example, when the road sign indicates the speed limit, the processing means may obtain the speed limit at the current location by character recognition and notify the driver of the speed limit. When the road sign indicates entry prohibition, the processing means may issue an alarm and notify the driver if the vehicle attempts to enter a road where entry is prohibited. For example when it is detected that the vehicle is about to drive in violation of the traffic rules indicated by the road sign or is already driving in violation of the traffic rules, it may notify the driver that it is a rule violation. This can prevent or suppress the occurrence of traffic rule violations. For example, when the processing means detects a sign indicating that the current driving section is an automatic speed enforcement section by recognizing the character string of the road sign, it may notify the driver to pay attention to excessive speed.

[0106] The image analysis function may include a function of detecting the congestion status of the lane adjacent to the own vehicle's driving lane. For example, whether congestion has occurred may be determined based on the average driving speed and inter-vehicle distance of the vehicles in the adjacent lane. For example, when the driving speed of the vehicles in the It is advisable to determine that a traffic jam is occurring. When it detects congestion in the adjacent lane, it detects vehicles jumping out from between the traffic jams. It is advisable to execute a process to alert the driver to be careful of the following: The lanes to be checked should include oncoming lanes and adjacent lanes traveling in the same direction as the vehicle. In particular, Shopping malls, theme parks, train stations and other crowded places, such as gas stations In areas where vehicles enter and exit, there is a high possibility that the lanes adjacent to the lane in which the vehicle is traveling in the same direction will also be congested. In such cases, it is effective to detect congestion in the adjacent lane traveling in the same direction as the vehicle. This can prevent accidents caused by sudden jumps out of the road. When the driving speed is below a certain speed, for example, 30 km / h, a warning sound is emitted to warn of pedestrians jumping out into the road. This will help to avoid unnecessary warning sounds. In addition, it is possible to detect situations where there is a high risk of the vehicle jumping out from the front image and warn the driver. For example, if there are a series of parked vehicles on the side of the road, parking meters are installed. If there are parked vehicles in the parking lot, it is a good idea to warn people not to rush out into the street.

[0107] For example, car navigation devices can provide traffic congestion probe information, general road and highway driving information, and surrounding facility information. The system has a function to alert drivers to driving caution based on information about residential areas, schools, etc. By using this system, it is possible to Even if the vehicle is in a traffic jam, a warning can be issued.

[0108] The image analysis function detects animals (dogs, cats, raccoons, wild boars, etc.) around the vehicle before starting off. It may be advisable to include a function for detecting animals or people. When the processing means detects an animal or a person around the vehicle, it may be advisable to inform the driver that there are animals or the like around the vehicle. This can suppress the occurrence of accidents immediately after starting.

[0109] The image analysis function may include a function for detecting that the vehicle is moving forward slowly from the flow of the surrounding scenery. When the processing means detects that the vehicle is moving forward slowly, it may issue an alarm. For example, after the vehicle has stopped once, when the vehicle starts to move little by little due to the relaxation of the force applied to the brake pedal in the creep phenomenon, it can inform the driver that the vehicle is moving and that there is a risk of collision. For example, when the vehicle speed is moving at a speed that allows for forward movement in the creep phenomenon, it may be advisable to determine that the vehicle is moving forward slowly. The speed of the flow of the surrounding scenery may be determined by detecting the optical flow.

[0110] The image analysis function may include a function for determining whether the vehicle in front is a large vehicle, such as a bus or a box truck with an aluminum loading platform. When the processing means detects that the vehicle in front is a large vehicle and approaches a traffic signal, it may be advisable to notify the driver of the presence of the traffic signal. For example, it may be advisable to emit a sound such as "Caution: Traffic Signal". The presence or absence of the traffic signal may be registered in the processing means in advance as POI information. For example, when a large vehicle is traveling ahead, the traffic signal in the traveling direction becomes difficult to see. It is preferable to predict the presence of the traffic signal by looking at the traffic signal in the oncoming lane or the pedestrian traffic signal, or to maintain a sufficient distance between vehicles so that the traffic signal in the traveling direction can be easily seen. However, if such attention is neglected, the traffic signal in the traveling direction may be overlooked. ​​​​​There is also a risk. In rural areas, there may be no traffic lights or pedestrian signals on the oncoming lane. When the leading vehicle is a large vehicle, notifying the driver of the presence of traffic lights can reduce the risk of the driver overlooking the traffic lights. Instead of directly notifying the presence of traffic lights, the processing means may issue a warning to prompt driving in a way that reduces the risk of overlooking traffic lights. For example, it may emit a voice such as "Please keep a sufficient distance." For example, it is advisable to issue a warning when the distance to the intersection where the traffic lights are installed is longer than the safe stopping distance of the own vehicle. The safe stopping distance of the own vehicle varies depending on the own vehicle speed and weather (whether it is rainy or not). It is advisable to set an appropriate distance based on the own vehicle speed and weather (whether it is rainy or not). For example, when the safe stopping distance on a sunny day is 50m, if the current weather is sunny, the distance to the leading large vehicle is less than 50m, and there is an intersection with traffic lights installed within 100m in the traveling direction outside the urban area as referred to by the car navigation device, a warning should be issued. When the leading vehicle is a large vehicle, notifying the driver of the presence of traffic lights can reduce the risk of the driver overlooking the traffic lights. Instead of directly notifying the presence of traffic lights, the processing means may issue a warning to prompt driving in a way that reduces the risk of overlooking traffic lights. For example, it may emit a voice such as "Please keep a sufficient distance." For example, it is advisable to issue a warning when the distance to the intersection where the traffic lights are installed is longer than the safe stopping distance of the own vehicle. The safe stopping distance of the own vehicle varies depending on the own vehicle speed and weather (whether it is rainy or not). It is advisable to set an appropriate distance based on the own vehicle speed and weather (whether it is rainy or not). For example, when the safe stopping distance on a sunny day is 50m, if the current weather is sunny, the distance to the leading large vehicle is less than 50m, and there is an intersection with traffic lights installed within 100m in the traveling direction outside the urban area as referred to by the car navigation device, a warning should be issued.

[0111] The image analysis function may include a function to determine whether the road ahead is on a slope, and if it is on a slope, a function to detect the degree of inclination. Depending on the degree of inclination of the slope, the processing means may prompt attention such as excessive speed, speed reduction, or setting to the hill climb mode. Depending on the degree of inclination of the slope, the processing means may prompt attention such as excessive speed, speed reduction, or setting to the hill climb mode.

[0112] If the vehicle is one that handles goods, such as a forklift or a truck, the processing means may have a function to recognize labels such as character strings or barcodes displayed on the goods from the captured image and determine whether the goods are the ones that the own vehicle should handle. If the vehicle is one that handles goods, such as a forklift or a truck, the processing means may have a function to recognize labels such as character strings or barcodes displayed on the goods from the captured image and determine whether the goods are the ones that the own vehicle should handle. It is advisable to attach​​​​ It is preferable to register in advance in the processing means the information represented by the sign. When the processing means detects that the vehicle is about to load a load that is not a load to be handled, it may execute processing to issue an alarm. When the processing means detects that the vehicle is about to load a load that is not a load to be handled, it may execute processing to issue an alarm.

[0113] The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow. The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow. The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow. The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow. The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow. The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered a tunnel, exited the tunnel, or is running inside the tunnel. In addition, the processing means may have a function of determining from the optical flow whether the vehicle is running under an elevated structure (under a parallel road). Further, the processing means may have a function of determining the presence or absence of a guardrail and the necessity of caution regarding a roadside ditch from the optical flow.

[0114] (12) As the function of the processing means for the image analysis, the processing means may have a function of recognizing characters written on the license plates of other vehicles captured in the image outside the vehicle, and perform processing corresponding to the analysis result, and record the information of the license plate identified by character recognition and / or transmit it via a network. As the function of the processing means for the image analysis, the processing means may have a function of recognizing characters written on the license plates of other vehicles captured in the image outside the vehicle, and perform processing corresponding to the analysis result, and record the information of the license plate identified by character recognition and / or transmit it via a network. As the function of the processing means for the image analysis, the processing means may have a function of recognizing characters written on the license plates of other vehicles captured in the image outside the vehicle, and perform processing corresponding to the analysis result, and record the information of the license plate identified by character recognition and / or transmit it via a network. As the function of the processing means for the image analysis, the processing means may have a function of recognizing characters written on the license plates of other vehicles captured in the image outside the vehicle, and perform processing corresponding to the analysis result, and record the information of the license plate identified by character recognition and / or transmit it via a network.

[0115] It is possible to identify the information of the license plate of another vehicle that is running near or parked near a vehicle equipped with an in-vehicle device having the function of the processing means of this system. The processing means may transmit the current position information of the vehicle equipped with this in-vehicle device and the date and time information when the license plate was imaged in association with the identified license plate information. The license plate information acquired together with the position information and the date and time information is installed on the road. It is possible to identify the information of the license plate of another vehicle that is running near or parked near a vehicle equipped with an in-vehicle device having the function of the processing means of this system. The processing means may transmit the current position information of the vehicle equipped with this in-vehicle device and the date and time information when the license plate was imaged in association with the identified license plate information. The license plate information acquired together with the position information and the date and time information is installed on the road. It is possible to identify the information of the license plate of another vehicle that is running near or parked near a vehicle equipped with an in-vehicle device having the function of the processing means of this system. The processing means may transmit the current position information of the vehicle equipped with this in-vehicle device and the date and time information when the license plate was imaged in association with the identified license plate information. The license plate information acquired together with the position information and the date and time information is installed on the road. It is possible to identify the information of the license plate of another vehicle that is running near or parked near a vehicle equipped with an in-vehicle device having the function of the processing means of this system. The processing means may transmit the current position information of the vehicle equipped with this in-vehicle device and the date and time information when the license plate was imaged in association with the identified license plate information. The license plate information acquired together with the position information and the date and time information is installed on the road. It is possible to identify the information of the license plate of another vehicle that is running near or parked near a vehicle equipped with an in-vehicle device having the function of the processing means of this system. The processing means may transmit the current position information of the vehicle equipped with this in-vehicle device and the date and time information when the license plate was imaged in association with the identified license plate information. The license plate information acquired together with the position information and the date and time information is installed on the road. ​The information of the license plate acquired by the vehicle number automatic reading device (N system) and both serve as useful information for knowing the driving routes of many vehicles. Thus, a system that can be regarded as a wide-area movement N system can be constructed by associating the vehicle number plate and the GPS information.

[0116] It is advisable to construct a center system that collects and associates position information, date and time information, and license plate information from the systems respectively installed in a plurality of vehicles via a network. This center system preferably has a function of extracting the position information and date and time information of a vehicle with specific license plate information and specifying the driving route of the vehicle.

[0117] (13) As a function of the image analysis, the processing means has a function of determining whether there is an edge of the window glass in the window portion of the vehicle, and as processing corresponding to the analysis result, when the edge of the window glass is detected in the window portion, it is advisable to use a system that executes a notification process. The presence of the edge of the window glass in the window portion means that the window glass has been forgotten to be closed. The driver etc. can be aware of forgetting to close the window glass by receiving a notification from this system.

[0118] To detect the edge of the window glass, for example, it is advisable to detect the discontinuity of the image due to the difference in refractive index between the window glass and the atmosphere. As the notification process, for example, it is advisable to perform processes such as wireless notification to the driver's mobile terminal etc. or notification by e-mail. In addition, it is advisable to perform the notification process by sounding a buzzer attached to the vehicle. A smoke film is pasted on the window of the vehicle on which the processing means is installed, and the processing means determines the area where the window is open and the window When it is detected that the window is open based on the difference in brightness between the image and the smoke area of the lattice That's fine.

[0119] (14) As the function of the image analysis, the processing means detects the presence or absence of the operation of the wiper, or the presence or absence of an increase or decrease in raindrops adhering to the windshield to determine whether it is raining has a function, and as a process corresponding to the analysis result, when it is determined that it is raining, it rains A system that executes a process of transmitting information notifying that it is raining from a communication device is preferable That's fine.

[0120] By receiving the information transmitted from this system, it is possible to know whether it is raining in the area where the vehicle is traveling. The processing means may notify an external device such as a server that it is raining and also notify the current position information of the vehicle. The external device can create a map of the area where it is raining by collecting information from a plurality of vehicles. Furthermore, the processing means may notify that it is raining and also notify the current time information. As a result, the external device can obtain information regarding the temporal change of the area where it is raining. It is preferable to notify an external device such as a server that it is raining and also notify the current position information of the vehicle. The external device can create a map of the area where it is raining by collecting information from a plurality of vehicles. Furthermore, the processing means may notify that it is raining and also notify the current time information. As a result, the external device can obtain information regarding the temporal change of the area where it is raining. Furthermore, the processing means may notify that it is raining and also notify the current time information. Thus, the external device can obtain information regarding the temporal change of the area where it is raining. information.

[0121] As the external device, for example, a server of an organization that provides information related to weather is preferable. The processing means may upload an image at a timing when the wiper is not captured, for example, at a timing when the wiper has moved down to the server. Information related to rainfall, such as rainfall amount, can be obtained from the image uploaded to the server. The server can utilize data such as rainfall amount collected from vehicles as big data in various application fields. For example, the bureau Regarding rainfall, such as rainfall amount, can be obtained. The server can utilize data such as rainfall amount collected from vehicles as big data in various application fields. For example, the bureau data such as rainfall amount collected from vehicles can be utilized as big data in various application fields. For example, the bureau ​It can be used for predicting local guerrilla heavy rain, etc. The processing means detects raindrops on the windshield If raindrops are attached but the number does not increase or decrease, it may be determined that the rain has stopped. If the processing means determines that the rain has stopped, it may notify the server that the rain has stopped. The server can obtain more detailed information about rainfall.

[0122] (15) As a function of the image analysis, the processing means has a function of determining the weather from the state of the sky captured in the image, and as a process corresponding to the analysis result, information about the weather is transmitted from the communication device A system that executes a process of transmitting may be used.

[0123] The information about the weather collected from multiple vehicles can be used as big data related to meteorological information. For example, from the color of the sky captured in the image, the presence or absence of clouds, etc., it is advisable to determine the weather such as clear sky, sunny , cloudy, etc. Furthermore, from the height of the sun and the brightness of the sky, detect the intensity of the morning sun or the evening sun, and it is advisable to transmit information that the morning sun or the evening sun is strong. Along with the information about the weather , it is advisable to transmit the current position information of the vehicle. The external device that has collected this information provides information that the morning sun or the evening sun is strong, and the current position information of the vehicle, to other vehicle drivers who will head to that location in the future. The driver who has received the provision of this information can notice that it is necessary to pay attention to the morning sun or the evening sun.

[0124] (16) The processing means has a function of detecting sound information from a microphone mounted on the vehicle, associates the detection result of the sound information with the analysis result of the image analysis function, and a system that executes a process according to the association result may be used.

[0125] Compared to processing based only on the results of sound detection or image analysis, By associating the two, more appropriate processing can be performed.

[0126] The processing means, as a function of image analysis, detects the state of a display unit of a device having a display unit mounted in the vehicle. It has the function of detecting changes in the state of the sound, and processes the detected sound information according to the analysis results. When the change in the display state is synchronized, the detected sound information and the change in the display state are It is advisable to execute a process of storing the information in association with the device.

[0127] When the processing means detects sound information from the microphone, the processing means associates the sound information with a change in the state of the display unit. Based on the attached and stored information, the device that emitted the sound can be identified. For example, the processing means may have a voice recognition function, and may recognize sound information from a microphone as being generated from a specific device. If it was issued from a specific device, it is recognized by voice recognition. This will allow you to avoid unnecessary voice recognition processing. The processing means can also detect whether the sound information from the microphone is generated from a specific device. If it is determined that the sound is generated, the area of ​​the image of the inside of the vehicle where the sound is generated is distinguishable from other areas. For example, the area where the device that generated the sound is captured can be recorded. It is a good idea to record an image of a surrounding rectangle or a closed curve of any shape on top of the original image. It is a good idea to record a semi-transparent geometric pattern over the area where the device generating the noise is located.

[0128] The processing means detects emergency vehicles, such as ambulances, fire engines, and police cars, in the image as a function of image analysis. The processing means may have a function to determine whether an emergency call is received from a microphone. When the sound of a vehicle siren is detected and an emergency vehicle is detected in the image, the two are associated and the self- It may be determined that there is an emergency vehicle near the vehicle. The processing means may notify the driver of a message prompting attention when it is determined that there is an emergency vehicle.

[0129] (17) There are a plurality of cameras installed inside the vehicle, As the function of the image analysis, the processing means has a function of detecting the positions of other cameras captured in the image, and as the processing corresponding to the analysis result, it is preferable to use a system that executes processing for specifying the positional relationship of the plurality of cameras.

[0130] Images of a wide area inside the vehicle that are difficult to capture with a single camera can be obtained. When the positional relationship of the plurality of cameras is specified, the positional relationship of the images captured by each camera can be specified.

[0131] The processing means preferably has a function of specifying the model of other cameras captured in the image and specifying the angle of view from the model. The processing means preferably has a function of synthesizing the images captured by the plurality of cameras. By synthesizing the images, the situation inside the vehicle can be intuitively grasped as compared with the case of displaying the individual images of the cameras.

[0132] For example, at least some of the functions of the processing means may be realized by a viewer such as a personal computer. It is preferable to display a label such as an icon of the camera so that the position of the camera can be understood in the displayed image. It is difficult to specify the position of the camera inside the vehicle from the image captured by the camera (itself). It is preferable to detect the position of the camera inside the vehicle from the images of other cameras captured by the camera. For example, synthesize and display the images captured by three cameras. ​​​​​​​​​​​ When using a composite image, it is recommended to display the positions of the three cameras using icons or other means. It is a good idea to use different icons depending on the model or angle of view.

[0133] The processing means displays the image captured by each camera and indicates the position of the camera within the image. It is recommended to have a function to display the camera's position and the field of view (area captured by the camera). For example, depending on the function, such as a full-sphere 360-degree camera, a half-sphere 360-degree camera, or a wide-angle camera, The user who sees the image can see which camera is which. It is possible to intuitively grasp the area being imaged.

[0134] (18) The camera has an image circle smaller than the diagonal length of the imaging surface. It is a 60 degree camera, The processing means may include a basic pattern for calculating a feature amount of an image as a function of the image analysis. The system has a function of deforming the image according to the position within the image circle. It is good.

[0135] The basic pattern is used to correct the image distortion within the image circle. This can reduce the load on the processing means. The angle is defined for images captured using a standard lens (a typical wide-angle lens). The projection methods to obtain the image circle include, for example, orthogonal projection, equidistant projection, and stereographic projection. There are projection methods such as equi-solid angle projection, and the basic pattern is transformed according to these projection methods. To obtain an image circle, it is recommended to use a circular fisheye lens. The imaging surface is rectangular, and the diameter of the image circle should be shorter than the long side. It is even better if it is equal to or shorter than the short side.

[0136] For example, as a basic pattern, it is advisable to prepare in advance a template pattern of the image to be detected. In this case, the template pattern may be deformed according to the position within the image circle. In addition, for example, when recognizing an image using Haar-like features, each of a plurality of rectangular patterns for obtaining Haar-like feature amounts may be deformed according to the position within the image circle. For each of the plurality of rectangular patterns for obtaining Haar-like feature amounts may be deformed according to the position within the image circle. When performing image recognition, among the regions within the image circle, it is advisable to limit the region in which a specific object to be detected is captured, and perform image analysis on the limited region. For example,

[0137] it is advisable to implement the function of the in-vehicle device of this system with a drive recorder. Generally, since the recommended installation position of the drive recorder is determined in advance, the regions in which the driver, traffic lights, passengers in the rear seat, etc. are captured within the image circle are almost determined. Therefore, according to the person or object to be detected, it is advisable to limit the region for performing image analysis to detect the person or object to the region in which the person or object is captured. For example, when analyzing an image in front of the vehicle, it is advisable to analyze the region in which the windshield is captured. when analyzing an image in front of the vehicle, it is advisable to analyze the region in which the windshield is captured. when analyzing an image in front of the vehicle, it is advisable to analyze the region in which the windshield is captured. Therefore, according to the person or object to be detected, it is advisable to limit the region for performing image analysis to detect the person or object to the region in which the person or object is captured. For example, when analyzing an image in front of the vehicle, it is advisable to analyze the region in which the windshield is captured. when analyzing an image in front of the vehicle, it is advisable to analyze the region in which the windshield is captured.

[0138] In addition, for example, it is advisable to convert a specific region of the image circle into a planar image (parallel projection image) and perform matching on the converted planar image using a normal basic pattern. In addition, for example, it is advisable to convert a specific region of the image circle into a planar image (parallel projection image) and perform matching on the converted planar image using a normal basic pattern.

[0139] Also, it is advisable to map the image within the image circle onto a sphere using a predetermined conversion function, zoom in on a part of the image mapped onto the sphere, and perform image analysis on the zoomed-in part as a planar image while ignoring distortion. Also, it is advisable to map the image within the image circle onto a sphere using a predetermined conversion function, zoom in on a part of the image mapped onto the sphere, and perform image analysis on the zoomed-in part as a planar image while ignoring distortion. It may be performed. When it is desired to detect a specific object, for example, a road sign, it may be advisable to zoom in on the area where the specific object is reflected and perform image analysis. When it is desired to detect a road sign, it may be advisable to zoom in on the area where the road sign is reflected in the forward image and perform image analysis. It may be performed. When it is desired to detect a specific object, for example, a road sign, it may be advisable to zoom in on the area where the specific object is reflected and perform image analysis. When it is desired to detect a road sign, it may be advisable to zoom in on the area where the road sign is reflected in the forward image and perform image analysis. It may be performed. When it is desired to detect a specific object, for example, a road sign, it may be advisable to zoom in on the area where the specific object is reflected and perform image analysis. When it is desired to detect a road sign, it may be advisable to zoom in on the area where the road sign is reflected in the forward image and perform image analysis.

[0140] (19) As the function of the processing means, from the perspective of privacy protection, it has a function of specifying an area that should not be made public in the image as a privacy protection area, and as a process corresponding to the analysis result, it generates, from the image captured by the camera, an image in which a person reflected in the privacy protection area cannot be identified by appearance. It is advisable to use a system that executes the process. By making it impossible to identify a person, the privacy of that person can be protected. As an area that should not be made public, for example, it may be advisable to use an area where the inside of the vehicle is reflected. In particular, when an omnidirectional camera is used to acquire an image, the inside of the vehicle may be captured, but there may be cases where the image of the inside of the vehicle is not desired to be left. In such cases, for example, it may be advisable to paint the area inside the vehicle black, or apply a mosaic to the area inside the vehicle, and record the image outside the vehicle as it is. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance.

[0141] A normal mode for recording an image in which a person within the privacy protection area can be identified, and it may be possible to protect the privacy of that person. As an area that should not be made public, for example, it may be advisable to use an area where the inside of the vehicle is reflected. In particular, when an omnidirectional camera is used to acquire an image, the inside of the vehicle may be captured, but there may be cases where the image of the inside of the vehicle is not desired to be left. In such cases, for example, it may be advisable to paint the area inside the vehicle black, or apply a mosaic to the area inside the vehicle, and record the image outside the vehicle as it is. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. In particular, when an omnidirectional camera is used to acquire an image, the inside of the vehicle may be captured, but there may be cases where the image of the inside of the vehicle is not desired to be left. In such cases, for example, it may be advisable to paint the area inside the vehicle black, or apply a mosaic to the area inside the vehicle, and record the image outside the vehicle as it is. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. In particular, when an omnidirectional camera is used to acquire an image, the inside of the vehicle may be captured, but there may be cases where the image of the inside of the vehicle is not desired to be left. In such cases, for example, it may be advisable to paint the area inside the vehicle black, or apply a mosaic to the area inside the vehicle, and record the image outside the vehicle as it is. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. In such cases, for example, it may be advisable to paint the area inside the vehicle black, or apply a mosaic to the area inside the vehicle, and record the image outside the vehicle as it is. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. The processing means may, for example, distinguish between the area where the inside of the vehicle is reflected and the area where the outside of the vehicle is reflected by performing image analysis. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. For example, it may be advisable to consider an area where an optical flow is generated at a speed corresponding to the vehicle speed as the outside of the vehicle. The processing means may have a function that allows the user to preset a privacy protection area in advance. The processing means may have a function that allows the user to preset a privacy protection area in advance.

[0142] A normal mode for recording an image in which a person within the privacy protection area can be identified, and An image in which a person captured in the privacy protection area cannot be identified by appearance It is preferable to allow the user to select a privacy mode for recording such an image. The user can switch between the normal mode and the privacy mode according to the purpose and usage situation of the vehicle .

[0143] The privacy area may be an area in which a person is captured. For example, the processing means detects a person by performing image analysis, and sets the area surrounded by the contour line of the detected person as the privacy area. Alternatively, only the face part of the detected person may be set as the privacy area . At this time, the person may be displayed as a silhouette. Alternatively, the outline of the person may be displayed, and an image when there is no person inside the outline may be displayed. At this time, the person will appear transparent. As the contour of the person, for example, the edge of something with movement may be displayed .

[0144] The processing means may display an avatar instead of the person in the area where the person is captured. The displayed avatar should be similar to the original person. For example, according to the gender, body shape, outer shape of the face, etc., an avatar reflecting those characteristics may be displayed. In advance, the person and the avatar are associated , and the processing means identifies the person captured in the image and displays the avatar associated with the identified person . The avatar may be created in advance for each person . The avatar may be moved in accordance with the movement of the person. For example, when the person raises a hand, the hand of the avatar is raised, and when the person turns sideways, the avatar is also turned sideways . The expression of the avatar may be changed according to the expression of the person. For example, the processing means determines that the person ​​When detecting expressions such as smiling, sad, or angry, the expression of the avatar may be changed to, respectively, a smiling expression, a sad expression, and an angry expression. The processing means may read the emotions of the person and change the color of the avatar according to the emotions of the person. For example, when it is detected that the person is angry, the color of the avatar may be changed to a color representing anger, for example, red.

[0145] Before recording the image, the processing means may generate an image in which a person captured in the image by the camera cannot be identified by appearance from the image captured by the camera, and record the generated image. Also, when playing back the image captured by the camera, it may be converted into an image in which a person captured in the privacy protection area cannot be identified by appearance and displayed.

[0146] (20) The processing means may be a system that executes a process of converting the result of image analysis into text

[0147] data or encoding and recording it as a process corresponding to the analysis result. By converting to text data or encoding, it becomes possible to easily find a desired image from the recorded image. For example, when a person is detected in the image, the text data may be "person detection", and when a red signal is detected in the image, the text data may be "red signal" as the text data. The generated text data may be recorded in association with the image that is the source of the generation of the text data. Instead of text data, the result of image analysis may be encoded (converted into binary data) by associating the result of image analysis with a code.

[0148] ​​​​​(21) As the image analysis function, the processing means has a function of detecting an area with black blurring in the image. As processing corresponding to the analysis result, it is preferable to use a system that executes processing for brightly restoring the area with black blurring in the image and displaying it on the display screen. The user who views the image can more easily recognize objects and people in the area with black blurring. In particular, when acquiring an image using an omnidirectional camera, since a bright outdoor scene and a dark interior of the vehicle are simultaneously imaged in the image, the area where the interior of the vehicle is reflected is likely to have black blurring. The process of detecting the area with black blurring, brightly restoring it, and displaying it is particularly effective when acquiring an image using an omnidirectional camera. As the process of brightly restoring, for example, gamma correction, color correction, conversion to black-and-white image, contrast enhancement, etc. may be performed.

[0149]

[0150] (22) The processing means has a function of dividing the imaging range imaged by the camera into a plurality of sections for the user, and a function of weighting each section. Based on the brightness of each of the sections and the weight assigned to the section, imaging conditions are determined, and it is preferable to use a system that has a function of causing the camera to image under the determined imaging conditions.

[0151] By increasing the weight of the section in the image that the user wants to view and decreasing the weight of the section that does not need to be viewed, the user can acquire an image under imaging conditions suitable for the area to be viewed. The user can divide the imaging range into an arbitrary shape according to the shape of the area to be viewed or the area that does not need to be viewed. For this reason, for example, compared to the case of weighting sections uniformly divided into a square lattice regardless of the content of the image, it becomes possible to perform more appropriate weighting for the user. ​​​​​​​​​​​​​​​

[0152] In particular, when imaging using a 360-degree camera, compared with the case of using a general camera, for each vehicle type the difference in the position within the imaging range where the vehicle parts are captured becomes larger. Therefore, in advance, the general weighting table set in the firmware of the processing means and the relationship with the actually expected imaging range are more likely to not match. Since a general weighting table is used, instead of that, a function that enables determining the weighting using PC browser software or the like has a particularly remarkable effect when using a 360 degree camera. For example, in the unweighted state, the image captured by the camera is analyzed using AI software on a PC or the like, and it is estimated what is captured in which part of the imaging range. For example, a weight of "5" is assigned to the road part, a weight of "3" is assigned to the face part, a weight of "1" is assigned to the instrument panel part, and a weight of "0" is assigned to outside the range of the round image. This would be appropriate.

[0153] The processing means may determine, as imaging conditions, for example, gain, exposure time, etc. Two half sky cameras are opposed to each other at the back to form an omnidirectional camera, and the imaging conditions of the two half sky cameras may be made the same. Thereby, when synthesizing the images captured by the two half sky cameras, an effect can be obtained that the boundary between the two images becomes less distinguishable.

[0154] When the processing means makes the user make a distinction, a part of the area within the imaging range may be surrounded by a closed curve, and the inner area surrounded by the closed curve may be made into one section. When the user divides the inner area surrounded by the closed curve into two with a curve, each of the divided areas may be made into one section. The processing means may make the user input a weight numerically for each set section. The numerical value to be input It is advisable to allow selection from a list such as a pull-down or pop-up.

[0155] Instead of, or in addition to, the function of allowing the input of weights numerically, the processing means may have a function of allowing the input of the type of section. For each type of section, a weight is associated in advance. When the type of section is input, the processing means may associate the weight associated with the type with the corresponding section. As the types of sections, for example, types such as driver, meter panel, car navigation screen, windshield, rear seat, passenger seat, etc. may be provided. The processing means may have a function that allows the user to change the correspondence between the type of section and the weight. A list of the types of sections may be made available for the user to register in advance. As a function for allowing the user to assign weights to each section, the processing means may have a function of setting sections, for example, by allowing the user to specify an area of interest within the imaging range. The weight of the section specified by the user as the area of interest may be made larger than the weights of other areas. This function corresponds to an example in which, in the function of allowing the user to input the type of section, the sections are classified into two types: the section the user wants to view and other sections.

[0156]

[0157]

[0157] The processing means may cause the user to divide the imaging range in an interactive form (wizard form). For example, the processing means may output a message such as "Please enclose the unnecessary area." or "Please fill in the unnecessary area.", and recognize the area enclosed or filled in by the user as the "unnecessary area". In addition, the processing means may output messages such as "Please fill in the area of the driver." and "Please fill in the road portion." Output a message and recognize the filled area as a section with a driver type or a section with a road type. The operation of surrounding with a closed curve or the operation of filling may be performed using a pointing device such as a touch panel or a mouse. It is advisable to do so. .

[0158] The processing means may realize the function of dividing the imaging range into a plurality of sections based on the captured image by artificial intelligence (AI). For example, from the image, automatically discriminate the inside and outside of the vehicle, seats, dashboards, instrument panels, etc., and divide the imaging range for each automatically discriminated component. The processing means may display the result of the division on the display screen and let the user determine whether there is a problem with this division. If the user determines that the division performed by the AI is incorrect, the processing means may have the function of allowing the user to correct the division. If the user determines that the weight or type corresponding to the section is incorrect, the processing means may allow the user to select the correct type from a pull-down or pop-up list.

[0159] (23) The processing means has a function of allowing the user to divide the imaging range captured by the camera into a plurality of sections and a function of weighting each section, and as a process corresponding to the analysis result, a system that executes a process of compressing by making the block at the time of compressing a section with a relatively small weight larger than the block at the time of compressing a section with a relatively large weight is preferable.

[0160] The user can increase the compression degree of a section with a low importance by increasing the weight of a section with a high importance and decreasing the weight of a section with a low importance. As a result, the entire image is uniformly ​​​​​​​​​​​While compressing, the quality of the image in the section with high importance can be maintained high, and the capacity of the image data can be reduced. For example, in the section with high weight, 8×8 pixels can be taken as one block, and in the section with low weight, 16×16 pixels can be taken as one block for compression.

[0161] In the area outside the circular image formed on the rectangular imaging surface of the camera, the image of the lens barrel part may be reflected. Since the image outside the circular image is unnecessary, this part may be filled with black . Thereby, the capacity of the image data including the pixels outside the circular image can be reduced . Also, excluding the pixels outside the circular image, only the pixels inside the circular image may be raster scanned to obtain the image data. Thereby, the capacity of the image data can be reduced.

[0162] (24) As the image analysis function, the processing means has a function of detecting a following vehicle and a function of detecting the distance to the following vehicle , and as processing corresponding to the analysis result, it is preferable to use a system that executes processing for detecting the tailgating of the following vehicle based on the distance to the following vehicle . .

[0163] By executing processing according to the determination result, it becomes possible to take appropriate measures against tailgating . For example, even though the speed of the vehicle is equal to or higher than a predetermined reference value , when the same following vehicle continues to exist within a predetermined determination distance for a predetermined determination time or more , it is preferable to determine that the following vehicle is performing tailgating.

[0164] As the camera, it is preferable to use one having a function of measuring the distance to the object for each pixel Yes. For example, as the image sensor of the camera, a back-illuminated time-of-flight distance image sensor may be used.

[0165] When the processing means determines that the following vehicle is performing weaving driving, as processing according to the determination result, for example, notification to the driver, recording of the rear image, advice to the driver, traffic control processing such as reporting to an agency (for example, the police) may be performed. When reporting to the traffic control agency, it is advisable to attach images of the vehicle performing weaving driving and its driver. The processing means, by image analysis, when detecting that the vehicle performing weaving driving has overtaken the own vehicle and cut in front of the own vehicle, notifies the driver that there is a possibility of being harassed by sudden braking.

[0166] The processing means may detect people and vehicles around the vehicle and control the vehicle according to the distances to the people and vehicles. For example, the processing means, based on the speed of the own vehicle, the distance to the person in front, and the determination result of whether the person is on the planned driving route of the own vehicle, if it is determined that there is a high possibility of contact with the person, may perform control to stop the vehicle.

[0167] (25) It is advisable to use a program for realizing the functions of the processing means described in any one of (1) to (24) above on a computer.

[0168] The inventions described in (1) to (24) above can be arbitrarily combined. For example, it may be configured to add at least a part of the configuration of at least one of the inventions from (2) onwards to all or a part of the configuration of the invention described in (1). In particular, it is advisable to make an invention in which at least a part of the configuration of at least one of the inventions from (2) onwards is added to the invention described in (1). Also ​ Any configuration may be extracted from the inventions shown in (1) to (24), and the extracted configurations may be combined. The applicant of the present application intends to obtain rights for inventions including these configurations.

Advantages of the Invention

[0169] Not only records the image captured by the camera attached to the vehicle, but also can utilize the image more effectively.

[0170] The effects of the invention of the present application are not limited to this, and the effects resulting from the disclosed configurations in this specification, drawings, etc. are also disclosed, and the applicant also intends to obtain rights for the configurations that exhibit such effects through divisional applications, amendments, etc. For example, in this specification, the parts described as "can" are descriptions that clearly indicate the effects, and even if there is no description of "can", there are parts that indicate the effects. Further, even if there is no such description, there are effects grasped by the said configuration.

Brief Description of the Drawings

[0171]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0172] [First Embodiment] Referring to FIGS. 1 and 2, a system according to the first embodiment of the present invention will be described. The functions of this system are realized by a drive recorder attached to a vehicle.

[0173] FIG. 1 is a diagram showing the front glass, dashboard, etc. inside a vehicle with a drive recorder 1 mounted on the vehicle. The vehicle on which the drive recorder 1 is mounted is a passenger car. The drive recorder 1 is attached to a position on the passenger seat side adjacent to the room mirror 3 near the center in the left - right direction at the upper part of the front glass 2 of the vehicle. The drive recorder 1 is fixed by being attached to the inner surface of the front glass 2 with an attachment member such as double - sided tape. The drive recorder 1 is connected to the cigarette socket 5 of the vehicle via a power cable 4. When the accessory power of the vehicle is turned on, power is supplied from the cigarette socket 5 to the drive recorder 1. The drive recorder 1 includes a main body 10 and a 360 - degree omnidirectional camera 11. The camera 11 has a fixed relative position and orientation with respect to the vehicle in a normal use state.

[0174] FIG. 2 is a block diagram of the drive recorder 1 as a system according to the first embodiment. The drive recorder 1 includes a processing unit 20, a camera 11, a GPS receiver 12, an SD card reader 13. An acceleration sensor, 14. A speaker, 15. A display, and 16. An operation button 17 are included. Note that the SD card reader 13 should strictly be called an "SD card reader / writer", but in this specification, it is simply called an "SD card reader".

[0175] The speaker 15 and the display 16 function as output means for notifying the user of various information. The speaker 15 emits alarms and various information by sound or voice based on the control by the processing unit 20. The display 16 displays various information as images based on the control by the processing unit 20. The operation button 17 functions as an input means for the user to give various commands to the drive recorder 1.

[0176] The camera 11 is a 360-degree omnidirectional camera and captures the entire area in the front-rear direction, up-down direction, and left-right direction of the vehicle as a video. Therefore, the camera 11 captures the outside of the vehicle through the front glass, rear glass, and side windows, and also captures the inside of the vehicle including the driver's seat, passenger seat, rear seat, instrument panel, center console, etc. The video captured by the camera 11 is taken into the processing unit 20 as image data. The processing unit 20 records the images on the SD card connected to the SD card reader 13 based on the image data input from the camera 11.

[0177] The GPS receiver 12 detects the position information of the own vehicle at the current time based on the instruction of the processing unit 20. The position information includes information such as the time obtained based on the signal from the GPS satellite, the speed of the own vehicle, longitude, latitude, altitude, etc. The processing unit 20 performs processing to record the history of these position information.

[0178] Based on the control by the processing unit 20, the SD card reader 13 reads the data of the SD card inserted into the SD card insertion slot or records data on the SD card.

[0179] As the acceleration sensor 14, a three-axis type sensor that detects the acceleration and inclination in the directions of each of the three axes (x-axis, y-axis, z-axis) is used. The acceleration sensor 14 always outputs the detected values to the processing unit 20. The processing unit 20 acquires the acceleration information of the three axes, for example, every 10 ms.

[0180] The processing unit 20 includes a well-known CPU 20a, memories such as a ROM 20b and a RAM 20c, a timer 20d, and other peripheral circuits. Various programs are stored in the ROM 20b of the processing unit 20. By executing these programs, the processing unit 20 realizes various functions. The various programs include an operating system (OS), a GPS information processing program, an image analysis program, and the like.

[0181] By executing the GPS information processing program, the processing unit 20 stores the GPS information received by the GPS receiver 12 in the SD card connected to the SD card reader 13. By executing the image processing program, the processing unit 20 always records the moving image captured by the camera 11 as image data associated with the time information and the position information in a predetermined recording area of the SD card.

[0182] By executing the image analysis program, the processing unit 20 performs image analysis of the image captured by the camera 11. Furthermore, the processing unit 20 executes various processes corresponding to the analysis results.

[0183] By mounting the system according to the first embodiment on a vehicle, not only can an image captured by a camera attached to the vehicle be recorded, but the image can also be utilized more effectively. When the image captured by the camera 11 is made into a moving image, by performing image analysis of the moving image, it becomes possible to easily determine the movement of the subject, the presence or absence of the subject before and after a certain point in time, and the like. When the image captured by the camera 11 is made into a moving image, by performing image analysis of the moving image, it becomes possible to easily determine the movement of the subject, the presence or absence of the subject before and after a certain point in time, and the like. When the image captured by the camera 11 is made into a moving image, by performing image analysis of the moving image, it becomes possible to easily determine the movement of the subject, the presence or absence of the subject before and after a certain point in time, and the like. .

[0184] Next, a modification of the first embodiment will be described. In the first embodiment, the drive recorder 1 is attached to a passenger car, but in the modification, it is attached to other types of vehicles, such as a freight car or a special vehicle. Also, in the first embodiment, the camera 11 captures both the inside and the outside of the vehicle, but in the modification, the camera 11 captures at least one of the inside and the outside of the vehicle. Regarding "inside the vehicle" and "outside the vehicle", for example, the following interpretation may be used. In the first embodiment, the drive recorder 1 is attached to a passenger car, but in the modification, it is attached to other types of vehicles, such as a freight car or a special vehicle. Also, in the first embodiment, the camera 11 captures both the inside and the outside of the vehicle, but in the modification, the camera 11 captures at least one of the inside and the outside of the vehicle. Regarding "inside the vehicle" and "outside the vehicle", for example, the following interpretation may be used. In the first embodiment, the drive recorder 1 is attached to a passenger car, but in the modification, it is attached to other types of vehicles, such as a freight car or a special vehicle. Also, in the first embodiment, the camera 11 captures both the inside and the outside of the vehicle, but in the modification, the camera 11 captures at least one of the inside and the outside of the vehicle. Regarding "inside the vehicle" and "outside the vehicle", for example, the following interpretation may be used. When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". .

[0185] When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". When the internal space and the external space are clearly separated by the body of the vehicle, the inside of the body may be referred to as "inside the vehicle", and the outside of the body may be referred to as "outside the vehicle". For example, in the case of a van where the passenger space and the cargo space are continuous, it may be appropriate to refer to the continuous passenger space and cargo space as "inside the vehicle". In the case of a sedan where the passenger space and the cargo space are partitioned, it may also be appropriate to refer to both the passenger space and the cargo space as "inside the vehicle". In a covered freight car with an enclosed cargo bed, it may be appropriate to refer to the inside of the passenger space and the enclosed cargo bed as "inside the vehicle". In an open-bed freight car (flat-bed vehicle) with an open cargo bed, it is difficult to clearly distinguish the space for accommodating the cargo on the cargo bed from the surrounding space, but it may also be appropriate to refer to the space for accommodating the open cargo as "inside the vehicle". In addition, the passenger space of an open-bed freight car may also be referred to as "inside the vehicle". In some special vehicles such as forklifts, both the passenger space and the cargo space are open and it may be difficult to clearly demarcate the passenger space and the cargo space from the surrounding space. Even in such cases, it is advisable to refer to both the passenger space and the cargo space as the interior of the vehicle. For example, it is advisable to refer to the space above the seat where the passenger sits and the space above the forks that hold and lift the cargo as the interior of the vehicle. Thus, as the vehicle moves, the space that can be considered to move along with the vehicle is referred to as the interior of the vehicle, and the surrounding space is referred to as the exterior of the vehicle

[0186] In the first embodiment, a 360-degree full spherical camera was used for the camera 11, but a 360-degree half spherical camera may also be used, or a normal wide-angle camera may be used. When using a wide-angle camera, it is advisable to attach a plurality of cameras to the vehicle so as to be able to image a wide area

[0187] In the first embodiment, the function of recording the image captured by the camera 11 was realized by the in-vehicle device, but it may also be realized by a server that communicates with the in-vehicle device. Also, in the first embodiment, the function of analyzing the image captured by the camera was realized by the in-vehicle device, but it may also be realized by a server that communicates with the in-vehicle device

[0188] As described above, the processing unit 20 of this system may be constituted by one in-vehicle device, or may be constituted by a plurality of devices. As in the first embodiment, the functions of the processing unit 20 may be realized by a microcomputer built into the drive recorder 1 and an application program stored in the memory. Alternatively, for example, the functions of the processing unit 20 may be realized by two devices, namely an in-vehicle device and a server, such that the in-vehicle device performs image recording and the server performs image analysis ​​​​​​​​​​​​​​ This is preferable. Also, it is preferable to implement the functions of the processing unit 20 with an in-vehicle device and an external device such as a personal computer, have the in-vehicle device record images, and have the external device perform image analysis and display.

[0189] FIG. 3 is a block diagram of a system according to a modified example in which the functions of the processing unit are implemented by two devices, an in-vehicle device and a server. The in-vehicle drive recorder 1 includes a communication device 21. The server 30 includes a processing unit 31, a communication device 32, an input device 33, and a display 34. The communication device 21 of the drive recorder 1 and the communication device 32 of the server 30 are both connected to a communication network 40 and perform data transmission and reception with other devices via the communication network 40. The processing unit 20 of the drive recorder 1 and the processing unit 31 of the server 30 perform data communication with each other via the communication network 40.

[0190] The input device 33 inputs various commands and data to the processing unit 31 when operated by the user. The processing unit 31 causes the display 34 to display processing results and various messages for the user.

[0191] FIG. 4 is a block diagram of a system according to a modified example in which the functions of the processing unit are implemented by an in-vehicle device and an external device 50 such as a personal computer. The external device 50 includes a processing unit 51, an SD card reader 52 , an input device 53, and a display 54. By attaching the SD card 60 on which image data is recorded to the SD card reader 13 of the drive recorder 1 to the SD card reader 52 of the external device 50, the processing unit 51 of the external device 50 can access the image data recorded on the SD card 60.

[0192] ​​​​​​​​Based on the command input from the input device 53, the processing unit 51 reads the image data recorded on the SD card 60 and performs image analysis or image processing. The processing unit 51 causes the display 54 to display the analysis result or the processing result. Next, the second to twenty-fourth embodiments will be described. Hereinafter, redundant descriptions of the configurations common to the system according to the first embodiment will be omitted. Also, the systems according to various embodiments described below are realized in any of the forms of the block diagrams shown in FIGS. 2, 3, and 4 as an example.

[0193] [Second Embodiment] Next, the second embodiment will be described. In the second embodiment, the processing unit of the system has a function of determining whether or not the movement of a person in an image corresponds to a specific movement registered in advance as a function of image analysis. When the movement of the person corresponds to the specific movement, the processing unit, as processing corresponding to the analysis result, performs at least one of recording an image of the specific movement of the person, transmitting the image of the specific movement of the person to the server 30 (FIG. 3), reporting that the person has made the specific movement, and issuing an alarm. The recorded image or the image transmitted to the server can be used as evidence that the person has made the specific movement. By reporting that the person has made the specific movement, it is possible to notify an external person that there is a person making the specific movement. Due to the issued alarm, the person who has made the specific movement is motivated to stop the specific movement by noticing that an alarm has been issued for himself / herself.

[0194]

[0195]

[0196] ​​​​​​​​​​​​​​Next, an example of "a person making a specific movement" will be described. As a person making a specific movement, for example, a driver of a vehicle shall be taken. As a specific movement, for example movements other than driving by the driver shall be taken. For example, the movement of looking at a mobile terminal such as a smartphone, eating the act of eating food, the act of looking around or looking sideways, dozing off, etc. For example, the presence or absence of the movement of looking sideways shall be determined based on the detection result of the position of the driver's eyes. The recorded image can be used for improving the driver's awareness , analyzing the cause of an accident, etc. When the processing unit detects that the driver is operating a mobile terminal such as a smartphone , it issues a warning to prompt the driver not to hold the mobile terminal while driving. When the driver notices the warning and stops making a specific movement, the danger can be avoided in advance . The server 30 of the data transmission destination is, for example, the server of the vehicle administrator or the server of the police. The vehicle administrator can evaluate the driving attitude of the driver based on the transmitted data . The police can evaluate the driving attitude of the driver, analyze the cause of an accident, etc. based on the transmitted image .

[0197] As a person making a specific movement, for example, a suspicious person outside the vehicle shall be taken. The specific movement is, for example, an act of trying to break into the vehicle , for example, an act of trying to break a glass window with a hammer, an act of peeping into the vehicle, etc. By issuing a warning, a suspicious person trying to break in can be made to refrain from entering the vehicle. The recorded image, or the image transmitted to the server 30 can be used as useful information for arresting a thief after a crime targeting a vehicle has occurred . By notifying the owner or driver of the vehicle that a person has made a specific movement , the owner or driver can immediately return to the vehicle and prevent damage caused by a crime targeting a vehicle It can be cut off. Also, the notified driver or owner can report it to the police. In this way, this function can be used for crime prevention. In this way, this function can be used for crime prevention.

[0198] As a person making specific movements, for example, a suspicious person inside the vehicle. As specific movements, various methods of targeting vehicles, such as illegal entry into the vehicle while parked, suspicious movements of a suspicious person inside the parked vehicle, movements of removing devices inside the vehicle, etc. By issuing an alarm, the thief will panic and flee, and the effect of reducing the damage of targeting vehicles can be obtained. The recorded image, or the image sent to the server 30, can be used as useful information for arresting the thief after the crime of targeting vehicles occurs. By notifying the vehicle owner or driver that a person has made specific movements, the owner or driver can immediately return to the vehicle or report to the police, thereby reducing the damage of targeting vehicles. Examples of external notifications include sending an email with an attached image of a suspicious person. The recipients of the email are, for example, vehicle administrators, drivers, owners, the police, etc. and the like, which can reduce the damage of targeting vehicles. The recorded image, or the image sent to the server 30, can be used as useful information for arresting the thief after the crime of targeting vehicles occurs. By notifying the vehicle owner or driver that a person has made specific movements, the owner or driver can immediately return to the vehicle or report to the police, thereby reducing the damage of targeting vehicles. Examples of external notifications include sending an email with an attached image of a suspicious person. The recipients of the email are, for example, vehicle administrators, drivers, owners, the police, etc. Examples of external notifications include sending an email with an attached image of a suspicious person. The recipients of the email are, for example, vehicle administrators, drivers, owners, the police, etc. For example, vehicle administrators, drivers, owners, the police, etc.

[0199] As a person making specific movements, for example, a passenger in a commercial vehicle where an unspecified person gets on. As specific movements, various methods of robbery, such as the movement of brandishing a knife, the movement of holding a firearm, the movement of using violence, etc. By the processing unit executing the process of issuing an alarm, if the robber stops the crime, the damage can be prevented. The recorded image, or the image sent to the server, can be used as useful information for arresting the robber. By the processing unit executing the process of issuing an alarm, if the robber stops the crime, the damage can be prevented. The recorded image, or the image sent to the server, can be used as useful information for arresting the robber. The recorded image, or the image sent to the server, can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made specific movements, the administrator can be aware of the occurrence of robbery. Notice it and report it to the police. Also, the processing unit has a function of reporting the occurrence of a robbery incident to the police using the communication function. to the police.

[0200] Take, for example, an infant in the car as a person making a specific movement. As the specific movement, for example, the movement when waking up from sleep. When detecting the awakening of the infant by image analysis, the processing unit notifies the guardian by e-mail or the like. There may be a case where the infant falls asleep in the car during driving and the driver wants to do some shopping or finish some errands with the engine running. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant. At this time, if the infant is woken up forcefully, the infant may get angry, so there is a desire to let the infant sleep as it is. However, when the guardian is away from the vehicle, the guardian is concerned about whether the infant has woken up. By using this system, when the infant wakes up, an e-mail is sent to the guardian's mobile terminal, so the parent can rush to the vehicle immediately and deal with the awakening of the infant.

[0201] Take, for example, a person in need of care or a person in need of rescue outside the vehicle as a person making a specific movement. As the specific movement, for example, the specific actions of a person in need of care or the actions corresponding to a situation where rescue is considered necessary. Examples of the specific actions of a person in need of care include, for example, walking unsteadily with a cane, bending the waist. Examples of the actions corresponding to a situation where rescue is considered necessary include the actions corresponding to a situation where a person has fallen and is foaming at the mouth and lying fallen. and the like.

[0202] As a function of detecting a person making a specific movement, the processing unit has a function of selecting at least one of the images in front of the vehicle, behind the vehicle, and inside the vehicle, and a function of selecting at least one of the images in front of the vehicle, behind the vehicle, and inside the vehicle, and It has a function of detecting the corresponding person. Further, the processing unit has a function of causing the user to specify at least one image. As a result, since the processing unit does not need to analyze unnecessary or less important images, there is no need to endow the processing unit with excessive processing capabilities. The processing unit has a function of detecting drowsiness of the driver as a function of detecting a person making a specific movement. For example, the processing unit measures the pulse from the variation in the redness of the driver's skin, and determines the presence or absence of the driver's drowsiness based on the variation in the pulse. Further, the processing unit has a function of enhancing the detection accuracy of drowsiness by coordinating the measurement of body temperature by a thermo camera, the measurement of pulse by microwaves, etc. When the processing unit of this system detects drowsiness, it executes processing to lower the set temperature of the air conditioner,

[0203] or processing to issue a warning by voice or sound. For example, the processing unit measures the pulse from the variation in the redness of the driver's skin, and determines the presence or absence of the driver's drowsiness based on the variation in the pulse. Further, the processing unit has a function of enhancing the detection accuracy of drowsiness by coordinating the measurement of body temperature by a thermo camera, the measurement of pulse by microwaves, etc. When the processing unit of this system detects drowsiness, it executes processing to lower the set temperature of the air conditioner, or processing to issue a warning by voice or sound. For example, the processing unit measures the pulse from the variation in the redness of the driver's skin, and determines the presence or absence of the driver's drowsiness based on the variation in the pulse. Further, the processing unit has a function of enhancing the detection accuracy of drowsiness by coordinating the measurement of body temperature by a thermo camera, the measurement of pulse by microwaves, etc. When the processing unit of this system detects drowsiness, it executes processing to lower the set temperature of the air conditioner, or processing to issue a warning by voice or sound. When the processing unit of this system detects drowsiness, it executes processing to lower the set temperature of the air conditioner, or processing to issue a warning by voice or sound.

[0204] [Third Embodiment] Next, the third embodiment will be described. The processing unit of the system according to the third embodiment has a function of determining whether a person in an image has a specific feature registered in advance as a function of image analysis, and as processing corresponding to the analysis result, executes processing according to the specific feature of the person in the image. The system according to the third embodiment can perform appropriate processing according to the features of the person captured in the image. The processing unit has a face recognition function as a function of determining whether a person in an image has a specific feature. The processing unit performs face recognition on the person sitting in the driver's seat, and if the person sitting in the driver's seat does not correspond to a person registered in advance,

[0205] access to the vehicle and starting the engine are not possible. The system according to the third embodiment can perform appropriate processing according to the features of the person captured in the image.

[0206] The processing unit has a face recognition function as a function of determining whether a person in an image has a specific feature. The processing unit performs face recognition on the person sitting in the driver's seat, and if the person sitting in the driver's seat does not correspond to a person registered in advance, access to the vehicle and starting the engine are not possible. access to the vehicle and starting the engine are not possible. For example, the processing unit may communicate with the engine control unit via an in-vehicle network, such as a CAN. A signal is sent to the engine control unit (ECU) to prohibit the engine from starting. This allows the system to register in advance who can drive a vehicle and prohibit unregistered people from driving the vehicle. As a result, the crime prevention function of the vehicle can be improved. It is also effective as a measure against pranks.

[0207] According to a modification of the third embodiment, the processing unit controls the side brake to prevent the accelerator from being depressed. Prevents the parking brake from being released when the shift lever is in the drive position. Prevent entry, sound an alarm, send an email with a picture to the registered email address The device has the function of performing at least one of the following processes:

[0208] According to another modification of the third embodiment, the processing unit performs face authentication of a person outside the vehicle, If the person corresponds to a pre-registered person, the system has the function of controlling the vehicle and unlocking the doors. For example, if the vehicle is provided to a car-sharing service, The registered members of the parking service are registered in the processing unit in advance. By moving your face within the field of view of both cameras, you can easily see your face even if you do not have a card key. This will allow people to use the vehicles of car sharing services. It will be easier to use.

[0209] According to yet another modification of the third embodiment, the vehicle is a business vehicle in which unspecified persons ride. The processing unit has a function of determining whether a passenger is a wanted criminal by face recognition. The facial photos of wanted criminals are sent from the police database via LTE or wireless LAN. The facial photo is then transmitted to the processing unit of the system via the communication. The processing unit registers the received facial photo and If the processing unit determines that the passenger is a wanted criminal, it will suspend the vehicle's management. The communication device 21 (FIG. 3) and the communication network (FIG. 3) are used to notify the person in question, the police, etc. Furthermore, the processing unit has a function of notifying the driver in a manner that is not noticeable to the passengers. The vehicle manager, police, driver, etc. must be aware that a wanted criminal has entered the vehicle and must act promptly. You can take appropriate action quickly.

[0210] For example, in a taxi, the driver and passengers are the only people in the vehicle, so the driver's safety is a concern. There is a high demand for this system in terms of securing the driver. When the system detects a criminal in a taxi, it notifies the taxi company and the police. This means that the criminal is arrested when he or she enters the vehicle before actually committing a crime inside the vehicle. It is possible to detect criminals getting on board. This makes it possible to prevent crimes from occurring. .

[0211] According to yet another variation of the third embodiment, the processing unit detects whether a person in the image has a particular characteristic. The function to determine whether a person is a human is to compare the appearance of the person in the image with a specific appearance registered in advance. It has a function to compare the appearance and determine whether they match (hereinafter referred to as the appearance authentication function). As the fixed appearance, the appearance specific to a wanderer, the appearance specific to a criminal, etc. are registered in the processing unit in advance. Examples of criminal appearances include those carrying knives on the street and those carrying dangerous objects. The processing unit also uses images from infrared cameras, ultraviolet cameras, etc. , in combination with the image of a normal camera, has the function of detecting a person in possession of illegal drugs. When the processing unit detects a person with a specific appearance in the image, it notifies the police and the vehicle administrator via the communication device 21 (Fig. 3) and the communication network 40 (Fig. 3).

[0212] According to still another modification of the third embodiment, the processing unit has a function of individually registering the appearance at the time when a specific missing person, loiterer, etc. goes missing as search information. For example, it has a function of registering individual search information such as the difference between male and female, height, type of clothing, color of clothing, presence or absence of wearing a hat, presence or absence of carrying a cane, and whether the waist is bent. Furthermore, the processing unit has a function of registering the area (search target area) where a person is likely to be loitering. When the vehicle is located in the search target area, the processing unit executes a detection process for a person who matches the individual search information. Such individual search information is received by the processing unit of this system from a database (server) such as the police and the regional support center. In this system, the processing unit of the in-vehicle device determines whether a person in the image has specific characteristics. When the vehicle enters the search area, the search information is downloaded from the server 30 (Fig. 3) to the processing unit of the vehicle.

[0213] According to still another modification of the third embodiment, as a function of determining whether a person in the image has specific characteristics, the processing unit has a function (hereinafter referred to as an appearance authentication function) of determining whether a person sitting in the passenger seat or the rear seat is wearing a seat belt. When the processing unit of this system detects a person not wearing a seat belt, it issues a message prompting the person to wear the seat belt.

[0214] ​​​​​​​​​​​​Furthermore, when a person in the rear seat approaches the entrance of a highway with the seat belt not fastened, the processing unit has a function of issuing a message prompting the fastening of the seat belt. When the distance from the current position of the vehicle to the entrance of the highway becomes equal to or less than a reference distance, the processing unit determines that it has approached the entrance of the highway. As another example, when the processing unit receives a signal for checking for forgetting to insert an ETC card, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. When a person in the rear seat approaches the entrance of a highway with the seat belt not fastened, the processing unit has a function of issuing a message prompting the fastening of the seat belt. When the distance from the current position of the vehicle to the entrance of the highway becomes equal to or less than a reference distance, the processing unit determines that it has approached the entrance of the highway. As another example, when the processing unit receives a signal for checking for forgetting to insert an ETC card, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. When the distance from the current position of the vehicle to the entrance of the highway becomes equal to or less than a reference distance, the processing unit determines that it has approached the entrance of the highway. As another example, when the processing unit receives a signal for checking for forgetting to insert an ETC card, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. As another example, when the processing unit receives a signal for checking for forgetting to insert an ETC card, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. As another example, when the processing unit receives a signal for checking for forgetting to insert an ETC card, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway. As yet another example, when the processing unit analyzes an image in front of the vehicle and detects the entrance of the highway in the image, it determines that it has approached the entrance of the highway.

[0215] [Fourth Embodiment] Next, the fourth embodiment will be described. The processing unit of the system according to the fourth embodiment has a function of counting the number of people in an image as an image analysis function, and as processing corresponding to the analysis result, executes processing of recording or transmitting the number of people data in the image. Next, the fourth embodiment will be described. The processing unit of the system according to the fourth embodiment has a function of counting the number of people in an image as an image analysis function, and as processing corresponding to the analysis result, executes processing of recording or transmitting the number of people data in the image. Next, the fourth embodiment will be described. The processing unit of the system according to the fourth embodiment has a function of counting the number of people in an image as an image analysis function, and as processing corresponding to the analysis result, executes processing of recording or transmitting the number of people data in the image.

[0216] With the system according to the fourth embodiment, it becomes possible to count the number of people without using a special device for counting the number of people. With the system according to the fourth embodiment, it becomes possible to count the number of people without using a special device for counting the number of people.

[0217] According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby According to a modification of the fourth embodiment, a camera is attached to the bus, and the processing unit adopts an image inside the vehicle as the image serving as the basis for counting the number of people. The processing unit displays the counting result on the display 16 (FIG. 2) to inform the crew. The crew can easily grasp the number of passengers (riders) in the vehicle. This function can be used instead of taking attendance on a tour. When all passengers should get off the vehicle and the counting result of the number of passengers does not become zero, the crew can know that there is a person left in the vehicle. For example, like a baby People who cannot express their own will, such as disabled people who cannot get off by themselves, etc. can detect those who stay behind.

[0218] According to another modification example of the fourth embodiment, the vehicle to which the camera is attached is a type of vehicle in which the number of passengers is predetermined, such as a sightseeing bus, a long-distance express bus, etc. The processing unit of this system has a function of registering the predetermined number of passengers. The processing unit compares the counting result of the number of people with the registered number of people, and if there is an excess or deficiency, it warns the crew about the excess or deficiency. This warning is issued by generating a warning sound from the speaker 15 (Fig. 2) or by displaying a warning message on the display 16 (Fig. 2). This prevents the vehicle from departing with the number of passengers insufficient. Also, it is possible to easily grasp that a person without the qualification to board has boarded and prompt them to get off. This warning is given by generating a warning sound from the speaker 15 (Fig. 2) or by displaying a warning message on the display 16 (Fig. 2). This prevents the vehicle from departing with the number of passengers insufficient. Also, it is possible to easily grasp that a person without the qualification to board has boarded and prompt them to get off. This warning is issued by generating a warning sound from the speaker 15 (Fig. 2) or by displaying a warning message on the display 16 (Fig. 2). This prevents the vehicle from departing with the number of passengers insufficient. Also, it is possible to easily grasp that a person without the qualification to board has boarded and prompt them to get off. This warning is issued by generating a warning sound from the speaker 15 (Fig. 2) or by displaying a warning message on the display 16 (Fig. 2). This prevents the vehicle from departing with the number of passengers insufficient. Also, it is possible to easily grasp that a person without the qualification to board has boarded and prompt them to get off. This warning is issued by generating a warning sound from the speaker 15 (Fig. 2) or by displaying a warning message on the display 16 (Fig. 2). This prevents the vehicle from departing with the number of passengers insufficient. Also, it is possible to easily grasp that a person without the qualification to board has boarded and prompt them to get off.

[0219] According to still another modification example of the fourth embodiment, a camera (for example, a hemispherical camera on the ceiling) is installed inside a route bus, and the processing unit performs image analysis to count the number of passengers on board, the number of standing passengers and the number of sitting passengers. Further, the processing unit acquires the current position information of the bus, the opening and closing signal of the bus door, the bus getting-off buzzer, the touch of the IC card when getting on and off the bus, and the signal of the fare box, and uses each of them as a trigger to associate and record these events and the images at the time of event occurrence. The server 30 (Fig. 3) accumulates these information in the data lake. Using this data lake, it becomes possible to perform big data analysis on where a person got on and where they got off, and what the seating situation inside the vehicle was during that time. Such data can be valuable information for consulting companies that conduct urban planning, etc. Further, the processing unit associate and record these events and the images at the time of event occurrence. The server 30 (Fig. 3) accumulates these information in the data lake. Using this data lake, it becomes possible to perform big data analysis on where a person got on and where they got off, and what the seating situation inside the vehicle was during that time. Such data can be valuable information for consulting companies that conduct urban planning, etc. Further, the processing unit associate and record these events and the images at the time of event occurrence. The server 30 (Fig. 3) accumulates these information in the data lake. Using this data lake, it becomes possible to perform big data analysis on where a person got on and where they got off, and what the seating situation inside the vehicle was during that time. Such data can be valuable information for consulting companies that conduct urban planning, etc. Further, the processing unit can be valuable information for consulting companies that conduct urban planning, etc. Further, the processing unit has a function of acquiring information transmitted through in-vehicle networks such as CAN and the vital information of the driver in combination. These pieces of information can be used to identify the causal relationship when some abnormality occurs in the vehicle, such as whether the abnormality is due to a driving factor, a passenger factor, a driver's own factor, a vehicle factor, etc.

[0220] According to still another modification of the fourth embodiment, the processing unit adopts an image outside the vehicle as an image serving as a basis for counting the number of people. The processing unit of this system has a function of recording the counting result as person number data. Based on the recorded person number data, it is possible to easily extract the locations where there are a large number of people. The processing unit has a function of recording the image serving as a basis for counting using the counting result as a trigger. For example, it has a function of recording an image when the number of people is equal to or greater than a predetermined threshold value.

[0221] [Fifth Embodiment] Next, the fifth embodiment will be described. The processing unit of the system according to the fifth embodiment has a function of detecting the display content of the vehicle's display device or the operation status of the vehicle's operating device from an image inside the vehicle as a function of image analysis, and as processing corresponding to the analysis result, executes processing according to the display content of the display device or the operation status of the operating device.

[0222] It is possible to acquire these data in this system without retrofitting a device for acquiring data that is the basis of the display content of the display device or data corresponding to the operation status of the operating device. For example, it is possible to acquire these data without connecting a retrofitted device to the OBD connector. In particular, it is possible to connect third-party products to the OBD connector. Even if the vehicle manufacturer prohibits it, these data can be obtained by this system. As vehicle display devices, various instruments, in-vehicle warning lights (warning lamps), and head-up displays that project images onto the front glass, etc. can be mentioned.

[0223] In a modification of the fifth embodiment, the processing unit of the system has a function of detecting the vehicle speed displayed on the speedometer as the display content of the display device. The processing unit has a function of recognizing the position of the needle of the analog speedometer as a pattern and recognizing the numbers on the dashboard to obtain vehicle speed data. Further, the processing unit has a function of recognizing the numbers displayed on the digital speedometer to obtain vehicle speed data. The obtained vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described.

[0224] The processing unit has a function of automatically detecting the speedometer by image recognition. By this function, the processing unit can detect the speedometer in various vehicles regardless of the vehicle type. In addition, the processing unit has a function of registering in advance the position and shape of the speedometer of the vehicle on which the processing unit is mounted within the image. Thereby, the processing unit can easily detect the speedometer.

[0225] General stand-alone car navigation devices use position information by GPS to obtain the current position. When GPS signals cannot be received, such as in tunnels, the current position of the vehicle must be determined by other methods. As an example, the current position of the vehicle in the map is moved at a constant speed of about 80% of the speed immediately before the GPS signal loss for 5 minutes. ​​​​​​​​The mode of performing such processing is referred to as the tunnel mode. The tunnel mode is effective not only when entering the tunnel but also when driving under the elevated road. However, when in a traffic jam while processing in the tunnel mode, assuming driving at a constant speed, the error of the current position becomes excessively large and it becomes practically useless. By attaching an acceleration sensor to the vehicle and using the acceleration data from the acceleration sensor, the accuracy of the current position information in the tunnel mode can be improved. However, it is difficult and costly for the user to attach the acceleration sensor, so it is not practical. This system has the function of a stand-alone car navigation. The processing unit reads the vehicle speed from the display content of the speedometer and, based on the read speed information, has the function of updating the current position information of the vehicle using the map matching function. Thereby, the accuracy of the current position information of the vehicle can be improved.

[0226] At locations where the road in the tunnel and the ground road run parallel, or where the elevated road and the ground road under the elevated road run parallel, the processing unit has the function of selecting which road to map based on whether the tunnel mode is activated. For example, when the tunnel mode is activated, the vehicle position is mapped to the road in the tunnel or the ground road under the elevated road, and when the tunnel mode is not activated, the vehicle position is mapped to the ground road running parallel to the road in the tunnel or the elevated road.

[0227] In another modification of the fifth embodiment, the processing unit uses, as the display content of the display device to be detected, the vehicle position is mapped to the road in the tunnel or the ground road under the elevated road, and when the tunnel mode is not activated, the vehicle position is mapped to the ground road running parallel to the road in the tunnel or the elevated road. For example, when the tunnel mode is activated, the vehicle position is mapped to the road in the tunnel or the ground road under the elevated road, and when the tunnel mode is not activated, the vehicle position is mapped to the ground road running parallel to the road in the tunnel or the elevated road. has the function of mapping.

[0228] In another modification of the fifth embodiment, the processing unit uses, as the display content of the display device to be detected, ​It has a function to detect the remaining fuel amount displayed on the fuel gauge. The processing unit performs image recognition of the fuel gauge to detect the remaining fuel amount. When the processing unit detects that the remaining fuel amount is below a predetermined reference value , it notifies the driver with a message prompting refueling. For example, when returning from an out-of-town location to near the vehicle storage location at home or at the company in conjunction with the current location information of the vehicle , if the remaining fuel amount is at the reference value, for example, 1 / 4 or less of a full tank, the processing unit issues a warning and prompts refueling . Also, by linking the car navigation device with this system, the processing unit compares the distance to the destination with the remaining fuel amount and prompts refueling either at the time of departure or along the route to the destination based on the comparison result . For example, near a gas station on the route, it outputs a message prompting refueling at a gas station near the current location . The function of extracting the speedometer and fuel gauge from the image is realized by the artificial intelligence (AI) installed in the processing unit of this system . The processing unit further has a function of registering in advance the positions and image information of the speedometer and fuel gauge in the image. When it is difficult to extract the speedometer and fuel gauge from the image by artificial intelligence

[0229] , the processing unit uses the registered image information to extract the speedometer and fuel gauge from the image . The processing unit reads the fuel gauge of the vehicle provided by the car-sharing service and, when detecting that the remaining amount is less than a predetermined reference value , has a function of notifying the driver that they can receive the benefits of the car-sharing service if they refuel . For example, it issues a message such as "If you refuel now

[0230] , you can receive a discount equivalent to the amount for 15 minutes." . .

[0231] ​​In yet another modification of the fifth embodiment, the processing unit has a function of determining whether a warning lamp in the instrument panel is lit or not. The warning lamps to be determined for lit or not include the headlamp indicator, turn signal indicator, high beam indicator (high beam warning lamp), parking brake indicator, half door warning lamp, fuel warning lamp, etc. The processing means of this system can detect the state of the vehicle corresponding to the warning lamp by determining whether these warning lamps are lit or not.

[0232] In yet another modification of the fifth embodiment, the processing unit has a function of detecting warning information displayed by a head-up display. The processing unit captures the change of the image displayed by the head-up display to detect the warning content and executes a predetermined process according to the warning content. For example, a warning is issued by sound or voice. By changing the visual information-based warning to an audible information-based warning and issuing it, the effect that the driver can easily notice the warning can be obtained.

[0233] In yet another modification of the fifth embodiment, the processing unit is associated with the ambient brightness information outside the vehicle and the current position information of the vehicle, and when it detects that the headlamp indicator is not lit even though it is a situation where the headlamp should be lit, it has a function of notifying the driver to turn on the headlamp. Situations where the headlamp should be lit include, for example, a situation where it is darker outside the vehicle than the ambient brightness, a situation where the current position of the vehicle is inside a tunnel or a dark road section, etc. When the headlamp indicator remains lit even though the situation where the headlamp should be lit has been resolved, the processing unit turns off the headlamp. It has a function of notifying the driver. Since there are also tunnels that are bright inside due to lighting, when determining whether it is a situation where the headlights should be turned on based only on the brightness information outside the vehicle, a situation may occur where it is erroneously determined that it is not a situation where the headlights should be turned on even though the vehicle is inside a tunnel. The processing unit can avoid such an erroneous determination by determining whether the vehicle is inside a tunnel based on the current position information of the vehicle. Furthermore, the processing unit has a function of notifying the driver to turn on the side marker lights (small lights) when it detects that the side marker light indicator is not lit even though it is a situation where not only the headlights but also the side marker lights should be turned on.

[0234] In still another modification of the fifth embodiment, the processing unit determines the presence or absence of an oncoming vehicle from an image in front of the vehicle, and when it detects that the high beam indicator is lit even though there is an oncoming vehicle, it has a function of warning the driver to switch to the low beam. Since the high-low beam switching operation of the beam varies depending on the vehicle type, an event such as the high beam remaining on even though the driver intends to switch to the low beam may occur when the driver is driving a vehicle of a different vehicle type from the one they usually drive. In such a case, this warning function is particularly effective.

[0235] In still another modification of the fifth embodiment, the processing unit detects that the turn signal is blinking in an image outside the vehicle when it is dark outside, and has a function of recognizing the indicator that is blinking inside the vehicle as a turn signal indicator in synchronization with the blinking period of the turn signal. By doing so, the processing unit can easily find the turn signal indicator inside the instrument panel.

[0236] In yet another modification of the fifth embodiment, when the processing unit detects that the headlights and the side marker lights are on even though it has detected that there is no one in the vehicle from the image inside the vehicle, it has a function of notifying the driver outside the vehicle or the vehicle administrator that the lights have been left on. This notification is performed, for example, by sending an email to a pre-registered email address. In addition, the processing unit has a function of notifying, by email or the like, that the hazard lamp has been left on or that the dome light has been left on. The lighting state of the hazard lamp is detected from the blinking state of the turn signal indicator, and the lighting state of the dome light is detected from the brightness of the dome light reflected in the image. The dome light generally has three setting modes: always-on mode, always-off mode, and door-opening / closing state interlock mode (hereinafter simply referred to as the interlock mode). In many vehicles, these three modes are cyclically updated each time the dome light on / off switch is pressed once. There are vehicles that have a function where the dome light comes on when the engine is stopped in a dark place (at night) and automatically goes out after a while (for example, after a few seconds have elapsed).

[0237] There are also cases where the driver wants to keep the dome light on for a longer time in order to prepare for getting out of the vehicle after the engine has been stopped. For example, when the driver wants to keep the dome light on during the period of organizing and taking out the purchased items and taking out the mobile terminal after the engine has been stopped. In such cases, the driver switches the dome light setting to the interlock mode or the always-on mode so that the dome light does not automatically turn off after a certain period has elapsed since the engine was stopped. However, when the engine is stopped and the driver wants to keep the dome light on for a longer time in order to prepare for getting out of the vehicle, there are also cases where the driver wants to keep the dome light on for a longer time. For example, when the driver wants to keep the dome light on during the period of organizing and taking out the purchased items and taking out the mobile terminal after the engine has been stopped. In such cases, the driver switches the dome light setting to the interlock mode or the always-on mode so that the dome light does not automatically turn off after a certain period has elapsed since the engine was stopped. However, when the driver wants to keep the dome light on during the period of organizing and taking out the purchased items and taking out the mobile terminal after the engine has been stopped. In such cases, the driver switches the dome light setting to the interlock mode or the always-on mode so that the dome light does not automatically turn off after a certain period has elapsed since the engine was stopped. However, when the driver wants to keep the dome light on during the period of organizing and taking out the purchased items and taking out the mobile terminal after the engine has been stopped. In such cases, When the door is open, it is difficult to determine whether the current room light setting is in the always-on mode or the linked mode. Even if the current room light setting is in the always-on mode, if the driver mistakenly believes it is in the linked mode and gets off the vehicle as it is, the room light will remain on. By using this system, the driver or administrator can notice if they forget to turn off the room light and turn it off, thereby preventing the vehicle battery from draining due to the room light being on for a long time. Just by looking at the operation switch, it is impossible to tell which mode the current room light setting is in. For example, in a vehicle with an operation switch structured such that the room light setting cyclically switches each time the push button is pressed, this function of this system is particularly beneficial. Even if the current room light setting is in the always-on mode, if the driver mistakenly believes it is in the linked mode and gets off the vehicle as it is, the room light will remain on. By using this system, the driver or administrator can notice if they forget to turn off the room light and turn it off, thereby preventing the vehicle battery from draining due to the room light being on for a long time. Just by looking at the operation switch, it is impossible to tell which mode the current room light setting is in. For example, in a vehicle with an operation switch structured such that the room light setting cyclically switches each time the push button is pressed, this function of this system is particularly beneficial. if the driver mistakenly believes it is in the linked mode and gets off the vehicle as it is, the room light will remain on. By using this system, the driver or administrator can notice if they forget to turn off the room light and turn it off, thereby preventing the vehicle battery from draining due to the room light being on for a long time. Just by looking at the operation switch, it is impossible to tell which mode the current room light setting is in. For example, in a vehicle with an operation switch structured such that the room light setting cyclically switches each time the push button is pressed, this function of this system is particularly beneficial. if the driver mistakenly believes it is in the linked mode and gets off the vehicle as it is, the room light will remain on. By using this system, the driver or administrator can notice if they forget to turn off the room light and turn it off, thereby preventing the vehicle battery from draining due to the room light being on for a long time. Just by looking at the operation switch, it is impossible to tell which mode the current room light setting is in. For example, in a vehicle with an operation switch structured such that the room light setting cyclically switches each time the push button is pressed, this function of this system is particularly beneficial. if the driver mistakenly believes it is in the linked mode and gets off the vehicle as it is, the room light will remain on.

[0238] When the room light setting is in the linked mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side may be open. When the room light setting is in the linked mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side may be open. When the room light setting is in the linked mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side may be open. When the room light setting is in the linked mode, if any door is open or in the half-door state, the room light will remain on. The driver or administrator notified that the room light is on can notice that it may be in the half-door state or that the door on the less visible side may be open.

[0239] When the room light setting is in the linked mode, some vehicles have a function of keeping the room light on until a certain time (the turn-off delay time ) elapses after the door is closed. In such vehicles, the processing unit of this system has a function of not determining whether it is forgotten to turn off the light until the turn-off delay time elapses after detecting that the door is closed. The processing unit has a function of setting the turn-off delay time based on an input from the user. In such vehicles, the processing unit of this system has a function of not determining whether it is forgotten to turn off the light until the turn-off delay time elapses after detecting that the door is closed. The processing unit has a function of setting the turn-off delay time based on an input from the user. In such vehicles, the processing unit of this system has a function of not determining whether it is forgotten to turn off the light until the turn-off delay time elapses after detecting that the door is closed. The processing unit has a function of setting the turn-off delay time based on an input from the user. In such vehicles, the processing unit of this system has a function of not determining whether it is forgotten to turn off the light until the turn-off delay time elapses after detecting that the door is closed. The processing unit has a function of setting the turn-off delay time based on an input from the user.

[0240] Also, even if a certain period of time has elapsed after no one is left in the vehicle, if the processing unit detects that the room light is still on, it has a function of transmitting a signal to the vehicle controller to turn off the room light.

[0241] In yet another modification of the fifth embodiment, the processing unit detects that the vehicle door has been opened while the headlight, side marker light, hazard light, room light, etc. are on or flashing, and has a function of issuing an alarm to notify that any of the lights is still on. Thereby, the driver can notice the omission of turning off the light before getting out of the vehicle, and can prevent the omission of turning off the light.

[0242] In yet another modification of the fifth embodiment, the processing unit, based on the illumination of the parking brake indicator, if it detects that the parking brake is engaged, has a function of not issuing an alarm even if it detects that the driver is dozing off from the driver's image. In addition, when the parking brake is engaged, even if it detects that the driver is performing an operation that would be regarded as a dangerous operation during driving, such as looking at a mobile terminal, it has a function of not issuing an alarm.

[0243] In yet another modification of the fifth embodiment, the processing unit, if it detects that a warning light indicating that early inspection or repair is necessary remains on for a certain period of time, has a function of notifying the vehicle dealer or the registered repair shop of the vehicle's condition. For example, users who are not interested in automobiles may often overlook the warning light and leave it unattended when the vehicle can still be driven even though the warning light is on. For example, when the warning light for the engine itself comes on, ​​​​​​​, although it usually becomes inoperable, it can still be driven even if the warning light indicating an abnormality in the engine oil is on. Also, in the case of an O2 sensor abnormality, the fuel consumption deteriorates, but driving itself is possible. If driving is possible and driving is continued without inspection and repair, a minor failure may become severe. For dealers and repairers, the notification of the warning light being on is useful information for conducting business activities for inspection and repair of the vehicle owners whose warning lights are on. Also, by having the vehicle owner perform inspection and maintenance early, it is possible to prevent the failure from becoming severe.

[0244] In still another modification of the fifth embodiment, the processing unit has a function of detecting the temperature displayed in the temperature display area in the instrument panel that displays the temperature measurement value measured by the temperature sensor that measures the outside temperature. The processing unit has a function of reading the temperature by character recognition from the image displayed in the temperature display area of the instrument panel as a function of image analysis. As processing corresponding to the analysis result, the processing unit executes processing of transmitting temperature information to an external server via the communication device and the communication network.

[0245] For example, in order to obtain data on pollen dispersal, sensors are installed in the houses of ordinary users, and by collecting measurement data from the sensors, a detailed pollen dispersal situation is predicted, and a service that distributes pollen dispersal information is provided by Weathernews Co., Ltd. and others. Also outdoor photos posted by ordinary users and images collected from outdoor live cameras installed in the houses of ordinary households are incorporated into weather forecasts. Also, services (for example, Weathernews SOLiVE24, etc.) that distribute these images are provided. These weather conditions A system that provides weather services can collect outside air temperature information measured by many vehicles. Furthermore, many vehicles can be used as temperature measurement probes. When many vehicles that transmit temperature information are distributed over a wide area, the temperature information collected from many vehicles can be utilized as spotty information. For example, it becomes possible to issue spotty heatstroke warnings and the like that cannot be expressed by general weather information. For a company that provides such weather information services, the effect of being able to collect temperature information over a wide area without installing temperature sensors by itself can be obtained. A company that receives the temperature information may provide the temperature information provider, such as the owner of the vehicle that provided the temperature information, with a free or discounted paid weather information service provided by the company as a

[0246] gift. The processing unit further has a function of obtaining outside air temperature information by recognizing the numbers displayed on the temperature display signs on the roadside that have moved into the image acquired by the camera, as well as the temperature information read from the temperature display area inside the instrument panel. Detection of the temperature display sign is performed by recognizing characters such as "Current Temperature" inside the sign. The obtained outside air temperature information is transmitted by the processing unit to an external server in the same way as the temperature information obtained from the temperature display area inside the instrument panel.

[0247] In still another modification of the fifth embodiment, the processing unit has a function of detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal as a function of detecting the operation status of the operating device by image analysis. As processing according to the operation status of the operating device, the processing unit determines that the driver's foot is detected. The obtained outside air temperature information is transmitted by the processing unit to an external server in the same way as the temperature information obtained from the temperature display area inside the instrument panel.

[0248] In still another modification of the fifth embodiment, the processing unit has a function of detecting the operation status of the operating device by image analysis, and has a function of detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal. As processing according to the operation status of the operating device, the processing unit determines that the driver's foot is Execute a process to notify the driver so that the driver can easily recognize whether the foot is on the accelerator pedal or the brake pedal. For example, when the foot is on the accelerator pedal and when it is on the brake pedal, notify the driver by changing the emission color of the light-emitting element. In still another modification of the fifth embodiment, the system is mounted on a vehicle having a function of detecting whether the foot is on the accelerator pedal or the brake pedal and displaying the detection result on a display as visual information. The processing unit of this system has a function of analyzing the content displayed on this display by image analysis and detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal. In still another modification of the fifth embodiment, the processing unit has a function of issuing an alarm by sound or the like when the foot is on the accelerator pedal even though an obstacle is detected ahead by image analysis. By the driver noticing the alarm, it is possible to suppress the occurrence of a mistake in stepping on the accelerator pedal and the brake pedal. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system.

[0249] In still another modification of the fifth embodiment, execute a process to notify the driver so that the driver can easily recognize whether the foot is on the accelerator pedal or the brake pedal. For example, when the foot is on the accelerator pedal and when it is on the brake pedal, notify the driver by changing the emission color of the light-emitting element. In still another modification of the fifth embodiment, the system is mounted on a vehicle having a function of detecting whether the foot is on the accelerator pedal or the brake pedal and displaying the detection result on a display as visual information. The processing unit of this system has a function of analyzing the content displayed on this display by image analysis and detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal. In still another modification of the fifth embodiment, the processing unit has a function of issuing an alarm by sound or the like when the foot is on the accelerator pedal even though an obstacle is detected ahead by image analysis. By the driver noticing the alarm, it is possible to suppress the occurrence of a mistake in stepping on the accelerator pedal and the brake pedal. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system.

[0250] In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of issuing an alarm by sound or the like when the foot is on the accelerator pedal even though an obstacle is detected ahead by image analysis. By the driver noticing the alarm, it is possible to suppress the occurrence of a mistake in stepping on the accelerator pedal and the brake pedal. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system.

[0251] In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation status of the operating device by image analysis. The processing unit executes a process of obtaining the traveling direction and the traveling distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as processing according to the operation status of the operating device. The processing unit provides the processing result to a stand-alone car navigation device. The stand-alone car navigation device uses the information on the traveling direction and the traveling distance provided from this system. Based on this, even in a situation where GPS signals cannot be received, the driving route of the vehicle can be calculated. This is achievable.

[0252] In still another modification of the fifth embodiment, as a function of performing image analysis, the processing unit has a function of detecting whether the vehicle is in an automated driving state (including automated driving levels 1 to 5) or a manual driving state (automated driving level 0). This detection is performed by the processing unit recognizing, from the image, the display state of an indicator indicating whether various functions of the automated driving displayed on the instrument panel are operating. As processing corresponding to the analysis result, when the processing unit detects that the automated driving function has been canceled, it executes processing to notify the driver that the automated driving function has been canceled. Also, when the processing unit detects that the driver is not touching the steering wheel even though the automated driving function has been canceled, it issues a warning prompting the driver to operate the steering wheel. Further, triggered by the cancellation of the automated driving function and the fact that the driver is not touching the steering wheel, the processing unit has a function of recording, as event recording separately from regular recording, the image captured by the camera. The recorded image can be used as evidence indicating that the driver's operation was inappropriate. This detection is performed by the processing unit recognizing, from the image, the display state of an indicator indicating whether various functions of the automated driving displayed on the instrument panel are operating. This detection is performed by the processing unit recognizing, from the image, the display state of an indicator indicating whether various functions of the automated driving displayed on the instrument panel are operating. This detection is performed by the processing unit recognizing, from the image, the display state of an indicator indicating whether various functions of the automated driving displayed on the instrument panel are operating. As processing corresponding to the analysis result, when the processing unit detects that the automated driving function has been canceled, it executes processing to notify the driver that the automated driving function has been canceled. As processing corresponding to the analysis result, when the processing unit detects that the automated driving function has been canceled, it executes processing to notify the driver that the automated driving function has been canceled. Also, when the processing unit detects that the driver is not touching the steering wheel even though the automated driving function has been canceled, it issues a warning prompting the driver to operate the steering wheel. Also, when the processing unit detects that the driver is not touching the steering wheel even though the automated driving function has been canceled, it issues a warning prompting the driver to operate the steering wheel. Also, when the processing unit detects that the driver is not touching the steering wheel even though the automated driving function has been canceled, it issues a warning prompting the driver to operate the steering wheel. Further, triggered by the cancellation of the automated driving function and the fact that the driver is not touching the steering wheel, the processing unit has a function of recording, as event recording separately from regular recording, the image captured by the camera. Further, triggered by the cancellation of the automated driving function and the fact that the driver is not touching the steering wheel, the processing unit has a function of recording, as event recording separately from regular recording, the image captured by the camera. The recorded image can be used as evidence indicating that the driver's operation was inappropriate.

[0253] In a vehicle equipped with a radar detector, when the automated driving function is valid, the processing unit has a function of disabling the function of the radar detector. This can eliminate the occurrence of unnecessary alarms from the radar detector. When the automated driving state is invalid, the processing unit enables the function of the radar detector. In a vehicle equipped with a radar detector, when the automated driving function is valid, the processing unit has a function of disabling the function of the radar detector. This can eliminate the occurrence of unnecessary alarms from the radar detector. When the automated driving state is invalid, the processing unit enables the function of the radar detector.

[0254] Whether the vehicle is in autonomous driving or manual driving, and which level (mode) of autonomous driving it is in is determined by the processing unit based on indicators, the movement of the steering wheel, the position of the driver's hand , the direction in which the driver's head is facing, etc.

[0255] [Sixth Embodiment] Next, the sixth embodiment will be described. The processing unit of the system according to the sixth embodiment has a function of recognizing the characters displayed on the sticker attached to the vehicle from the image inside the vehicle as a function of image analysis , and as processing corresponding to the analysis result, executes processing for giving a warning according to the information indicated by the recognized characters.

[0256] By being warned, the driver, administrator, or owner of the vehicle can direct their attention to the character information of the sticker attached to the vehicle and perform appropriate processing according to the character information.

[0257] In a modification of the sixth embodiment, the processing unit has a function of recognizing the characters displayed on the inspection sticker (vehicle inspection sticker). On the inner surface of the vehicle of the inspection sticker, the date of expiration of the vehicle inspection is displayed. The processing unit reads the information of the date of expiration of the vehicle inspection by character recognition, and if the period from the current time to the expiration date of the vehicle inspection (the deadline of the vehicle inspection) becomes shorter than a predetermined period, it gives a warning to have the vehicle inspection. For example, the processing unit gives a warning by emitting a voice saying, "The expiration date of the vehicle inspection is approaching." When the period until the expiration date of the vehicle inspection becomes even shorter, for example, the processing unit issues a warning in a voice saying, "Please have the vehicle inspected."

[0258] In another modification of the sixth embodiment, the processing unit has a function of recognizing the characters displayed on the maintenance inspection sticker. The maintenance inspection sticker displays the deadline for the next regular inspection in year, month, and day. The processing unit reads the information of the year, month, and day indicating the deadline for the regular inspection by character recognition. If the period from the current time to the deadline for the regular inspection is shorter than a predetermined period, it issues a reminder to undergo a regular inspection. For example, the processing unit issues a reminder by emitting a voice saying, "The deadline for the regular inspection is approaching." When the period until the deadline for the regular inspection becomes even shorter, for example, the processing unit issues a warning by emitting a voice saying, "Please undergo a regular inspection."

[0259] In yet another modification of the sixth embodiment, as a function of image analysis, the processing unit has a function of determining whether an inspection mark appears in the image captured by the camera. As processing corresponding to the analysis result, when the inspection mark does not appear in the image, the processing unit executes a process of issuing a reminder to adjust the mounting position and posture of the camera so that the inspection mark appears. For example, the processing unit may emit a voice saying, "The inspection mark does not appear. Please adjust the position and posture of the camera so that the inspection mark appears." This enables the position and posture of the camera to be easily adjusted when the camera is installed. The processing unit has a function of reading the content displayed on the display devices inside the vehicle, such as the speedometer and fuel gauge, and has a function of issuing a reminder so that the display devices inside the vehicle are reflected in the image.

[0260] In yet another modification of the sixth embodiment, as a function of image analysis, the processing unit analyzes the front windshield, the side windows of the driver's seat and the passenger seat, for stickers other than those recognized by law. ​​​​​​​​​It has a function to determine whether a sticker is pasted or not. As a process corresponding to the analysis result when the processing unit determines that a sticker other than the sticker approved by the law is pasted it executes a process of notifying the driver or the like to peel off the sticker.

[0261] [Example 7] Next, the seventh embodiment will be described. The processing unit of the system according to the seventh embodiment has a function of analyzing an image of an area where the passenger seat is reflected as an image analysis function, and as a process corresponding to the analysis result it executes a process according to the situation of the passenger seat.

[0262] For example, appropriate processing can be executed according to whether a person is sitting in the passenger seat or whether an object is placed on the passenger seat.

[0263] In a modification of the seventh embodiment, the drive recorder 1 (FIG. 1) is mounted on a vehicle having a function of issuing a warning when a person sitting in the passenger seat is not wearing a seat belt. The pro cessing unit has a function of determining whether a person is sitting in the passenger seat from an image of an area where the passenger seat is reflected as an image analysis function. As a process corresponding to the analysis result, the processing unit when detecting that no person is sitting in the passenger seat, invalidates the function of issuing a warning for non-wearing of the seat belt of the passenger seat. The processing unit further has a function of determining whether an object is placed on the passenger seat as an image analysis function. As a process corresponding to the analysis result, the processing unit when it is determined that an object is placed on the passenger seat, executes a process of invalidating the function of issuing a warning for non-wearing of the seat belt of the passenger seat.

[0264] When an object equivalent to the weight of a human is placed on the passenger seat, the vehicle may erroneously determine that a person is riding However, there may be cases where a warning for unfastened seat belts is issued. When this system is installed in a vehicle, if there are objects placed on the passenger seat, the vehicle will not issue a warning for unfastened seat belts. Thus, the driver will not be confused by the warning. In particular, in a vehicle with a specification that the warning buzzer continues to sound when heavy objects are placed on the passenger seat, this function is effective. To prevent the vehicle from issuing a warning for unfastened seat belts, the processing unit transmits a signal to the vehicle's controller through the in-vehicle network (CAN) to invalidate the warning. In vehicles with a similar function for the rear seats, when the processing unit detects that there are objects placed on the rear seats, the processing means cancels the unfastened seat belt warning function.

[0265]

[0266] [Eighth Embodiment] Next, the eighth embodiment will be described. The processing unit of the system according to the eighth embodiment has a function of comparing the current in-vehicle image with the past in-vehicle image as a function of image analysis, and as processing corresponding to the analysis result, executes processing to notify when the degree of difference between the current in-vehicle image and the past in-vehicle image exceeds a predetermined level.

[0267] By receiving the notification, the vehicle owner or administrator can know that the in-vehicle situation has changed beyond a certain level.

[0268] In a modification of the eighth embodiment, the processing unit uses, as the past in-vehicle image, an image in a state where objects other than those originally attached to the vehicle are not brought into the vehicle, and determines whether there are objects brought into the vehicle by comparing this image with the current in-vehicle image. ​​​​​​​​​​​​​ has a function.

[0269] The processing unit further calculates the time that an object brought into the vehicle remains in the vehicle by comparing images at a plurality of past time points with the current image. The processing unit has a function of calculating the time that an object brought into the vehicle remains in the vehicle. When an object that has been left in the vehicle for a long time is detected, the processing unit has a function of notifying the driver of the object. has a function.

[0270] Next, the specific processing of the processing unit will be described. First, with no luggage loaded on the vehicle and preferably with only the driver in the vehicle, press the initial setting button of this system and drive the vehicle. The processing unit recognizes the part within the image with an optical flow corresponding to the vehicle speed as the window. The parts with other movements are recognized as the driver. The parts other than the recognized window and driver are recognized as areas where objects may be placed, and are recorded as an image (video in the no-luggage state) in the no-luggage state with no luggage loaded on the vehicle. The processing of the initial setting is completed up to this point. The parts within the image with an optical flow corresponding to the vehicle speed are recognized as the window. The parts with other movements are recognized as the driver. The parts other than the recognized window and driver are recognized as areas where objects may be placed, and are recorded as an image (video in the no-luggage state) in the no-luggage state with no luggage loaded on the vehicle. The parts within the image with an optical flow corresponding to the vehicle speed are recognized as the window. The parts with other movements are recognized as the driver. The parts other than the recognized window and driver are recognized as areas where objects may be placed, and are recorded as an image (video in the no-luggage state) in the no-luggage state with no luggage loaded on the vehicle. The parts other than the recognized window and driver are recognized as areas where objects may be placed, and are recorded as an image (video in the no-luggage state) in the no-luggage state with no luggage loaded on the vehicle. The parts other than the recognized window and driver are recognized as areas where objects may be placed, and are recorded as an image (video in the no-luggage state) in the no-luggage state with no luggage loaded on the vehicle. The processing of the initial setting is completed up to this point.

[0271] After that, when the engine is started or stopped, etc., the processing unit calculates the difference between the no-luggage state image and the current image to identify the presence of an object brought into the vehicle. When there is no change in the position of the object identified as being brought into the vehicle for a certain period, the processing unit issues a warning. Thereby, the driver and others can be made aware of an object that has been left in the vehicle for a long time. This function is particularly effective for people who tend to quickly make the vehicle messy with garbage. the driver and others can be made aware of an object that has been left in the vehicle for a long time. This function is particularly effective for people who tend to quickly make the vehicle messy with garbage.

[0272] [Example 9] Next, Example 9 will be described. The processing unit of the system according to Example 9 performs image analysis. As a function, it has a function of determining whether a pre-registered object appears in the image inside the vehicle. As a process corresponding to the analysis result, when a pre-registered object appears in the image inside the vehicle, an alarm is issued.

[0273] The driver or the like can be aware that there is a pre-registered object inside the vehicle due to the alarm.

[0274] In a modification of the ninth embodiment, the processing unit has a function of registering an image of an object to be checked for forgetting (a specific bag, wallet, etc.). When the processing unit detects that the registered object appears in the image in a state where no one is inside the vehicle, it issues an alarm or notifies of the forgotten object. As an example of issuing an alarm, for example, a buzzer or the like attached inside the vehicle is sounded. As an example of notifying of the forgotten object, for example, it is notified by e-mail or the like to the smartphone or mobile phone of the driver or the like wirelessly.

[0275] In another modification of the ninth embodiment, the processing unit has a function of issuing an alarm when it detects a forgotten object inside the vehicle and a movement of opening the door. Due to the alarm, the driver or the like can be aware of the forgotten object before leaving the vehicle. The processing unit further has a function of notifying the driver or the like by e-mail or the like that there is a forgotten object when it detects a forgotten object inside the vehicle and a movement of closing the door. The driver or the like can be aware that there is a forgotten object after leaving the vehicle. Detection of the movement of opening and closing the door is performed, for example, by detecting optical flow.

[0276] In still another modification of the ninth embodiment, the processing unit has a function of detecting other movements inside the vehicle by detecting optical flow. For example, the processing unit ​​​​​​​​​​​​ Analysis functions include door opening and closing movements, window opening and closing movements, turn signal operation, steering The system has the function of detecting wheel operation, raising and lowering of sun visors, etc. This can be done without connecting to the vehicle's in-car network connector (e.g. OBD connector). You can obtain information about them.

[0277] When the sun visors are down, it is very dangerous to use a 360-degree camera mounted on the windshield. When capturing an image of the driver's side, the sun visor gets in the way and it is impossible to capture an image in that direction. Therefore, when the brightness of the front of the vehicle is such that sun visors are not necessary, However, if the processing unit detects that the sun visor is down, it issues a warning. Whether the sun visor is down or not is determined by the processing unit based on the ratio of the sun visor area to the image. is determined.

[0278] [Tenth Example] Next, a tenth embodiment will be described. The processing unit of the system according to the tenth embodiment processes an image The analysis function is to capture an object that the user brings close to the camera and recognize the image. As a processing method for the analysis results, processing according to the information written on the object that the user brings close to the camera is performed. Execute.

[0279] By bringing an object close to the camera, the user can receive information written on the object. The corresponding processing can be executed.

[0280] In a modification of the tenth embodiment, the processing unit is provided with a function for performing image recognition of the driver's license of the driver. The processing unit recognizes the character of the validity date written on the driver's license and calculates the validity date from the current date. When the period until the expiration date is shorter than the predetermined period, it notifies the user that the renewal date of the driver's license is approaching. Notify the user of this.

[0281] In another modification of the tenth embodiment, the processing unit has a function of recognizing an image of a receipt from a gas station. The processing unit recognizes the date, fuel quantity, and amount displayed on the receipt as characters, records this information, and creates a database. Record these information and create a database.

[0282] In still another modification of the tenth embodiment, the processing unit has a function of performing image recognition on an identity certificate (for example, a driver's license, employee ID card, resident registration card, etc.). The processing unit recognizes the name and the like written on the identity certificate as characters and determines whether the person is a pre-registered person. If the person in the identity certificate is not registered, the processing unit performs processing to prohibit the engine from starting. By this, it is possible to limit the persons who can drive the vehicle to pre-registered persons. Furthermore, the processing unit records the name and the like of the person in the identity certificate. The recorded information can be used to identify the driver when performing the operation management of the vehicle. Furthermore, the processing unit records the name and the like of the person in the identity certificate. The recorded information can be used to identify the driver when performing the operation management of the vehicle.

[0283] In still another modification of the tenth embodiment, as a function of image recognition, the processing unit has a function of recognizing an object that the user brings close to the camera outside the vehicle and a function of recognizing an object that the user brings close to the camera inside the vehicle. For example, when the processing unit compares with a normal image and an object that occupies a wider portion than a predetermined range within the angle of view of the camera is not in the normal image, it recognizes that there is an object brought close to the camera. In particular, when the processing unit recognizes that the user is holding an object that occupies a wider portion than a predetermined range within the angle of view of the camera, it determines that the user has brought the object close to the camera. it recognizes that there is an object brought close to the camera. In particular, when the processing unit recognizes that the user is holding an object that occupies a wider portion than a predetermined range within the angle of view of the camera, it determines that the user has brought the object close to the camera.

[0284] [Example 11] Next, Example 11 will be described. The processing unit of the system according to Example 11 has a function of performing image analysis of objects captured in the front image as an image analysis function, and as a corresponding process, executes a process of providing the driver with information to be noted regarding the driving of the vehicle.

[0285] By prompting the driver according to the situation ahead, it is possible to suppress the occurrence of accidents that may occur in the future when the vehicle moves forward.

[0286] In a modification of Example 11, the processing unit has a function of calculating at least one of the inter-vehicle distance and the relative speed with the vehicle ahead as an image analysis function. The processing unit detects a situation with a high risk of collision based on the inter-vehicle distance or the ...

Claims

1. A function for detecting the presence of a specific object in the vehicle; a function of sounding a buzzer installed in the vehicle when a door of the vehicle is opened in a state where the presence of the specific object in the vehicle is detected; Equipped A system characterized by:

2. The vehicle door is closed while the specific object is detected to be present in the vehicle, and the vehicle door is notified to the driver via a mobile device.

2. The system of claim 1 .

3. 3. A program for causing a computer to realize the functions of the system according to claim 1 or 2.

Images