Systems and programs, etc.

Abstract

Images acquired by cameras mounted on vehicles can be used to determine the cause of accidents more accurately than before. The goal is to use this information to aid in identification, and to utilize it more effectively, not just limited to identifying the cause of the accident. Let's assume that. [Solution] The processing means of this system processes images captured by a camera mounted on the vehicle. The camera performs recording and image analysis of captured images, and based on the analysis results, it performs a solution. It has the function to perform processing corresponding to the analysis results.

Description

Technical Field

[0001] The present invention relates to a system, a program, etc. for processing an image captured by a camera or the like attached to a vehicle or the like.

Background Art

[0002] The following Patent Document 1 discloses a system applied to a drive recorder or the like. This system repeats a cycle of recording video data in a recording area. When a predetermined cycle end condition is satisfied, the currently ongoing cycle is ended, and a new cycle is started in which the video data recorded in the ended cycle is overwritten with new video data in the order from the oldest. Information about the overwrite position is output to output means based on at least one of the capacity of the video data recorded in the currently ongoing cycle and the capacity of the video data recorded in the immediately preceding cycle that has not been overwritten.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The images recorded by a drive recorder are effectively used to identify the cause of an accident or the like. It is desired to make the images acquired by a camera attached to a vehicle or the like more useful for identifying the cause of an accident than before, and not limited to identifying the cause of an accident, but to utilize them more effectively.

[0005] ​​​​​​​​ The object of the present invention is not limited thereto, but includes any part of the configuration disclosed in this specification and the drawings. Regarding configurations aimed at achieving a specific effect, rights can be obtained through divisional applications, amendments, etc. To have the intention to do so. For example, in this specification, the phrase "can do" should be changed to "the challenge is that..." The problems disclosed in this specification are those that have been reinterpreted as "ru". Each problem is independent of the others. This is described in detail, and the configuration for solving this problem is also subject to separate divisional applications. They intend to acquire the rights through amendments, etc. Although the issues are implicitly understood from the description in the specification, Even if so, the applicant may request a patent for a portion of the configuration described herein by amending or filing a divisional application. We intend to include this within the scope of the request. Furthermore, issues combining these independent issues have also been disclosed. ru. [Means for solving the problem]

[0006] (1) A function to record images captured by a camera mounted on the vehicle, and Image analysis is performed on the images captured by the aforementioned camera, and based on the analysis results, the corresponding analysis results are performed. A function that executes the process. It is preferable to have a system that has processing means that have [the specified function / function].

[0007] Not only does it record images captured by cameras mounted on the vehicle, but it also uses those images to... It can be used more effectively.

[0008] Examples of vehicles include passenger cars, trucks, and special-purpose vehicles. The camera is... For example, it is advisable to image at least one of the interior and exterior of the vehicle. Next, "interior" and " Regarding "outside the vehicle," it can be interpreted as follows, for example:

[0009] When the interior space and exterior space are clearly separated by the vehicle body, It's best to refer to the interior as the "interior" and the exterior as the "exterior." For example, the passenger space. In the case of a van where the passenger space and cargo space are continuous, the continuous passenger space and cargo space It's better to call it the interior of the car. Even in the case of a sedan where the passenger space and the cargo space are separated, the passenger It is appropriate to refer to both the passenger space and the cargo space as the interior of the vehicle. Covered cargo with a sealed cargo bed. In automobiles, the passenger compartment and the enclosed cargo area should be referred to as the interior of the vehicle. In open cargo trucks (flatbed trucks) with a flatbed, the empty space on the cargo bed is used to hold the cargo. It is difficult to clearly distinguish the space between the compartment and the surrounding area, but it accommodates open cargo. The space inside the vehicle can also be called the interior. Furthermore, the passenger compartment of an open-top freight truck can also be called the interior. In some special vehicles such as forklifts, both the passenger compartment and the cargo compartment are open. The passenger compartment and cargo compartment are clearly separated from the surrounding space. There are times when this is difficult. Even in such cases, both the passenger space and the cargo space can be It would be better to call it the interior of the vehicle. For example, the space above the seats where the passengers sit, and the space used to lift and lower cargo. The space above the forks can be called the interior of the vehicle. In this way, as the vehicle moves, The space that can be considered to move along with the vehicle is called the interior of the vehicle, and the surrounding space is called the exterior of the vehicle. That's good.

[0010] As a "camera mounted on a vehicle," under normal operating conditions, its relative position to the vehicle and It is best to use a camera with a fixed posture. Also, "a camera mounted on a vehicle" Therefore, it is recommended to use a hemispherical or full-spherical 360-degree camera. When using a 360-degree camera of a certain type, a wider range of images can be obtained compared to using a normal camera. For example, with one camera, images of both the inside and outside of the vehicle can be obtained. Note that instead of a 360-degree camera, multiple normal cameras may be attached to the vehicle.

[0011] The image captured by the camera is preferably made into a video. When the image captured by the camera is made into a video, the processing means can easily determine the movement of the subject, the presence or absence of the subject before and after a certain point, etc. by performing image analysis.

[0012] The function of recording the image captured by the camera may be realized by a server that communicates with the in-vehicle device of the vehicle on which the camera is mounted, but it is even better to perform it on the in-vehicle device. As the recording medium, a removable medium such as an SD card may be used. When using a removable medium as the recording medium, the removable medium on which the image is recorded can be removed from the in-vehicle device, and the image can be displayed on other terminals such as a personal computer.

[0013] The function of analyzing the image captured by the camera may be realized by a server that communicates with the in-vehicle device of the vehicle on which the camera is mounted, but it is even better to perform it on the in-vehicle device. When the in-vehicle device performs image analysis, processing corresponding to the analysis result can be quickly executed in the vehicle.

[0014] The processing means may be composed of one device or multiple devices. For example, a drive recorder has a function of recording the image captured by the camera. The function of the processing means of this system is the microcomputer and memory built into this drive recorder. ​​​​​​​​​​​This can be implemented by an application program stored in the system. Alternatively, for example, processing... The system consists of two devices: an in-vehicle device and a server. The in-vehicle device records the images, and the image resolution is performed by the server. It would be best to have the server perform the analysis.

[0015] Furthermore, it has a function to record images captured by cameras attached to objects other than vehicles, and front The camera performs image analysis on the captured images, and based on the analysis results, it performs corresponding actions. It may also be a system having processing means that have the function of executing processing. However, in particular (1) It would be good to do so.

[0016] (2) The processing means has an image analysis function in which the movement of a person in the image is registered in advance. It has a function to determine whether or not a person's movement corresponds to a specific movement. When this is the case, the processing corresponding to the analysis result is to capture images of the person's specific movements. Recording, sending images of specific movements to a server, and notifying that a person has made a specific movement. The system according to claim 1, which performs at least one of the processes of issuing a report and an alarm. That's good.

[0017] For example, recorded images, or images sent to a server, show a person performing a specific movement. It can be used as evidence that someone was there. For example, evidence that a person made a specific movement. By reporting this, it is possible to inform outsiders that there is a person making specific moves. For example, if an alarm is issued, a person who is performing a specific action will be notified that an alarm has been issued to them. Being aware of what is being displayed motivates one to stop a particular movement.

[0018] A person performing a specific action could be, for example, a vehicle driver. As a specific action, For example, this could be an action other than driving. For instance, looking at a mobile device such as a smartphone. Examples of actions that can be used include: eating, looking away or distracted, napping, etc. Whether or not the driver is distracted should be determined based on the detection results of the driver's eye position. The processed images can be used to improve driver awareness, analyze the causes of accidents, etc. The processing means is: If the system detects that the driver is operating a mobile device such as a smartphone, the system will stop the operation of the mobile device while driving. It would be good to issue a warning to caution drivers against taking the end off the vehicle. By stopping certain movements, danger can be avoided in advance. The server to which the data is sent and The server should be either the vehicle administrator's server or the police server. The vehicle administrator will send Based on the collected data, the driver's driving attitude can be evaluated. The police will receive the data. Based on the images, it is possible to evaluate the driver's driving attitude, analyze the cause of accidents, and so on.

[0019] A person performing a specific action, for example, a suspicious person outside the car, and a person performing a specific action, for example, a car Actions that attempt to illegally enter the premises, such as attempting to break a glass window with a hammer. Actions such as peering into the car are appropriate. By issuing an alarm, it will deter anyone attempting to illegally enter. This can deter suspicious individuals from entering the vehicle. Recorded images or screenshots Images sent to the car thief contain useful information for apprehending the thief after a car break-in has occurred. It can be used as a means to notify the vehicle owner or driver that a person has made a specific movement. By providing this information, owners and drivers can immediately return to their vehicles, preventing them from becoming victims of car break-ins. In some cases, this may be possible. Furthermore, drivers or owners who receive notification can also report it to the police. This is a function that can be used for crime prevention.

[0020] A person performing specific actions, for example, a suspicious person inside a car, and specific actions, various cars Methods targeting the upper body, for example, illegal entry into a parked car, the movements of a suspicious person inside a parked car, inside the car This could be used for actions such as removing the device. By issuing an alarm, the thief will panic and leave. By scattering them, the damage from car break-ins can be reduced. Recorded images, or Images sent to the server provide useful information for apprehending thieves after a car break-in has occurred. It can be used as a report. It can inform the vehicle owner or driver that a person has made a specific movement. By notifying the police, the owner or driver can immediately return to the vehicle or report the incident to the police. This makes it possible to reduce the damage caused by car break-ins. An example of reporting to an external party is: Examples include sending an email with an attached image of the suspect. It would be appropriate to include the vehicle manager, driver, owner, police, etc.

[0021] As a person performing a specific action, for example, a passenger on a commercial vehicle carrying an unspecified number of people, Typical actions include various robbery techniques, such as brandishing a knife or drawing a gun. This could be an action that involves committing violence, etc. The processing means will execute a process that issues an alarm. If the robber is deterred, the crime can be prevented. Recorded images, Alternatively, the images sent to the server can be used as useful information for the arrest of robbers. By reporting to the vehicle manager that a person has made a specific move, the manager can detect a robbery. If you notice the occurrence, you can report it to the police. Also, the means of processing can use communication functions to contact the police. It would be good to have a function to report incidents.

[0022] A person performing a specific movement, for example, an infant or toddler in a car, and a person performing a specific movement For example, it would be good to use the movement when an infant wakes up from sleep. Image analysis can detect when an infant wakes up. In that case, the appropriate course of action would be to notify the parents via email or similar means. I fell asleep and left the engine running while I went out to do some shopping or ran some errands. There are times when you want to do this. However, if you forcibly wake an infant, they will become fussy, so There is a request to leave the child asleep as is. However, when the guardian is away from the vehicle, I'm worried about whether my infant is awake. This system will help me find out if my infant is awake. An email will be sent to the parent's mobile device at that moment, allowing the parent to immediately rush to the vehicle. This allows you to address the awakening of infants and toddlers.

[0023] For example, a person performing a specific movement, such as a person requiring care or rescue outside the vehicle, and their specific movements For example, if we define these as actions specific to people requiring care, or actions that correspond to situations where rescue is deemed necessary, then Good. For example, as an example of the specific movements of a person requiring assistance, for example, leaning on a cane, bending at the waist, One example is a wobbly gait. An example of a movement that indicates a situation requiring rescue is falling. Examples include actions such as collapsing while foaming at the mouth.

[0024] The function detects people performing specific movements, using images from the front of the vehicle, the rear of the vehicle, and the interior of the vehicle. Includes a function to select at least one image from the images, and from the selected image, the person in question It would be good to enable object detection. The user should be able to specify the selection of at least one image. This is a good approach. This will prevent the processing device from analyzing unnecessary or unimportant images. Because this is eliminated, there is no longer a need to give processing devices excessive processing capacity.

[0025] It would be good to include a function to detect driver drowsiness in the function that detects people performing specific movements. For example, by measuring the pulse rate from the degree of change in the redness of the driver's skin, and based on the change in pulse rate, It is good to determine whether the driver is drowsy. This can be done by measuring body temperature with a thermal camera and using microwaves. It would be good to improve the accuracy of sleepiness detection by linking it with pulse rate measurement and other data. If drowsiness is detected, the air conditioner's temperature setting will be lowered, or a voice or sound warning will be issued. good.

[0026] (3) The processing means, as an image analysis function, determines that a person in the image is a person who has been registered in advance. It has a function to determine whether or not it has certain features, and as a processing corresponding to the analysis result, within the image It would be ideal to have a system that performs processing according to the specific characteristics of the person.

[0027] The system can perform appropriate processing based on the characteristics of the people captured in the image.

[0028] Functions that determine whether or not a person in an image possesses specific characteristics include, for example, facial recognition. It would be good to do this. For example, perform facial recognition on the person sitting in the driver's seat, and if the person sitting in the driver's seat is pre-registered If the person is not a registered individual, the processing method will be to access the vehicle and start the engine. It is best to prevent this from happening. For example, from the processing unit to the engine control unit (ECU). It is advisable to send a signal to the vehicle that prohibits engine starting. This will allow it to operate in advance. It is possible to register authorized individuals and prohibit unregistered individuals from driving the vehicle. As a result, the vehicle's security features can be enhanced. Also, as a measure against children's mischief... This is also effective. In addition to the process of prohibiting engine starting, it also prevents the accelerator from being pressed. To prevent the handbrake (parking brake) from being released, the shift lever is driven Avoid entering the Eve location, sound an alarm, and send an image along with the message to the registered email address. You may perform actions such as sending an email.

[0029] In addition, for example, facial recognition of a person outside the vehicle can be performed to determine if that person matches a pre-registered individual. In such cases, it is advisable to have the processing device control the vehicle and unlock the doors. For example, If the vehicle is to be provided to a car-sharing service, the car-sharing service It is recommended to register as a registered member in advance. Registered members can approach the vehicle and view the vehicle's camera image. By moving your face into the corner, you can use the vehicle even without possessing a key card or similar. This makes it possible to use the vehicles in car-sharing services more convenient. It will get worse.

[0030] The vehicle is a commercial vehicle used by an unspecified number of people, and the means of handling it is to identify the passengers as wanted criminals. It would be good to determine whether or not it is a wanted criminal using facial recognition technology. The facial photographs of wanted criminals are stored by the police. The data is transmitted from the database to the processing means of this system via communication such as LTE or Wi-Fi. It is advisable to register the passenger in advance with the processing system. The processing system determines that the passenger is a wanted criminal. If this occurs, the vehicle manager and the police will be notified via the communication device and communication network. It is best to do this via the system. Furthermore, the processing method should notify the driver in a way that the passengers will not notice. It would be good if the vehicle manager, police, driver, etc., acknowledge that a wanted criminal has boarded the vehicle. They can recognize the issue and take prompt and appropriate action.

[0031] For example, a taxi is a confined space with only the driver and passenger inside, so the driver's safety... In terms of apprehension, there seems to be a high demand for this system. When the processing system detects this, it should notify the taxi company or the police that a criminal is in the taxi. Furthermore, according to this system, before a criminal actually commits a crime inside a vehicle, the system can detect if the criminal is in the vehicle. The system can detect criminals boarding the vehicle at the moment they get inside. Therefore, it helps to prevent crimes from happening in the first place. It will also become possible to do so.

[0032] A function that determines whether or not a person in an image has certain characteristics, for example, the external characteristics of a person in an image. A function that compares the appearance with a specific appearance that has been registered in advance and determines whether the two match or not (hereinafter It would be good to include a visual authentication function. Specific visual characteristics include the appearance of a wanderer, a criminal It is advisable to register in advance the distinctive appearance of criminals. Examples of distinctive appearances of criminals include: Examples include carrying a knife or other dangerous object on the street. By combining images from external cameras, ultraviolet cameras, etc., with images from regular cameras, illegal drugs It would be good to be able to detect a person who possesses it. The processing means detects a specific appearance in the image. If a person carrying the device is detected, the police or vehicle administrator will be notified via the communication device and communication network. It would be good to know.

[0033] If you want to find a specific missing person or wanderer, for example, their appearance at the time they went missing. It is advisable to register this information individually as search data in the processing system. For example, gender, height, clothing. Type of attire, color of clothing, whether or not a hat is worn, whether or not a cane is being carried, whether or not the person is bent over It is recommended to register individual search information such as the area where the person is thought to be wandering. It is recommended to register the area (elephant area). The processing means that when a vehicle is located in the search area, individual It is advisable to perform a detection process for individuals that match the search information. Such individual search information is used by the police. The system transmits data from the database (server) of local support centers, etc., to the processing unit of this system. It would be good to have a vehicle-mounted device that performs a function to determine whether or not a person in an image has specific characteristics. Once the vehicle enters the search area, search information is downloaded from the server to the vehicle's processing system. It would be good to do so.

[0034] The function to determine whether a person in an image possesses specific characteristics, for example, the passenger seat or the back seat. A function that determines whether or not a person sitting in a seat is wearing a seatbelt (hereinafter referred to as appearance recognition). It would be good to include the function. The processing means of this system detects a person not wearing a seatbelt. When detected, it would be good to display a message prompting the user to fasten their seatbelt.

[0035] In particular, when approaching a highway entrance with a passenger in the back seat not wearing a seatbelt, It would be good to display a message encouraging the driver to fasten their seatbelt. For example, from the vehicle's current location, When the distance to the highway entrance falls below a certain distance, it means you are approaching the highway entrance. It is good to make a determination. In addition, if you receive a signal to check if you have forgotten to insert your ETC card, then you should proceed to the highway It is good to determine when the vehicle is approaching the entrance to the road. Also, analyze the image in front of the vehicle and look within the image. When a highway entrance is detected, it should be determined that the vehicle is approaching the highway entrance.

[0036] (4) The processing means has a function of counting the number of people in the image as part of the image analysis function. Then, as a process corresponding to the analysis results, the system executes a process to record or transmit the number of people in the image. It would be good to have a system that does this.

[0037] This system counts the number of people without using any special equipment for counting. It is possible.

[0038] For example, a camera could be attached to a bus, and images of the interior of the bus could be used as the source of images for counting the number of people. It would be good to adopt this. The processing method should be to display the counting results and inform the crew. The crew This function makes it easy to determine the number of passengers (occupants) on board the vehicle. It can be used in place of a tour roll call. When all passengers should disembark, the passengers When the counting result is not zero, the crew knows that someone is trapped inside the vehicle. This is possible. For example, for people who cannot express their own will, like babies, or people with disabilities. It can detect people who are unable to disembark on their own and remain behind.

[0039] For example, a vehicle to which a camera is installed is a type of vehicle where the number of passengers is predetermined, for example This includes tour buses, long-distance express buses, etc., and the number of passengers is predetermined and processed by this system. It's a good idea to register it in the section. The processing method involves comparing the count result with the registered number of people. If there is a surplus or shortage, it is a good idea to warn the crew about it. This will help the passengers This prevents the train from departing with insufficient passengers. It is easy to identify if someone has boarded the vehicle and to ask them to get off.

[0040] For example, a camera (for example, a hemispherical camera on the ceiling) is installed inside a route bus, and processing is done The system performs image analysis to count the number of passengers, the number of standing passengers, and the number of sitting passengers. It would be good to have the following functions: Bus current location information, bus door opening / closing signals, bus disembarkation information. The processing device acquires signals from the buzzer, IC card touch when boarding or alighting the bus, and the fare box, and then It would be good to have a function to record using that as a trigger. This information can be stored in a data lake. Big data analysis to determine where people boarded and alighted, and what the seating situation was like inside the vehicle during that time. This makes analysis possible. Such data can be used by consulting firms that handle urban planning, etc. Furthermore, it can be valuable information when transmitted via in-vehicle networks such as CAN. It is best to obtain information and the driver's vital signs together. This information is useful if the vehicle has any issues. When an abnormality occurs, it is determined whether the abnormality is due to a driving factor, a passenger factor, or a driver factor. It can be used to identify causal relationships, such as whether both factors are involved.

[0041] For example, it would be good to use an image of the outside of the car as the source image for counting the number of people. The processing method should involve recording the counting results as people data. Therefore, it is possible to easily extract locations with large crowds. Using the counting results as a trigger Therefore, it is a good idea to record the image that was used as the basis for the count. For example, if the number of people exceeds a predetermined threshold. It's a good idea to record images at the time of the above.

[0042] (5) The processing means, as an image analysis function, processes images from inside the vehicle to the vehicle's display device. It has a function to detect the displayed content or the operating status of the vehicle's control devices, and processes the results accordingly. In principle, the system executes processing according to the display content of the display device or the operating status of the operating device. It would be good to have a system like this.

[0043] A device that acquires the data that forms the basis of the display content of a display device, or the operating status of an operating device. This system acquires the corresponding data without requiring the addition of separate data acquisition equipment. This is possible. For example, without connecting aftermarket devices to the OBD connector, these This allows you to obtain data. In particular, when connecting a third-party product to the OBD connector... Even if the vehicle manufacturer prohibits it, this system can collect this data. This can be obtained. This display device can display various instruments, in-car warning lights (warning lamps), and floor It would be good to include a head-up display that projects images onto the headlight glass or similar.

[0044] If the content of the display device to be detected is, for example, the vehicle speed displayed on the speedometer, Good. The processing means recognizes the position of the needle of the analog speedometer as a pattern and the numbers on the dial as characters. It is recommended to acquire vehicle speed data by recognizing the numbers displayed on the digital speedometer. It is recommended to obtain vehicle speed data. The obtained vehicle speed data can be used for various processes. This is possible. Below is an example of how vehicle speed data can be used.

[0045] The processing device should ideally have a function to automatically detect the speedometer using image recognition. Therefore, the processing means can detect the speedometer in various vehicles, regardless of the vehicle type. This can be done. In addition, the processing means can process the image of the speedometer of the vehicle on which the processing means is installed. It is advisable to pre-register the position and shape in the location. This allows the processing means to easily speed It can detect the meter.

[0046] Standalone car navigation devices use GPS to obtain their current location. It uses location information. If GPS signals cannot be received, such as in tunnels, other methods will be used. The vehicle's current position must be determined. For example, the speed immediately before the GPS signal is lost. The vehicle's current position on the map is moved at a constant speed of approximately 80% for 5 minutes. The mode in which processing is performed is called tunnel mode. This is effective not only when entering an overpass, but also when traveling under one. However, tunnel When processing in automatic mode, if you encounter traffic congestion, you will be driving at a constant speed. Using this assumption would result in an excessively large error in the current position, making it practically unusable. No. If you attach an acceleration sensor to the vehicle and use the acceleration data from the acceleration sensor... This allows for improved accuracy of current location information in tunnel mode. However, For Za, installing an accelerometer is difficult and expensive, making it impractical.

[0047] The vehicle's speed is read from the speedometer display, and based on this read speed, the car navigation system... The current location information of the vehicle is obtained using the map matching function implemented in the system. Updating this will improve the accuracy of your vehicle's current location information.

[0048] Where a road inside a tunnel runs parallel to a ground-level road, and where an elevated road runs parallel to a ground-level road beneath it. In locations where this is the case, the system will match you to either road based on whether or not tunnel mode is activated. You can choose to enable or disable this feature. For example, when tunnel mode is activated, the vehicle's position is displayed. Map the road inside the tunnel or the ground road under the elevated structure, and activate tunnel mode. If not available, the vehicle's position is mapped to the ground road or elevated road running parallel to the road inside the tunnel. It would be good to let them do it.

[0049] The display content of the display device to be detected may include, for example, the fuel level displayed on the fuel gauge. It would be good to do so. The processing means would detect the remaining amount of fuel by performing image recognition of the fuel gauge. i. When the processing means detects that the remaining fuel level has fallen below a predetermined standard value, it will instruct the driver to refuel. It would be good to notify users of a message prompting them to do so while they are out and about. For example, it could be linked to the current location information of the vehicle. When you return from your home or near the company's vehicle storage area, the remaining fuel level should be at the standard level, for example. For example, if the tank is less than 1 / 4 full, the system should issue a warning and prompt refueling. - The navigation device and this system are linked, and the processing means calculates the distance to the destination and the fuel consumption. The system compares the remaining fuel level with the current level and prompts refueling at the start of the journey or along the route to the destination based on the comparison results. It's a good idea to do this. For example, near a gas station along your route, check the gas station near your current location. It would be good to display a message prompting the driver to refuel.

[0050] The function to extract the speedometer and fuel gauge from the image is installed in the processing means of this system. This should be achieved using artificial intelligence (AI). Artificial intelligence can be used to extract information such as speedometers and fuel gauges from images. If this is difficult, the positions of the speedometer and fuel gauge in the image and the image information may be pre-recorded in the processing means. It's a good idea to register.

[0051] The processing means reads the fuel gauge of the vehicle provided to the car-sharing service, When the remaining fuel level is detected to be below a predetermined standard value, refueling will trigger the car-sharing service It's a good idea to inform drivers that they can receive a special benefit if they refuel now. The system displays a message such as, "You can receive a discount equivalent to 15 minutes." It would be good to do so.

[0052] Examples of display content on the display device to be detected include warnings on the instrument panel. It would be best to determine whether the light is on or off. The warning light to be used to determine whether it is on or off is the headlight. Light indicator, turn signal indicator, high beam indicator (high beam warning) It would be good to include lights, parking brake indicator, door ajar warning light, fuel warning light, etc. The processing means of this system determines whether these warning lights are illuminated or not, and then... It can detect the status of the corresponding vehicle.

[0053] The content displayed on the display device to be detected is, for example, a head-up display. The warning information displayed is a head-up display. The system detects changes in the displayed image to identify the warning content and executes a predetermined process corresponding to the warning content. It would be good to do so. For example, it would be good to issue warnings using sound or voice. Warnings using visual information, By converting the information into sensory information and issuing it, the effect is obtained that drivers will be more likely to notice the warning. It can be done.

[0054] For example, by linking this information with ambient light outside the vehicle and the vehicle's current location, the processing steps of this system can be improved. Even though the situation requires the headlights to be turned on, the headlights do not turn on. If the indicator light is not illuminated, it will instruct the driver to turn on the headlights. It would be good to notify them. Situations in which headlights should be turned on include, for example, when there is more light outside the car than This includes situations that are dark, where the vehicle is currently located inside a tunnel or culvert, etc. The headlight indicator remains on even though the situation requiring the headlights to be turned on has been resolved. When the light remains on, the processing means of this system will turn off the headlights. It would be good to notify the driver. Some tunnels are brightly lit inside due to the lighting, so the outside light is also bright. If we determine whether or not the headlights should be turned on based solely on the information provided, then... Despite being inside the vehicle, the system mistakenly determined that it was not a situation where the headlights should be turned on. This situation could arise. The determination of whether or not the vehicle is inside a tunnel is based on the vehicle's current location information. This helps to avoid such misjudgments. Furthermore, it applies not only to headlights, but also to the width of the vehicle. Even though the lights (small lights) should be on, the indicator light for the parking lights is not on. If the system detects that the marker lights are not illuminated, it will notify the driver to turn on the parking lights. That's good.

[0055] The processing means of this system determines the presence or absence of an oncoming vehicle from the image in front of the vehicle, and if an oncoming vehicle is present... Nevertheless, if the high beam indicator is detected to be lit, it will switch to low beam. It is a good idea to warn the driver to change the settings. The operation for switching between high and low beams varies depending on the vehicle model. Therefore, when a driver is driving a vehicle of a different type than the one they normally use, they should use low beams. It's easy to accidentally leave the high beams on when you intended to switch them off. Therefore, the function that issues this warning is particularly effective.

[0056] The processing means of this system detects when the turn signals are flashing from an image of the outside of the vehicle when it is dark outside. It detects the turn signal and flashes an indicator inside the vehicle in sync with the turn signal's flashing cycle. It is best to recognize it as a ca indicator. In this way, the processing means is instrument The turn signal indicator can be easily found within the control panel.

[0057] The processing means of this system detects from the images inside the vehicle that there is no one inside the vehicle, Furthermore, if the headlights or parking lights are detected to be on, the system will notify the driver or vehicle management outside the vehicle. It is a good idea to notify the person that they forgot to turn off the lights. This notification can be sent, for example, to a pre-registered email address. This can be done by sending an email to the email address. Other processing methods include: Set up a system to send email notifications if you forget to turn off the hazard lights or interior lights. It would be good to do this. The status of the hazard lights can be detected from the flashing status of the turn signal indicator. That's good. The illumination status of the room light can be determined from the brightness of the room light visible in the image. It would be good to know.

[0058] Room light settings typically include always-on mode, always-off mode, and door open / closed status. There are three types of linked modes (hereinafter simply referred to as linked modes). In many vehicles, These three modes cycle through and are refreshed each time the room light on / off switch is pressed. When the engine is stopped in a dark place (at night), the interior lights turn on, and after a while... Some vehicles have a feature that automatically turns off (for example, after a few seconds). If you want to keep the interior lights on for a longer time to prepare to get out of the car, There are also cases where, after turning off the engine, you organize and take out the purchased items, and then use your mobile phone. During the period when removing the last part of the car, etc., I would like to keep the room lights on. The driver wanted to prevent the interior lights from automatically turning off after a certain period of time has elapsed since the engine was stopped. To do this, switch the room light setting to either linked mode or always-on mode. However, When the door is open, the current setting of the interior light is either always on or on. It is difficult to determine this. Even if the current room light setting is in constant-on mode. Regardless, if you mistakenly think it's in linked mode and get out of the car, the interior lights will remain on. This system allows drivers and administrators to control the interior lights. Realizing that I had forgotten to turn off the lights, I turned off the room lights, which meant that the room lights were not left on for a long time. This can prevent the vehicle's battery from running out. Just by looking at the control switch, the room The current mode of the light is not clear from the structure, for example, the push button. A vehicle with an operating switch that cycles through the room light settings each time it is pressed. In this context, this function of the system is particularly useful.

[0059] When the interior lights are set to linked mode, if any door is open or ajar, In that case, the room light will remain on. A notification will be sent indicating that the room light is on. If a driver or manager is found to have a door ajar or an open door on the side that is difficult to see, You can realize that there is a possibility that this is the case.

[0060] When the room light is set to linked mode, there is a certain delay after closing the door (off delay time). Some vehicles have a function that keeps the interior lights on until a certain amount of time has passed. In such vehicles, the processing means of this system detects that the door has been closed and then proceeds to... It is best not to determine whether or not the lights were forgotten until the delayed light-off time has elapsed. The delay time should be set by this system based on user input.

[0061] Furthermore, the processing method is designed so that even after a certain amount of time has passed since no one has left the car, the interior lights remain on. If it detects that the light remains on, the vehicle's controller will turn off the interior light. It would be good to send a signal to turn it on.

[0062] The processing means of this system is for headlights, side marker lights, hazard lights, interior lights, etc. When the vehicle door is opened while the light is on or flashing, one of the lights will activate. It would be good to have a function that issues an alarm when the light remains lit. The driver can notice they've forgotten to turn off the lights before getting out of the vehicle, thus preventing the problem from occurring. .

[0063] The processing means of this system, based on the illumination of the parking brake indicator, When it detects that the brakes are applied, it uses the driver's image to determine where the driver is located. It would be good to prevent alarms from being triggered even if sleep is detected. In addition, park When the brakes are applied, the driver may be looking at a mobile device or otherwise using other means while driving. Even if it detects an action that is considered dangerous, it will not issue an alarm. It would be good to do so.

[0064] The processing mechanism of this system provides a warning indicating that inspection or repair is needed as soon as possible. If the light remains illuminated for a certain period of time, the vehicle's dealer or registered It would be good to have a function that notifies the registered repair shop of the vehicle's condition. For example, if the car is interesting to... Users who do not have a warning light may not notice it is on if the vehicle is still drivable even when the warning light is illuminated. Often, this is left unattended. For example, when the engine warning light illuminates, it usually means the vehicle is not drivable. However, even if the warning light indicating an abnormal engine oil is illuminated, you can continue driving. In addition, if the O2 sensor is malfunctioning, fuel efficiency will decrease, but the vehicle will still be able to run. If you continue driving without performing inspections and repairs when the vehicle is still drivable, minor malfunctions may worsen. In some cases, the notification of a warning light being illuminated is for dealers and repair shops. This information will be useful for sales activities to vehicle owners regarding inspections and repairs. By performing inspections and maintenance early, it is possible to prevent malfunctions from becoming more severe.

[0065] For example, the display content of the display device to be detected could be a temperature sensor that measures the outside temperature of the vehicle. Therefore, the measured temperature value is displayed in the temperature display area on the instrument panel. The temperature should be set accordingly. The processing means, as an image analysis function, is the instrument panel It would be good to have a function that reads the temperature from an image displayed in the temperature display area using character recognition. The processing means, as a process corresponding to the analysis results, communicates via a communication device and a communication network. It would be a good idea to implement a process that sends temperature information to an external server.

[0066] For example, sensors are installed in the homes of ordinary users to obtain data on pollen dispersion. Furthermore, by collecting measurement data from sensors, it is possible to predict the detailed pollen dispersal situation and the flowers Services that provide information on dust dispersion are offered by companies such as Weathernews Inc. This includes outdoor photos submitted by general users, and outdoor live cameras installed at private residences. The images collected are incorporated into weather forecasts. Furthermore, the servers that distribute these images... Services (such as Weathernews SOLiVE24) are available. The system providing the information service collects outside temperature information measured by many vehicles. This allows more vehicles to be used as temperature measurement probes. It transmits temperature information. When many vehicles are distributed over a wide area, the temperature information collected from many vehicles is not localized to specific spots. It can be useful as information. For example, it can provide localized information that cannot be expressed in general weather reports. It will become possible to issue heatstroke warnings and other similar alerts.

[0067] For companies providing such weather information services, installing temperature sensors in-house is a necessary step. This allows for the collection of temperature information over a wide area without requiring any special measures. The company that received the temperature information will, in return, give a gift to the owner of the vehicle that provided the information. It would be beneficial to offer the paid weather information service free of charge or at a discounted price.

[0068] In addition to temperature information read from the temperature display area in the instrument panel, the camera The numbers displayed on the roadside temperature sign captured in the image were recognized using character recognition. By doing so, it is good to obtain outside temperature information. Detection of temperature display signs can be done, for example, within the sign This can be done by recognizing text such as "Current Temperature". The acquired outside temperature information is Similar to the temperature information obtained from the temperature display area in the instrument panel, this system The processing device should send the data to an external server.

[0069] The function detects the operating status of control devices through image analysis, including the accelerator pedal and brake. It would be good to include a function to detect the position of the driver's foot relative to the pedal. Depending on the situation, the driver's foot may be positioned on the accelerator pedal or the brake pedal. It would be good to implement a process that notifies the driver of their location so that they can easily recognize where the vehicle is located. For example, when your foot is in the position of the accelerator pedal and when it is in the position of the brake pedal. The driver should be notified of this by changing the color of the light emitted by the light-emitting element. The system detects whether the foot is on the handlebars or the brake pedal, and the detection result... This system is installed in a vehicle that has the function of displaying visual information on a display unit, and this system The processing means performs image analysis on the content displayed on this display and determines the accelerator pedal and brake pedal It would be good to detect the driver's foot position relative to the dial.

[0070] Furthermore, even though an obstacle was detected ahead through image analysis, the foot was still on the accelerator pedal. When the device is in the "dull" position, it would be good to issue a warning with sound or other means. By the driver noticing the warning... This helps to prevent accidental misapplication of the accelerator pedal and the brake pedal.

[0071] The function detects the operating status of control devices through image analysis, including the steering wheel (handle It would be good to include a function to detect the rotation angle of the dollar. Processing according to the operating status of the aforementioned operating device and Based on the steering wheel rotation angle and vehicle speed, the vehicle's direction of travel and distance are determined. It is recommended to perform a process to obtain the result. The processing result can be used, for example, with a standalone car navigation system. It is recommended to provide this to the device. Standalone car navigation devices can be accessed from this system. Based on the given information regarding the direction of travel and distance traveled, the GPS signal cannot be received. Even under these circumstances, the vehicle's travel route can be calculated.

[0072] The image analysis function determines whether the vehicle is in autonomous driving mode (including autonomous driving levels 1-5) or manually driven. It would be good to include a function to detect whether the vehicle is in transition mode (autonomous driving level 0). This detection could be, for example, by using an insulator. The indicators displayed on the control panel show whether or not the various autonomous driving functions are in operation. This should be done by recognizing the display state of the data from the image. The processing should correspond to the analysis results. For example, if it detects that the autonomous driving function has been deactivated, It would be good to perform a process that notifies the driver of this. Also, even if the automatic driving function has been deactivated Furthermore, if it is detected that the driver does not have their hands on the steering wheel, processing means It would be good to issue a warning to remind the driver to operate the steering wheel. Furthermore, the autonomous driving function was deactivated, and the driver's hands were on the steering wheel. When this is not triggered, the processing means should record the image captured by the camera. The recorded images can be used as evidence to show that the driver's actions were inappropriate. can.

[0073] In a vehicle equipped with a radar detector, the processing means is that the autonomous driving function is enabled. In that case, it is best to disable the radar detector's function. This will prevent unwanted signals from the radar detector. This eliminates the need for alarms. When the automatic driving state is disabled, processing steps It's a good idea to enable the radar detector function.

[0074] Is the vehicle in autonomous driving mode or manual driving mode? What level (mode) of autonomous driving is it in? ) whether it is, indicators, steering wheel movement, driver's hand position, driver It's best to determine this based on the direction the head is facing, etc.

[0075] (6) The processing means, as an image analysis function, processes images from inside the vehicle and pastes them onto the vehicle. It has the function to recognize the characters displayed on the sticker and processes the analysis results accordingly. The system then performs a process to issue a warning based on the information indicated by the recognized characters. It would be good to do so.

[0076] By being alerted, the driver, manager, or owner of the vehicle will affix the notice to the vehicle. Attention is drawn to the textual information on the sticker, and appropriate processing is performed according to that textual information. can.

[0077] For example, if you include inspection stickers (vehicle inspection stickers) in the stickers to be analyzed by image analysis... The inspection sticker, on the inside of the vehicle, displays the date the vehicle inspection expires. The method involves reading the date of expiration of the vehicle inspection using character recognition, and then determining the current date. If the period until the expiration date of the vehicle inspection (vehicle inspection deadline) becomes shorter than the prescribed period, It would be good to remind them to get their vehicle inspected. For example, the processing method could be, "The expiration date of the vehicle inspection is..." It is advisable to issue a warning by emitting a voice message saying, "It is approaching." If the period until the expiration date becomes even shorter, for example, the processing method will be, "Please get your car inspected." It would be good to issue a warning with the voice message, "Please stop."

[0078] For example, it would be good to include the maintenance and inspection sticker among the stickers to be analyzed in the image. The inspection completion sticker displays the date of the next scheduled inspection (year, month, and day). The system reads the date information indicating the deadline for periodic inspections using character recognition, and then calculates the date from the current time to the periodic inspection deadline. If the period until the inspection deadline is shorter than the prescribed period, a reminder will be issued to have the vehicle inspected. It would be good if the processing device could emit a voice message saying, "The deadline for your periodic inspection is approaching." It would be good to raise awareness by issuing a notice. If the period until the next scheduled inspection becomes even shorter... For example, the processing device will issue a warning in voice saying, "Please undergo regular inspection." good.

[0079] The image analysis function determines whether or not an inspection mark is visible in an image captured by a camera. It would be good to include the function. As a processing step corresponding to the analysis results, if the inspection mark is not visible in the image... In that case, the processing means adjusts the mounting position and orientation of the camera so that the inspection mark is captured in the image. It is advisable to implement a process that alerts the user to the following: "If the inspection mark is visible..." "There is no one. Please adjust the camera's position and orientation so that the inspection sticker is visible." It is helpful to make a sound. This makes it easier to adjust the position and orientation of the camera when setting it up. This is possible. The processing means can be used to process data displayed on display devices inside the vehicle, such as a speedometer, fuel gauge, etc. It has a function to read content and a function to warn users that display devices inside the vehicle may be visible in the image. It's even better if you have one.

[0080] The image analysis function includes the windshield, driver's side window, and passenger side window, as required by law. If we include a function to determine whether or not stickers other than approved stickers are attached, Good. As a process corresponding to the analysis results, stickers other than those permitted by law are... If it is determined that a sticker is present, the system will notify the driver or other relevant person to remove the sticker. It would be good to do so.

[0081] (7) The processing means, as an image analysis function, analyzes the image of the area in which the passenger seat is reflected. It includes an analysis function, and as a processing step corresponding to the analysis results, it executes processing according to the situation in the passenger seat. It would be good to have a system like this.

[0082] For example, depending on whether there is a person in the passenger seat, or whether there are objects placed on the passenger seat , the appropriate processing can be performed.

[0083] The system will issue a warning if the person sitting in the front passenger seat is not wearing a seatbelt. It would be good to install it in a vehicle that has the function to do so. The processing means is an image analysis function that captures the passenger seat It would be good to have a function that can determine whether or not someone is sitting in the passenger seat based on the image of the area being entered. As a processing step corresponding to the analysis results, the processing means detects that there is no one sitting in the passenger seat. It would be good to disable the function that issues a warning if the passenger seatbelt is not fastened. The processing method is It would be good to have a function in the image analysis that can determine whether or not there is an object placed on the passenger seat. As a processing step corresponding to the analysis results, the processing means, when an object is placed on the passenger seat, the passenger It would be a good idea to disable the function that issues warnings when a seatbelt is not fastened.

[0084] The vehicle mistakenly identifies a person as being in the passenger seat when an object equivalent to a person's weight is placed on the passenger seat. This may result in the system issuing a seatbelt-unfastened warning. If an object is placed on the passenger seat, the vehicle will issue a seatbelt unfastened warning. This eliminates the problem. As a result, drivers will no longer be misled by warnings. Especially in the passenger seat. In vehicles that have a feature that causes a warning buzzer to sound continuously when a heavy object is placed on them, this device The function is effective. To prevent the vehicle from issuing a seatbelt unfastened warning: For example, the processing means sends a warning to the vehicle's controller via the in-vehicle network (CAN). It would be best to send a signal that disables the transmission of that signal.

[0085] Regarding the rear seats, in vehicles with similar functions, items are placed in the rear seats. When this is detected, the processing device should cancel the seatbelt unfastened warning function.

[0086] (8) The processing means, as an image analysis function, uses images of the car interior at the present time and images of the car interior in the past. It has a function to compare with images inside the vehicle, and as a process corresponding to the analysis results, it displays the current image of the vehicle's interior. The system then executes a process to notify the user if the degree of discrepancy with past in-car images exceeds a predetermined level. It would be good to have a system like this.

[0087] The vehicle owner or manager will be notified if the conditions inside the vehicle exceed a certain level. You can see that it has changed. Images of the interior of the car from the past are included with the vehicle. It is best to use an image showing that nothing other than objects has been brought into the vehicle. This image and the current By comparing it with images of the car's interior, it is possible to determine whether or not something was brought into the car. It is possible.

[0088] By comparing images from multiple past points in time with images from the present, it is possible to determine what was brought into the vehicle. It is best to calculate the amount of time the item remains inside the vehicle. The processing method involves leaving the item inside the vehicle for an extended period of time. It would be good to have a function that notifies you when an item that has been left unattended is detected.

[0089] For example, when the vehicle is empty and, ideally, only the driver is inside. Then, press the initial setup button of this system and start driving the vehicle. The processing means corresponds to the vehicle speed. Areas of the image with optical flow are recognized as windows. Other areas with movement are not recognized as windows. The driver is identified as the driver. Areas other than the windows and the driver's area may be used for placing objects. It recognizes a certain area and processes it as an image of a vehicle without cargo (unloaded image). Record. The initial setup process is now complete. From now on, when starting the engine, When the vehicle is stopped, the difference between the image of the vehicle without luggage and the current image is taken to determine what objects have been brought into the vehicle. Identify its presence. If there is no change in the position of the object identified as having been brought into the vehicle for a certain period of time... In such cases, a warning will be issued. This will alert the driver or other relevant parties to any objects left inside the vehicle for an extended period of time. It can be made to recognize the contents. This feature is especially useful for people whose cars quickly become cluttered with trash. It is effective.

[0090] (9) The processing means, as an image analysis function, checks if pre-registered items appear in the image of the vehicle interior. It has a function to determine whether or not something is captured in the image, and the processing corresponding to the analysis result is pre-registered. Let's say the system executes a process to issue an alarm when the object is visible in the image inside the car. stomach.

[0091] Drivers and other personnel can become aware of the presence of a pre-registered item inside the vehicle through the alarm. For example, you can process images of items you want to check for forgotten items (specific bags, wallets, etc.). It is a good idea to register it in the system. The processing method is registered when no one is inside the vehicle. If an object is detected in the image, an alarm will be issued or a notification of a lost item will be sent. For example, an alarm could be sounded by installing a buzzer inside the vehicle. As an example of knowledge, for instance, wirelessly sending emails to drivers' smartphones or mobile phones, etc. It would be good to notify them.

[0092] For example, if the processing means detects an item left behind in the vehicle and also detects the movement of opening the door, an alarm will sound. It would be good to issue a warning. The warning will allow the driver or other personnel to realize they have left something behind before leaving the vehicle. Yes, it is possible. The processing means could, for example, detect an item left behind in the car and also detect the movement of closing the door. It is a good idea to notify the driver or other relevant person via email or other means that an item has been left behind. The driver or other relevant person should check the vehicle or You can realize you've left something behind after you've left. The detection of door opening and closing motions is... For example, this can be done by detecting optical flow.

[0093] The processing means detects optical flow and other movements inside the vehicle. It would be good to have a detection function. For example, the processing means could have an image analysis function to detect the opening and closing of a door. Movement, window opening and closing, turn signal operation, steering wheel operation, sun visor It would be good to detect operations such as lowering the vehicle. This system connects to the vehicle's in-vehicle network connector. This information can be obtained without connecting to, for example, an OBD connector.

[0094] When the sun visor is down, especially with a 360-degree camera mounted on the windshield... When trying to capture images of the driver's side, the sun visor gets in the way and it's impossible to get an image in that direction. Therefore, the front of the vehicle should be bright enough that a sun visor is deemed unnecessary. Nevertheless, if the processing device detects that the sun visor is down, it will issue a warning. Good. Whether the sun visor is down or not is based on the proportion of the image occupied by the sun visor. The processing means should then make the determination.

[0095] (10) The processing means, as an image analysis function, when the user brings the camera closer It has the function of capturing images of objects and performing image recognition, and as a processing corresponding to the analysis results, the user can... A good system would be one that performs actions based on the information written on objects brought close to Mera.

[0096] By bringing an object they possess close to the camera, the user can respond to the information written on that object. It is possible to execute the specified process.

[0097] For example, the object to be brought close to the camera could be the driver's license. The processing method is The system recognizes the expiration date written on the driver's license and calculates the period from the current time to the expiration date. If the interval is shorter than the specified period, the user will be notified that the driver's license renewal date is approaching. It would be good to do so.

[0098] For example, a gas station receipt would be a good object to hold up to the camera. The process involves recognizing the date, fuel quantity, and amount displayed on the receipt, and then processing this information. It's a good idea to record it and create a database.

[0099] For example, you could bring identification documents close to the camera (e.g., driver's license, employee ID, address). It is preferable to use a base card, etc. The processing method involves character recognition of the name etc. written on the identification document. It is best to determine whether the person is already registered. If the person on the identification document is not registered... In such cases, it is advisable to perform a process that prohibits engine starting. This will prevent the vehicle from being driven. The people who can do this can be limited to those who have been registered in advance. Furthermore, the processing means is It is advisable to record the name and other details of the person on the identification document. The recorded information will be used for vehicle operation management. It can be used to identify the driver in such situations.

[0100] The processing means, as an image recognition function, detects objects that the user brings close to the camera outside the vehicle. It is desirable to have a recognition function, such as a function that recognizes objects that the user brings close to the camera inside the car. It would be even better if it had. For example, the processing means has features that are not present in a normal image compared to a normal image. When an object occupies a larger portion of the camera's field of view than a predetermined range, it can be brought closer to the camera. It is good to be aware that such things exist. In particular, the processing means is wider than a predetermined range within the camera's field of view. When it is recognized that the user is holding something that occupies a large portion of the screen, the system will recognize that the user is holding that item. It's best to determine that you've gotten close to Mera.

[0101] (11) The processing means, as an image analysis function, processes the objects captured in the forward image. It has the function to perform image analysis, and as a process corresponding to the analysis results, it takes precautions regarding the vehicle's movement. It would be ideal to have a system that performs the process of providing the driver with the necessary information.

[0102] By alerting the driver according to the situation ahead, the vehicle will proceed forward. This can help prevent accidents that may occur in the future.

[0103] The image analysis function calculates at least one of the following: the distance to the vehicle ahead and the relative speed. It would be good to include a feature that detects situations with a high risk of collision based on the distance between vehicles or relative speed. It is advisable to notify the driver of the high risk of collision through the issuance of a warning, etc. (Follow-up distance) The distance is determined by the size of the vehicle in front in the image, for example, the size of the license plate, and the tread (tire). It is best to calculate the relative speed from information such as the size of the gap between vehicles and the width of the taillights. It is best to calculate it from the rate of change.

[0104] The image analysis function should include a function to detect the color of the traffic light ahead. The processing means is: When the vehicle indicator light is red, it is advisable to notify the driver to stop.

[0105] The image analysis function should include a function to recognize road signs ahead. The processing means is road It is a good idea to alert drivers to the information indicated by the signs. For example, road signs may indicate speed limits. If so, the processing means obtains the speed limit of the current location by character recognition, and the speed limit It is advisable to notify the driver of the degree. If the road sign indicates a no-entry zone, the means of handling it is: If a vehicle attempts to enter a road where entry is prohibited, it would be good to issue a warning to notify the driver. For example If a vehicle is attempting to drive in violation of traffic rules indicated by road signs, or if the vehicle is crossing roads If the system detects that the vehicle is driving in violation of traffic rules, it will notify the driver of the rule violation. It is good to know this. This can help prevent or reduce traffic rule violations. For example, the processing means recognizes the text of the road sign, thereby determining the current section of travel. If a sign indicating that an automatic speed enforcement camera is installed is detected, the driver will be warned for exceeding the speed limit. It would be good to notify them to be careful.

[0106] If the image analysis function includes a feature to detect congestion in lanes adjacent to the vehicle's lane, Good. For example, whether or not there is congestion can be determined by the average speed of vehicles in the adjacent lanes, and It is best to make the determination based on the distance between vehicles. For example, if the speed of the vehicles in the adjacent lane is a predetermined base When the value is below the standard value, and the distance between vehicles is below the specified standard value and a specified number of vehicles are lined up, It is good to determine that a traffic jam is occurring. As a corresponding process to the analysis result, in the lane in which the vehicle is traveling... When congestion is detected in an adjacent lane, the vehicle will suddenly emerge from between the congested vehicles. It would be good to implement a process to alert the driver to pay attention to the following: The lane should include the oncoming lane and the adjacent lane traveling in the same direction as your vehicle. In particular, Places where people gather, such as shopping malls, theme parks, train stations, and gas stations. In areas where vehicles are entering and exiting, there is a high probability that the adjacent lane going in the same direction will also become congested. In such cases, it is effective to detect congestion in the adjacent lane traveling in the same direction as your own vehicle. Yes, this can reduce the occurrence of accidents caused by vehicles suddenly running out into the road. When the vehicle speed is below a specified speed, for example, below 30 km / h, a warning sound will be emitted to indicate the vehicle may suddenly appear. It's best to prevent it from being emitted. This will help avoid the generation of unnecessary warning sounds. In addition, it can detect situations with a high risk of sudden appearances from images taken in front of the vehicle and issue a warning. For example, if there are parked vehicles in a row along the side of the road, parking meters are installed in the area. - If there are parked vehicles in the parking lot, it is a good idea to warn people about the risk of suddenly running out into the road.

[0107] For example, car navigation devices may include traffic congestion probe information, general road and highway driving information, and surrounding area information. It is equipped with a function that alerts the driver to driving caution based on location information (such as residential areas and schools). There is a way to obtain traffic congestion probe information in places where it is not available. However, it is possible to issue warnings due to traffic congestion.

[0108] The image analysis function detects animals (dogs, cats, raccoons, wild boars, etc.) around the vehicle before it starts moving. It may be preferable to include a function for detecting animals or people. When the processing means detects an animal or a person around the vehicle, it may be good to inform the driver that there are animals or the like around the vehicle. This can suppress the occurrence of accidents immediately after starting.

[0109] The image analysis function may include a function for detecting that the vehicle is slowly moving forward from the flow of the surrounding scenery. When the processing means detects that the vehicle is slowly moving forward, it may issue an alarm. For example, when the vehicle has once stopped and then starts to move little by little due to the loosening of the force applied to the brake pedal (creep phenomenon), it can inform the driver that the vehicle is moving and there is a risk of collision. For example, when the vehicle speed is moving at a speed that allows forward movement due to the creep phenomenon, it may be determined that the vehicle is slowly moving forward. The speed of the flow of the surrounding scenery may be detected by detecting the optical flow.

[0110] The image analysis function may include a function for determining whether the vehicle in front is a large vehicle, such as a bus, a box-type truck with an aluminum loading platform, etc. When the processing means detects that the vehicle in front is a large vehicle and approaches a traffic signal, it may be good to notify the driver of the presence of the traffic signal. For example, it may be good to emit a voice such as "Attention to traffic signal". The presence or absence of the traffic signal may be registered in the processing means in advance as POI information. For example, when a large vehicle is traveling ahead, the traffic signal in the traveling direction becomes difficult to see. It is preferable to predict the presence of the traffic signal by looking at the traffic signal in the oncoming lane or the pedestrian traffic signal, or to keep a sufficient distance so that the traffic signal in the traveling direction can be easily seen. However, if such attention is neglected, the traffic signal in the traveling direction may be overlooked. ​​​​​There is a risk of this happening. Also, in rural areas, traffic lights on the opposite lane and pedestrian signals are not installed. There are also cases where this is not the case. When the vehicle in front is a large vehicle, by notifying it of the presence of a traffic light, This can reduce the risk of drivers overlooking traffic lights. It also directly indicates the presence of traffic lights. Instead of directly informing the driver, the processing method involves driving in a way that reduces the risk of missing traffic signals. It's a good idea to remind them to do so. For example, "Please maintain a safe following distance." It is good to emit a voice message indicating the distance to the intersection with traffic lights, and the safety of your vehicle. It is best to issue a warning at a stage longer than the stopping distance. The safe stopping distance of your vehicle depends on your vehicle speed. It varies depending on the weather (whether it's raining or not). The safe stopping distance depends on the vehicle's speed and the weather (whether it's raining or not). It is best to set an appropriate distance based on the following. For example, if the safe stopping distance in clear weather is 50m In this case, the processing method is as follows: the current weather is clear, the distance to the preceding large vehicle is less than 50m, - Outside of the urban area as defined by the navigation system, there is no traffic light within 100m in the direction of travel. When a safety device is installed at an intersection, it is a good idea to issue a warning to the public.

[0111] The image analysis function determines whether the road ahead is sloped or not, and if it is sloped... It would be good to include a function to detect the degree of incline. The processing means depends on the degree of the slope. This provides warnings about excessive speed, slowing down, and setting the device to hill climb mode. That's good.

[0112] The vehicle is one that handles cargo, such as a forklift or truck, and the processing means is From the captured image, the system recognizes the text, barcodes, and other markings displayed on the package, and the vehicle... It would be good to have a function to determine whether or not the cargo should be handled by the vehicle. It is advisable to register the information represented by the displayed sign in advance in the processing device. The processing device is responsible for the information that the vehicle has taken If it detects that you are attempting to load cargo that should not be handled, it will issue an alarm. It is best to act according to reason.

[0113] The processing method should have the function of detecting window regions in the image from the optical flow. Furthermore, the processing means determines from the optical flow whether a vehicle has entered the tunnel or if the vehicle is It would be good to have a function that determines whether the vehicle has exited the tunnel or is still traveling inside it. Another processing method involves optical flow, where the vehicle is traveling under an elevated structure (below parallel roads). It would be good to have a function to determine whether it is in or out. Furthermore, the processing means is from the optical flow, It would be beneficial to have a function that determines whether or not there are guardrails and whether or not caution is needed regarding drainage ditches.

[0114] (12) The processing means, as an image analysis function, detects other vehicles that appear in the image outside the vehicle. It has the function to recognize the characters written on both license plates and processes the results accordingly. The process involves recording the license plate information identified by character recognition and networking. The system should perform at least one of the processes that send data via the network.

[0115] Is the vehicle driving near a vehicle equipped with an in-vehicle device that has the processing functions of this system? Alternatively, it can identify the license plate information of other vehicles parked nearby. The method involves capturing the current location information and license plate of the vehicle equipped with this in-vehicle device. It is best to transmit date and time information in association with the information of the identified license plate. The license plate information obtained along with the report and date / time information is from the self-service station installed on the road. The information of the license plate obtained by the vehicle number automatic reading device (N system) and both become useful information for knowing the driving routes of many vehicles. Thus, by associating the vehicle number plate and the GPS information, a system that can be called a wide-area movement N system can be constructed.

[0116] It is advisable to construct a center system that collects and associates position information, date and time information, and license plate information through a network from the systems respectively installed in a plurality of vehicles. This center system may have a function of extracting the position information and date and time information of a vehicle having specific license plate information and specifying the driving route of the vehicle.

[0117] (13) As a function of the image analysis, the processing means has a function of determining whether there is an edge of the window glass in the window portion of the vehicle, and as a processing corresponding to the analysis result, when the edge of the window glass is detected in the window portion, it is advisable to adopt a system that executes a notification process.

[0118] The presence of the edge of the window glass in the window portion means that the window glass has been forgotten to be closed. The driver or the like can be alerted to the fact that the window glass has been forgotten to be closed by receiving the notification from this system. For the detection of the edge of the window glass, for example, it may be performed by detecting the discontinuity of the image due to the difference in refractive index between the window glass and the atmosphere. As the notification process, for example, processing such as wireless notification to the driver's mobile terminal or the like or notification by e-mail may be performed. In addition, it is advisable to perform the notification process by sounding a buzzer attached to the vehicle. A smoke film is pasted on the window of the vehicle on which the processing means is installed, and the processing means ​Based on the difference in brightness between the smoked area of ​​the glass and the image, it detects that the window is open. good.

[0119] (14) The processing means, as an image analysis function, determines whether or not the wiper is operating, or The system detects whether or not it is raining by checking for an increase or decrease in raindrops on the windshield. It has a function and, as a process corresponding to the analysis result, if it is determined that it is raining, it will process the rain Let's say the system executes a process to send information from a communication device to notify that something is happening. stomach.

[0120] By receiving information transmitted from this system, it is possible to determine if it is raining in the area where the vehicle is traveling. It is possible to know whether or not it is raining. The processing method is to determine whether or not it is raining from an external server, etc. It is recommended to notify the internal equipment and also notify the vehicle's current location information. The external equipment may be multiple By collecting information from vehicles, it is possible to create a map of areas where it is raining. Furthermore, the processing means notifies that it is raining and transmits the current time information. It is good to know this. This allows external devices to receive information about the temporal changes in areas where it is raining. You can obtain the information.

[0121] As an external device, it would be good to use, for example, a server for an organization that provides weather-related information. The method is to capture the image when the wiper is not visible, for example, when the wiper is down. It is recommended to upload the image to the server. From the image uploaded to the server, rainfall Information such as rainfall can be obtained. The server collects rainfall from the vehicle. Such data can be used as big data in various application fields. For example, It can be used to predict localized torrential downpours, etc. The processing method involves applying raindrops to the windshield. If there are some particles attached, but the number of particles does not increase or decrease, it is safe to assume that the rain has stopped. The processing method should be to notify the server that the rain has stopped once it is determined that the rain has stopped. B allows you to obtain more detailed information about rainfall.

[0122] (15) The processing means, as an image analysis function, determines the state of the sky captured in the image It has a function to determine the weather, and as a process corresponding to the analysis results, it communicates weather-related information via a communication device. It would be best to have a system that performs the process of sending data from a location.

[0123] Weather information collected from multiple vehicles is considered big data related to weather information. It can be put to good use. For example, from the color of the sky and the presence or absence of clouds in the image, it can determine whether the weather is clear or sunny. It is also good to determine the weather, such as cloudy. Furthermore, you can determine the sunrise and west from the height of the sun and the brightness of the sky. It would be good to detect the intensity of sunlight and transmit information about strong morning and afternoon sun. Weather information and It would be good to transmit the vehicle's current location information. An external device that collects this information will then use it. Information that the setting sun is strong, and the current location of the vehicle, will be shared with other vehicles heading towards that location. This information should be provided to drivers. Drivers who receive this information should be careful of the rising or setting sun. You can realize that it is necessary.

[0124] (16) The processing means has a function to detect sound information from a microphone mounted on the vehicle. The sound information detection results and the analysis results of the image analysis function are correlated, and the correlation results are used. It would be best to have a system that performs the appropriate processing accordingly.

[0125] Compared to performing processing based solely on the detection results of sound information or the analysis results of images, By linking the two, it becomes possible to perform more appropriate processing.

[0126] The processing means, as an image analysis function, analyzes the state of the display unit of a device with a display unit installed inside the vehicle. It has a function to detect changes in state, and as a processing step corresponding to the analysis results, it uses the detected sound information and When the state of the display unit changes in sync with the detected sound information and the state of the display unit has changed. It is recommended to perform a process that associates and stores the device.

[0127] The processing means detects sound information from the microphone and correlates the sound information with the change in the state of the display unit. Based on the information attached and stored, the device that emitted the sound can be identified. For example, the processing means has a voice recognition function, and the sound information from the microphone is emitted from a specific device. It determines whether or not it was emitted from a specific device, and if so, it performs voice recognition. It's best to cancel the function. This will prevent unnecessary speech recognition processing from being performed. Yes, it is possible. Furthermore, the processing method involves the audio information from the microphone being emitted from a specific device. If this is determined, then the area of ​​the in-car image containing the sound-generating device can be distinguished from other areas. It would be good to have a function to add information and record it. For example, the area in which the device that produced the sound is shown. It is best to record an image of a rectangle or a closed curve of any shape overlaid on the original image. Alternatively, sound It is recommended to overlay a semi-transparent geometric pattern onto the area where the device generating the phenomenon is visible during recording.

[0128] The processing means, as an image analysis function, identifies emergency vehicles in the image, such as ambulances, fire trucks, and police cars. It would be good to have a function to determine whether or not things like the above are captured in the image. The processing means will receive emergency information from the microphone. When the sound of a vehicle siren is detected and an emergency vehicle is detected in the image, the two are associated and the system is then used to... It is good to determine if there is an emergency vehicle in the vicinity of the vehicle. The processing method is to determine if there is an emergency vehicle. It would be good to notify the driver with a message urging them to pay attention.

[0129] (17) There are multiple cameras installed inside the vehicle, The processing means, as part of the image analysis function, detects the positions of other cameras captured in the image. It has the function of knowing and, as a process corresponding to the analysis results, identifies the positional relationship of multiple cameras. It would be best to use it as a system that executes processing.

[0130] It is possible to obtain images of a wide area inside the car that is difficult to capture with a single camera. Once the relative positions of the cameras are determined, the relative positions of the images captured by each camera can be determined. Cut.

[0131] The processing means identifies the models of other cameras that are visible in the image and determines the field of view from the models. It would be good to have the following function: The processing means has the function of combining images captured by multiple cameras. This is good. By combining images, it is more intuitive compared to displaying individual images from the camera. It allows you to accurately assess the situation inside the vehicle.

[0132] For example, it would be good to implement at least some of the functions of the processing means using a viewer such as a personal computer. If you display a camera icon or other indicator within the image to show the camera's location, Good. The camera's position inside the vehicle can be determined from the images captured by the camera (itself). This is difficult. From images of other cameras that capture the camera in question, it is possible to determine the interior of the vehicle from the camera in question. It would be good to detect the position in that location. For example, images captured by three cameras could be combined and displayed. When doing so, it is a good idea to display the positions of the three cameras in the combined image using icons or similar symbols. It would be good to use different icons depending on the camera model or field of view.

[0133] The processing means displays the images captured by each camera, and also displays the camera position within the image. It should have a function to display the camera's position and its field of view (the area the camera is capturing). For example, depending on the functions of a 360-degree full-spherical camera, a 360-degree hemispherical camera, a wide-angle camera, etc. Then, the field of view of each camera should be displayed within the image. Users viewing the image will be able to see the field of view of each camera. It allows you to intuitively understand whether a region is being imaged.

[0134] (18) The camera has an image circle smaller than the length of the diagonal of the imaging plane. It is a 60-degree camera, The processing means, as a function of image analysis, calculates basic patterns for image feature quantities. The system has the function of deforming the circle according to its position within the image circle. That's good.

[0135] Without correcting the distortion of the image within the image circle, the basic pattern is used to define the characteristics of the image. It is good to calculate the quantity. This can reduce the load on the processing equipment. Basic pattern The term "angle" is defined for images obtained by capturing images using a standard lens (a typical wide-angle lens). The projection methods used to obtain the image circle include, for example, orthogonal projection, equidistant projection, and stereographic projection. There are projection methods such as equisolid angle projection, and the basic pattern is modified according to these projection methods. It is good to do this. To obtain an image circle, it is good to use a circular fisheye lens. Camera The imaging surface is rectangular, and the diameter of the image circle should be shorter than the longer side, and the diameter of the shorter side is It is even better if the sides are equal or shorter than the shorter side.

[0136] For example, as a basic pattern, you can prepare a template pattern of the image you want to detect in advance. It is a good idea to do so. In this case, change the template pattern according to its position within the image circle. It is good to shape it. In addition, for example, when performing image recognition using Har-like features, Each of the multiple rectangular patterns used to find the like features is assigned a position within the image circle. It's best to deform it.

[0137] When performing image recognition, the area within the image circle that contains the specific object to be detected... It is best to limit the crowded areas and perform image analysis only within those limited areas. In this case, the functions of the in-vehicle unit of this system can be implemented using a dashcam. Generally, Because the recommended installation location of the recorder is predetermined, within the image circle, the driver The areas that are reflected, the areas where traffic lights are reflected, the areas where people in the back seat are reflected, etc., are all pretty much fixed. Therefore, depending on the person or object to be detected, an image resolution is used to detect that person or object. It is best to limit the area of ​​analysis to the area in which the person or object is captured. For example, the area in front of a vehicle. When analyzing the image, it is best to analyze the area where the windshield is reflected.

[0138] In addition, for example, converting a specific region of an image circle into a planar image (parallel projection image). Furthermore, it is advisable to perform matching on the converted planar image using the usual basic patterns.

[0139] Furthermore, the images within the image circle are mapped to a sphere using a predetermined transformation function, and the sphere Zoom in on a portion of the image mapped to a surface and perform image analysis as a planar image, ignoring distortion. It is recommended to do this. If you want to detect a specific object, such as a road sign, then the area in which that specific object is captured should be... It is best to zoom in on the area and perform image analysis. If you want to detect road signs, zoom in on the image ahead. It is best to zoom in on the area where the road sign is visible and perform image analysis.

[0140] (19) The processing means, as an image analysis function, from the viewpoint of protecting privacy It has a function to identify areas of an image that should not be made public as privacy-protected areas. As a process corresponding to the analysis results, the appearance of the person captured in the aforementioned privacy protection area is determined. A process for generating images that cannot be identified by the aforementioned method from images captured by the camera. It would be good to have a system that executes this.

[0141] By making it impossible to identify a person, the privacy of that person is protected. This is possible. Areas that should not be made public include, for example, areas where the interior of the car is visible. However, when acquiring images using a 360-degree camera, the interior of the car will be captured in the image, There may be times when you don't want to keep the image. In such cases, for example, you can black out the interior area of ​​the car. Alternatively, you can blur the interior of the vehicle while recording the exterior images as they are. One method is to perform image analysis to distinguish between the area showing the interior of the car and the area showing the exterior of the car. It is helpful to distinguish between different regions. For example, optical flow is occurring at a speed equivalent to the vehicle speed. The area should be outside the vehicle. The processing method is that the user pre-sets the privacy protection area. It would be good to have a function that allows for this.

[0142] A normal mode that records images that can identify individuals within a privacy-protected area, Images in which individuals within the privacy-protected area cannot be identified by their appearance. It would be good to allow users to choose between a privacy mode that records data and a mode that records data in the car. Switch between normal mode and privacy mode depending on the purpose and usage situation. It is possible.

[0143] The privacy area should be defined as the area in which a person is pictured. For example, the processing method is image By performing analysis, a person is detected, and the area enclosed by the outline of the detected person is protected from privacy. It would be good to designate this as the privacy area. Alternatively, designate only the face portion of the detected person as the privacy area. It would be good to do this. At this time, it would be good to display the person as a silhouette. Alternatively, the outline of the person It is best to display the image, and show the image when there is no person inside the outline. Objects appear transparent. This is useful for displaying the outlines of people, or, for example, the edges of moving objects. .

[0144] The processing method involves displaying an avatar in place of the person in the area where the person is visible. The avatar should resemble the original person. For example, the gender, body shape, facial features, etc. Accordingly, it is advisable to display an avatar that reflects those characteristics. The relationship between the person and the avatar should be established beforehand. The processing method involves identifying the person in the image and associating it with that identified person. It's a good idea to display the avatars that have been set up. It's best to create avatars for each person in advance. Yes. It's best to move the avatar in accordance with the character's movements. For example, when the character raises their hand, It's a good idea to have the avatar raise its hand, and when the person turns to the side, have the avatar turn to the side as well. It would be good to change the avatar's facial expression in accordance with the person's facial expression. For example, the processing method could be: When facial expressions such as laughing, sadness, or anger are detected, the avatar's facial expression is changed. It is best to change the expressions to smile, sad, and angry, respectively. Processing method It would be good to read the person's emotions and change the avatar's color according to those emotions. For example For example, when it is detected that a person is angry, the avatar's color will be changed to a color that represents anger. It would be good to change it to red.

[0145] The processing means, before recording the image, processes the image captured by the camera to remove privacy-protected areas The generated image makes it impossible to identify the person in the area based on their appearance. It is a good idea to record the images you capture. Also, when playing back images captured by the camera, privacy is important. - Convert the image so that the person captured in the protected area cannot be identified by their appearance. It would be good to display it.

[0146] (20) The processing means, as a process corresponding to the analysis result, converts the image analysis result into text It would be ideal to have a system that performs the process of digitizing or coding and recording the data.

[0147] By converting the recorded image into text data or encoding it, the desired image can be obtained from the recorded image. It becomes easy to find. For example, when a person is detected in an image, the text data The text data for detecting a red light in an image is "person detection". It is good to generate ". The generated text data is the source of that text data generation It is best to record the data in association with the image. Instead of text data, use the image analysis results and code. It is recommended to associate the data points and encode (convert to binary data) the image analysis results.

[0148] (21) The processing means, as an image analysis function, processes the blacked-out areas in the image. It has a function to detect this, and as a processing step corresponding to the analysis result, it processes the areas of the image that are blacked out. The system should perform a process to restore brightness and display the image on the screen.

[0149] Users viewing the image will find it easier to recognize objects and people within the blacked-out areas. In particular, all When acquiring images using a spherical camera, the image should contain both a bright exterior view and a dark interior view. Because images are captured simultaneously, areas where the interior of the car is visible tend to appear as blacked-out black areas. The process of detecting an area, brightening it, and displaying it uses a 360-degree camera to acquire images. This is particularly effective in the following cases. Brightening restoration processes include, for example, gamma correction, color correction, and black and white. It is recommended to convert the image to a digital format and enhance the contrast.

[0150] (22) The processing means allows the user to select from the imaging range captured by the camera, and then selects a number of sections. It has a function to divide into sections and a function to weight each section, and the brightness of each section Based on the weight assigned to the area, imaging conditions are determined, and the determined imaging conditions are used. The system should have a function to allow the aforementioned camera to take images.

[0151] Users can increase the weight of areas they want to see within the image and decrease the weight of areas they don't need to see. This allows the user to acquire images under imaging conditions suitable for the area they want to view. Dividing the imaging area into arbitrary shapes according to the shape of the areas you want to view and the areas that do not need to be viewed. This is possible. For example, regardless of the content of the image, the section is uniformly divided into a square grid. Compared to simply assigning weights, this allows for more appropriate weighting for the user. .

[0152] In particular, when imaging with a 360-degree camera, the vehicle type is more accurately represented compared to when using a regular camera. Each time, the difference in the position of the vehicle within the imaging range becomes larger. Therefore, processing is performed in advance. A general weighting table set in the firmware of the device, and the actual expected imaging range. The relationship with the partitions is likely to be inconsistent. There is no such feature, but there is a function that allows you to determine the weights using PC browser software, etc. The effect is particularly noticeable when using a 3D camera. For example, without weighting. In this state, the images captured by the camera are analyzed using AI software on a PC, and which of the imaging range... We estimate what is depicted in each part, for example, assigning a weight of "5" to the road part and a weight of "5" to the face part. A weight of "3" is assigned to the instrument panel area, a weight of "1" is assigned to the area outside the circular image. It is best to assign a weight of "0" to it.

[0153] The processing means may determine imaging conditions such as gain and exposure time. By placing the backs of the two celestial cameras facing each other, a full 360-degree camera is constructed, and the imaging rays of the two hemispherical cameras are captured. It is best to make the items identical. This will allow you to combine images taken by two hemispherical cameras. This has the effect of making the boundary between the two images less distinct.

[0154] The processing means, when having the user divide the area, encloses a portion of the imaging range with a closed curve, and close It is best to consider the area enclosed by the curve as a single section. The user is inside the closed curve If the region is divided into two by a curve, each of the divided regions can be considered a single section. For each section of the level, it would be best to have the user input a numerical weight. The numerical values ​​to be input are as follows: It's best to allow users to select from a list, such as a dropdown menu or a pop-up window.

[0155] The processing means replaces, or replaces, a function that allows users to input weights numerically. In addition, it would be good to have a function to input the type of plot. The type of plot should be assigned a weight in advance. When the type of section is entered, the processing means is associated with that type. It is good to associate the weights with the respective compartments. Examples of compartment types include driver, meter panel, etc. It would be good to categorize them as follows: rear, car navigation screen, windshield, rear seats, passenger seat, etc. The processing mechanism should ideally have a function that allows the user to change the correspondence between the type of section and its weight. It would be a good idea to allow users to register a list of image types in advance.

[0156] The processing means provides a function that allows the user to weight each section, for example, within the imaging range. It would be good to have a function that allows users to specify the area they want to view, thereby setting up partitions. It is recommended to make the weight of the area you want to view greater than the weight of the other areas. This feature is for the user In the function that allows users to input the type of section, the sections are divided into sections the user wants to view and other sections. This is an example of classifying things into two categories.

[0157] The processing method should ideally involve an interactive (wizard-style) approach, allowing the user to define the imaging area. For example, the processing method could be, "Surround the unnecessary area," or "Paint the unnecessary area." It displays the message "Please crush it," and then fills in the area surrounded by the user. It is good to recognize the area that has been crushed as "unnecessary area". In addition, the processing means is " "Please fill in the area of ​​the train," and "Please fill in the road area." The message is output, and the filled area is a section with a driver type, or a road type. It is best to recognize it as a separate section. The operation of enclosing with a closed curve, or filling it, For example, it is a good idea to use a pointing device such as a touch panel or mouse. .

[0158] The processing means has a function of dividing the imaging range into multiple sections based on the captured image, artificial It would be ideal if this could be achieved using AI. For example, from an image, it could analyze the interior and exterior of the car, the seats, the dashboard, etc. The system automatically identifies components such as the driver, instrument panel, etc., and sets the imaging range for each automatically identified component. It is best to categorize them. The processing method displays the results of the categorization on the display screen and checks whether this categorization is acceptable. It's best to let the user make the judgment. If the user determines that the classification made by the AI ​​is incorrect, then... The management mechanism should have a function that allows the user to modify the classification. The classification and the corresponding classification If the user determines that the type is incorrect, the processing method will be a pull-down or pop-up. It's best to have the user select the correct type from a list of options provided.

[0159] (23) The processing means allows the user to select from the imaging range captured by the camera, and then selects a number of sections. It has a function to divide into sections and a function to weight each section, and corresponds to the analysis results. As a process, the blocks with relatively small weights during compression are processed, and the blocks with relatively large weights are processed. It would be best to implement a system that compresses images by making the image size larger than the block size used for compression.

[0160] Users can increase the weight of high-priority parcels and decrease the weight of low-priority parcels. This allows for increased compression of less important sections. As a result, the entire image can be compressed uniformly. Compared to compression, this method maintains high image quality for high-priority sections of the image data. The capacity can be reduced. For example, in a heavy section, the processing means can use 8x8 drawings. Each element is treated as one block, and in lighter weighted sections, 16x16 pixels are treated as one block. It is recommended to compress the file after doing so.

[0161] The area outside the circular image formed on the camera's rectangular imaging plane contains the image of the lens barrel. This may appear in the image. The image outside the circular image is unnecessary, so this part is blacked out. This is a good idea. This will reduce the size of the image data, including pixels outside the circular image. Furthermore, raster scanning is performed on only the pixels inside the circular image, excluding the pixels outside the circular image. It is advisable to acquire image data in this way. This will reduce the size of the image data.

[0162] (24) The processing means has the function of detecting a following vehicle and the function of detecting a following vehicle as part of the image analysis function. It has a function to detect the distance to the next vehicle, and as a processing step corresponding to the analysis result, it detects the distance to the following vehicle. A system that detects aggressive driving by a following vehicle based on the distance would be ideal. .

[0163] The processing means performs appropriate processing according to the judgment result in response to aggressive driving. It becomes possible to take action. For example, even if the vehicle's speed is above a predetermined standard value, Furthermore, if the same following vehicle continues to exist within a predetermined judgment distance for a predetermined judgment time or longer. It is best to determine that the following vehicle is engaging in aggressive driving.

[0164] As a camera, it is recommended to use one that has a function to measure the distance to the object for each pixel. For example, as an image sensor for a camera, a back-illuminated time-of-flight distance image sensor is used. It is best to use Nsa.

[0165] When the processing means determines that a following vehicle is engaging in aggressive driving, it takes action according to the determination result. In principle, for example, notifying the driver, recording rear-view images, advising the driver, and traffic enforcement. It is advisable to take action such as reporting to an agency (e.g., the police). When reporting to a traffic enforcement agency... It is recommended to attach images of the vehicle and its driver engaging in aggressive driving. The processing method is image Image analysis revealed that the vehicle engaging in aggressive driving overtook the other vehicle and then moved in front of it. If this is detected, the driver will be notified that they may be subjected to harassment through sudden braking. That would be good.

[0166] The processing means detects people and vehicles around the vehicle and controls the vehicle according to the distance to the people and vehicles. It would be good to do this. For example, the processing method is the speed of your vehicle, the distance to the person in front, and the person in front of you Based on the determination of whether or not the vehicle is on the planned route, it is determined that there is a high probability of contact with a person. Once the vehicle is set, it is advisable to implement a control mechanism to stop it.

[0167] (25) The function of the processing means described in any one of (1) to (24) above is computed It would be good to create a program to achieve this.

[0168] The inventions described in (1) to (24) above can be combined in any way. For example If, then, all or part of the configuration of the invention shown in (1) is at least one of the inventions from (2) onward A configuration may be made that includes at least some of the components of (1). In particular, the invention shown in (1) may be modified to include (2 It is preferable that the invention be an invention that incorporates at least a part of the configuration of at least one of the inventions described above. From the inventions shown in (1) to (24), any configuration is extracted and the extracted configuration is combined. It is permissible to do so. The applicant of this application intends to obtain rights to the invention including these elements. do [Effects of the Invention]

[0169] Not only does it record images captured by cameras mounted on the vehicle, but it also uses those images to... It can be used more effectively.

[0170] The effects of the present invention are not limited thereto, but also include the effects of the components disclosed in this specification and the drawings. The effects produced by this are also disclosed, and the configuration that produces these effects is also disclosed in the divisional application. They intend to acquire rights through amendments, etc. For example, in this specification, they may write "can do ~". The parts marked with an asterisk clearly indicate the effect it will produce, and even if it does not say "can do ~", the effect will still be present. There is a section that indicates this. Furthermore, even without such a description, the effects that can be understood from this configuration are also present. exist. [Brief explanation of the drawing]

[0171] [Figure 1] Figure 1 shows the interior of a vehicle, including the windshield, dashboard, etc., with the drive recorder system according to the first embodiment installed in the vehicle. [Figure 2] Figure 2 is a block diagram of the drive recorder system according to the first embodiment. [Figure 3] Figure 3 is a block diagram of a modified system in which the processing unit's functions are implemented using two devices: an in-vehicle device and a server. [Figure 4] Figure 4 is a block diagram of a modified system in which the functions of the processing unit are realized by an in-vehicle device and external devices such as a personal computer. [Figure 5]Figure 5 is a diagram of the imaging surface illustrating the method of dividing the imaging range into multiple sections using the comparative example method. [Figure 6] Figure 6 is a diagram of the imaging surface illustrating a method for dividing the imaging range into multiple sections in the system according to the 22nd embodiment. [Modes for carrying out the invention]

[0172] [First Embodiment] The system according to the first embodiment of the present invention will be described with reference to Figures 1 and 2. The function of the system is achieved by a drive recorder mounted on the vehicle.

[0173] Figure 1 shows the interior of the vehicle with the drive recorder 1 installed on the windshield and dashcam. This is a diagram showing a driveboard, etc. The vehicle equipped with drive recorder 1 is a passenger car. The iBreCorder 1 is located at the top of the vehicle's windshield 2, near the center in the left-right direction. It will be installed on the passenger side, adjacent to the rear 3. Drive recorder 1 is attached with double-sided tape. It is attached and secured to the inner surface of the windshield 2 using mounting components such as the drive. Recorder 1 is connected to the vehicle's cigarette lighter socket 5 via power cable 4. When the accessory power is turned on, power is supplied from cigarette lighter socket 5 to dashcam 1. The drive recorder 1 includes a main unit 10 and a 360-degree spherical camera 11. Camera 11 is fixed in its relative position and orientation to the vehicle under normal operating conditions. .

[0174] Figure 2 is a block diagram of the drive recorder 1 as a system according to the first embodiment. Drive recorder 1 consists of a processing unit 20, a camera 11, a GPS receiver 12, and an SD card reader. Includes 13, an accelerometer 14, a speaker 15, a display 16, and control buttons 17. More precisely, the SD card reader 13 should be called an "SD card reader / writer." However, in this specification, it is simply referred to as "SD card reader".

[0175] The speaker 15 and the display 16 are output means for informing the user of various information. It functions as follows. The speaker 15, based on control by the processing unit 20, provides warnings via sound or voice. It emits reports and various information. The display 16 displays various information based on the control of the processing unit 20. The information is displayed as an image. Operation button 17 allows the user to perform various actions on drive recorder 1. It functions as an input means for giving commands.

[0176] Camera 11 is a 360-degree spherical camera that captures images of the vehicle in the front-to-back, up-and-down, and left-to-right directions. The entire area is captured as a video. For this purpose, camera 11 captures the windshield, rear window, and In addition to capturing images of the outside of the vehicle through the side windows, the driver's seat, passenger seat, rear seats, and interior The interior of the vehicle, including the storage panel and center console, is also imaged by camera 11. The captured video is taken into the processing unit 20 as image data. The processing unit 20 processes the camera 1 Based on the image data input from 1, the SD card connected to the SD card reader 13 Record the image.

[0177] The GPS receiver 12, based on instructions from the processing unit 20, provides the vehicle's current location information. Detect. Location information is determined based on signals from GPS satellites, including the time, the vehicle's speed, and This includes information such as longitude, latitude, and altitude. The processing unit 20 records the history of this location information. To carry out the right thing.

[0178] The SD card reader 13 is inserted into the SD card slot based on the control of the processing unit 20. Read data from an SD card or write data to an SD card.

[0179] The acceleration sensor 14 measures acceleration and tilt in each of the three axes (x-axis, y-axis, z-axis). A 3-axis type sensor is used to detect the accelerometer 14. The accelerometer 14 constantly processes the detected values. The output is sent to unit 20. Processing unit 20 acquires 3-axis acceleration information, for example, every 10ms.

[0180] The processing unit 20 includes a well-known CPU 20a, memory such as ROM 20b and RAM 20c, and a ty It has a motor 20d and other peripheral circuits. Various programs are stored in the ROM 20b of the processing unit 20. These are stored. The processing unit 20 performs various functions by executing these programs. It will be realized. Various programs include an operating system (OS) and GPS information processing software. This includes programs, image analysis programs, etc.

[0181] The processing unit 20 executes a GPS information processing program, thereby receiving the GPS information received by the GPS receiver 12. The transmitted GPS information is saved to the SD card connected to the SD card reader 13. Unit 20 executes an image processing program to process the video captured by camera 11, time Image data linked to information and location information is continuously recorded in a designated recording area on the SD card. .

[0182] The processing unit 20 executes an image analysis program to analyze the image captured by the camera 11. The image analysis of the image is performed. Furthermore, the processing unit 20 executes various processes corresponding to the analysis results. .

[0183] By installing the system according to the first embodiment into a vehicle, the camera mounted on the vehicle It not only records the images captured, but also allows for more effective use of those images. When the images captured by camera 11 are converted into a video, by performing image analysis on the video, This makes it easy to determine the movement of the subject, the presence or absence of the subject before and after a certain point in time, and so on. .

[0184] Next, a modified example of the first embodiment will be described. In the first embodiment, the drive recorder 1 was installed in a passenger car, but in the modified example, other types It is attached to vehicles such as cargo trucks and special vehicles. Also, in the first embodiment, camera 1 Camera 1 captured images both inside and outside the vehicle, but in a modified example, camera 11 captured images both inside and outside the vehicle. Both sides are imaged. Regarding "inside the vehicle" and "outside the vehicle," they can be interpreted, for example, as follows: good.

[0185] When the interior space and exterior space are clearly separated by the vehicle body, It's best to refer to the interior as the "interior" and the exterior as the "exterior." For example, the passenger space. In the case of a van where the passenger space and cargo space are continuous, the continuous passenger space and cargo space It's better to call it the interior of the car. Even in the case of a sedan where the passenger space and the cargo space are separated, the passenger It is appropriate to refer to both the passenger space and the cargo space as the interior of the vehicle. Covered cargo with a sealed cargo bed. In automobiles, the passenger compartment and the enclosed cargo area should be referred to as the interior of the vehicle. In open cargo trucks (flatbed trucks) with a flatbed, the empty space on the cargo bed is used to hold the cargo. It is difficult to clearly distinguish the space between the compartment and the surrounding area, but it accommodates open cargo. The space inside the vehicle can also be called the interior. Furthermore, the passenger compartment of an open-top freight truck can also be called the interior. In some special vehicles such as forklifts, both the passenger compartment and the cargo compartment are open. The passenger compartment and cargo compartment are clearly separated from the surrounding space. There are times when this is difficult. Even in such cases, both the passenger space and the cargo space can be It would be better to call it the interior of the vehicle. For example, the space above the seats where the passengers sit, and the space used to lift and lower cargo. The space above the forks can be called the interior of the vehicle. In this way, as the vehicle moves, The space that can be considered to move along with the vehicle is called the interior of the vehicle, and the surrounding space is called the exterior of the vehicle. That's good.

[0186] In the first embodiment, a 360-degree full-spherical camera was used for camera 11, but a 360-degree hemispherical camera was also used. You may use a wide-angle camera, or you may use a regular wide-angle camera. If you use a wide-angle camera, multiple It would be beneficial to mount a camera on the vehicle to enable imaging of a wide area.

[0187] In the first embodiment, the function of recording images captured by the camera 11 was realized in the in-vehicle device. Alternatively, this may be implemented by a server that communicates with the in-vehicle device. In the first embodiment, the image captured by the camera The function to analyze the images was implemented in the in-vehicle device, but it was implemented on the server that communicates with the in-vehicle device. That's good too.

[0188] As described above, the processing unit 20 of this system may be composed of one in-vehicle device, or multiple The device may be configured as follows. As in the first embodiment, the function of the processing unit 20 is the drive recorder 1 The application program stored in the built-in microcomputer and memory This would be a good way to achieve this. In addition, for example, the functions of the processing unit 20 can be shared between the in-vehicle device and the server. This is achieved using two devices: the in-vehicle device records the images, and the server performs the image analysis. This is good. Also, the functions of the processing unit 20 can be realized by the in-vehicle device and external devices such as a personal computer, and the image recording It is preferable to have the in-vehicle device perform the recording, while external equipment handles the image analysis and display.

[0189] Figure 3 shows a modified example in which the processing unit's functions are implemented using two devices: an in-vehicle device and a server. This is a block diagram of the system. The in-vehicle drive recorder 1 includes a communication device 21. 30 includes a processing unit 31, a communication device 32, an input device 33, and a display 34. The communication device 21 of Ible Recorder 1 and the communication device 32 of Server 30 are both connected via a communication network. It is connected to device 40 and transmits and receives data with other devices via the communication network 40. The processing unit 20 of drive recorder 1 and the processing unit 31 of server 30 are connected via the communication network 40 They communicate data with each other via this.

[0190] The input device 33, when operated by the user, sends various commands and data to the processing unit 31. The input is entered. The processing unit 31 displays the processing results and various messages to the user on the display 34. Display sage.

[0191] Figure 4 shows a modified example in which the functions of the processing unit are realized by an in-vehicle device and an external device 50 such as a personal computer. This is a block diagram of the system. External device 50 is the processing unit 51, SD card reader 52 , including input device 53 and display 54. SD card reader of drive recorder 1 The SD card 60 on which the image data is recorded in 13 is connected to the SD card reader 52 of the external device 50. By attaching it, the processing unit 51 of the external device 50 can access the image recorded on the SD card 60. This will allow access to the image data.

[0192] The processing unit 51 records data on the SD card 60 based on the command input from the input device 53. The processing unit 51 reads the recorded image data and performs image analysis or image processing. The analysis results or processing results are displayed on the display 54.

[0193] Next, we will describe the second to twentieth embodiments. The following describes the system according to the first embodiment. For components common to the above, redundant explanations will be omitted. Also, for the various embodiments described below... The system, as an example, is one of the block diagrams shown in Figures 2, 3, and 4. It will be realized in this way.

[0194] [Second Example] Next, a second embodiment will be described. In the second embodiment, the system's processing unit performs image analysis One of its functions is to determine whether the movement of a person in an image corresponds to a specific movement that has been pre-registered. It has the function of analyzing the results. When a person's movement corresponds to a specific movement, the processing unit analyzes the results. The corresponding process involves recording images of a person's specific movements, and capturing images of specific movements. Sending images to server 30 (Figure 3), reporting when a person makes a specific movement, and issuing an alarm. Execute at least one of the output processes.

[0195] The recorded images, or images sent to the server, indicate that the person was performing a specific movement. It can be used as evidence. By reporting that a person has made a specific move. It can alert outsiders that there is a person making specific movements. As a result, a person performing a specific action will realize that an alarm has been issued against them. This motivates them to stop a particular movement.

[0196] Next, I will explain an example of a "person who performs a specific action." Let's consider a person performing a specific action, for example, a vehicle driver. This refers to actions other than driving by the driver. For example, looking at a mobile device such as a smartphone, or eating. This includes actions such as eating, looking away or distracted, and napping. For example, whether or not someone is looking away. This is determined based on the detection results of the driver's eye position. The recorded images are based on the driver's awareness. It can be used for improvement, accident cause analysis, etc. The processing unit is used when the driver uses a smartphone, etc. If the system detects that a mobile device is being used, it will warn the driver not to carry a mobile device while driving. An alarm will be issued. If the driver notices the alarm and stops a specific movement, the danger will be avoided in advance. This is possible. The server 30 to which the data is sent may be, for example, the server of the vehicle administrator, or This is a police server. Based on the transmitted data, the vehicle administrator can monitor the driver's driving behavior. The police can evaluate the driver's driving behavior based on the submitted images. This allows for accident cause analysis and other related activities.

[0197] A person performing a specific action could be, for example, a suspicious person outside the car. A person performing a specific action could be, for example, inside the car. Actions that attempt to illegally enter, for example, actions such as trying to break a glass window with a hammer, This includes actions such as peering into the vehicle. By issuing an alarm, it will deter suspicious individuals attempting to illegally enter the vehicle. This can deter intruders from entering the vehicle. Recorded images, or on server 30 The transmitted images are used as useful information for apprehending thieves after a car break-in crime has occurred. It can be used to notify the vehicle owner or driver when a person makes a specific movement. This allows the owner or driver to immediately return to the vehicle, thus preventing theft. It is possible. Furthermore, drivers and owners who receive notification can also report it to the police. This function can be used for crime prevention.

[0198] A person performing specific actions, for example, a suspicious person inside a car. Various specific actions. Car break-in methods, for example, illegal entry into a parked car, suspicious activity inside a parked car, the car This involves actions such as removing internal devices. By issuing an alarm, the thief will panic and flee. This has the effect of reducing the damage from car break-ins. Recorded images or service Images sent to Ba30 are useful information for the arrest of thieves after a car break-in crime has occurred. It can be used as a report. It can inform the vehicle owner or driver that a person has made a specific movement. By notifying the police, the owner or driver can immediately return to the vehicle or report the incident to the police. This makes it possible to reduce the damage caused by car break-ins. An example of reporting to an external party is: Examples include sending an email with an attached image of the suspect. The recipient of the email is: For example, this includes vehicle managers, drivers, owners, and the police.

[0199] A person performing a specific action could be, for example, a passenger on a commercial vehicle carrying an unspecified number of people. Specific actions include various robbery techniques, such as brandishing a knife or drawing a handgun. Actions such as committing violence. The processing unit executes a process to issue an alarm, If a thief is deterred from committing a crime, the damage can be prevented. Recorded images or... Images sent to the robber can be used as useful information for the arrest of robbers. By reporting to the vehicle manager that a person has made a specific movement, the manager can detect a robbery. If you notice something, you can report it to the police. Also, the processing unit uses its communication function to detect robbery cases. It has a function to report the occurrence of such an event to the police.

[0200] A person performing a specific movement could be, for example, an infant or toddler in a car. For example, consider the movement when waking up from sleep. Image analysis is used to detect when an infant wakes up. The processing unit will then notify the parents via email or other means. Sometimes you want to leave the engine running while you go shopping or run errands. At this time, forcibly waking an infant will make them fussy, so let them sleep. There is a request to keep this in place. However, parents must be careful not to let their infants or toddlers see the vehicle when they are away from it. I'm worried they might be awake. This system will protect infants once they wake up. An email will be sent to the person's mobile device, allowing the parent to immediately rush to the vehicle and check on the infant. It can help with waking up.

[0201] A person performing a specific movement is, for example, a person outside the vehicle who requires care or rescue. Specific movement For example, these actions could be specific to individuals requiring care, or actions that indicate a need for assistance. Examples of the unique movements of people requiring assistance include, for example, walking with a cane, a bent back, and an unsteady gait. Examples of actions that indicate a situation requiring rescue include falling and foam coming out of the mouth. Actions such as spraying something and then falling over are examples of such situations.

[0202] The processing unit has a function to detect people performing specific movements, using images from the front of the vehicle and the rear of the vehicle. A function to select at least one image from the images of the front and the interior of the car, and the selected image It has the function to detect the person in question from the image. Furthermore, the processing unit uses at least one image. It has a function to allow the user to specify which images are unnecessary or of low importance. Since the analysis will no longer be performed, there will be no need to give the processing unit excessive processing power.

[0203] The processing unit has a function to detect a person performing a specific movement, and a function to detect driver drowsiness. It has, for example, the processing unit measures the pulse rate from the degree of change in the redness of the driver's skin and the pulse rate Based on the fluctuations, the system determines whether the driver is drowsy. Furthermore, the processing unit uses a thermal camera to determine the presence or absence of drowsiness. This device improves the accuracy of drowsiness detection by integrating body temperature measurement, microwave pulse measurement, and other methods. It has the ability to lower the air conditioner's set temperature when the system's processing unit detects drowsiness. Alternatively, the system can execute a process that issues a warning via voice or sound.

[0204] [Third Embodiment] Next, a third embodiment will be described. The processing unit of the system according to the third embodiment is image analysis One of its functions is to determine whether or not a person in an image possesses certain pre-registered characteristics. It has the capability to perform processing corresponding to the analysis results, and processes that are specific to the characteristics of the people in the image. Execute.

[0205] The system according to the third embodiment performs appropriate processing according to the characteristics of the person captured in the image. It is possible.

[0206] The processing unit has a function to determine whether or not a person in an image has specific features, such as a facial recognition device. It has the ability. The processing unit performs facial recognition of the person sitting in the driver's seat, and if the person sitting in the driver's seat is If you are not a pre-registered person, you will not be able to access the vehicle or start the engine. For example, the processing unit controls the engine via an in-vehicle network, such as CAN. A signal is sent to the engine control unit (ECU) to prohibit engine starting. Therefore, it is necessary to register authorized drivers in advance and prohibit unregistered individuals from driving the vehicle. It can be stopped. As a result, the vehicle's security function can be enhanced. Also, children It is also effective as a measure against vandalism.

[0207] According to a modified example of the third embodiment, the processing unit prevents the accelerator from being pressed, and the side brake -Prevents the parking brake from being released when the shift lever is in the drive position. Prevent entry, sound an alarm, and send an email with an image to the registered email address. It has the functionality to perform at least one of the following processes: sending data, etc.

[0208] According to another modification of the third embodiment, the processing unit performs facial recognition of a person outside the vehicle, and the person outside the vehicle If the person is a pre-registered individual, the vehicle has a function to control the vehicle and unlock the doors. For example, if the vehicle is to be provided to a car-sharing service, Registered members of the service are pre-registered in the processing unit. Registered members approach the vehicle and then... By moving your face within the field of view of both cameras, you can also avoid having a key card or similar item in your possession. However, it will become possible to use the vehicles. This will allow the vehicles of the car-sharing service to be used. It will become easier to use.

[0209] According to yet another modification of the third embodiment, the vehicle is a commercial vehicle used by an unspecified number of people. The processing unit has a function that uses facial recognition to determine whether or not a passenger is a wanted criminal. They possess it. The facial photographs of wanted criminals are accessed from the police database via LTE, Wi-Fi, etc. The data is transmitted to the processing unit of this system via communication. The processing unit registers the received facial photograph. It has the function of controlling the vehicle if the processing unit determines that the passenger is a wanted criminal. Notifications to the person concerned, notifications to the police, etc., are made using the communication device 21 (Figure 3) and the communication network (Figure 3). This is done through a third party. Furthermore, the processing unit has a function to notify the driver in a way that passengers will not notice. Vehicle managers, police, drivers, etc., should be aware that a wanted criminal has boarded the vehicle and act quickly. We can take prompt and appropriate action.

[0210] For example, a taxi is a confined space with only the driver and passenger inside, so the driver's safety... There is a high demand for this system in terms of securing criminals. The processing unit detects when a criminal gets into a taxi. When detected, the system notifies the taxi company and the police of the presence of a criminal. This means that the crime is committed not before the criminal actually commits the crime inside the vehicle, but the moment the criminal gets into the vehicle. It can detect when a criminal is on board. Therefore, it becomes possible to prevent crimes from occurring in the first place. .

[0211] According to yet another modification of the third embodiment, the processing unit determines that a person in the image has certain features. As a function to determine whether or not a person is present, the appearance of a person in an image is compared to a specific appearance that has been registered in advance. It has a function to compare and determine whether the two are the same or not (hereinafter referred to as the appearance authentication function). As a fixed appearance, the appearance specific to a wanderer, the appearance specific to a criminal, etc., are pre-registered in the processing unit. Examples of appearances specific to criminals include, for example, carrying a knife on the street or carrying dangerous objects. These include the fact that the processing unit uses images from infrared cameras, ultraviolet cameras, etc. It combines images from a regular camera with a function to detect people in possession of illegal drugs. The processing unit, upon detecting a person with a specific appearance in the image, communicates with the communication device 21 (Figure 3) and The police and vehicle managers are notified via the communication network 40 (Figure 3).

[0212] According to yet another modification of the third embodiment, the processing unit is used to track the actions of a specific missing person, wanderer, etc. It has a function to individually register the appearance of a person at the time they went missing as search information. For example, male or female. Distinction, height, type of clothing, color of clothing, whether or not a hat is worn, whether or not a cane is used, whether or not the back is bent. It has a function to register individual search information, such as whether or not it is present. Furthermore, the processing unit wanders around It has a function to register the area where the person is thought to be located (the area to be searched). The processing unit is the area to be searched When a vehicle is located in this position, the system performs a process to detect individuals that match the individual search information. Individual search information is collected from databases (servers) of the police, local support centers, etc. The stem's processing unit receives the data. In this system, a person in an image has certain characteristics. The processing unit of the in-vehicle device performs a function to determine whether or not the vehicle is present, and once the vehicle enters the search area, the server... Search information is downloaded from unit 30 (Figure 3) to the vehicle's processing unit.

[0213] According to yet another modification of the third embodiment, the processing unit determines that a person in the image has certain features. As a function to determine whether or not a person is wearing a seatbelt, the system checks if the person sitting in the front or back seat is wearing a seatbelt. This system has a function to determine whether or not it is being used (hereinafter referred to as the appearance authentication function). When the processing unit detects a person not wearing a seatbelt, it prompts them to fasten their seatbelt. Send a message.

[0214] Furthermore, the processing unit detects when a person in the rear seat is approaching a highway entrance without wearing a seatbelt. It has a function that issues a message prompting the driver to fasten their seatbelt when the vehicle is in a seat. The processing unit is located in the vehicle. If the distance from both current locations to the highway entrance falls below a certain distance, the highway will be closed. The system determines that the user is approaching the entrance. Another example is the processing unit checking for forgotten ETC card insertion. When a signal for this purpose is received, it is determined that the vehicle is approaching the highway entrance. As another example, The processing unit analyzes the image in front of the vehicle and, when it detects a highway entrance within the image, it detects the highway entrance. The system determines that it is approaching the entrance to the road.

[0215] [Fourth embodiment] Next, a fourth embodiment will be described. The processing unit of the system according to the fourth embodiment is for image analysis. It has a function to count the number of people in an image, and as a processing step corresponding to the analysis result, within the image Perform the process of recording or transmitting the number of people data.

[0216] The system according to the fourth embodiment does not require the use of a special device for counting people. It becomes possible to count the number of people.

[0217] According to a modified example of the fourth embodiment, a camera is attached to the bus, and the processing unit counts the number of people. The interior image of the vehicle is used as the source image. The processing unit counts the data on the display 16 (Figure 2). Display the results and inform the crew. The crew will inform the passengers (boarders) on board the vehicle. The number can be easily determined. This function can be used as a substitute for roll call during tours. To do so. When all passengers should disembark, if the passenger count does not result in 0, the crew This allows you to know that there are people trapped inside the vehicle. For example, like a baby. People who cannot express their own will, people with disabilities who cannot get off on their own, etc. It can detect students staying after school.

[0218] According to another modification of the fourth embodiment, the vehicle on which the camera is attached has a predetermined number of passengers. These are types of vehicles, such as tour buses and long-distance express buses, and the processing unit of this system It has a function to register a predetermined number of passengers. The processing unit processes the counting result of the number of people and the registration The system compares the number of personnel with the number of people present and, if there is a surplus or shortage, warns the crew about the discrepancy. This warning is indicated by the sound of a warning from speaker 15 (Figure 2) or by a message displayed on display 16 (Figure 2). This is done by displaying a warning message. This prevents the train from departing with insufficient passengers. This prevents people from boarding without the proper qualifications. It is easy to identify the passenger and encourage them to get off.

[0219] According to yet another modification of the fourth embodiment, a camera (for example, half-length on the ceiling) is installed inside the bus. A spherical camera is attached, and the processing unit performs image analysis to determine the number of passengers and the number of standing passengers. It has a function to count the number of seated passengers. Furthermore, it displays the bus's current location and the bus doors. The system controls the opening and closing signals, the bus stop request buzzer, the IC card touch system for boarding and alighting buses, and the signals for the fare box. The science department acquires these, and each of them acts as a trigger, along with the images at the time the event occurs. It has the function to associate and record this information. Server 30 (Figure 3) puts this information into a data lake. The data is accumulated. Using this data lake, it is possible to determine where people boarded and alighted, and the number of passengers inside the vehicle during that time. It will become possible to analyze seating arrangements using big data. This kind of data will This could be valuable information for consulting firms involved in urban planning, etc. This combines information transmitted via in-vehicle networks such as CAN with the driver's vital information. It has the function to acquire this information when some kind of abnormality occurs in the vehicle. This is useful for identifying causal relationships such as whether the cause is driving-related, passenger-related, driver-related, or vehicle-related. It can be used.

[0220] According to yet another modification of the fourth embodiment, the processing unit uses an image as the source for counting the number of people. The system uses images from outside the vehicle. The processing unit of this system records the counting results as people data. It has the functionality to easily extract locations with large crowds based on recorded people data. The processing unit records the image that was used for the count, triggered by the counting result. It has a function. For example, a function to record an image when the number of people exceeds a predetermined threshold. It has.

[0221] [Fifth Example] Next, a fifth embodiment will be described. The processing unit of the system according to the fifth embodiment is image analysis As part of its functionality, it can extract images from inside the vehicle to display content on the vehicle's display devices or the operating status of the vehicle's control devices. It has a function to detect the situation, and as processing corresponding to the analysis results, it displays the content or operation of the display device. The system executes processing according to the operating status of the equipment.

[0222] A device that acquires the data that forms the basis of the display content of a display device, or the operating status of an operating device. This system acquires the corresponding data without requiring the addition of separate data acquisition equipment. This is possible. For example, without connecting aftermarket devices to the OBD connector, these This allows you to obtain data. In particular, when connecting a third-party product to the OBD connector... Even if the vehicle manufacturer prohibits it, this system can collect this data. It can be obtained. As a vehicle display device, various instruments, in-vehicle warning lights (warning lamps) Examples include head-up displays that project images onto the windshield or other surfaces.

[0223] In a modified version of the fifth embodiment, the system's processing unit displays the following as the display content of the display device: It has a function to detect the displayed vehicle speed. The processing unit uses the position of the analog speedometer needle as a pattern. It has the function of recognizing and characterizing the numbers on the dial to acquire vehicle speed data. It has a function that recognizes the numbers displayed on the digital speedometer and acquires vehicle speed data. The acquired vehicle speed data can be used for various processes. The following describes the uses of vehicle speed data. Let me describe an example of its form.

[0224] The processing unit has a function to automatically detect the speedometer using image recognition. This function allows, The processing unit can detect the speedometer in various vehicles, regardless of the vehicle type. In addition, the processing unit determines the position and shape of the speedometer of the vehicle on which the processing unit is installed within the image. It has a function to register the status in advance. This allows the processing unit to easily detect the speedometer. It is possible.

[0225] Typical standalone car navigation devices use G to obtain the current location. It uses location information from PS. In cases where GPS signals cannot be received, such as in tunnels, The vehicle's current location must be determined by another method. For example, immediately after GPS signal loss... Move your vehicle's current position on the map for 5 minutes at a constant speed of approximately 80% of its previous speed. The mode in which this type of processing is performed is called tunnel mode. This is effective not only when entering a tunnel, but also when traveling under an elevated structure. If you encounter traffic congestion while processing in tunnel mode, you will drive at a constant speed. Using the assumption that this is the case would result in an excessively large error in the current position, making it practically unusable. It won't become a useful thing. An acceleration sensor is attached to the vehicle, and acceleration data from the acceleration sensor is collected. By using this feature, the accuracy of current location information in tunnel mode can be improved. However, installing an accelerometer is difficult and expensive for users, making it impractical. There isn't one.

[0226] This system has the functionality of a standalone car navigation system. The processing unit is the speedometer. The system reads the vehicle's speed from the displayed information and uses this speed information to perform map matching. It has a function to update the vehicle's current location information using the ping function. This can improve the accuracy of location information.

[0227] The processing areas include places where roads inside tunnels and ground-level roads run parallel, and elevated roads and ground-level roads beneath them. Where two roads run parallel to each other, the system determines which road to take based on whether or not tunnel mode is activated. It has a function to select whether to enable or disable switching. For example, when tunnel mode is activated The vehicle's position is mapped to the road inside the tunnel or the ground road under the elevated structure, and tunnel mode is activated. When it is not activated, the vehicle's position is determined by the ground road running parallel to the road inside the tunnel or the elevated road. It has a function to map the roadway.

[0228] In another modification of the fifth embodiment, the processing unit determines the display content of the display device to be detected as follows: It has a function to detect the remaining fuel amount displayed on the fuel gauge. The processing unit performs image recognition of the fuel gauge. This detects the remaining fuel level. The processing unit then detects when the remaining fuel level falls below a predetermined standard value. When it detects this, it notifies the driver with a message prompting them to refuel. For example, it can detect the vehicle's current location. In conjunction with the notification system, when you return from an outing to near your home or the company's vehicle storage area, If the remaining fuel level falls below the standard value, for example, 1 / 4 of a full tank, the processing unit will issue a warning and prompt you to refuel. It prompts the user to move to their destination. Furthermore, by linking the car navigation system with this system, the processing unit will guide the user to their destination. The distance and remaining fuel are compared, and based on the comparison results, the route from departure to destination is determined. It prompts you to refuel inside. For example, near a gas station along the route, it prompts you to refuel near your current location. It displays a message prompting the user to refuel at the gas station.

[0229] The function to extract the speedometer and fuel gauge from the image is installed in the processing unit of this system. This is achieved through artificial intelligence (AI). The processing unit further determines the position of the speedometer and fuel gauge in the image in advance. It has a function to register location and image information. Artificial intelligence extracts speedometer and fuel gauge data from images. If it is difficult to output, the processing unit will use the registered image information to output within the image. Extract the speedometer and fuel gauge from it.

[0230] The processing unit reads the fuel gauge of the vehicle provided for the car-sharing service and determines the remaining fuel level. When the amount is detected to be below a predetermined standard value, refueling will be performed and the car-sharing service will be activated. It has a function that notifies the driver that they can receive a special benefit. For example, "If you refuel now..." The message displayed reads: "And you can receive a discount equivalent to 15 minutes." do.

[0231] In yet another modification of the fifth embodiment, the processing unit controls the warning lights in the instrument panel. It has a function to determine whether or not it is lit. The warning lights that are subject to the determination of whether or not they are lit include the headlights. Light indicator, turn signal indicator, high beam indicator (high beam This includes warning lights, parking brake indicator, door ajar warning light, fuel warning light, etc. The system's processing means determines whether these warning lights are illuminated, and in response to the warning lights... It can detect the status of the corresponding vehicle.

[0232] In yet another modification of the fifth embodiment, the processing unit is displayed by a head-up display. It has a function to detect the displayed warning information. The processing unit is controlled by the head-up display. The system detects changes in the displayed image to identify warning content and executes predetermined processing according to the warning content. For example, issuing a warning using sound or voice. Converting a visual warning into an auditory warning. By issuing the warning, the effect is obtained that drivers will be more likely to notice the warning.

[0233] In yet another modification of the fifth embodiment, the processing unit receives information on the brightness outside the vehicle and the current position of the vehicle. Even though the situation should be such that the headlights should be turned on in conjunction with the information, the headlights When it detects that the headlight indicator is not lit, it will turn on the headlights. It has a function to notify the driver. Situations in which headlights should be turned on include, for example, when driving. Situations where the outside is darker than a certain level of brightness, or where the vehicle is currently located inside a tunnel or culvert. This includes the following: the headlights remain on even though the situation requiring them to be turned on has been resolved. When the headlight indicator remains lit, the processing unit will turn off the headlights. It has a function to notify the driver. There are also tunnels that are brightly lit inside due to the lighting. If you determine whether or not the headlights should be turned on based solely on the brightness information outside the vehicle, The system incorrectly determined that it was not appropriate to turn on the headlights, even though it was inside a tunnel. This situation could occur. The processing unit determines whether or not the vehicle is inside a tunnel based on its current location information. By making this determination, such misjudgments can be avoided. Furthermore, the processing unit, Even though it was a situation where not only the headlights but also the parking lights should be turned on, If the system detects that the indicator light for the parking lights is not illuminated, it will illuminate the parking lights. It has a function that notifies the driver to do so.

[0234] In yet another modification of the fifth embodiment, the processing unit determines from the image in front of the vehicle whether or not there is an oncoming vehicle. The system detects that the high beam indicator is illuminated despite the presence of an oncoming vehicle. It has a function that warns the driver to switch to low beams. Depending on the vehicle model, the beam Because the operation for switching between high and low beams is different, the driver will be driving a vehicle of a different type than the one they normally use. When this happens, you might intend to switch to low beams but end up with high beams on. It's inexpensive. This warning function is particularly useful in such situations.

[0235] In yet another modification of the fifth embodiment, the processing unit uses images of the outside of the vehicle when it is dark outside the vehicle. The system detects that the turn signal is flashing and flashes an indicator light inside the vehicle in sync with the turn signal's flashing cycle. It has the function to recognize the indicator as a turn signal indicator. The processing unit makes it easy to locate the turn signal indicator in the instrument panel. It is possible.

[0236] In yet another variation of the fifth embodiment, the processing unit determines from the image of the car interior that there is no one inside the car. Even if the vehicle is detected to be unresponsive, if the headlights or parking lights are detected to be on, the vehicle will be turned on. It has a function to notify the driver or vehicle manager that the lights have been forgotten to be turned off. For example, this is done by sending an email to a pre-registered email address. In addition, the processing unit will notify you via email, etc., if you forget to turn off the hazard lights or interior lights. It has a notification function. The hazard light status is indicated by the blinking status of the turn signal indicator. It detects the status of the room light. The brightness of the room light reflected in the image is used to determine if it is on or off. It detects it.

[0237] Room light settings typically include always-on mode, always-off mode, and door open / closed status. There are three types of linked modes (hereinafter simply referred to as linked modes). In many vehicles, These three modes cycle through and are refreshed each time the room light on / off switch is pressed. When the engine is stopped in a dark place (at night), the interior lights turn on, and after a while... Some vehicles have a feature that automatically turns off (for example, after a few seconds). If you want to keep the interior lights on for a longer time to prepare to get out of the car, There are also cases where, after turning off the engine, you organize and take out the purchased items, and then use your mobile phone. During the period when removing the last part of the car, etc., I would like to keep the room lights on. The driver wanted to prevent the interior lights from automatically turning off after a certain period of time has elapsed since the engine was stopped. To do this, switch the room light setting to either linked mode or always-on mode. However, When the door is open, the current setting of the interior light is either always on or on. It is difficult to determine this. Even if the current room light setting is in constant-on mode. Regardless, if you mistakenly think it's in linked mode and get out of the car, the interior lights will remain on. This system allows drivers and administrators to control the interior lights. Realizing that I had forgotten to turn off the lights, I turned off the room lights, which meant that the room lights were not left on for a long time. This can prevent the vehicle's battery from running out. Just by looking at the control switch, the room The current mode of the light is not clear from the structure, for example, the push button. A vehicle with an operating switch that cycles through the room light settings each time it is pressed. In this context, this function of the system is particularly useful.

[0238] When the interior lights are set to linked mode, if any door is open or ajar, In that case, the room light will remain on. A notification will be sent indicating that the room light is on. If a driver or manager is found to have a door ajar or an open door on the side that is difficult to see, You can realize that there is a possibility that this is the case.

[0239] When the room light is set to linked mode, there is a certain delay after closing the door (off delay time). Some vehicles have a function that keeps the interior lights on until a certain amount of time has passed. In such vehicles, the processing unit of this system detects that the door has been closed and then turns off It has a function that prevents the system from determining whether or not the lights were left on until the light delay period has elapsed. The control unit has a function to set the light-off delay time based on user input.

[0240] Furthermore, the processing unit will not turn on the interior lights even after a certain amount of time has passed since no one has left the car. If it detects that the lights are still on, the vehicle's controller will turn off the interior lights. It has the function of transmitting a signal to cause something to happen.

[0241] In yet another modification of the fifth embodiment, the processing unit controls the headlights, side marker lights, and hazard lights. The system detects when a vehicle door is opened while the interior lights, etc., are on or flashing. It also has a function that issues an alarm if any of the lights remain lit. This allows the driver to notice they've forgotten to turn off the lights before getting out of the car, thus preventing the incident from happening again. It is possible.

[0242] In yet another modification of the fifth embodiment, the processing unit is a point of the parking brake indicator. Based on the lights, if it detects that the parking brake is engaged, the driver A feature that prevents the alarm from being issued even if it detects from an image that the driver is drowsy. In addition, when the parking brake is applied, the driver may have a mobile device. Even if it detects that the driver is performing an action that would be considered dangerous while driving, such as looking at something, It has a function that prevents alarms from being issued.

[0243] In yet another modification of the fifth embodiment, the processing unit requires early inspection or repair. If a warning light indicating that something has happened remains illuminated for a certain period of time, the system will detect that It has a function to notify the vehicle's dealer or registered repair shop of the vehicle's condition. For example, Users who are not interested in cars will not notice the warning light if the car is still drivable. Often, people don't notice the light and leave it unattended. For example, when the engine's own warning light comes on... Normally, the vehicle would become inoperable, but even if the warning light indicating an engine oil abnormality is illuminated... It can still be driven in this condition. Also, in the case of an O2 sensor malfunction, fuel efficiency will decrease, but driving itself will still be possible. It is possible. If the vehicle is still drivable, continuing to drive without inspection and repair may result in minor malfunctions. It can sometimes become more serious. For dealers and repair shops, a warning light illumination is a warning. This provides useful information for conducting sales activities to the owners of vehicles with illuminated lights, enabling them to offer inspection and repair services. Vehicle owners can prevent serious breakdowns by having their vehicles inspected and maintained early. ru.

[0244] In yet another modification of the fifth embodiment, the processing unit is provided by a temperature sensor that measures the outside temperature of the vehicle. The temperature measurement value is displayed in the temperature display area within the instrument panel. It has a function to detect temperature. The processing unit has an image analysis function, and the instrument panel It has a function to read the temperature from an image displayed in the temperature display area using character recognition. The Science Department, as a processing step corresponding to the analysis results, communicates externally via communication devices and communication networks. The process of sending temperature information to the server is executed.

[0245] For example, sensors are installed in the homes of ordinary users to obtain data on pollen dispersion. Furthermore, by collecting measurement data from sensors, it is possible to predict the detailed pollen dispersal situation and the flowers Services that provide information on dust dispersion are offered by companies such as Weathernews Inc. This includes outdoor photos submitted by general users, and outdoor live cameras installed at private residences. The images collected are incorporated into weather forecasts. Furthermore, the servers that distribute these images... Services (such as Weathernews SOLiVE24) are available. The system providing the information service collects outside temperature information measured by many vehicles. This allows more vehicles to be used as temperature measurement probes. It transmits temperature information. When many vehicles are distributed over a wide area, the temperature information collected from many vehicles is not localized to specific spots. It can be useful as information. For example, it can provide localized information that cannot be expressed in general weather reports. It will become possible to issue heatstroke warnings and other similar alerts.

[0246] For companies providing such weather information services, installing temperature sensors in-house is a necessary step. This allows for the collection of temperature information over a wide area without requiring any special measures. The company that received the temperature information will, in return, give a gift to the owner of the vehicle that provided the information. It would be beneficial to offer the paid weather information service free of charge or at a discounted price.

[0247] The processing unit further reads temperature information from the temperature display area in the instrument panel. Furthermore, it is displayed on the temperature display sign on the side of the road that is captured in the image taken by the camera. It has the function of acquiring outside temperature information by recognizing the numbers. The process involves recognizing the text "Current Temperature" on the sign. The acquired outside temperature... The information is processed in the same way as temperature information obtained from the temperature display area in the instrument panel. The department sends it to an external server.

[0248] In yet another modification of the fifth embodiment, the processing unit analyzes the operating status of the operating device based on image analysis. As a function to detect the position of the driver's feet relative to the accelerator and brake pedals, It has a detection function. As processing according to the operating status of the control device, the processing unit detects the driver's foot. The driver can easily recognize whether the pedal is in the accelerator pedal position or the brake pedal position. The system executes a process to notify the driver so that they can proceed. For example, if the driver's foot is in the position of the accelerator pedal... The light-emitting color of the light source changes depending on whether the brake pedal is in the position or not. Therefore, the driver will be notified.

[0249] In yet another variation of the fifth embodiment, the foot is on the accelerator pedal or the brake. A device that detects whether a foot is on the pedal and displays the detection result as visual information on a display. This system will be installed in vehicles equipped with this capability. The processing unit of this system is displayed on this display. The image data was analyzed to determine the position of the driver's feet relative to the accelerator and brake pedals. It has the function of knowing.

[0250] In yet another modification of the fifth embodiment, the processing unit detects an obstacle in front by image analysis. Even if the foot is in the position of the accelerator pedal, the device will emit a warning sound or otherwise. It has the ability to activate when the driver notices the warning and presses the accelerator and brake pedals. This can help reduce the occurrence of errors.

[0251] In yet another modification of the fifth embodiment, the processing unit analyzes the operating status of the operating device based on image analysis. As a function to detect this, it has a function to detect the rotation angle of the steering wheel (handle). The processing unit processes the steering wheel rotation angle according to the operating status of the control device. Based on the vehicle speed, the processing unit performs a process to determine the vehicle's direction of travel and distance traveled. The processing results are provided to a standalone car navigation system. The navigation device will use the information on the direction of travel and distance traveled provided by this system. Based on this, the vehicle's travel route can be calculated even in situations where GPS signals cannot be received. can.

[0252] In yet another modification of the fifth embodiment, the processing unit has a function to perform image analysis, and the vehicle itself It detects whether the vehicle is in manual operation (including autonomous driving levels 1-5) or manual operation (autonomous driving level 0). It has the function to do this. This detection is performed by the processing unit, which displays the automatic information on the instrument panel. Recognizing the display status of indicators that show whether various driving functions are operating or not, from an image. This is done by the processing unit, which, as a process corresponding to the analysis results, determines that the automatic driving function has been deactivated. When this is detected, the system executes a process to notify the driver that the autonomous driving function has been deactivated. Even though the autonomous driving function was deactivated, the driver kept their hands on the steering wheel. If the processing unit detects that the steering wheel is not being operated, it will instruct the driver to operate the steering wheel. A warning will be issued to draw attention. Furthermore, it will indicate that the autonomous driving function has been deactivated and that the driver has stopped When the ring wheel is not being held, the processing unit is triggered by the camera's image. It has a function to record images as event records, separate from continuous recording. This can be used as evidence that the driver's actions were inappropriate.

[0253] The processing unit operates when the autonomous driving function is enabled in a vehicle equipped with a radar detector. It has a function to disable the radar detector's function. This allows you to access the radar detector. Unnecessary alarms can be eliminated. When the autonomous driving state is disabled, processing The unit enables the radar detector function.

[0254] Is the vehicle in autonomous driving mode or manual driving mode? What level (mode) of autonomous driving is it in? Whether this is the case depends on the processing unit's observations of indicator and steering wheel movement, and the driver's hand position. The determination is made based on factors such as the driver's head position and direction.

[0255] [Sixth Example] Next, a sixth embodiment will be described. The processing unit of the system according to the sixth embodiment is image analysis One of its functions is to extract text from images of the car's interior that is displayed on stickers attached to the vehicle. It has a recognition function, and as a processing step corresponding to the analysis result, it responds to the information indicated by the recognized characters. Then, the process of issuing a warning is executed.

[0256] By being alerted, the driver, manager, or owner of the vehicle will affix the notice to the vehicle. Attention is drawn to the textual information on the sticker, and appropriate processing is performed according to that textual information. can.

[0257] In the modified version of the sixth embodiment, the processing unit is shown on the inspection sticker (vehicle inspection sticker). It has the function to recognize the characters present. The inspection sticker has a mark on the inside of the vehicle indicating when the vehicle inspection is valid. The date is displayed. The processing unit uses character recognition to process the information of the date the vehicle inspection expires. The reading is performed, and the period from the present time until the expiration date of the vehicle inspection (vehicle inspection deadline) is determined by the specified method. If the period is shorter than expected, a reminder will be issued to get the vehicle inspected. For example, the processing unit will say, "Vehicle inspection required." The system will alert users by emitting a voice message saying, "The expiration date of the validity period is approaching." If the period until the expiration date of the vehicle inspection becomes even shorter, for example, the processing unit will say, "The vehicle inspection will be It issues a warning with the voice message, "Please do not do this."

[0258] In another modification of the sixth embodiment, the processing unit is indicated on the serviced sticker. It has the function to recognize characters. The maintenance inspection sticker has the deadline for the next periodic inspection. The date is displayed in year, month, and day format. The processing unit uses character recognition to process the year, month, and day information indicating the deadline for periodic inspections. The system reads the data, and if the period from the current time until the due date for the periodic inspection becomes shorter than the specified period, the system will be closed. The system will issue a reminder to undergo inspections. For example, the processing unit will state, "The deadline for the periodic inspection is approaching." The system will alert users by emitting the voice message, "Please be careful." The period until the deadline for the regular inspection is If the message becomes even shorter, for example, the processing unit will issue a voice warning saying, "Please undergo regular maintenance." Issue.

[0259] In yet another modification of the sixth embodiment, the processing unit has a camera as an image analysis function. It has a function to determine whether or not the inspection mark is visible in the image captured. As part of the process, if the inspection mark is not visible in the image, the processing unit will take steps to ensure the inspection mark is visible. The system will execute a process to remind users to adjust the camera's mounting position and orientation accordingly. For example, the processing unit might say, "The inspection sticker is not visible. Adjust the camera position so that the inspection sticker is visible." It would be good to emit a voice message saying, "Please adjust your posture." This will help the camera's positioning. At times, its position and orientation can be easily adjusted. The processing unit controls the display equipment inside the vehicle. For example, it has a function to read the information displayed on the speedometer, fuel gauge, etc., and the display devices inside the vehicle are digital. It has a function to warn people about things that might be reflected in the image.

[0260] In yet another modification of the sixth embodiment, the processing unit has an image analysis function, which is fluorocarbon On the windshield, driver's side window, and passenger side window, stickers other than those permitted by law may be displayed. It has a function to determine whether or not a sticker is attached. The processing corresponding to the analysis result is as follows: The processing unit will determine if a sticker other than those permitted by law is affixed to the vehicle. In that case, the system will execute a process to notify the driver or other relevant person to remove the sticker.

[0261] [Seventh Example] Next, the seventh embodiment will be described. The processing unit of the system according to the seventh embodiment is image analysis One of its functions is to analyze images of the area where the passenger seat is visible, and respond to the analysis results. The process will execute according to the situation in the passenger seat.

[0262] For example, depending on whether there is a person in the passenger seat, or whether there are objects placed on the passenger seat , the appropriate processing can be performed.

[0263] In a modified example of the seventh embodiment, the drive recorder 1 (Figure 1) is placed in the passenger seat. To be installed in vehicles equipped with a function that issues a warning when a person is not wearing a seatbelt. The science department, as part of its image analysis function, uses images of the area where the passenger seat is visible to determine if a person is sitting in the passenger seat. It has a function to determine whether or not it is present. As a process corresponding to the analysis result, the processing unit is located in the passenger seat. This feature detects that no one is seated in the passenger seat and issues a warning that the seatbelt is not fastened. This disables it. The processing unit also performs image analysis, detecting if there is an object on the passenger seat. It has a function to determine whether or not there is an object. As a process corresponding to the analysis result, the processing unit determines whether there is an object in the passenger seat. When the vehicle is in place, the function that issues a warning for unfastened seatbelts in the passenger seat will be disabled. Execute the process.

[0264] The vehicle mistakenly identifies a person as being in the passenger seat when an object equivalent to a person's weight is placed on the passenger seat. This may result in the system issuing a seatbelt-unfastened warning. If an object is placed on the passenger seat, the vehicle will issue a seatbelt unfastened warning. This eliminates the problem. As a result, drivers will no longer be misled by warnings. Especially in the passenger seat. In vehicles that have a feature that causes a warning buzzer to sound continuously when a heavy object is placed on them, this device The function is effective. It prevents the vehicle from issuing a seatbelt-unfastened warning. The processing unit then issues a warning to the vehicle's controller via the in-vehicle network (CAN). It sends a signal to disable it.

[0265] In vehicles with similar functions for the rear seats, the processing unit detects when an object is placed in the rear seat. When the system detects that the seatbelt is not fastened, it cancels the seatbelt unfastened warning function.

[0266] [Eighth Example] Next, the eighth embodiment will be described. The processing unit of the system according to the eighth embodiment is image analysis One of its functions is to compare current images of the car's interior with past images of the car's interior and analyze them. As a corresponding process to the results, the degree of discrepancy between the current image of the car's interior and past images of the car's interior is assessed. If the level exceeds a predetermined level, a notification process will be executed.

[0267] The vehicle owner or manager will be notified if the conditions inside the vehicle exceed a certain level. This allows us to see that it has changed.

[0268] In the modified version of the 8th embodiment, the processing unit is attached to the vehicle in advance as a past image of the interior of the vehicle. Using an image showing that nothing other than the item being shown has been brought into the vehicle, and comparing this image with the current situation By comparing it with images of the car's interior, it is possible to determine whether or not something was brought into the car. It has a function.

[0269] The processing unit further compares images from multiple past points in time with the current image. It has a function to calculate the amount of time that items brought into the vehicle remain inside the vehicle. If an object is detected that has been left inside the vehicle for an extended period of time, the system will notify you of that object. He possesses talent.

[0270] Next, we will explain the specific processing of the processing unit. First, the state when the vehicle is not loaded with cargo. Then, if possible, with only the driver in the vehicle, press the initial setup button of this system, and the car The two vehicles are driven. The processing unit is located in the part of the image where there is optical flow corresponding to the vehicle speed. It is recognized as a window. Any other moving part is recognized as the driver. Window and driver recognized Areas other than those identified are recognized as areas where objects may be placed, and the vehicle is not loaded with luggage. The image (video) of the vehicle without cargo is recorded. This completes the initial setup process. The process is completed.

[0271] From then on, when the engine starts, when the engine stops, etc., the processing unit will display an image of the unloaded state and the current image. The difference between the image and the actual object is calculated to identify the presence of an object brought into the vehicle. If the position of a specified object does not change for a certain period of time, the processing unit will issue a warning. This feature allows the driver or other personnel to recognize objects that have been left inside the vehicle for an extended period of time. This is especially useful for people whose cars quickly become cluttered with trash.

[0272] [Ninth Example] Next, the ninth embodiment will be described. The processing unit of the system according to the ninth embodiment is image analysis One of its functions is to determine whether or not pre-registered items are visible in the images inside the vehicle. As a process corresponding to the analysis results, if a pre-registered object is visible in the image inside the vehicle, an alarm will be issued. The process of issuing a report is executed.

[0273] Drivers and other personnel can become aware of the presence of a pre-registered item inside the vehicle through the alarm. Cut.

[0274] In a modified example of the 9th embodiment, the processing unit checks for items that should be forgotten (specific items) It has a function to register images of items (bags, wallets, etc.). The processing unit is set to the state when no one is inside the car. If the system detects that a registered object is visible in the image, it will issue an alarm or forget It notifies you of a lost item. An example of an alarm is sounding, such as a buzzer installed inside the car. Examples of notifications for lost items include, for example, wirelessly sending electronic messages to the driver's smartphone or mobile phone. We will notify you via email or other means.

[0275] In another modification of the ninth embodiment, the processing unit detects an item left in the vehicle and opens the door. It has a function that issues an alarm when it detects movement. The alarm will alert the driver or other personnel to move away from the vehicle. You can notice your forgotten item before it gets lost. The processing unit further detects forgotten items inside the vehicle, and When the system detects the movement of closing a door, it notifies the driver or other relevant person via email or other means that an item has been left behind. It has a function that allows the driver or other personnel to realize they have left something behind after leaving the vehicle. The detection of door opening and closing motions is performed, for example, by detecting optical flow.

[0276] In yet another modification of the ninth embodiment, the processing unit detects optical flow. This enables the detection of other movements inside the vehicle. For example, the processing unit detects images The analysis functions include door opening and closing movements, window opening and closing movements, turn signal operation, and steering. This system has the function to detect wheel operation, sun visor raising and lowering, etc. Without connecting to the vehicle's in-vehicle network connector (e.g., OBD connector), You can obtain their information.

[0277] When the sun visor is down, especially with a 360-degree camera mounted on the windshield... When trying to capture images of the driver's side, the sun visor gets in the way and it's impossible to get an image in that direction. Therefore, the front of the vehicle should be bright enough that a sun visor is deemed unnecessary. Nevertheless, if the processing unit detects that the sun visor is down, it will issue a warning. Whether the sun visor is down or not is determined by the processing unit based on the proportion of the image occupied by the sun visor. The judge will make the decision.

[0278] [Tenth embodiment] Next, the tenth embodiment will be described. The processing unit of the system according to the tenth embodiment is shown in the image. As an analysis function, it has the ability to capture images of objects that the user brings close to the camera and perform image recognition. The processing corresponding to the analysis results involves processing based on the information written on the object the user brings close to the camera. Execute this.

[0279] By bringing an object they possess close to the camera, the user can respond to the information written on that object. It is possible to execute the specified process.

[0280] In a modified example of the 10th embodiment, the processing unit performs image recognition of the driver's license. It has the ability. The processing unit recognizes the expiration date written on the driver's license and calculates from the present time. When the period until the expiration date is shorter than the prescribed period, the driver's license renewal date is approaching. Inform the user of this.

[0281] In another modification of the 10th embodiment, the processing unit processes an image of a gas station receipt. It has a recognition function. The processing unit reads the date, fuel amount, and amount displayed on the receipt into text. Recognize this information, record it, and create a database.

[0282] In yet another modification of the 10th embodiment, the processing unit is an identification card (e.g., driver's license). It has the function of image recognition for identification documents such as driver's licenses, employee IDs, and resident registration cards. The processing unit reads the information on the identification document. The system recognizes the names and other information displayed and determines whether or not the person is a pre-registered individual. If the person on the identification document is not registered, the system will prevent the engine from starting. This allows us to limit the people who can drive the vehicle to those who have been registered in advance. The processing unit then records the name and other information of the person on the identification document. The recorded information is then used to inspect the vehicle. It can be used to identify the driver when managing the operation of both vehicles.

[0283] In yet another modification of the 10th embodiment, the processing unit has an image recognition function, which is a car A function that recognizes objects the user brings close to the camera outdoors, and a function that recognizes objects the user brings close to the camera inside the vehicle. It has the function of recognizing objects that come close to the camera. For example, the processing unit compares it with a normal image. When something not normally present in an image occupies a larger portion of the camera's field of view than a predetermined range, The camera recognizes that something has been brought close to it. In particular, the processing unit recognizes a predetermined object within the camera's field of view. When it recognizes that the user is holding something that occupies a larger area than the specified range, it will then consider that item. The system detects when the user has moved closer to the camera.

[0284] [Embodiment 11] Next, the 11th embodiment will be described. The processing unit of the system according to the 11th embodiment is shown in the image. The analysis function includes the ability to perform image analysis on objects captured in the forward image, and the analysis results are... As a corresponding process, the system will execute a process to provide the driver with information that they should be aware of regarding the vehicle's operation. do.

[0285] By alerting the driver according to the situation ahead, the vehicle will proceed forward. This can help prevent accidents that may occur in the future.

[0286] In a modified example of the 11th embodiment, the processing unit has an image analysis function that compares with the vehicle in front. It has a function to calculate at least one of the following distance and relative speed. The processing unit calculates the following distance and Based on relative velocity, it detects situations with a high risk of collision and warns of the high risk of collision. The driver will be notified by issuing a report, etc. The following distance is determined by the size of the vehicle in front in the image, for example. Information such as the size of the license plate, the width of the tread (distance between tires), and the width of the taillights. It is calculated from the report. Relative speed is calculated from the rate of change in the distance between vehicles.

[0287] In another modification of the 11th embodiment, the processing unit, as an image analysis function, processes the forward signal It has a function to detect the color of the traffic light. The processing unit tells the driver to stop when the traffic light is red. Notify them to prompt them.

[0288] In yet another modification of the 11th embodiment, the processing unit, as an image analysis function, forward It has the function of recognizing road signs. The processing unit provides the driver with information according to the information indicated by the road sign. To prompt attention. For example, if a road sign indicates a speed limit, the processing unit will use character recognition to The system retrieves the speed limit at the current location and notifies the driver of the speed limit. Road signs indicate no entry. If this is the case, the processing unit will issue an alarm if the vehicle attempts to enter a road where entry is prohibited. The driver will be notified. The processing unit will determine if the vehicle is driving in violation of traffic rules indicated by road signs. If it detects that you are about to do so or are driving in violation of traffic rules, the rules The driver will be notified that a violation has occurred. This will prevent traffic rule violations from occurring, or It can be suppressed. The processing unit recognizes the text of the road sign and determines the current driving If a sign indicating that the section is a zone with an automatic speed enforcement camera is detected, the driver will be instructed to speed We will notify them to be careful not to overuse it.

[0289] In yet another modification of the 11th embodiment, the processing unit has an image analysis function, which is the vehicle It has a function to detect the congestion status of lanes adjacent to the driving lane. Whether or not congestion is occurring The processing unit makes a determination based on the average speed and distance between vehicles in the adjacent lane. The speed of vehicles in adjacent lanes is below a predetermined standard value, and the distance between vehicles is below a predetermined standard value. When a predetermined number of vehicles are lined up, the processing unit determines that a traffic jam has occurred. The department, as a process corresponding to the analysis results, determines that congestion has occurred in the lane adjacent to the vehicle's lane. When this is detected, the system will warn the driver to be careful of vehicles suddenly emerging from between other vehicles in a traffic jam. The process to trigger a warning is executed. In lanes adjacent to the vehicle's lane, the oncoming lane and the same lane as the vehicle are detected. This includes adjacent lanes traveling in the same direction. In particular, this applies to places like shopping malls, theme parks, and train stations. In places where vehicles gather, such as gas stations, where vehicles come and go, adjacent to your own lane The adjacent lane going in the same direction is also likely to be congested. In such cases, if you are traveling in the same direction as your vehicle... Detecting congestion in the adjacent lane is effective. This helps prevent accidents caused by sudden movements. This can be suppressed. The processing unit detects when the vehicle's speed is below a predetermined speed, for example, 30 km / h. When the distance is less than m, the system will not emit a warning sound for sudden pedestrians. This can avoid the generation of warning sounds. In addition, the processing unit takes the image from the front and jumps It has a function to detect situations where there is a high risk of accidents and issue a warning. For example, the processing unit detects road conditions. If there are consecutive parked vehicles, park in the parking lot where a parking meter is installed. When there are vehicles present, a warning will be issued to warn against sudden movements.

[0290] For example, car navigation devices may include traffic congestion probe information, general road and highway driving information, and surrounding area information. It is equipped with a function that alerts the driver to driving caution based on location information (such as residential areas and schools). There is a way to obtain traffic congestion probe information in places where it is not available. However, it is possible to issue warnings due to traffic congestion.

[0291] In yet another modification of the 11th embodiment, the processing unit has an image analysis function, launch Previously, it had a function to detect animals (dogs, cats, raccoons, wild boars, etc.) and people around the vehicle. The processing unit detects animals or people around the vehicle and then activates the system to indicate that animals or people are present in the vicinity of the vehicle. This will inform the driver. This will help prevent accidents that occur immediately after starting.

[0292] In yet another modification of the 11th embodiment, the processing unit, as an image analysis function, surround It has a function that detects that the vehicle is moving slowly forward from the flow of the scenery. The processing unit is the vehicle If it detects that both vehicles are moving slowly forward, it will issue an alarm. For example, if the vehicle comes to a complete stop. Afterward, the vehicle slowly begins to move due to the creep phenomenon caused by the release of pressure on the brake pedal. In such a case, the driver should be notified that the vehicle is moving and that there is a risk of a rear-end collision. This is possible. The processing unit is located when the vehicle is moving at a speed such that it is moving forward due to the creep phenomenon. The system determines that the vehicle is moving forward slowly. The processing unit determines the speed of the surrounding scenery flow. This is done by detecting the flow.

[0293] In yet another modification of the 11th embodiment, the processing unit has the function of image analysis, foreground A machine that determines whether a vehicle is a large vehicle, such as a bus or a box truck with an aluminum cargo bed. It has the ability. When the processing unit detects that the vehicle in front is a large vehicle and is approaching a traffic light, The system notifies the driver of the presence of traffic lights. For example, it might emit a voice message such as "Caution: Traffic Lights." The presence or absence of an aircraft is pre-registered in the processing unit as POI information. For example, if there is a large vehicle ahead When driving, it becomes difficult to see traffic lights in the direction of travel. Traffic lights on the opposite lane and pedestrian traffic lights are also difficult to see. By looking at the traffic lights, you can anticipate their presence and maintain a sufficient distance from the vehicle ahead. It is preferable to make it easily visible. However, if such care is neglected, progress There is a risk of overlooking the traffic light in the direction you are going. Also, in rural areas, there is a risk of overlooking the traffic light on the opposite lane. Pedestrian traffic signals may not be installed. When the vehicle in front is a large vehicle, By notifying drivers of the presence of traffic lights, the risk of drivers overlooking them can be reduced. In addition to the function of directly notifying the presence of traffic lights, the processing unit also has a function to prevent traffic lights from being overlooked. It has a function to warn the driver to drive in a way that reduces safety. For example, the processing unit says, It will emit a voice message such as, "Please maintain a safe distance between vehicles." For example, when a traffic light is installed... A warning is issued when the distance to the intersection exceeds the vehicle's safe stopping distance. The safe stopping distance varies depending on the vehicle's speed and weather conditions (whether it is raining or not). The processing unit performs safe stopping. The appropriate distance is set based on the vehicle's speed and weather conditions (whether it's raining or not). For example, If the safe stopping distance in clear weather is 50m, the processing unit will determine that the current weather is clear and the preceding large When the distance to the vehicle is less than 50m and outside the urban area according to the car navigation system. If there is an intersection with traffic lights within 100 meters in the direction of travel, a warning will be issued.

[0294] In yet another modification of the 11th embodiment, the processing unit, as an image analysis function, forward It determines whether the road is on a slope, and if so, it detects the degree of the slope. It has the function of doing so. The processing unit detects excessive speed and speed depending on the degree of the slope. The system warns of a decrease in performance and the need to switch to hill climb mode.

[0295] In yet another modification of the 11th embodiment, the vehicle is a cargo handling device, for example, a f This includes forklifts, trucks, etc., and the processing unit processes the captured image to determine the text displayed on the cargo. The system recognizes labels such as letters and barcodes and determines whether or not the cargo should be handled by the vehicle. The information indicated by the labels attached to the cargo to be handled by the vehicle is pre-registered in the processing unit. The processing unit detects that the vehicle is attempting to load cargo that it is not supposed to handle. Upon detection, the system executes a process to issue an alarm.

[0296] In yet another modification of the 11th embodiment, the processing unit is configured to move from optical flow to image It has the function of detecting window regions in the image. Furthermore, the processing unit receives data from the optical flow, and then the car Whether the vehicle entered the tunnel, exited the tunnel, or is currently traveling inside the tunnel. It has a function to determine that. In addition, the processing unit determines from optical flow whether the vehicle is under the elevated structure ( It has a function to determine whether or not the vehicle is traveling on a parallel road (below the one above or below). Furthermore, the processing unit opti It has a function that uses Calflow to determine the presence or absence of guardrails and whether caution is needed regarding drainage ditches.

[0297] [Twelfth Example] Next, the 12th embodiment will be described. The processing unit of the system according to the 12th embodiment is shown in the image. One of the analysis functions is to analyze the characters written on the license plates of other vehicles that appear in images taken outside the vehicle. It has the function of recognizing, and as a process corresponding to the analysis result, it identifies the character through character recognition. The process of recording information on a card and the process of transmitting it over the network are at least We will also execute the other option.

[0298] Is the vehicle driving near a vehicle equipped with an in-vehicle device that has the processing functions of this system? It can identify the license plate information of other vehicles parked nearby. The unit records the current location information of the vehicle equipped with this in-vehicle device and the date and time when the license plate was captured. The information is transmitted in association with the information of the identified license plate. Location information and date and time are also transmitted. The license plate information obtained along with the information is the vehicle number plate installed on the road. - Along with the license plate information acquired by the automatic reading device (N system), This provides useful information for determining the travel routes of many vehicles. In this way, the vehicle's license plate and GP By linking S information, we can construct a system that can be called a wide-area mobile N system. It is possible.

[0299] In a modified example of the 12th embodiment, the system is installed in multiple vehicles, and Location information, date and time information, and license plate information are collected and linked via the network. A central system will be built to collect specific license plate information. It has the function of extracting the location and date / time information of the vehicle it owns to identify the vehicle's travel route.

[0300] [13th Example] Next, the 13th embodiment will be described. The processing unit of the system according to the 13th embodiment is shown in the image. The analysis function includes the ability to determine whether or not a window frame exists around the vehicle's window. Then, as a process corresponding to the analysis results, if the edge of the window glass is detected in the window area, a notification process is implemented. To do.

[0301] The presence of a window frame around the window indicates that the window glass has not been closed. The driver, etc. By receiving notifications from this system, you can become aware if you have forgotten to close a window. The processing unit detects the discontinuity in the image caused by the difference in refractive index between the window glass and the atmosphere, The edge of the window glass is detected. The processing unit, as a notification process, wirelessly sends a notification to the driver's mobile device, etc. It processes notifications such as those sent via email. In addition, the processing unit is installed in the vehicle. The notification process is handled by sounding a buzzer. A smoke screen is displayed on the window of the vehicle equipped with the processing unit. When the protective film is applied, the processing unit is located in the area where the window is open and the window glass. The system detects whether a window is open based on the difference in brightness between the image and the smoked area.

[0302] [14th Example] Next, the 14th embodiment will be described. The processing unit of the system according to the 14th embodiment is shown in the image. The analysis function includes detecting whether the wipers are operating or whether the amount of raindrops on the windshield is increasing or decreasing. It has a function to detect the presence or absence of [unclear] and determine whether or not it is raining, and processes the analysis results accordingly. If it is determined that it is raining, the system will send information to notify that it is raining. The process of sending from the location is executed.

[0303] By receiving information transmitted from this system, it is possible to determine if it is raining in the area where the vehicle is traveling. It can determine whether or not it is raining. The processing unit can tell an external server, etc., if it is raining. It has the function of notifying the device and also notifying the vehicle's current location information. Multiple external devices are available. By collecting information from vehicles, it is possible to create a map of areas where it is raining. Furthermore, the processing unit notifies that it is raining and transmits the current time information. It has the function of knowing. This allows external devices to know about the temporal changes in areas where it is raining. It is possible to obtain the necessary information.

[0304] In a modified example of the 14th embodiment, the external device is a service of an organization that provides weather-related information. It is a wiper. The processing unit takes a picture when the wiper is not visible, for example when the wiper is down. It has a function to upload images to the server at the moment they are taken. It has the function to obtain information about rainfall, such as rainfall amount, from the image. The server is a car The function allows the rainfall and other data collected from both sources to be used as big data in various application fields. It has. For example, the server is used for predicting localized torrential downpours. The processing unit is the front-end. If raindrops are present on the glass, but their number remains unchanged, it is determined that the rain has stopped. The processing unit, once it determines that the rain has stopped, notifies the server that the rain has stopped. This allows us to obtain more detailed information about rainfall.

[0305] [Example 15] Next, the 15th embodiment will be described. The processing unit of the system according to the 15th embodiment is shown in the image. The analysis function includes the ability to determine the weather from the sky conditions captured in the image, and the analysis results As a corresponding process, the system will execute a process to transmit weather-related information from the communication device.

[0306] Weather information collected from multiple vehicles is considered big data related to weather information. This information can be put to good use. The server that collected this information can then analyze the color of the sky and the clouds captured in the image. Based on the presence or absence of [unclear], the system determines the weather, such as clear skies, partly cloudy, etc. Furthermore, the processing unit determines the height of the sun. Furthermore, it detects the intensity of the rising and setting sun from the brightness of the sky and transmits information indicating that the rising or setting sun is strong. It has the function of transmitting the vehicle's current location information along with weather information. The server that collected this information also collected information about the strength of the morning and afternoon sun, as well as the vehicle's current location. It has the function of providing this information to drivers of other vehicles heading to that location. Drivers who receive this information will be aware that they need to be careful of the rising or setting sun.

[0307] [16th Example] Next, the 16th embodiment will be described. The processing unit of the system according to the 16th embodiment is located in the vehicle It has a function to detect sound information from the microphone installed on it, and the sound information detection result and the image solution The analysis results of the analytical function are correlated, and processing is performed according to the results of the correlation.

[0308] Compared to performing processing based solely on the detection results of sound information or the analysis results of images, By linking the two, it becomes possible to perform more appropriate processing.

[0309] In a modified example of the 16th embodiment, the processing unit is mounted inside the vehicle as an image analysis function. It has a function to detect changes in the state of the display unit of a device that has a display unit, and processes the results in accordance with the analysis. In principle, when the detected sound information and the change in the state of the display unit are synchronized, the detected sound The system performs a process to associate and store information with the device whose display state has changed.

[0310] In another modification of the 16th embodiment, when the processing unit detects sound information from the microphone, Based on the information stored in which sound information and changes in the state of the display unit are associated, a sound is emitted. It has the function to identify the device that emitted the signal. For example, the processing unit has a voice recognition function and uses a microphone. It determines whether the sound information originated from a specific device, and if it originated from a specific device... If it is found to be a voice recognition function, the voice recognition function will be canceled. This will prevent unnecessary voice recognition processing. The execution of the logic can be avoided. Also, the processing unit determines whether the sound information from the microphone is for a specific device. If it is determined that the sound was emitted from the device, the area of ​​the in-car image corresponding to the device that generated the sound will be removed from the image. It has the function of recording information that can distinguish it from the surrounding area. For example, if the device that produced the sound The image of a rectangle or a closed curve of any shape surrounding the captured area is overlaid onto the original image and recorded. Alternatively, a semi-transparent geometric pattern can be overlaid and recorded over the area where the sound-generating device is visible. .

[0311] In yet another modification of the 16th embodiment, the processing unit has an image analysis function, and A function that determines whether or not emergency vehicles, such as ambulances, fire trucks, and police cars, are visible in the image. The processing unit detects the sound of an emergency vehicle siren from the microphone and also detects the emergency vehicle in the image. When it detects both, it associates them and determines that there is an emergency vehicle in the vicinity of its own vehicle. The processing unit then If an emergency vehicle is detected, a message will be sent to the driver to alert them.

[0312] [17th Example] Next, the 17th embodiment will be described. In the 17th embodiment, a turtle is installed inside the vehicle. There are multiple "Ra"s, and the processing unit, as part of its image analysis function, determines the position of other cameras captured in the image. It has a function to detect and, as a process corresponding to the analysis results, identifies the positional relationship of multiple cameras. Execute the process.

[0313] It is possible to obtain images of a wide area inside the car that is difficult to capture with a single camera. Once the relative positions of the cameras are determined, the relative positions of the images captured by each camera can be determined. Cut.

[0314] According to a modification of the 17th embodiment, the processing unit detects the models of other cameras that are visible in the image. It has the function to identify and determine the field of view from the model. The processing unit further processes images captured by multiple cameras. It has the function of combining images. By combining images, it displays the individual images from the camera. Compared to other methods, it allows for a more intuitive understanding of the situation inside the vehicle.

[0315] According to another modification of the 17th embodiment, at least a part of the processing unit functions as a personal computer or the like. This is achieved by the JUWA. The processing unit displays the camera's position within the image to be displayed. Display icons or other signs. The camera's position inside the vehicle is determined by the camera's (self) position. It is difficult to identify from the captured image. The processing unit detects other objects that the camera is capturing. The processing unit detects the position of the camera inside the vehicle from the camera image. When images captured by cameras are combined and displayed, the positions of the three cameras are visible within the combined image. This will be displayed using icons, etc. Furthermore, the icons will differ depending on the camera model or field of view. ru.

[0316] According to yet another modification of the 17th embodiment, the processing unit displays the images captured by each camera. In addition to showing the camera's position and its field of view (the angle of view the camera captures) within the image, the camera's position and its field of view are also indicated within the image. It has the function of displaying the area being captured. For example, a 360-degree panoramic camera, a 360-degree hemisphere camera. Depending on the camera's functions, such as the camera or wide-angle camera, the camera's field of view is displayed within the image. Users can intuitively understand which area each camera is capturing.

[0317] [Example 18] Next, the 18th embodiment will be described. In the 18th embodiment, the camera mounted on the vehicle is It is a 360-degree camera with an image circle smaller than the length of the diagonal of the imaging plane, and The science department, as a function of image analysis, has identified basic patterns for calculating image features. It has the function of deforming according to its position within the circle.

[0318] The processing unit uses the basic pattern without correcting the distortion of the image within the image circle. It has a function to calculate image features. This reduces the load on the processing unit. The basic pattern is to obtain an image by capturing it using a standard lens (a typical wide-angle lens). It is defined for. Projection methods for obtaining the image circle include, for example, orthogonal projection and equidistant projection. There are projection methods such as shadow projection, stereographic projection, and equisolid angle projection, and the processing unit processes these projection methods accordingly. ...and modify the basic pattern. As a camera lens, a circle is used to obtain the image circle. A fisheye lens is used. The camera's imaging surface is rectangular, and the diameter of the image circle is long. It is shorter than the side, and equal to or shorter than the shorter side.

[0319] As a basic pattern, the template pattern of the image to be detected is pre-registered in the processing unit. The processing unit deforms the template pattern according to its position within the image circle. In addition, the processing unit has a function that performs image recognition using Harlike features, for example. Each of the multiple rectangular patterns used to obtain the Harlike features is placed within the image circle. It has the function of deforming according to its position.

[0320] In a modified example of the 18th embodiment, the processing unit, when performing image recognition, uses an image circle. It has a function to limit the area within the frame to the area in which the specific object to be detected is captured, Image recognition is performed within the selected area. This function is implemented in the dashcam. Generally, the recommended installation location for dashcams is predetermined, so the image circle Within the image, areas where the driver is visible, areas where traffic lights are visible, areas where people in the back seat are visible, etc. It's pretty much decided. Therefore, depending on the person or object to be detected, the system will detect that person or object. To do this, the area in which image analysis is performed is limited to the area in which the person or object is captured. For example, a car When analyzing both forward-facing images, the processing unit analyzes the area where the windshield is reflected. Perform the analysis.

[0321] In another modification of the 18th embodiment, the processing unit flattens a specific area of ​​the image circle. Convert to a planar image (parallel projection image), and then use the usual basic patterns on the resulting planar image. Then, matching is performed.

[0322] In yet another variation of the 18th embodiment, the processing unit processes the image within the image circle. A predetermined transformation function is used to map the image onto a sphere, and a portion of the image mapped onto the sphere is then applied to the sphere. Zoom in and perform image analysis as a flat image, ignoring distortion. For specific objects, such as road signs... To detect something, zoom in on the area where the specific object is visible and perform image analysis. If you want to detect it, zoom in on the area of ​​the forward image where the road sign is visible and perform image analysis. conduct. [19th Example] Next, the 19th embodiment will be described. In the 19th embodiment, the processing unit has an image analysis function. Therefore, from the perspective of protecting privacy, areas of the image that should not be made public should be designated as privacy It has the function to identify protected areas, and as a process corresponding to the analysis results, it protects privacy. Images are captured by a camera in which it is not possible to identify people within the area based on their appearance. The process of generating data from the provided image is executed.

[0323] By making it impossible to identify a person, the privacy of that person is protected. This is possible. The processing unit has the function of distinguishing between areas where the car interior is visible and areas that should not be made public. In particular, when acquiring images using a 360-degree camera, the interior of the car will be captured in the image, There may be cases where you don't want to keep the image. In such cases, the processing unit will black out the area inside the car. A function to blur the interior of the car, or a function to blur the interior while recording the exterior image as is. The processing unit performs image analysis to distinguish between areas showing the interior and exterior of the vehicle. It has a function to distinguish between areas. For example, the processing unit performs opti at a speed equivalent to the vehicle speed. It has a function to determine that the area where calorie flow is occurring is outside the vehicle. The processing unit allows the user to privacy It has a function that allows you to pre-set the sea protection area.

[0324] In a modified example of the 19th embodiment, the processing unit identifies a person within the privacy protection area. Normal mode records images that can be captured, and privacy mode records images that capture people in the privacy-protected area. A privacy mode that records images in which objects cannot be identified by their appearance, and a user It has a function that allows users to select according to the purpose and usage of the vehicle. You can switch between normal mode and privacy mode.

[0325] According to another modification of the 19th embodiment, the processing unit processes the area in which a person is pictured to protect privacy. It has the function of defining a region. The processing unit detects people by performing image analysis, and the detected It has a function that designates the area enclosed by the outline of a person as a privacy area. Furthermore, the processing unit, It has a function that designates only the face portion of the detected person as a privacy area. It has a function to display people as silhouettes. Alternatively, the processing unit displays only the outline of the person. It has the function of displaying an image when there is no person inside the outline. It appears transparent. The processing unit has the function of displaying the edges of moving objects as the outline of a person. It has.

[0326] As yet another modification of the 19th embodiment, the processing unit is applied to the area where the person is located. It has a function to display an avatar instead. The displayed avatar will resemble the original person. For example, it displays avatars that reflect the user's characteristics, such as gender, body type, and facial features. The processing unit first associates a person with an avatar, then identifies the person in the image. It has the function to display the avatar associated with the identified person. The avatar is, A pre-created sequence is created for each person and registered in the processing unit. The processing unit adjusts to the person's movements. It has the function to move the avatar. For example, when a person raises their hand, the avatar's hand will also be raised. The processing unit responds to the person's facial expression. And it has the function to change the avatar's facial expression. For example, the processing unit will change the expression of the person when they are smiling. When it detects expressions such as sadness or anger, it changes the avatar's expression to smile. It has the function to change facial expressions to a cheerful, sad, or angry expression. The processing unit is responsible for the person's It has the ability to read emotions and change the avatar's color according to the person's emotions. For example, a person When an object is detected as angry, the avatar's color will change to a color that represents anger, such as red. Change it.

[0327] As yet another modification of the 19th embodiment, the processing unit takes a picture with a camera before recording the image. From the captured image, individuals appearing within the privacy-protected area are identified based on their appearance. It has the function of generating an image that cannot be reproduced and recording the generated image. Furthermore, the processing unit When playing back images captured by the camera, people who appear in the privacy protection area may be detected. It has a function that converts and displays objects into images that cannot be identified by their appearance.

[0328] [Example 20] Next, the 20th embodiment will be described. In the 20th embodiment, the processing unit corresponds to the analysis results. The process involves converting the results of image analysis into text data or encoding them for recording. Execute.

[0329] By converting the recorded image into text data or encoding it, the desired image can be obtained from the recorded image. It becomes possible to find them easily. For example, when the processing unit detects a person in the image, As text data, "person detection" is used, and when a red light is detected in the image, text data is used. The processing unit generates a "red light". The processing unit then processes the generated text data into the text data. It has a function to record the generated image in association with the original image. Furthermore, in other variations, the processing unit Instead of text data, image analysis results are associated with codes, and the image analysis results are coded It has the function to convert data into binary data.

[0330] [Example 21] Next, the 21st embodiment will be described. In the 21st embodiment, the processing unit has an image analysis function. It has a function to detect areas of black crushing within an image and processes the results accordingly. This process involves brightening and restoring the blacked-out areas of the image and displaying them on the screen. do.

[0331] Users viewing the image will find it easier to recognize objects and people within the blacked-out areas. In particular, all When acquiring images using a spherical camera, the image should contain both a bright exterior view and a dark interior view. Because images are captured simultaneously, areas where the interior of the car is visible tend to appear as blacked-out black areas. The process of detecting an area, brightening it, and displaying it uses a 360-degree camera to acquire images. This is particularly effective in the following cases. As a process to restore brightness, the processing unit performs, for example, gamma correction and color Performs corrections, conversion to black and white, contrast enhancement, etc.

[0332] [Example 22] Next, the 22nd embodiment will be described. In the 22nd embodiment, the processing unit provides the user with a turtle A function to divide the imaging range captured by the camera into multiple sections, and to weight each section. It has the function of determining imaging conditions based on the brightness of each section and the weight assigned to that section. It has the function of determining and instructing the camera to take images under the determined imaging conditions.

[0333] Users can increase the weight of areas they want to see within the image and decrease the weight of areas they don't need to see. This allows the user to acquire images under imaging conditions suitable for the area they want to view. Dividing the imaging area into arbitrary shapes according to the shape of the areas you want to view and the areas that do not need to be viewed. It is possible.

[0334] Figure 5 illustrates the method of dividing the imaging area into multiple sections using the comparative example method. A diagram of the image plane is shown. Images are projected onto the rectangular or square imaging plane 71 of the forward-facing 360-degree camera. A circle 73 is formed on the rectangular or square imaging surface 72 of the rear-facing 360-degree camera. An image circle 74 is formed. Within the image circle 73 that images the area in front, a fluorocarbon The view ahead is reflected through the window, as well as the dashboard and instrument panel inside the car. The rear view is visible, including the instrument panel and the display screen of the car navigation system. Within the image circle 74, the driver, passenger seat, rear seats, and ceiling of the car are captured. Furthermore, the scenery to the side and rear of the vehicle is reflected through the side windows and rear window. It is.

[0335] The imaging surfaces 71 and 72 are divided into multiple sections by a square grid. The user can select a section Weights can be assigned to each element.

[0336] Figure 6 shows a method for dividing the imaging range into multiple sections in the system according to the 22nd embodiment. A diagram of the imaging surface for explanation is shown. Similar to the example shown in Figure 5, a 360-degree camera for forward use An image circle 73 is formed on the imaging surface 71, and on the imaging surface 72 of the rear 360-degree camera. Image circle 74 is formed.

[0337] When the user is asked to divide the area, the processing unit 51 (Figure 4) creates a closed curve 7 in a portion of the imaging range. The area enclosed by the closed curve 75, enclosed by 5, is set as a single partition. The user, Multiple sections are defined by drawing multiple closed curves 75. The user can define the area enclosed by the closed curves. When the inner region is divided into two by a curve, each of the divided regions becomes a single section.

[0338] As shown in the comparative example in Figure 5, the sections are uniformly divided into a square grid pattern regardless of the image content. Compared to weighting by other factors, it becomes possible to perform weighting that is more appropriate for the user. ru.

[0339] In particular, when imaging with a 360-degree camera, the vehicle type is more accurately represented compared to when using a regular camera. Each time, the difference in the position of the vehicle within the imaging range becomes larger. Therefore, processing is performed in advance. The general weighting table set in the firmware of the unit and the actual expected imaging range The likelihood of discrepancies in the relationship with the plot increases. Using a general-purpose weighting table would be better. There is no function to determine weighting using PC browser software, etc., 360 degrees This effect is particularly noticeable when using a camera.

[0340] The processing unit determines imaging conditions such as gain and exposure time. Two hemispherical cameras In a configuration where the backs of two cameras are facing each other to form a 360-degree camera, the processing unit consists of two hemispherical cameras. The imaging conditions are made identical. This allows the images captured by the two hemispherical cameras to be combined. Sometimes, this can produce the effect of making the boundary between two images less distinct.

[0341] In a modified example of the 22nd embodiment, the processing unit (for example, AI software on a PC) performs weighting. Without this information, the camera analyzes the captured image to determine what is captured in which part of the image area. It has an estimation function. For example, the processing unit assigns a weight of "5" to the road part and a weight to the face part. The number "3" is assigned to the instrument panel area, and the area outside the circular image is weighted "1". Assign a weight of "0".

[0342] The processing means, when having the user divide the area, encloses a portion of the imaging range with a closed curve, and close The area enclosed by the curve can be considered as one section. The processing means is for each set section. It would be good to have the user input the weight as a numerical value. The numerical value to be entered should be in a dropdown menu or a pop-up menu. It would be best to allow users to select from a list of options such as app upgrades.

[0343] In another modification of the 22nd embodiment, the processing unit, instead of having a function to input weights numerically, Alternatively, in addition to the function to input numerical values, it also has a function to input the type of plot. Type of Plot Each element has a weight assigned to it in advance, and when the type of section is input, the processing unit will determine the type The weights associated with the section are then associated with the section in question. The type of section can be, for example, driving. The types of parts include the driver, meter panel, car navigation screen, windshield, rear seats, passenger seat, etc. It is provided. The processing unit has a function that allows the user to change the correspondence between the type of partition and its weight. It can hold. A list of partition types can be pre-registered by the user in the processing unit.

[0344] In yet another variation of the 22nd embodiment, the processing unit allows the user to weight each section. One of its functions is the ability to define a section by specifying the area you want to view within the imaging range. This function increases the weight of the area specified by the user that they want to view compared to the weights of other areas. The function allows the user to input the type of partition, and the partition is selected based on the partition the user wants to view. This is an example of dividing a section into two categories: one for this section and one for other sections.

[0345] In yet another modification of the 22nd embodiment, the processing unit is interactive (wizard-style), It has a function to divide the imaging area into sections. The processing unit says, "Please surround the area you do not want to use." It outputs a message such as "." or "Please fill in the unnecessary areas." The area surrounded or filled in by "Za" is recognized as "unnecessary area". The processing unit then instructs the user to "fill in the driver's area" and "fill in the road area." It outputs the message "Please do so," and the filled area is the section with the driver type. It recognizes it as a plot or a section of road. The processing unit performs an operation to enclose it with a closed curve, Alternatively, the user can perform a fill operation using a pointing device such as a touch panel or mouse. It has the function of using and performing actions.

[0346] In yet another modification of the 22nd embodiment, the processing unit determines the imaging range based on the captured image. The function of dividing the image into multiple sections is realized by artificial intelligence (AI). The processing unit uses the image to... It automatically detects the interior and exterior of the vehicle, seats, dashboard, instrument panel, etc., and automatically determines It has a function to divide the imaging area for each separate component. The processing unit displays the results of the division. The classification is displayed on the surface, and the user is asked to determine if this classification is correct. The processing unit then corrects the classification for the user. It has a function to allow the user to make a classification. If the user determines that the classification made by the AI ​​is incorrect, the user can make a classification. The minutes can be corrected. The processing unit corrects the value from a pull-down or pop-up list. It has a function that allows the user to select the type. The weight or type corresponding to the section is incorrect. If the user determines this, they can easily correct the weight or type.

[0347] [Example 23] Next, the 23rd embodiment will be described. In the 23rd embodiment, the processing unit provides the user with a turtle A function to divide the imaging range captured by the camera into multiple sections, and to weight each section. It has the function of performing a process corresponding to the analysis results, and when compressing sections with relatively small weights, The process of compressing by making the lock larger than the block size when compressing sections with relatively large weights. Execute this.

[0348] Users can increase the weight of high-priority parcels and decrease the weight of low-priority parcels. This allows for increased compression of less important sections. As a result, the entire image can be compressed uniformly. Compared to compression, this method maintains high image quality for high-priority sections of the image data. The capacity can be reduced. For example, in the heavy weight section, the processing unit uses 8x8 pixels. Each block consists of 16x16 pixels, and in lighter weighted sections, each block consists of 16x16 pixels. Then perform compression.

[0349] In the modified version of the 23rd embodiment, the processing unit is painted black in the area outside the image circle. It has a crushing function. The area outside the image circle formed on the rectangular imaging surface of the camera. The image may include the lens barrel. By blacking out unnecessary images, the image data including pixels outside the image circle is removed. The capacity can be reduced. In other variations, the processing unit is located at the image circle. By raster scanning only the pixels inside the image circle, excluding the outer pixels, the image data is created. It has a function to acquire data. This can reduce the size of the image data.

[0350] [Example 24] Next, the 24th embodiment will be described. In the 24th embodiment, the processing unit has an image analysis function. It has the function of detecting following vehicles and the function of detecting the distance to following vehicles, and the analysis results As a corresponding process, the system detects aggressive driving by the following vehicle based on the distance to that vehicle. Execute the process.

[0351] The processing unit executes processing according to the judgment result to take appropriate action against aggressive driving. This makes it possible to perform the following actions. For example, the processing unit will determine if the vehicle speed is above a predetermined reference value. Nevertheless, the same following vehicle continues to be present within a predetermined detection distance for a predetermined detection time or longer. When this happens, the system determines that the following vehicle is engaging in aggressive driving.

[0352] In a modified example of the 24th embodiment, the camera measures the distance to the object for each pixel. A device that has the function of doing so is used. For example, as the image sensor of a camera, a back-illuminated time sensor is used. An obflight-type distance image sensor is used.

[0353] In another modification of the 24th embodiment, the processing unit detects when a following vehicle is tailgating. When this is determined, the appropriate action based on the determination result may be taken, for example, notifying the driver or recording the rear view image. This involves recording incidents, providing advice to drivers, and reporting to traffic enforcement agencies (e.g., the police). When reporting to the enforcement agency, the processing unit will identify the vehicle and its driver that are engaging in aggressive driving. The image is attached. The processing unit analyzes the image to determine if the vehicle engaging in aggressive driving is the vehicle itself. If it detects that you have overtaken it and moved in front of its vehicle, it will harass you by slamming on the brakes. The driver will be notified of the possibility.

[0354] In yet another modification of the 24th embodiment, the processing unit detects people and vehicles around the vehicle. It has a function to control the vehicle according to the distance to people and vehicles. For example, the processing unit has a function to control the vehicle according to the distance to people and vehicles. The results of determining the car's speed, the distance to the person ahead, and whether or not the person is on the car's planned route. Based on this, if it is determined that there is a high probability of contact with a person, the system will take control to stop the vehicle.

[0355] The scope of this invention is not limited to the configurations explicitly described in this specification, but is not limited to those described in this specification. This also includes combinations of various aspects of the present invention disclosed in this book. Of the details, the configuration for which patent protection is sought is specified in the attached claims, but the current situation Even configurations not specified in the claims, the configurations disclosed herein shall We intend to include this in the scope of the patent claims.

[0356] The present invention is not limited to the configuration described in the embodiments described above. The components of the modified form can be arbitrarily selected and combined. Any component of the example and any component or invention described in the means for solving the invention Any component described in the means for solving the problem can be combined with any component that embodies that component. It is best to structure them accordingly. Rights will also be obtained in the amendment or divisional application of this application. They have the intention to do so.

[0357] Furthermore, by filing an application to amend to a design application, it is possible to acquire rights for the entire design or a partial design. The drawing depicts the entire device with solid lines, but it shows not only the overall design but also the device itself. The drawing also includes partial designs claimed for a portion of the device. This includes not only making components into partial designs, but also making parts of the device into partial designs, regardless of the components themselves. These are drawings that include a divided design. The parts of the device include some components of the device. It is acceptable to do so, or to use it as a part of the component. This applies to the overall design as well as the solid lines in the drawing. I intend to obtain a patent for a partial design in which any part of the design is represented by a dashed line. [Explanation of symbols]

[0358] 1. Dashcam 2 Windshield 3. Rearview mirror 4 Power Cables 5. Cigarette lighter socket 10. Dashcam unit 11 Cameras 12 GPS receivers 13 SD card reader 14. Accelerometer 15 speakers 16 displays 17 Operation buttons 20 Processing Units 21 Communication equipment 30 servers 31 Processing Unit 32 Communication equipment 33 Input device 34 displays 40 Communication Networks 50 External equipment 52 SD card reader 53 Input device 54 displays 60 SD cards 71, 72 Imaging plane 73, 74 Image Circle 75 Closed curves

Claims

1. A system having a processing means that performs image analysis on images captured by a camera mounted on a vehicle, and then performs processing corresponding to the analysis results.

2. The processing means includes, as an image analysis function, a function to detect animals and people around the vehicle before starting. As a process corresponding to the aforementioned analysis results, if animals or people are detected around the vehicle, the system will notify the driver that there are animals or people around the vehicle. The system according to feature 1.

3. The processing means includes, as an image analysis function, a function to detect that the vehicle is moving slowly forward from the flow of the surrounding scenery. As a process corresponding to the aforementioned analysis results, if it detects that the vehicle is moving slowly forward, an alarm will be issued. The system according to claim 1 or 2, characterized in that it is the same as described in claim 1 or 2.

4. The processing means includes a function to determine whether the preceding vehicle is a large vehicle as part of the image analysis function. The system according to any one of claims 1 to 3.

5. The processing means includes, as an image analysis function, a function to determine whether the road ahead is on a slope, and if it is on a slope, a function to detect the degree of incline. As a response to the aforementioned analysis results, the system will issue warnings such as excessive speed, reduced speed, or switching to hill climb mode, depending on the degree of the slope. The system according to any one of claims 1 to 4.

6. The vehicle is for handling cargo. The processing means includes, as an image analysis function, a function to recognize a string of characters, a barcode, or other markings displayed on the package from the captured image, and to determine whether or not the package is to be handled by the vehicle. As a process corresponding to the aforementioned analysis results, if it detects that the vehicle is attempting to load cargo that it is not supposed to handle, it will execute a process to issue an alarm. The system according to any one of claims 1 to 5.

7. The processing means has the function of detecting window regions in the image from the optical flow. The system according to any one of claims 1 to 6.

8. The processing means has the function of determining from the optical flow whether a vehicle has entered the tunnel, exited the tunnel, or is currently traveling inside the tunnel. The system according to feature 7.

9. The processing means has the function of determining from the optical flow whether or not the vehicle is traveling under an elevated structure. The system according to feature 7 or 8.

10. The processing means has the function of determining the presence or absence of guardrails and whether caution is needed regarding side ditches based on optical flow. The system according to any one of 7 to 9, characterized by the features described herein.

11. A program for a computer to implement the functions of the processing means described in any one of claims 1 to 10.

Images