Information processing device, driving diagnostic method and computer-readable non-transitory recording medium

Abstract

This invention provides an information processing apparatus, a driving diagnostic method, and a computer-readable non-transitory recording medium. The information processing apparatus includes: an acquisition unit that acquires driving information associated with vehicle driving and information on whether a target is identified from images captured by a camera mounted on the vehicle; a preprocessing unit that, if the information acquired within a certain interval including a certain time period includes information indicating that the target has been identified, updates the information to indicate that the target has been identified even if the information at that time indicates that the target has not been identified; and a diagnostic unit that, at each time period, diagnoses an event that should be noted as having occurred if information indicating that the target has been identified exists and detection conditions are met, the detection conditions being based on the shape of the target and the driving information.

Description

Technical Field

[0001] This invention relates to information processing devices, driving diagnostic methods and procedures. Background Technology

[0002] Japanese Patent Application Publication No. 2021-118002 discloses a method for determining vehicle distance based on vehicle images. This method includes: obtaining a driving image captured by a camera of a moving first vehicle; detecting a second vehicle from the obtained driving image; if the second vehicle is not detected from the driving image, detecting a first feature point of a second vehicle region from a first frame corresponding to the frame in which the second vehicle was detected, preceding the frame in which the second vehicle was not detected among a plurality of frames constituting the driving image; detecting a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point; calculating a feature point change value between the first and second feature points; and calculating the distance from the camera of the first vehicle to the second vehicle based on the calculated feature point change value. Summary of the Invention

[0003] In the past, when using external servers or other devices for driver diagnostics, it was necessary to send image data from the vehicle to the server, which placed a heavy burden on the communication circuitry.

[0004] The purpose of this invention is to provide an information processing device, driving diagnosis method, and program capable of suppressing communication traffic when performing driving diagnosis outside the vehicle.

[0005] The information processing apparatus of technical solution 1 includes: an acquisition unit that acquires driving information associated with the driving of a vehicle and information on whether a target is identified from an image captured by a camera mounted on the vehicle; a preprocessing unit that, when the information acquired within a certain interval including a certain time includes information indicating that the target has been identified, updates the information to indicate that the target has been identified, even if the information at that time indicates that the target has not been identified; and a diagnosis unit that, at each time, diagnoses an event that should be noted as having occurred if information indicating that the target has been identified exists and detection conditions are met, the detection conditions being based on the shape of the target and the driving information.

[0006] In the information processing apparatus described in technical solution 1, the acquisition unit acquires driving information related to the vehicle's movement and information on whether a target has been identified from an image captured by a camera mounted on the vehicle. Here, the target may be, for example, a vehicle ahead or a traffic signal. If the preprocessing unit acquires information indicating that the target has been identified within a certain interval including a certain time, it updates the information to indicate that the target has been identified, even if the information at that time indicates that the target has not been identified. Regarding each time interval, if information indicating that the target has been identified exists and detection conditions are met, the diagnosis unit diagnoses an event that should be noted as having occurred, the detection conditions being based on the shape of the target and the driving information. Here, events that should be noted include, for example, shortening the inter-vehicle distance, significantly delaying the start relative to the vehicle ahead, or failing to stop despite a red light. The conditions based on the target's shape and driving information are conditions specified using the target's shape and driving information for diagnosing an event that should be noted as having occurred.

[0007] This information processing device acquires driving information related to the vehicle's movement and information on whether a target has been identified from images captured by a camera mounted on the vehicle. Therefore, according to this information processing device, communication volume can be suppressed when performing driving diagnostics outside the vehicle.

[0008] The information processing device described in technical solution 2 is based on the information processing device described in technical solution 1, wherein the information processing device further includes an extraction unit, which extracts the captured image based on the condition that the detection conditions are met.

[0009] In the information processing apparatus described in technical solution 2, the captured image is extracted based on the condition that the detection conditions are met. According to this information processing apparatus, it is possible to record captured images when an event that should be noted occurs.

[0010] The information processing apparatus of technical solution 3 is based on the information processing apparatus of technical solution 2, wherein the extraction unit terminates the extraction of the captured image when the detection conditions are no longer met.

[0011] In the information processing apparatus described in technical solution 3, the extraction of the captured image is terminated when the detection conditions are no longer met. According to this information processing apparatus, the extraction of the captured image can be appropriately terminated when an event that should be noted has been diagnosed.

[0012] The information processing device described in technical solution 4 is based on the information processing device described in technical solution 2 or 3, wherein the extraction unit deletes the extracted captured image when the specified detection release condition based on the driving information is met.

[0013] In the information processing apparatus described in technical solution 4, the extraction unit deletes the extracted captured image when a predetermined detection release condition based on the driving information is met. According to this information processing apparatus, the predetermined detection release condition is a condition for deleting a diagnosis that an event that should have been noted had occurred, and then diagnosing that the event had not occurred. According to this information processing apparatus, when an event that should have been noted was temporarily diagnosed as having occurred, and then is later diagnosed as not having occurred, the extracted captured image can be deleted.

[0014] In the driving diagnosis method described in technical solution 5, the computer performs the following processing: obtaining driving information associated with the vehicle's driving and information on whether a target is identified from images captured by a camera mounted on the vehicle; if the information obtained within a certain interval including a certain moment includes information indicating that the target has been identified, even if the information at that moment indicates that the target has not been identified, it is updated to indicate that the target has been identified; regarding each moment, if there is information indicating that the target has been identified and detection conditions are met, an event that should be noted has occurred, the detection conditions being based on the shape of the target and the driving information.

[0015] In the driving diagnosis method described in technical solution 5, the computer acquires driving information associated with the vehicle's movement and information on whether a target has been identified from images captured by a camera mounted on the vehicle. If the information acquired by the computer within a certain interval including a certain moment includes information indicating that the target has been identified, even if information indicating that the target was not identified at that moment, the computer updates it to indicate that the target has been identified. Regarding each moment, if information indicating that the target has been identified exists and detection conditions are met, the computer diagnoses an event that should be noted as having occurred, the detection conditions being based on the target's shape and the driving information.

[0016] This driving diagnostic method acquires driving information related to the vehicle's movement and information on whether a target is identified from images captured by a camera mounted on the vehicle. Therefore, this driving diagnostic method can suppress communication overhead when performing driving diagnostics outside the vehicle.

[0017] The program described in technical solution 6 causes a computer to perform the following processing: acquiring driving information associated with the vehicle's movement and information on whether a target is identified from images captured by a camera mounted on the vehicle; if the information acquired within a certain interval including a certain time includes information indicating that the target has been identified, even if the information at that time indicates that the target has not been identified, it is updated to indicate that the target has been identified; and regarding each time, if there is information indicating that the target has been identified and detection conditions are met, an event that should be noted has occurred, the detection conditions being based on the shape of the target and the driving information.

[0018] The computer executing the program described in technical solution 6 acquires driving information associated with the vehicle's movement and information on whether a target has been identified from images captured by a camera mounted on the vehicle. If the information acquired by the computer within a certain interval including a certain moment includes information indicating that the target has been identified, it updates the information to indicate that the target has been identified, even if the information at that moment indicates that the target has not been identified. Regarding each moment, if information indicating that the target has been identified exists and detection conditions are met, the computer diagnoses an event that should be noted as having occurred, the detection conditions being based on the target's shape and the driving information.

[0019] This program obtains driving information related to the vehicle's movement and information on whether a target is identified from images captured by a camera mounted on the vehicle. Therefore, this program can suppress communication overhead when performing driving diagnostics outside the vehicle.

[0020] According to the present invention, it is possible to suppress the amount of communication when performing driving diagnostics outside the vehicle. Attached Figure Description

[0021] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which the same reference numerals denote the same elements, wherein:

[0022] Figure 1 This is a diagram showing the general structure of the driving diagnostic system according to this embodiment.

[0023] Figure 2 This is a block diagram illustrating the hardware structure of the vehicle according to this embodiment.

[0024] Figure 3 This is a block diagram showing the structure of the ROM of the vehicle-mounted device according to this embodiment.

[0025] Figure 4 This is a block diagram illustrating the structure of the storage device of the vehicle-mounted device according to this embodiment.

[0026] Figure 5 This is a block diagram illustrating the functional structure of the vehicle-mounted device according to this embodiment.

[0027] Figure 6 This is a block diagram illustrating the hardware structure of the central server in this embodiment.

[0028] Figure 7 This is a block diagram illustrating the functional structure of the central server in this embodiment.

[0029] Figure 8 This is a flowchart illustrating the process of target information transmission processing performed in the vehicle-mounted device according to this embodiment.

[0030] Figure 9 This is a flowchart illustrating the process of driving diagnostics performed in the central server of this embodiment.

[0031] Figure 10 This is a flowchart illustrating the process of driving diagnostics performed in the central server of this embodiment. Detailed Implementation

[0032] A driving diagnostic system incorporating the information processing apparatus of the present invention will be described. The driving diagnostic system is a system that performs driving diagnostics using information about targets identified in images captured by a camera mounted on the vehicle and vehicle driving information. Furthermore, the driving diagnostic system records images captured when an event is determined to have occurred that requires attention.

[0033] (Overall structure)

[0034] like Figure 1 As shown, the driving diagnostic system 10 of this embodiment is configured to include a vehicle 12 and a central server 30 as an information processing device. Additionally, a vehicle-mounted unit 20 is mounted on the vehicle 12. The vehicle-mounted unit 20 and the central server 30 are interconnected via a network N. It should be noted that... Figure 1 The diagram shows one vehicle 12 and vehicle-mounted device 20 relative to one central server 30, but the number of vehicles 12 and vehicle-mounted devices 20 is not limited to this.

[0035] Central server 30 may be located, for example, at the manufacturer, car dealership, or any business that manufactures vehicle 12.

[0036] (vehicle)

[0037] like Figure 2 As shown, the vehicle 12 involved in this embodiment is configured to include an onboard unit 20, multiple ECUs (Electronic Control Units) 22 and multiple onboard devices 24.

[0038] The vehicle-mounted unit 20 is configured to include a CPU (Central Processing Unit) 20A, a ROM (Read Only Memory) 20B, a RAM (Random Access Memory) 20C, a storage device 20D, a wireless communication I / F (Interface) 20E, and an in-vehicle communication I / F 20F. The CPU 20A, ROM 20B, RAM 20C, storage device 20D, wireless communication I / F 20E, and in-vehicle communication I / F 20F are interconnected via an internal bus 20G in a manner that enables them to communicate with each other.

[0039] CPU 20A is the central processing unit, which executes various programs and controls various parts. That is, CPU 20A reads programs from ROM 20B or storage device 20D and uses RAM 20C as the working area to execute the programs.

[0040] ROM20B stores various programs and data. For example... Figure 3 As shown, in this embodiment, ROM 20B stores a control program 100. The control program 100 is a program for sending information about a target identified from an image captured by the camera to a central server 30.

[0041] like Figure 2 As shown, RAM20C temporarily stores programs or data as a work area.

[0042] The storage device 20D, serving as a memory, is composed of HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data. For example... Figure 4 As shown, the storage device 20D in this embodiment stores a data set containing information about the identified target, namely, target information DB (database) 120. It should be noted that the storage device 20D may also store the control program 100 and the target information DB 120.

[0043] like Figure 2 As shown, the wireless communication I / F20E is a wireless communication module used for communication with the central server 30. This wireless communication module uses communication standards such as 5G, LTE, and Wi-Fi (registered trademark). The wireless communication I / F20E is connected to the network N.

[0044] The in-vehicle communication I / F20F is an interface used to connect to each ECU 22. This interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F20F is connected to the external bus 20H.

[0045] ECU22 includes at least ADAS (Advanced Driver Assistance System) - ECU22A.

[0046] ADAS-ECU22A controls the advanced driver assistance system. Connected to ADAS-ECU22A are a vehicle speed sensor 24A and a camera 24C, which constitute the on-board equipment 24. Camera 24C is a camera that captures images of the surroundings of the vehicle 12.

[0047] like Figure 5 As shown, in the vehicle-mounted device 20 of this embodiment, the CPU 20A functions as the acquisition unit 200, the processing unit 210, and the transmission unit 220 by executing the control program 100.

[0048] The acquisition unit 200 has the function of acquiring vehicle information and capturing images. The vehicle information includes the status of the on-board device 24 and the status of the vehicle 12 obtained from the on-board device 24. The captured images are images of the exterior of the vehicle 12 captured by the camera 24C of the vehicle 12. In this embodiment, the vehicle information includes location information or driving information such as vehicle speed.

[0049] The processing unit 210 has the function of storing vehicle information acquired in the acquisition unit 200 to the storage device 20D and identifying targets from images captured by the camera 24C to generate target information. Specifically, in this embodiment, the processing unit 210 temporarily stores vehicle information and target information in the storage device 20D, and then transmits the vehicle information and target information to the central server 30 using the transmission unit 220. In addition, when requested by the central server 30, the processing unit 210 transmits captured images to the central server 30 using the transmission unit 220. Furthermore, the processing unit 210 generates target information from the captured images by the camera 24C through image recognition processing.

[0050] More specifically, the processing unit 210 identifies the signals of the vehicles ahead and traffic lights as the target represented by the captured image, and converts this information into information about the vehicles ahead (signs indicating the presence or absence of the vehicles ahead, the distance between the vehicles ahead, and the time between the vehicles ahead) and information about the traffic lights (signs indicating the presence or absence of red lights, signs indicating the presence or absence of green lights, and the distance between the vehicles and the traffic lights). The time between the vehicles is calculated based on the distance between the vehicles and the vehicle speed.

[0051] The sending unit 220 has the function of sending vehicle information and target information stored in the storage device 20D to the central server 30. Specifically, in this embodiment, the sending unit 220 sends vehicle information and target information stored in the storage device 20D to the central server 30 upon request from the central server 30.

[0052] In addition, when requested by the central server 30, the sending unit 220 sends the captured images by the camera 24C to the central server 30.

[0053] (Central Server)

[0054] like Figure 6 As shown, the central server 30 is configured to include a CPU 30A, ROM 30B, RAM 30C, storage device 30D, and communication I / F 30E. The CPU 30A, ROM 30B, RAM 30C, storage device 30D, and communication I / F 30E are interconnected via an internal bus 30G in a manner capable of communication. The functions of the CPU 30A, ROM 30B, RAM 30C, storage device 30D, and communication I / F 30E are the same as those of the CPU 20A, ROM 20B, RAM 20C, storage device 20D, and wireless communication I / F 20E of the vehicle-mounted device 20 described above. It should be noted that the communication I / F 30E can also perform wired communication. The CPU 30A is an example of a processor.

[0055] In this embodiment, the storage device 30D stores a processing program 150, target information DB160, and extracted image DB170. It should be noted that the processing program 150 may also be stored in the ROM 30B.

[0056] Processor 150 is a program used to control central server 30. Accompanying the execution of processor 150, central server 30 obtains target information and driving information from vehicle 12, and performs various processes for driving diagnosis on the driving information of vehicle 12.

[0057] The target information DB160 stores the target information obtained from vehicle 12.

[0058] Specifically, it stores information obtained from vehicle 12 about the vehicle ahead (indicating the presence or absence of the vehicle ahead, the distance between the two vehicles, and the time between the two vehicles), and information about the traffic signal (indicating the presence or absence of the red light signal, the presence or absence of the green light signal, and the distance between the two vehicles).

[0059] The extracted image DB170 contains images taken when an event that should be noted has occurred.

[0060] like Figure 7 As shown, in the central server 30 of this embodiment, the CPU 30A functions as the acquisition unit 260, the preprocessing unit 270, the diagnostic unit 280, and the extraction unit 290 by executing the processing program 150.

[0061] The acquisition unit 260 has the function of acquiring vehicle information of the vehicle 12 and target information from the vehicle-mounted unit 20 of the vehicle 12. The acquisition unit 260 stores the acquired target information in the target information DB160.

[0062] If the preprocessing unit 270 obtains information indicating that a target has been identified within a certain interval including a certain time, it updates the information to indicate that a target has been identified, even if the information at that time indicates that a target has not been identified.

[0063] Specifically, when the identified target is a vehicle ahead and the target information is a sign indicating the presence of a vehicle ahead, the preprocessing unit 270 records the vehicle ahead sign. Furthermore, to avoid false detection or non-detection of the vehicle ahead, a filtering process is performed to replace the sign indicating the presence of a vehicle ahead with the maximum value within a certain interval of 0.2 seconds before and after, thereby updating all signs within the certain interval with this maximum value.

[0064] Furthermore, when the detected target is a signal from a traffic light and the target information indicates a red light signal, the preprocessing unit 270 records the red light signal flag. Moreover, to avoid false detection or non-detection of red light signals, a filtering process is performed that replaces the flag indicating a red light signal with the maximum value of the preceding and following seconds within a certain interval, thereby updating all flags within that interval with this maximum value.

[0065] Furthermore, when the identified target is a signal from a traffic light and the target information is a green light signal, the preprocessing unit 270 records the green light signal flag. Moreover, to avoid false detection or non-detection of the green light signal, a filtering process is performed that replaces the flag indicating a green light signal with the maximum value of one second before and after a certain interval, thereby updating all flags within that interval with that maximum value.

[0066] Based on vehicle information and target information obtained from the vehicle-mounted unit 20 of vehicle 12, the diagnostic unit 280 diagnoses an event that should be noted as having occurred at any given time if the target information obtained from the vehicle-mounted unit 20 of vehicle 12 indicates that the target has been identified and the detection conditions are met. The detection conditions are based on the shape of the target and the driving information contained in the vehicle information obtained from the vehicle-mounted unit 20 of vehicle 12.

[0067] Specifically, the diagnostic unit 280 uses the shortening of the inter-vehicle distance while driving or parked as a detection condition to diagnose whether an event requiring attention has occurred.

[0068] For example, the detection conditions would be a sign indicating the presence of a vehicle ahead, vehicle 12 being in motion (speed > 10 km / h), and the time between the vehicle and the sign being less than 0.5 seconds. Alternatively, the detection conditions would be a sign indicating the presence of a vehicle ahead, vehicle 12 being stationary (speed < 0.5 km / h), and the distance between the vehicle and the sign being less than 1500 mm.

[0069] In addition, the diagnostic unit 280 will use a significant delay relative to the vehicle in front when starting at an intersection or in a traffic jam as a detection condition to diagnose whether an event that should be noted has occurred.

[0070] For example, if a sign indicates the presence of a vehicle ahead and the vehicle speed is <0.5km / h, a temporary diagnosis of an event requiring attention is made. If a vehicle ahead is present while the vehicle is stopped and the maximum inter-vehicle distance minus the minimum inter-vehicle distance is ≥25000mm, then an event requiring attention is diagnosed.

[0071] Additionally, if the parking time exceeds 120 seconds, it is considered a parked stop, and the temporary diagnosis of an event that should have been noted is deleted, with the diagnosis being no event that should have been noted. Similarly, if the distance to the vehicle in front is greater than 15000mm, the temporary diagnosis of an event that should have been noted is deleted, with the diagnosis being no event that should have been noted. Furthermore, when turning left or right (with the turn signal switch on), there is a high probability of a start delay; therefore, the temporary diagnosis of an event that should have been noted is deleted, with the diagnosis being no event that should have been noted.

[0072] In addition, the diagnostic unit 280 uses the fact that the vehicle did not stop despite a red light signal as a detection condition to diagnose whether an event requiring attention has occurred.

[0073] For example, if the indicator for a red light signal is active (ON), a temporary diagnosis is made that an event that should be noted has occurred. If the minimum vehicle speed is below the threshold (1 km / h), this temporary diagnosis is deleted, and the diagnosis is that no event that should be noted has occurred.

[0074] Additionally, if the turn signal switch is not turned on, it is considered that an arrow signal has appeared, and the temporary diagnosis that an event requiring attention has occurred is deleted, and the diagnosis is no longer considered an event requiring attention.

[0075] In addition, when the red light signal is consistently more than 10 meters away from the traffic signal controller, the temporary diagnosis that an event that should have been noted has been removed, and the diagnosis is changed to no event that should have been noted.

[0076] In addition, when the light immediately switches from red to green, the temporary diagnosis that an event that should have been noted has occurred is removed, and the diagnosis is changed to no event that should have been noted has occurred.

[0077] When the detection conditions are met, the extraction unit 290 requests an image to be captured from the vehicle-mounted device 20 of the vehicle 12, extracts the captured image from the vehicle-mounted device 20 of the vehicle 12, and stores it in the extracted image DB170.

[0078] In addition, the extraction unit 290 ends the extraction of the captured image obtained from the vehicle-mounted device 20 of the vehicle 12 when the detection conditions are no longer met, and stores the extracted captured image in the extracted image DB170.

[0079] In addition, the extraction unit 290 deletes the extracted captured image if the detection release conditions specified based on the driving information are met.

[0080] Specifically, the extraction of images from the on-board unit 20 of vehicle 12 is initiated when either the detection condition of a sign indicating the presence of a vehicle ahead and vehicle 12 is moving (speed > 10 km / h) and the time between vehicles is < 0.5 seconds, or the detection condition of a sign indicating the presence of a vehicle ahead and vehicle 12 is stationary (speed < 0.5 km / h) and the distance between vehicles is < 1500 mm, and the extraction of images is terminated when the detection condition is no longer met. The extracted images are then stored in the extracted image DB170.

[0081] The extraction of images from the on-board unit 20 of vehicle 12 begins when the detection condition of a sign indicating the presence of a vehicle ahead and a vehicle speed of <0.5 km / h is met. If this detection condition is no longer met, the image extraction ends. Furthermore, if a vehicle ahead is present while the vehicle is parked and the maximum inter-vehicle distance minus the minimum inter-vehicle distance is ≥25000 mm, the extracted image is stored in the extracted image DB170.

[0082] In addition, a parking time of >120 seconds is used as a condition for disabling the detection. If the condition for disabling the detection is met, the extracted captured image will be deleted.

[0083] In addition, a distance of >15000mm between the vehicle in front is used as a condition for disabling the detection. If the condition for disabling the detection is met, the extracted captured image will be deleted.

[0084] In addition, the detection release condition will be when turning left or right (with the turn signal switch on). If the detection release condition is met, the extracted captured image will be deleted.

[0085] Furthermore, when the detection condition indicating that the red light signal is valid (ON) is met, the extraction of the captured image obtained from the vehicle-mounted device 20 of vehicle 12 begins. When the detection condition is no longer met, the extraction of the captured image ends, and the extracted captured image is stored in the extracted image DB170.

[0086] Furthermore, the minimum vehicle speed is set below a threshold (1 km / h) as the detection release condition. If the detection release condition is met, the extracted captured image will be deleted.

[0087] In addition, turning on the turn signal switch is used as the detection release condition. If the detection release condition is met, it is considered that an arrow signal has appeared, and the extracted captured image is deleted.

[0088] In addition, the detection release condition will be that the signal controller is always more than 10 meters away from the red light signal. If the detection release condition is met, the extracted captured image will be deleted.

[0089] In addition, the detection will be released by immediately switching from a red light signal to a green light signal. Once the detection release condition is met, the extracted captured image will be deleted.

[0090] (Control process)

[0091] Regarding the process flow performed by the driving diagnostic system 10 of this embodiment, using Figures 8-10 The process is illustrated using a flowchart. The processing in the vehicle-mounted unit 20 is performed by the CPU 20A of the vehicle-mounted unit 20, which functions as the acquisition unit 200, processing unit 210, and transmission unit 220. Similarly, the processing in the central server 30 is performed by the CPU 30A of the central server 30, which functions as the acquisition unit 260, preprocessing unit 270, diagnostic unit 280, and extraction unit 290.

[0092] First, while vehicle 12 is in motion, CPU 20A in onboard unit 20 repeatedly executes... Figure 8 The target information sending process is shown.

[0093] exist Figure 8 In step S100, the acquisition unit 200 acquires images of the exterior of the vehicle 12 captured by the camera 24C, and acquires driving information.

[0094] In step S102, the processing unit 210 identifies the target from the captured image by the camera 24C and converts it into target information, which is then temporarily stored in the target information DB120.

[0095] In step S104, the processing unit 210 determines whether a target has been identified through the processing in step S102. If a target has been identified through the processing in step S102, the process proceeds to step S106. On the other hand, if a target has not been identified through the processing in step S102, the target information transmission process ends.

[0096] In step S106, the sending unit 220 sends the target information obtained as the recognition result of step S102 along with the driving information to the central server 30, thus ending the target information sending process.

[0097] In addition, in the central server 30, CPU 30A repeatedly executes... Figures 9-10 The driving diagnostic process shown.

[0098] Specifically, the CPU30A repeatedly executes commands to diagnose events that are causing a decrease in workshop distance as events that warrant attention. Figure 9 The driving diagnostic process shown.

[0099] exist Figure 9 In step S110, the acquisition unit 260 obtains the driving information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.

[0100] In step S112, the acquisition unit 260 obtains target information from the vehicle-mounted device 20 of the vehicle 12.

[0101] In step S114, if the detected target is a vehicle ahead, the preprocessing unit 270 records the vehicle ahead sign if it indicates the presence of a vehicle ahead. Furthermore, to avoid false detection or failure to detect a vehicle ahead, a filtering process is performed to replace the sign indicating the presence of a vehicle ahead with the maximum value within a certain interval of 0.2 seconds before and after, thereby updating all signs within the certain interval with this maximum value.

[0102] In step S116, the diagnostic unit 280 uses the condition that the inter-vehicle distance is decreasing while driving or parked as a detection condition to diagnose whether an event requiring attention has occurred. If the detection condition is met, an event requiring attention is diagnosed, and the process moves to step S118. On the other hand, if the detection condition is not met, the process moves to step S120.

[0103] In step S118, the extraction unit 290 requests an image to be captured from the vehicle-mounted device 20 of the vehicle 12 and begins the extraction of the captured image obtained from the vehicle-mounted device 20 of the vehicle 12.

[0104] In step S120, the extraction unit 290 determines whether it is currently extracting a captured image. If it is, the process proceeds to step S122, ends the image extraction, and stores the extracted image in the extracted image DB170. Otherwise, if it is not, the driving diagnostic process ends.

[0105] In addition, the CPU30A repeatedly executes commands to diagnose situations where there is a significant delay relative to the vehicle in front during start-up, identifying these as events that warrant attention. Figure 10 The driving diagnostic process shown is as described above. It should be noted that, regarding the above... Figure 9 The same treatment is applied, with the same labels used for explanation.

[0106] exist Figure 10 In step S110, the acquisition unit 260 obtains the driving information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.

[0107] In step S112, the acquisition unit 260 obtains target information from the vehicle-mounted device 20 of the vehicle 12.

[0108] In step S114, if the identified target is a vehicle ahead, the preprocessing unit 270 records the vehicle ahead sign if it indicates the presence of a vehicle ahead. Furthermore, it performs a filtering process that replaces the signs indicating the presence of a vehicle ahead with the maximum value within a certain interval (0.2 seconds before and after), thereby updating all signs within that interval with this maximum value.

[0109] In step S116, the diagnostic unit 280 uses a significant delay relative to the vehicle in front during a start at an intersection or in congestion as a detection condition to diagnose whether an event requiring attention has occurred. If the detection condition is met, an event requiring attention is diagnosed, and the process moves to step S118. On the other hand, if the detection condition is not met, the process moves to step S120.

[0110] In step S118, the extraction unit 290 requests an image to be captured from the vehicle-mounted device 20 of the vehicle 12 and begins the extraction of the captured image obtained from the vehicle-mounted device 20 of the vehicle 12.

[0111] In step S120, the extraction unit 290 determines whether it is currently extracting a captured image. If it is, the process proceeds to step S122, ends the image extraction, and stores the extracted image in the extracted image DB170. Otherwise, if it is not, the driving diagnostic process ends.

[0112] In step S200, the extraction unit 290 determines whether the detection release conditions are met by using the following criteria: parking time > 120 seconds, distance to the vehicle in front > 15000mm, or turning left or right (turn signal switch on). If the detection release conditions are not met, the driving diagnostic process ends. On the other hand, if the detection release conditions are met, the process proceeds to step S202.

[0113] In step S202, the extraction unit 290 deletes the diagnosis of the incident that should be noted and the extracted captured image, and ends the driving diagnosis process.

[0114] In addition, the CPU30A repeatedly executes the same driving diagnostic processing used to diagnose situations where vehicles fail to stop despite a red light, as well as the aforementioned events that should be noted, similar to the processing used to diagnose significant delays relative to the vehicle in front during start-up. Figure 10 The driving diagnostic process shown.

[0115] Here, in step S114, if the detected target is a signal from a traffic light, the preprocessing unit 270 records the red light signal if it is a red light signal. Furthermore, by performing a filtering process that replaces the red light signal with the maximum value of one second before and after a certain interval, all signals within that interval are updated with that maximum value.

[0116] Furthermore, if the flag indicates a green light signal, the preprocessing unit 270 records the green light signal flag. Then, by performing a filtering process that replaces the flag indicating a green light signal with the maximum value of one second before and after a certain interval, all flags within that interval are updated with that maximum value.

[0117] In step S116, the diagnostic unit 280 uses the fact that the red light signal has not stopped as a detection condition to diagnose whether an event requiring attention has occurred. If the detection condition is met, an event requiring attention is diagnosed, and the process moves to step S118. On the other hand, if the detection condition is not met, the process moves to step S120.

[0118] In step S200, the extraction unit 290 determines whether the detection release conditions are met by using the following criteria: minimum vehicle speed below a threshold (1 km / h), turn signal switch on, maintaining a distance of more than 10 meters from the traffic signal controller during red light signal recognition, or immediate switch from red light to green light. If the detection release conditions are not met, the driving diagnostic process ends. On the other hand, if the detection release conditions are met, the process proceeds to step S202.

[0119] (Summary of Implementation Methods)

[0120] In this embodiment, the central server 30 acquires driving information associated with the vehicle's movement and information on whether a target is identified from images captured by a camera mounted on the vehicle. If detection conditions are met, it diagnoses an event requiring attention, based on the target's shape and the driving information. Therefore, according to the central server 30, there is no need to transmit captured images for driving diagnosis, thus reducing communication volume with the vehicle when driving diagnosis is performed outside the vehicle.

[0121] Furthermore, in this embodiment, the central server 30 extracts captured images when the detection conditions for an event that should be noted are met. Therefore, according to the central server 30, captured images can be recorded when an event that should be noted is diagnosed to have occurred.

[0122] Furthermore, in this embodiment, the central server 30 deletes the extracted captured image when the detection release conditions based on driving information are met. Therefore, according to the central server 30, when an event that should be noted has been temporarily diagnosed as having occurred, and then is diagnosed as not having occurred, the extracted captured image can be deleted.

[0123] [Note]

[0124] The various processes executed by CPU20A and CPU30A in the above embodiments can also be performed by various processors other than the CPU, which read the software (program). Examples of processors in this case include FPGAs (Field-Programmable Gate Arrays) and PLDs (Programmable Logic Devices) whose circuit structures can be modified after manufacturing, and ASICs (Application Specific Integrated Circuits) and other processors with circuit structures specifically designed for performing specific processes, i.e., dedicated circuits. Furthermore, the aforementioned processes can be performed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor elements.

[0125] Furthermore, in the above embodiments, each program is described as being pre-stored (installed) on a computer-readable non-transitory recording medium. For example, the control program 100 in the vehicle loader 20 is pre-stored in the ROM 20B, and the processing program 150 in the central server 30 is pre-stored in the storage device 30D. However, it is not limited to this; each program may also be provided as a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Additionally, the program may be configured to be downloaded from an external device via a network.

[0126] The processing flow described in the above embodiments is an example, and unnecessary steps can be deleted, new steps can be added, and the processing order can be changed without departing from the main idea.

Claims

1. An information processing apparatus, comprising: The acquisition unit acquires driving information related to the vehicle's movement and information on whether the traffic signal is identified as red from images captured by a camera mounted on the vehicle. The preprocessing unit, when the information obtained within a certain interval including a certain moment includes information indicating that the signal of the traffic light has been identified as red, updates the information to indicate that the signal of the traffic light has been identified as red, even if at that moment the information indicates that the signal of the traffic light has not been identified as red. The diagnostic department, at any given time, diagnoses an event that warrants attention as having occurred if there is information indicating that the signal of the traffic light is red and the detection conditions indicating that the signal of the traffic light is red are met. and The extraction unit begins extracting the captured image when the detection conditions are met, and ends extracting the captured image when the detection conditions are no longer met. The extraction unit uses the vehicle's minimum speed being below a threshold as a first detection release condition. If the first detection release condition is met, the extracted captured image is deleted. The extraction unit will use the left or right turn as the second detection release condition, and delete the extracted captured image when the second detection release condition is met.

2. A driving diagnostic method, wherein, The computer performs the following processing: Obtain driving information associated with the vehicle's movement and information on whether the traffic light signal is red is identified from images captured by a camera mounted on the vehicle. If the information obtained within a certain interval at a certain moment includes information indicating that the signal light of the traffic light has been detected as red, even if at that moment the information indicates that the signal light of the traffic light has not been detected as red, the information will still be updated to indicate that the signal light of the traffic light has been detected as red. At any given time, if there is information indicating that the signal light is red and the detection conditions for the signal light being red are met, an event requiring attention is diagnosed as having occurred. The extraction of the captured image begins when the detection conditions are met and ends when the extraction fails to meet the detection conditions. The first detection release condition is that the vehicle's minimum speed is below a threshold. If the first detection release condition is met, the extracted captured image is deleted. The second detection release condition will be when turning left or right. If the second detection release condition is met, the extracted captured image will be deleted.

3. A computer-readable non-transitory recording medium storing a program that causes a computer to perform the following processes: Obtain driving information associated with the vehicle's movement and information on whether the traffic light signal is red is identified from images captured by a camera mounted on the vehicle. If the information obtained within a certain interval at a certain moment includes information indicating that the signal light of the traffic light has been detected as red, even if at that moment the information indicates that the signal light of the traffic light has not been detected as red, the information will still be updated to indicate that the signal light of the traffic light has been detected as red. At any given time, if there is information indicating that the signal light is red and the detection conditions for the signal light being red are met, an event requiring attention is diagnosed as having occurred. The extraction of the captured image begins when the detection conditions are met and ends when the extraction fails to meet the detection conditions. The first detection release condition is that the vehicle's minimum speed is below a threshold. If the first detection release condition is met, the extracted captured image is deleted. The second detection release condition will be when turning left or right. If the second detection release condition is met, the extracted captured image will be deleted.

Images