Information processing device, driving diagnosis method and program
The information processing device addresses communication burdens in driving diagnosis by locally processing travel and target recognition data, enabling efficient event detection and reduced data transfer.
Patent Information
- Application Number
- JP2022076207
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-05-02
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2042-05-02
AI Technical Summary
Conventional driving diagnosis methods require significant communication bandwidth for sending image data to external servers, placing a burden on communication circuits.
An information processing device that acquires and processes travel information and target recognition data from vehicle-mounted cameras, updating target recognition status and diagnosing events requiring attention based on detection conditions, thereby reducing the need for image data transmission.
This approach suppresses communication load during driving diagnosis by processing data locally, allowing for efficient event detection and image recording without excessive data transfer.
Smart Images

Figure 0007578117000001 
Figure 0007578117000002 
Figure 0007578117000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a driving diagnosis method, and a program. [Background technology]
[0002] Patent Document 1 discloses a method for measuring a vehicle distance based on vehicle images. This method includes the steps of acquiring a driving image captured by a camera of a first vehicle while the vehicle is moving, detecting a second vehicle from the acquired driving image, and if the second vehicle is not detected from the driving image, detecting a first feature point in the second vehicle area from a first frame corresponding to a frame in which the second vehicle was detected and preceding the frame in which the second vehicle was not detected among a plurality of frames constituting the driving image, detecting a second feature point in a second frame corresponding to the current frame by tracking the detected first feature point, calculating a feature point change value between the first feature point and the second feature point, and calculating a distance from the camera of the first vehicle to the second vehicle based on the calculated feature point change value. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-118002 Summary of the Invention [Problem to be solved by the invention]
[0004] When driving diagnosis is performed on a server or the like external to the vehicle, in the conventional technology, image data must be sent from the vehicle to the server, which places a large burden on the communication circuit.
[0005] An object of the present invention is to provide an information processing device, a driving diagnosis method, and a program that can reduce the amount of communication when driving diagnosis is performed outside a vehicle. [Means for solving the problem]
[0006] The information processing device described in claim 1 includes an acquisition unit that acquires driving information related to the driving of a vehicle and information on whether a target has been recognized from an image captured by a camera mounted on the vehicle, a pre-processing unit that updates the information acquired within a certain section including a certain time to information indicating that the target has been recognized, even if the information indicates that the target has not been recognized at that time, to information indicating that the target has been recognized, when the information acquired within a certain section including a certain time includes information indicating that the target has been recognized, and a diagnosis unit that diagnoses that a noteworthy event has occurred when there is information indicating that the target has been recognized for each time and a detection condition based on the state of the target and the driving information is satisfied.
[0007] In the information processing device according to claim 1, an acquisition unit acquires driving information related to the driving of the vehicle and information on whether or not a target has been recognized from an image captured by a camera mounted on the vehicle. The target here refers to, for example, a vehicle ahead, a traffic light, etc. The preprocessing unit updates the information acquired in a certain section including a certain time to information indicating that the target has been recognized, even if the information indicates that the target has not been recognized at the time, to information indicating that the target has been recognized. The diagnosing unit diagnoses that an event requiring attention has occurred when there is information indicating that the target has been recognized for each time and a detection condition based on the target's appearance and the driving information is satisfied. The event requiring attention here refers to, for example, shortening the inter-vehicle distance, being significantly behind the vehicle ahead when starting, not stopping despite a red light, etc. The condition based on the target's appearance and driving information is a condition specified using the target's appearance and driving information for diagnosing that an event requiring attention has occurred.
[0008] The information processing device acquires driving information related to the driving of the vehicle and information on whether a target is recognized from an image captured by a camera mounted on the vehicle. Therefore, the information processing device can reduce the amount of communication when performing driving diagnosis outside the vehicle.
[0009] The information processing device according to a second aspect of the present invention is the information processing device according to the first aspect, further including an extraction unit that extracts the captured image when the detection condition is satisfied.
[0010] In the information processing device according to claim 2, the captured image is extracted when the detection condition is satisfied. According to the information processing device, it is possible to record a captured image when a noteworthy event occurs.
[0011] The information processing device according to a third aspect of the present invention is the information processing device according to the second aspect, wherein the extraction section ends the extraction of the captured image when the detection condition is no longer satisfied.
[0012] In the information processing device according to claim 3, the extraction of the captured images is terminated when the detection condition is no longer satisfied. According to the information processing device, it is possible to appropriately terminate the extraction of captured images when it is determined that a noteworthy event has occurred.
[0013] The information processing device according to a fourth aspect of the present invention is the information processing device according to the second or third aspect, wherein the extraction section deletes the extracted captured image when a predetermined detection cancellation condition based on the traveling information is satisfied.
[0014] In the information processing device according to claim 4, the extraction unit deletes the extracted captured image when a predetermined detection cancellation condition based on the driving information is satisfied. According to the information processing device, the predetermined detection cancellation condition is a condition for deleting a diagnosis that an event requiring attention has occurred and diagnosing that an event requiring attention has not occurred. According to the information processing device, after it has been once diagnosed that an event requiring attention has occurred, when it is diagnosed that an event requiring attention has not occurred, the extracted captured image can be deleted.
[0015] The driving diagnosis method described in claim 5 acquires driving information related to the vehicle's driving and information on whether a target has been recognized from an image captured by a camera mounted on the vehicle, and if the information acquired within a certain section including a certain time includes information indicating that the target has been recognized, even if the information indicates that the target has not been recognized at that time, it updates it to information indicating that the target has been recognized, and if there is information indicating that the target has been recognized for each time and a detection condition based on the state of the target and the driving information is satisfied, a computer executes a process to diagnose that an event requiring attention has occurred.
[0016] In the driving diagnosis method described in claim 5, a computer acquires driving information related to the driving of a vehicle and information on whether a target has been recognized from an image captured by a camera mounted on the vehicle. If the information acquired by the computer in a certain section including a certain time includes information indicating that the target has been recognized, even if the information indicates that the target has not been recognized at that time, the computer updates the information to information indicating that the target has been recognized. If the computer has information indicating that the target has been recognized for each time and a detection condition based on the state of the target and the driving information is satisfied, the computer diagnoses that a noteworthy event has occurred.
[0017] In the driving diagnosis method, driving information related to the driving of the vehicle and information on whether a target is recognized from an image captured by a camera mounted on the vehicle are acquired. Therefore, according to the driving diagnosis method, it is possible to reduce the amount of communication when performing driving diagnosis outside the vehicle.
[0018] The program described in claim 6 is a program that causes a computer to execute a process of acquiring driving information related to the driving of a vehicle and information on whether or not a target has been recognized from an image captured by a camera mounted on the vehicle, and if the information acquired during a certain period including a certain time includes information indicating that the target has been recognized, updating the information to information indicating that the target has been recognized even if the information indicates that the target has not been recognized at that time, and diagnosing that a noteworthy event has occurred when there is information indicating that the target has been recognized for each time and a detection condition based on the state of the target and the driving information is satisfied.
[0019] A computer that executes the program according to claim 6 acquires driving information related to the driving of a vehicle and information on whether a target has been recognized from an image captured by a camera mounted on the vehicle. If the information acquired by the computer in a certain section including a certain time includes information indicating that the target has been recognized, even if the information indicates that the target has not been recognized at that time, the computer updates the information to information indicating that the target has been recognized. If the computer has information indicating that the target has been recognized for each time and a detection condition based on the state of the target and the driving information is satisfied, the computer diagnoses that a noteworthy event has occurred.
[0020] The program acquires driving information related to the driving of the vehicle and information on whether a target is recognized from an image captured by a camera mounted on the vehicle. Therefore, the program can reduce the amount of communication when performing driving diagnosis outside the vehicle. Effect of the Invention
[0021] According to the present invention, it is possible to reduce the amount of communication when driving diagnosis is performed outside the vehicle. [Brief description of the drawings]
[0022] [Figure 1] 1 is a diagram showing a schematic configuration of a driving diagnosis system according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing a hardware configuration of a vehicle according to an embodiment of the present invention; [Diagram 3] 2 is a block diagram showing the configuration of a ROM of the vehicle-mounted device of the embodiment; FIG. [Figure 4] FIG. 2 is a block diagram showing a configuration of a storage of the vehicle-mounted device according to the embodiment; [Diagram 5] FIG. 2 is a block diagram showing a functional configuration of the vehicle-mounted device according to the embodiment; [Figure 6] FIG. 2 is a block diagram showing a hardware configuration of a center server of the present embodiment. [Figure 7] FIG. 2 is a block diagram showing the functional configuration of a center server of the present embodiment. [Figure 8] 5 is a flowchart showing a flow of a target information transmission process executed in the vehicle-mounted device of the present embodiment. [Figure 9] 4 is a flowchart showing the flow of a driving diagnosis process executed in the center server of the present embodiment. [Figure 10] 4 is a flowchart showing the flow of a driving diagnosis process executed in the center server of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0023] A driving diagnosis system including an information processing device of the present invention will be described. The driving diagnosis system is a system that performs driving diagnosis using information of a target recognized in an image captured by a camera mounted on a vehicle and driving information of the vehicle. In addition, the driving diagnosis system records the captured image when it is determined that an event requiring attention has occurred.
[0024] (Overall composition) As shown in Fig. 1, a driving diagnosis system 10 of this embodiment includes a vehicle 12 and a center server 30 as an information processing device. The vehicle 12 is equipped with an on-board device 20. The on-board device 20 and the center server 30 are connected to each other via a network N. Although Fig. 1 illustrates one vehicle 12 and one on-board device 20 for one center server 30, the numbers of vehicles 12 and on-board devices 20 are not limited to this.
[0025] The center server 30 is installed, for example, at a manufacturer that produces the vehicle 12, a car dealer, or any other business entity.
[0026] (vehicle) As shown in FIG. 2, the vehicle 12 according to this embodiment includes an in-vehicle device 20, a plurality of ECUs (Electronic Control Units) 22, and a plurality of in-vehicle devices 24.
[0027] The vehicle-mounted device 20 includes a central processing unit (CPU) 20A, a read only memory (ROM) 20B, a random access memory (RAM) 20C, a storage 20D, a wireless communication interface (I / F) 20E, and an in-vehicle communication I / F 20F. The CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, the wireless communication interface (I / F) 20E, and the in-vehicle communication I / F 20F are connected to each other so as to be able to communicate with each other via an internal bus 20G.
[0028] The CPU 20A is a central processing unit that executes various programs and controls each part. That is, the CPU 20A reads a program from the ROM 20B or the storage 20D, and executes the program using the RAM 20C as a working area.
[0029] ROM 20B stores various programs and various data. As shown in Fig. 3, ROM 20B of this embodiment stores a control program 100. The control program 100 is a program for transmitting information on a target recognized from an image captured by the camera to the center server 30.
[0030] As shown in FIG. 2, the RAM 20C temporarily stores programs or data as a working area.
[0031] The storage 20D, which is a memory, is configured with a HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs and various data. As shown in Fig. 4, the storage 20D of this embodiment stores a target information DB (database) 120, which is a data group of information on recognized targets. The storage 20D may store the control program 100 and the regulation information DB 120.
[0032] 2, the wireless communication I / F 20E is a wireless communication module for communicating with the center server 30. The wireless communication module uses a communication standard such as 5G, LTE, or Wi-Fi (registered trademark). The wireless communication I / F 20E is connected to a network N.
[0033] The in-vehicle communication I / F 20F is an interface for connecting to each ECU 22. The interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F 20F is connected to the external bus 20H.
[0034] The ECU 22 includes at least an ADAS (Advanced Driver Assistance System)-ECU 22A.
[0035] The ADAS-ECU 22A controls the advanced driving assistance system. A vehicle speed sensor 24A and a camera 24C that configure the in-vehicle device 24 are connected to the ADAS-ECU 22A. The camera 24C is a camera that captures images of the surroundings of the vehicle 12.
[0036] As shown in FIG. 5, in the vehicle-mounted device 20 of this embodiment, a CPU 20A executes a control program 100 to function as an acquisition unit 200, a processing unit 210, and a transmission unit 220.
[0037] The acquisition unit 200 has a function of acquiring vehicle information and captured images. The vehicle information is information such as the state of the in-vehicle device 24 and the state of the vehicle 12 obtained from the in-vehicle device 24. The captured images are images of the outside of the vehicle 12 captured by the camera 24C of the vehicle 12. The vehicle information in this embodiment includes position information or traveling information such as vehicle speed.
[0038] The processing unit 210 has a function of storing the vehicle information acquired by the acquisition unit 200 in the storage 20D, and recognizing the target from the captured image captured by the camera 24C, thereby converting the information into information on the target. Specifically, the processing unit 210 of this embodiment temporarily stores the vehicle information and the target information in the storage 20D, and then transmits the vehicle information and the target information to the center server 30 by the transmission unit 220. Furthermore, when a request is received from the center server 30, the processing unit 210 transmits the captured image to the center server 30 by the transmission unit 220. Furthermore, the processing unit 210 converts the captured image captured by the camera 24C into information on the target by image recognition processing.
[0039] More specifically, the processing unit 210 recognizes the vehicle ahead and the traffic light as targets represented by the captured image, and converts them into information on the vehicle ahead (a flag indicating the presence or absence of a vehicle ahead, the distance to the vehicle ahead, and the time between vehicles) and information on the traffic light (a flag indicating the presence or absence of a red light, a flag indicating the presence or absence of a green light, and the distance to the traffic light). The time between vehicles is calculated from the distance between vehicles and the vehicle speed.
[0040] The transmission unit 220 has a function of transmitting the vehicle information and the target information stored in the storage 20D to the center server 30. Specifically, the transmission unit 220 in this embodiment transmits the vehicle information and the target information stored in the storage 20D to the center server 30 when requested by the center server 30.
[0041] Furthermore, the transmission unit 220 transmits the captured image captured by the camera 24C to the center server 30 upon receiving a request from the center server 30.
[0042] (Center server) As shown in Fig. 6, the center server 30 includes a CPU 30A, a ROM 30B, a RAM 30C, a storage 30D, and a communication I / F 30E. The CPU 30A, the ROM 30B, the RAM 30C, the storage 30D, and the communication I / F 30E are connected to each other so as to be able to communicate with each other via an internal bus 30G. The functions of the CPU 30A, the ROM 30B, the RAM 30C, the storage 30D, and the communication I / F 30E are the same as the functions of the CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, and the wireless communication I / F 20E of the vehicle-mounted device 20 described above. The communication I / F 30E may perform wired communication. The CPU 30A is an example of a processor.
[0043] The storage 30D of this embodiment stores a processing program 150 as a program, a target information DB 160, and an extracted image DB 170. The processing program 150 may be stored in the ROM 30B.
[0044] The processing program 150 is a program for controlling the center server 30. With the execution of the processing program 150, the center server 30 acquires target information and driving information from the vehicle 12, and executes each process for performing driving diagnosis on the driving information of the vehicle 12.
[0045] The target information DB 160 stores information about targets acquired from the vehicle 12.
[0046] Specifically, information on the vehicle ahead (a flag indicating whether or not there is a vehicle ahead, the distance from the vehicle ahead, and the time between vehicles) and traffic light information (a flag indicating whether or not there is a red light, a flag indicating whether or not there is a green light, and the distance from the traffic light) obtained from vehicle 12 are stored.
[0047] The extracted image DB 170 stores captured images when a diagnosis is made that a noteworthy event has occurred.
[0048] As shown in FIG. 7, in the center server 30 of this embodiment, a CPU 30A executes a processing program 150 to function as an acquisition unit 260, a preprocessing unit 270, a diagnosis unit 280, and an extraction unit 290.
[0049] The acquisition unit 260 has a function of acquiring vehicle information of the vehicle 12 and information on the target from the vehicle-mounted device 20 of the vehicle 12. The acquisition unit 260 stores the acquired information on the target in the target information DB160.
[0050] When the target information acquired during a certain period including a certain time includes information indicating that the target has been recognized, the pre-processing unit 270 updates the information to indicate that the target has been recognized, even if the information indicates that the target has not been recognized at that time.
[0051] Specifically, when the recognized target is a vehicle ahead and the target information is a flag indicating the presence of a vehicle ahead, the pre-processing unit 270 records a vehicle ahead flag. Then, in order to avoid erroneous detection or non-detection of a vehicle ahead, the pre-processing unit 270 performs a filter process to replace the flag indicating the presence of a vehicle ahead with the maximum value in a certain section of 0.2 seconds before and after the vehicle ahead, and updates all flags in the certain section with the maximum value.
[0052] Furthermore, when the recognized target is a traffic signal and the target information is a flag indicating a red light, the preprocessing unit 270 records a red light flag. In order to avoid erroneous detection or non-detection of a red light, the preprocessing unit 270 performs a filter process to replace the flag indicating a red light with the maximum value in a certain section, which is one second before and after the red light, and updates all flags in the certain section with the maximum value.
[0053] Furthermore, when the recognized target is a traffic signal and the target information is a flag indicating a green light, the preprocessing unit 270 records a green light flag. In order to avoid erroneous detection or non-detection of a green light, the preprocessing unit 270 performs a filter process to replace the flag indicating a green light with the maximum value in a certain section, which is one second before and after the green light, and updates all flags in the certain section with the maximum value.
[0054] Based on the vehicle information and target information acquired from the vehicle-mounted device 20 of the vehicle 12, the diagnosis unit 280 diagnoses that a noteworthy event has occurred when the target information acquired from the vehicle-mounted device 20 of the vehicle 12 at each time indicates that the target has been recognized and detection conditions based on the target's behavior and the driving information included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 are satisfied.
[0055] Specifically, the diagnosis unit 280 diagnoses whether or not a noteworthy event has occurred, using as a detection condition that the vehicle-to-vehicle distance is shortened while the vehicle is moving or stopped.
[0056] For example, the detection conditions are that the flag indicates the presence of a vehicle ahead, the vehicle 12 is moving (vehicle speed>10km / h), and the inter-vehicle time is <0.5 seconds. Also, the detection conditions are that the flag indicates the presence of a vehicle ahead, the vehicle 12 is stopped (vehicle speed<0.5km / h), and the inter-vehicle distance is <1500mm.
[0057] Furthermore, when starting at an intersection or in a traffic jam, the diagnosis unit 280 diagnoses whether or not a cautionary event has occurred, using as a detection condition that the vehicle is significantly lagging behind the vehicle ahead.
[0058] For example, if the flag indicates that there is a vehicle ahead and the vehicle speed is <0.5km / h, it is provisionally diagnosed that an event requiring attention has occurred; if there is a vehicle ahead while the vehicle is stopped and the maximum inter-vehicle distance - minimum inter-vehicle distance while the vehicle is stopped is ≥ 25,000mm, it is diagnosed that an event requiring attention has occurred.
[0059] Furthermore, if the stopping time is >120 seconds, it is considered to be parking, the provisional diagnosis that an event requiring attention has occurred is deleted, and it is diagnosed that an event requiring attention has not occurred. Furthermore, if the distance to the vehicle ahead is >15,000 mm, the provisional diagnosis that an event requiring attention has occurred is deleted, and it is diagnosed that an event requiring attention has not occurred. Furthermore, when turning right or left (with the turn signal switch on), since there is a high possibility of a start delay, the provisional diagnosis that an event requiring attention has occurred is deleted, and it is diagnosed that an event requiring attention has not occurred.
[0060] Furthermore, the diagnosis unit 280 diagnoses whether or not a noteworthy event has occurred, using the detection condition that the vehicle has not stopped despite the red light.
[0061] For example, if a flag indicating that the light is red is on, it is provisionally diagnosed that an event requiring attention has occurred, and if the minimum vehicle speed is less than a threshold value (1 km / h), the provisional diagnosis that an event requiring attention has occurred is deleted, and it is diagnosed that an event requiring attention has not occurred.
[0062] Also, when the winker switch is on, it is assumed that an arrow signal is being issued, the provisional diagnosis that an event requiring attention has occurred is deleted, and it is diagnosed that an event requiring attention has not occurred.
[0063] In addition, if the vehicle is consistently more than 10 meters away from a traffic light while it is recognizing a red light, the provisional diagnosis that an event requiring attention has occurred is removed and the system determines that no event requiring attention has occurred.
[0064] In addition, if the light changes from red to green immediately, the provisional diagnosis that an event requiring attention has occurred is deleted, and a diagnosis is made that no event requiring attention has occurred.
[0065] When the detection condition is satisfied, the extraction unit 290 requests captured images from the vehicle-mounted device 20 of the vehicle 12, extracts captured images acquired from the vehicle-mounted device 20 of the vehicle 12, and stores them in the extracted image DB170.
[0066] Furthermore, when the detection condition is no longer satisfied, the extraction unit 290 stops the extraction of the captured images acquired from the vehicle-mounted device 20 of the vehicle 12, and stores the extracted captured images in the extracted image DB 170.
[0067] Furthermore, the extraction unit 290 deletes the extracted captured image when a predetermined detection cancellation condition based on the driving information is satisfied.
[0068] Specifically, when the detection conditions are met that a flag is set indicating that there is a vehicle ahead, the vehicle 12 is moving (vehicle speed > 10 km / h), and the time gap is < 0.5 seconds, or when the detection conditions are set that a flag is set indicating that there is a vehicle ahead, the vehicle 12 is stopped (vehicle speed < 0.5 km / h), and the distance gap is < 1500 mm, extraction of captured images obtained from the vehicle-mounted device 20 of the vehicle 12 is started, and when the detection conditions are no longer met, extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170.
[0069] When the detection conditions, that is, a flag indicating the presence of a vehicle ahead and the vehicle speed is <0.5km / h, are satisfied, the extraction of captured images acquired from the vehicle-mounted device 20 of the vehicle 12 is started, and when the detection conditions are no longer satisfied, the extraction of captured images is terminated. Then, if there is a vehicle ahead while the vehicle is stopped and the maximum inter-vehicle distance - the minimum inter-vehicle distance while the vehicle is stopped is ≥ 25000mm, the extracted captured images are stored in the extracted image DB 170.
[0070] In addition, the detection release condition is set to be that the stop time is greater than 120 seconds, and when the detection release condition is satisfied, the extracted captured image is deleted.
[0071] In addition, the detection release condition is set to be a distance from the vehicle ahead > 15000 mm, and when the detection release condition is satisfied, the extracted captured image is deleted.
[0072] In addition, the detection release condition is when the vehicle is turning right or left (the turn signal switch is on), and when the detection release condition is satisfied, the extracted captured image is deleted.
[0073] In addition, when the detection condition that a flag indicating that the light is red is on is satisfied, extraction of captured images obtained from the vehicle-mounted device 20 of the vehicle 12 is started, and when the detection condition is no longer satisfied, extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170.
[0074] A minimum vehicle speed less than a threshold (1 km / h) is set as a detection release condition, and when the detection release condition is satisfied, the extracted captured image is deleted.
[0075] In addition, the detection release condition is that the turn signal switch is on. When the detection release condition is satisfied, it is determined that an arrow signal is being output, and the extracted captured image is deleted.
[0076] In addition, the detection cancellation condition is that the vehicle is always more than 10 meters away from the traffic light while the red light is recognized, and when the detection cancellation condition is met, the extracted captured image is deleted.
[0077] In addition, the detection cancellation condition is that the light changes from red to green immediately, and when the detection cancellation condition is satisfied, the extracted captured image is deleted.
[0078] (Flow of Control) The flow of processing executed in the driving diagnosis system 10 of this embodiment will be described with reference to the flowcharts of Fig. 8 to Fig. 10. The processing in the vehicle-mounted device 20 is executed by the CPU 20A of the vehicle-mounted device 20 functioning as an acquisition unit 200, a processing unit 210, and a transmission unit 220. The processing in the center server 30 is executed by the CPU 30A of the center server 30 functioning as an acquisition unit 260, a pre-processing unit 270, a diagnosis unit 280, and an extraction unit 290.
[0079] First, while the vehicle 12 is traveling, the CPU 20A of the vehicle-mounted device 20 repeatedly executes the target information transmission process shown in FIG.
[0080] In step S100 of FIG. 8, the acquisition unit 200 acquires a captured image of the outside of the vehicle 12 captured by the camera 24C, and also acquires driving information.
[0081] In step S102, the processing unit 210 recognizes the target from the captured image captured by the camera 24C, converts the target into information, and stores the information in the target information DB 120 temporarily.
[0082] In step S104, the processing unit 210 determines whether or not a target has been recognized by the processing in step S102. If a target has been recognized by the processing in step S102, the process proceeds to step S106. On the other hand, if a target has not been recognized by the processing in step S102, the target information transmission process is terminated.
[0083] In step S106, the transmission unit 220 transmits the information on the target obtained as the recognition result in step S102 above, together with the traveling information, to the center server 30, and ends the target information transmission process.
[0084] Moreover, in the center server 30, the CPU 30A repeatedly executes the driving diagnosis process shown in FIG. 9 and FIG.
[0085] Specifically, CPU 30A repeatedly executes a driving diagnosis process shown in FIG. 9 for diagnosing a situation in which the vehicle-to-vehicle distance is shortened as a noteworthy event.
[0086] In step S110 of FIG. 9, the acquisition unit 260 acquires driving information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.
[0087] In step S112 , the acquisition unit 260 acquires information about the target from the vehicle-mounted device 20 of the vehicle 12 .
[0088] In step S114, if the recognized target is a vehicle ahead, the pre-processing unit 270 records a flag indicating the presence of a vehicle ahead if the flag is a flag indicating the presence of a vehicle ahead. Then, in order to avoid erroneous detection or non-detection of a vehicle ahead, the pre-processing unit 270 performs a filter process to replace the flag indicating the presence of a vehicle ahead with the maximum value in a certain section of 0.2 seconds before and after the vehicle ahead, and updates all flags in the certain section with the maximum value.
[0089] In step S116, diagnosis unit 280 diagnoses whether an event requiring attention has occurred, using a detection condition that the vehicle distance is shortened while the vehicle is moving or stopped. If the detection condition is satisfied, it is diagnosed that an event requiring attention has occurred, and the process proceeds to step S118. On the other hand, if the detection condition is not satisfied, the process proceeds to step S120.
[0090] In step S118, the extraction unit 290 requests the captured images from the vehicle-mounted device 20 of the vehicle 12, and starts extracting the captured images acquired from the vehicle-mounted device 20 of the vehicle 12.
[0091] In step S120, the extraction unit 290 determines whether or not a captured image is being extracted. If a captured image is being extracted, the process proceeds to step S122, where the extraction of the captured image is terminated, and the extracted captured image is stored in the extracted image DB 170. On the other hand, if a captured image is not being extracted, the driving diagnosis process is terminated.
[0092] In addition, the CPU 30A repeatedly executes the driving diagnosis process shown in Fig. 10 for diagnosing that the vehicle is significantly behind the vehicle ahead when starting off as an event requiring attention. Note that the same processes as those in Fig. 9 will be described with the same reference numerals.
[0093] In step S110 of FIG. 10, the acquisition unit 260 acquires driving information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.
[0094] In step S112 , the acquisition unit 260 acquires information about the target from the vehicle-mounted device 20 of the vehicle 12 .
[0095] In step S114, if the recognized target is a vehicle ahead, the pre-processing unit 270 records a flag indicating the presence of a vehicle ahead if the flag is a flag indicating the presence of a vehicle ahead, and updates all flags in the fixed section with the maximum value by performing a filter process to replace the flag indicating the presence of a vehicle ahead with the maximum value in a fixed section of 0.2 seconds before and after the target.
[0096] In step S116, diagnosis unit 280 diagnoses whether an event requiring attention has occurred, using a detection condition that the vehicle is significantly behind the vehicle ahead when starting at an intersection or in a traffic jam. If the detection condition is met, it is diagnosed that an event requiring attention has occurred, and the process proceeds to step S118. On the other hand, if the detection condition is not met, the process proceeds to step S120.
[0097] In step S118, the extraction unit 290 requests the captured images from the vehicle-mounted device 20 of the vehicle 12, and starts extracting the captured images acquired from the vehicle-mounted device 20 of the vehicle 12.
[0098] In step S120, the extraction unit 290 determines whether or not a captured image is being extracted. If a captured image is being extracted, the process proceeds to step S122, where the extraction of the captured image is terminated, and the extracted captured image is stored in the extracted image DB 170. On the other hand, if a captured image is not being extracted, the driving diagnosis process is terminated.
[0099] In step S200, the extraction unit 290 determines whether or not the detection release conditions are met, with the detection release conditions being that the vehicle is stopped for more than 120 seconds, that the distance to the vehicle ahead is more than 15,000 mm, or that the vehicle is turning right or left (the turn signal switch is on). If the detection release conditions are not met, the driving diagnosis process is terminated. On the other hand, if the detection release conditions are met, the process proceeds to step S202.
[0100] In step S202, the extraction unit 290 diagnoses that a noteworthy event has occurred, deletes the extracted captured image, and ends the driving diagnosis process.
[0101] In addition, similar to the driving diagnosis process for diagnosing whether the vehicle is significantly lagging behind the vehicle ahead when starting off, the CPU 30A repeatedly executes the driving diagnosis process shown in FIG. 10 for diagnosing whether the vehicle is not stopping despite a red light, which is an event that requires attention.
[0102] Here, in step S114, if the recognized target is a traffic light, the pre-processing unit 270 records a red light flag if the flag indicates a red light. Then, by performing a filter process that replaces the flag indicating a red light with the maximum value in a certain section, which is one second before and after the traffic light, all flags in the certain section are updated with the maximum value.
[0103] Furthermore, if the flag indicates a green light, the pre-processing unit 270 records the green light flag. Then, the pre-processing unit 270 performs a filter process to replace the flag indicating a green light with the maximum value in a certain section, which is one second before and after the green light, and updates all flags in the certain section with the maximum value.
[0104] In step S116, diagnosis unit 280 diagnoses whether an event requiring attention has occurred, using the detection condition that the vehicle is not stopped despite the red light. If the detection condition is met, it is diagnosed that an event requiring attention has occurred, and the process proceeds to step S118. On the other hand, if the detection condition is not met, the process proceeds to step S120.
[0105] In step S200, the extraction unit 290 determines whether or not the detection release conditions are met, with the minimum vehicle speed being less than the threshold (1 km / h), the turn signal switch being on, always being 10 m or more away from the traffic light while recognizing the red light, or the red light immediately changing to green. If the detection release conditions are not met, the driving diagnosis process is terminated. On the other hand, if the detection release conditions are met, the process proceeds to step S202.
[0106] (Summary of the embodiment) The center server 30 of this embodiment acquires driving information related to the driving of the vehicle and information on whether a target has been recognized from an image captured by a camera mounted on the vehicle, and diagnoses that a noteworthy event has occurred when a detection condition based on the target's state and the driving information is satisfied. Therefore, the center server 30 does not need to transmit captured images for driving diagnosis, and can reduce the amount of communication between the vehicle and the center server 30 when driving diagnosis is performed outside the vehicle.
[0107] In addition, the center server 30 of the present embodiment extracts a captured image when a detection condition for a noteworthy event is satisfied. Therefore, the center server 30 can record a captured image when it is diagnosed that a noteworthy event has occurred.
[0108] Furthermore, the center server 30 of the present embodiment deletes the extracted captured image when a predetermined detection cancellation condition based on the driving information is satisfied. Therefore, according to the center server 30, after it has been diagnosed that an event requiring attention has occurred, it is possible to delete the extracted captured image when it is diagnosed that an event requiring attention has not occurred.
[0109] [remarks] In the above embodiment, various processes executed by the CPU 20A and the CPU 30A by reading the software (programs) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. In addition, each of the above-mentioned processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0110] In the above embodiment, each program has been described as being prestored (installed) in a non-transitory computer-readable recording medium. For example, the control program 100 in the vehicle-mounted device 20 is prestored in the ROM 20B, and the processing program 150 in the center server 30 is prestored in the storage 30D. However, the present invention is not limited to this, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Also, the program may be downloaded from an external device via a network.
[0111] The process flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be changed, without departing from the spirit of the invention. [Explanation of symbols]
[0112] 12 Vehicles 20 Onboard equipment 30 Center server (information processing device) 30A CPU (processor) 30D Storage (Memory) 150 Processing Program (Program) 260 Acquisition Department 270 Pretreatment section 280 Diagnostic Department 290 Extraction part
Claims
1. an acquisition unit that acquires driving information related to the driving of a vehicle and information on whether a traffic light is recognized as being red from an image captured by a camera mounted on the vehicle; a pre-processing unit that, when the information acquired during a certain period including a certain time includes information indicating that the traffic light has been recognized as a red light, updates the information to information indicating that the traffic light has been recognized as a red light even if the information indicates that the traffic light has not been recognized as a red light at that time; a diagnosis unit that diagnoses that a noteworthy event has occurred when there is information indicating that the traffic light has been recognized as a red light for each time and when a detection condition indicating that the traffic light is a red light is satisfied; When the detection condition is satisfied, the extraction of the captured image is started, and when the detection condition is no longer satisfied, the extraction of the captured image is ended. a first detection release condition being that a minimum vehicle speed of the vehicle is less than a threshold, and when the first detection release condition is satisfied, deleting the extracted captured image; an extraction unit that deletes the extracted captured image when a second detection release condition is satisfied, the second detection release condition being a right or left turn; An information processing device comprising:
2. Acquire driving information related to the driving of the vehicle and information on whether or not a traffic light is recognized as being red from an image captured by a camera mounted on the vehicle, When the information acquired during a certain period including a certain time includes information indicating that the traffic light is recognized as a red light, even if the information indicates that the traffic light is not recognized as a red light at that time, the information is updated to information indicating that the traffic light is recognized as a red light, For each time, when there is information indicating that the traffic light is recognized as a red light and a detection condition indicating that the traffic light is a red light is satisfied, a diagnosis is made that a noteworthy event has occurred; When the detection condition is satisfied, the extraction of the captured image is started, and when the detection condition is no longer satisfied, the extraction of the captured image is ended. a first detection release condition being that a minimum vehicle speed of the vehicle is less than a threshold, and when the first detection release condition is satisfied, deleting the extracted captured image; A second detection release condition is when the vehicle is turning right or left, and when the second detection release condition is satisfied, the extracted captured image is deleted. A driving diagnosis method in which processing is performed by a computer.
3. Acquire driving information related to the driving of the vehicle and information on whether or not a traffic light is recognized as being red from an image captured by a camera mounted on the vehicle, When the information acquired during a certain period including a certain time includes information indicating that the traffic light is recognized as a red light, even if the information indicates that the traffic light is not recognized as a red light at that time, the information is updated to information indicating that the traffic light is recognized as a red light, For each time, when there is information indicating that the traffic light is recognized as a red light and a detection condition indicating that the traffic light is a red light is satisfied, a diagnosis is made that a noteworthy event has occurred; When the detection condition is satisfied, the extraction of the captured image is started, and when the detection condition is no longer satisfied, the extraction of the captured image is ended. a first detection release condition being that a minimum vehicle speed of the vehicle is less than a threshold, and when the first detection release condition is satisfied, deleting the extracted captured image; A second detection release condition is when the vehicle is turning right or left, and when the second detection release condition is satisfied, the extracted captured image is deleted. A program that causes a computer to carry out processing.
Citation Information
Patent Citations
Drive support device
JP2016206970A
Dashboard camera and image storage system
JP2020052634A
Measuring method and measuring device for vehicular distance based on vehicle video, electronic apparatus, computer program and computer readable recording medium
JP2021118002A
