Information processing device, driving diagnosis method and program
The information processing device enhances driving diagnosis accuracy by filtering and updating regulatory values based on reliability thresholds, addressing the issue of blurring in regulatory display reading from captured images.
Patent Information
- Application Number
- JP2022076202
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-05-02
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2042-05-02
AI Technical Summary
Existing systems struggle to accurately read regulatory displays from captured images for driving diagnosis without blurring.
An information processing device that acquires and processes travel information and regulatory display data from a vehicle-mounted camera, setting regulation values based on reliability thresholds and updating them to perform precise driving diagnoses.
Enables clear and accurate driving diagnoses by filtering and updating regulatory values to prevent blurring, ensuring reliable violation detection and image recording.
Smart Images

Figure 0007582247000001 
Figure 0007582247000002 
Figure 0007582247000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a driving diagnosis method, and a program. [Background technology]
[0002] Patent Document 1 discloses a vehicle cruise control device that conveys speed limit information to the driver based on a speed limit value obtained from a car navigation system. This cruise control device prompts the driver to decelerate the cruise speed setting to a safe speed that is equal to or lower than the speed limit. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2006-321354 A Summary of the Invention [Problem to be solved by the invention]
[0004] It is conceivable to obtain regulatory signs from captured images and perform driving diagnosis based on the obtained regulatory signs. However, there is room for improvement in accurately reading the various regulatory signs and performing driving diagnosis without blurring.
[0005] An object of the present invention is to provide an information processing device, a driving diagnosis method, and a program that enable a driving diagnosis without shaking to be performed based on a restriction display obtained from a captured image. [Means for solving the problem]
[0006] The information processing device described in claim 1 includes an acquisition unit that acquires driving information related to the driving of a vehicle and information on a regulatory display that has been digitized from an image of the regulatory display captured by a camera mounted on the vehicle, a pre-processing unit that sets the information to a regulatory value if the reliability of the digitized information on the regulatory display is equal to or higher than a threshold, and updates the regulatory value at a certain time with any one of the regulatory values in a certain section including the time, and a diagnosis unit that diagnoses a violation if the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulatory value.
[0007] In the information processing device according to claim 1, an acquisition unit acquires driving information related to the driving of a vehicle and information on a regulatory display that has been digitized from an image of the regulatory display captured by a camera mounted on the vehicle. When the reliability of the digitized information on the regulatory display is equal to or higher than a threshold, a preprocessing unit sets the information to a regulatory value and updates the regulatory value at a certain time with any one of the regulatory values in a certain section including the time. Here, the reliability of the information on the regulatory display is a value indicating the likelihood of the information on the regulatory display that has been digitized from the image of the regulatory display. A diagnosis unit diagnoses a violation when the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulatory value. Here, the violation condition based on the regulatory value is a condition specified using a regulatory value for diagnosing that the driving of the vehicle is a violation.
[0008] In the information processing device, when the reliability of the digitized information of the restriction display is equal to or higher than a threshold, the information is set to a restriction value, and the restriction value at a certain time is updated to any one of the restriction values in a certain interval including the time. Therefore, according to the information processing device, it is possible to perform a driving diagnosis without blurring based on the restriction display acquired from the captured image.
[0009] The information processing device of claim 2 is the information processing device of claim 1, wherein the restriction display is an display related to a speed limit, the information of the restriction display is a speed limit, and the pre-processing unit updates all restriction values in the certain section with the maximum value of the restriction values in the certain section.
[0010] In the information processing device according to claim 2, when the reliability of the digitized information on the display related to the speed limit is equal to or higher than a threshold, the speed limit is set to a regulation value, and all regulation values in the fixed section are updated with the maximum regulation value in the fixed section. With this information processing device, it is possible to perform a steady driving diagnosis based on the display related to the speed limit acquired from the captured image.
[0011] The information processing device of claim 3 is the information processing device of claim 1, wherein the restriction display is a display related to a prohibited act, and the information of the restriction display is a flag indicating the presence or absence of a display related to the prohibited act, and the pre-processing unit sets the flag which is the restriction value to on when the reliability of the information of the digitized restriction display is equal to or greater than a threshold, and updates the flags which are all restriction values in the certain section to on when the flag which is the restriction value in the certain section is on.
[0012] In the information processing device according to claim 3, when the reliability of the information of the display related to the computerized prohibited act is equal to or higher than a threshold, the flag which is the regulated value is set to ON, and when the flag which is the regulated value in the certain section is ON, the flags which are all regulated values in the certain section are updated to ON. Here, the display related to the prohibited act refers to, for example, a stop sign, a parking sign, and a no turn sign. According to the information processing device, it is possible to perform a driving diagnosis without shaking based on the display related to the prohibited act acquired from the captured image.
[0013] The information processing device according to a fourth aspect of the present invention is the information processing device according to the first aspect, further including an extraction unit that extracts the captured image when the violation condition is satisfied.
[0014] In the information processing device according to claim 4, the captured image is extracted when the violation condition is satisfied. According to the information processing device, the captured image when a violation is diagnosed can be recorded.
[0015] The information processing device according to a fifth aspect of the present invention is the information processing device according to the fourth aspect, wherein the extraction section ends the extraction of the captured image when the violation condition is no longer satisfied.
[0016] In the information processing device according to claim 5, the extraction of the captured images is terminated when the violation condition is no longer satisfied. According to the information processing device, it is possible to appropriately terminate the extraction of captured images when a violation is diagnosed.
[0017] The information processing device according to a sixth aspect of the present invention is the information processing device according to the fourth aspect, wherein the extraction section deletes the extracted captured image when a predetermined violation cancellation condition based on the driving information is satisfied.
[0018] In the information processing device according to claim 6, when a predetermined violation cancellation condition based on the driving information is satisfied, the extracted captured image is deleted. Here, the predetermined violation cancellation condition is a condition for deleting the diagnosis of a violation and diagnosing that there is no violation. According to the information processing device, after a violation has been diagnosed, when it is diagnosed that there is no violation, the extracted captured image can be deleted.
[0019] The driving diagnosis method described in claim 7 acquires information on a regulatory display that has been digitized from driving information related to the driving of the vehicle and an image of the regulatory display captured by a camera mounted on the vehicle, and if the reliability of the digitized information on the regulatory display is equal to or higher than a threshold, sets the information to a regulatory value, updates the regulatory value at a certain time with any one of the regulatory values in a certain section including the time, and if the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulatory value, a computer executes a process to diagnose a violation.
[0020] In the driving diagnosis method described in claim 7, a computer acquires information on a restriction sign that has been digitized from driving information related to the driving of the vehicle and an image of the restriction sign captured by a camera mounted on the vehicle. If the reliability of the digitized information on the restriction sign is equal to or higher than a threshold, the computer sets the information to a restriction value and updates the restriction value at a certain time with any one of the restriction values in a certain section including the time. If the driving of the vehicle based on the driving information satisfies a violation condition based on the updated restriction value, the computer diagnoses a violation.
[0021] In the driving diagnosis method, when the reliability of the digitized information of the restriction display is equal to or higher than a threshold, the information is set to a restriction value, and the restriction value at a certain time is updated to any one of the restriction values in a certain interval including the time. Therefore, according to the driving diagnosis method, it is possible to perform a driving diagnosis without any blurring based on the restriction display acquired from the captured image.
[0022] The program described in claim 8 is a program that causes a computer to execute a process of acquiring driving information related to the driving of a vehicle and information on a regulatory display that has been digitized from an image of the regulatory display captured by a camera mounted on the vehicle, and if the reliability of the digitized information on the regulatory display is equal to or higher than a threshold, setting the information to a regulatory value, updating the regulatory value at a certain time with any one of the regulatory values in a certain section including the time, and diagnosing a violation if the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulatory value.
[0023] A computer that executes the program according to claim 8 acquires information on a regulation sign that is digitized from driving information related to the driving of the vehicle and an image of the regulation sign captured by a camera mounted on the vehicle. If the reliability of the digitized information on the regulation sign is equal to or higher than a threshold, the computer sets the information to a regulation value and updates the regulation value at a certain time with any one of the regulation values in a certain section including the time. If the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulation value, the computer diagnoses a violation.
[0024] In the program, when the reliability of the digitized information of the restriction display is equal to or higher than a threshold, the information is set to a restriction value, and the restriction value at a certain time is updated to any one of the restriction values in a certain interval including the time. Therefore, according to the driving diagnosis method, it is possible to perform a driving diagnosis without any blurring based on the restriction display acquired from the captured image. Effect of the Invention
[0025] According to the present invention, it is possible to carry out a driving diagnosis without shaking based on a restriction display acquired from a captured image. [Brief description of the drawings]
[0026] [Figure 1] 1 is a diagram showing a schematic configuration of a driving diagnosis system according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing a hardware configuration of a vehicle according to an embodiment of the present invention; [Diagram 3] 2 is a block diagram showing the configuration of a ROM of the vehicle-mounted device of the embodiment; FIG. [Figure 4] FIG. 2 is a block diagram showing a configuration of a storage of the vehicle-mounted device according to the embodiment; [Diagram 5] FIG. 2 is a block diagram showing a functional configuration of the vehicle-mounted device according to the embodiment; [Figure 6] FIG. 2 is a block diagram showing a hardware configuration of a center server of the present embodiment. [Figure 7]FIG. 2 is a block diagram showing the functional configuration of a center server of the present embodiment. [Figure 8] 4 is a flowchart showing a flow of a restriction display information transmission process executed in the vehicle-mounted device of the present embodiment. [Figure 9] 4 is a flowchart showing the flow of a driving diagnosis process executed in the center server of the present embodiment. [Figure 10] 4 is a flowchart showing the flow of a driving diagnosis process executed in the center server of the present embodiment. [Figure 11] 4 is a flowchart showing the flow of a driving diagnosis process executed in the center server of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0027] A driving diagnosis system including an information processing device of the present invention will be described. The driving diagnosis system is a system that performs driving diagnosis using traffic sign information collected in a vehicle and vehicle running information. In addition, the driving diagnosis system records a captured image when a violation is determined.
[0028] (Overall composition) As shown in Fig. 1, a driving diagnosis system 10 of this embodiment includes a vehicle 12 and a center server 30 as an information processing device. The vehicle 12 is equipped with an on-board device 20. The on-board device 20 and the center server 30 are connected to each other via a network N. Although Fig. 1 illustrates one vehicle 12 and one on-board device 20 for one center server 30, the numbers of vehicles 12 and on-board devices 20 are not limited to this.
[0029] The center server 30 is installed, for example, at a manufacturer that produces the vehicle 12, a car dealer, or any other business entity.
[0030] (vehicle) As shown in FIG. 2, the vehicle 12 according to this embodiment includes an in-vehicle device 20, a plurality of ECUs (Electronic Control Units) 22, and a plurality of in-vehicle devices 24.
[0031] The vehicle-mounted device 20 includes a central processing unit (CPU) 20A, a read only memory (ROM) 20B, a random access memory (RAM) 20C, a storage 20D, a wireless communication interface (I / F) 20E, and an in-vehicle communication I / F 20F. The CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, the wireless communication interface (I / F) 20E, and the in-vehicle communication I / F 20F are connected to each other so as to be able to communicate with each other via an internal bus 20G.
[0032] The CPU 20A is a central processing unit that executes various programs and controls each part. That is, the CPU 20A reads a program from the ROM 20B or the storage 20D, and executes the program using the RAM 20C as a working area.
[0033] ROM 20B stores various programs and various data. As shown in Fig. 3, ROM 20B of this embodiment stores a control program 100. The control program 100 is a program for transmitting information on restriction displays that is digitized from an image captured by a camera to the center server 30.
[0034] As shown in FIG. 2, the RAM 20C temporarily stores programs or data as a working area.
[0035] The storage 20D, which is a memory, is configured with a HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs and various data. As shown in Fig. 4, the storage 20D of this embodiment stores a regulation information DB (database) 120, which is a data group of information on digitized regulation display. The storage 20D may store the control program 100 and the regulation information DB 120.
[0036] 2, the wireless communication I / F 20E is a wireless communication module for communicating with the center server 30. The wireless communication module uses a communication standard such as 5G, LTE, or Wi-Fi (registered trademark). The wireless communication I / F 20E is connected to a network N.
[0037] The in-vehicle communication I / F 20F is an interface for connecting to each ECU 22. The interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F 20F is connected to the external bus 20H.
[0038] The ECU 22 includes at least an ADAS (Advanced Driver Assistance System)-ECU 22A, a steering ECU 22B, a brake ECU 22C, and an engine ECU 22D.
[0039] The ADAS-ECU 22A controls the advanced driving assistance system. A vehicle speed sensor 24A, a yaw rate sensor 24B, and a camera 24C that configure the in-vehicle device 24 are connected to the ADAS-ECU 22A. The camera 24C is a camera that captures images of the surroundings of the vehicle 12.
[0040] The steering ECU 22B controls the power steering. A steering angle sensor 24D constituting the in-vehicle device 24 is connected to the steering ECU 22B. The steering angle sensor 24D is a sensor that detects the steering angle of the steering wheel.
[0041] As shown in FIG. 5, in the vehicle-mounted device 20 of this embodiment, a CPU 20A executes a control program 100 to function as an acquisition unit 200, a processing unit 210, and a transmission unit 220.
[0042] The acquisition unit 200 has a function of acquiring vehicle information. The vehicle information is information such as the state of the in-vehicle device 24, the state of the vehicle 12 obtained from the in-vehicle device 24, and captured images captured in the vehicle 12. The vehicle information in this embodiment includes driving information such as position information, vehicle speed, yaw rate, and steering angle. The vehicle information also includes captured images of the outside of the vehicle 12 captured by the camera 24C.
[0043] The processing unit 210 has a function of storing the vehicle information acquired by the acquisition unit 200 in the storage 20D, and converting the information into information on restriction signs by recognizing restriction signs from an image captured by the camera 24C. Specifically, the processing unit 210 of this embodiment temporarily stores the vehicle information in the storage 20D, and then transmits the information to the center server 30 by the transmission unit 220 when requested by the center server 30. The processing unit 210 also converts the image captured by the camera 24C into information on restriction signs by image recognition processing, and derives the reliability of the information on restriction signs.
[0044] More specifically, the processing unit 210 recognizes a speed limit sign or a prohibition sign, which is a regulatory display shown in the captured image, and converts the information into information on the speed limit, stop sign, no parking, and no turn.
[0045] The transmission unit 220 has a function of transmitting the vehicle information and the restriction display information stored in the storage 20D to the center server 30. Specifically, the transmission unit 220 of the present embodiment transmits the vehicle information and the restriction display information stored in the storage 20D to the center server 30 when requested by the center server 30.
[0046] (Center server) As shown in Fig. 6, the center server 30 includes a CPU 30A, a ROM 30B, a RAM 30C, a storage 30D, and a communication I / F 30E. The CPU 30A, the ROM 30B, the RAM 30C, the storage 30D, and the communication I / F 30E are connected to each other so as to be able to communicate with each other via an internal bus 30G. The functions of the CPU 30A, the ROM 30B, the RAM 30C, the storage 30D, and the communication I / F 30E are the same as the functions of the CPU 20A, the ROM 20B, the RAM 20C, the storage 20D, and the wireless communication I / F 20E of the vehicle-mounted device 20 described above. The communication I / F 30E may perform wired communication. The CPU 30A is an example of a processor.
[0047] The storage 30D of this embodiment stores, as programs, a processing program 150, a regulated value information DB 160, and an extracted image DB 170. The processing program 150 and the regulated value information DB 160 may be stored in the ROM 30B.
[0048] The processing program 150 is a program for controlling the center server 30. With the execution of the processing program 150, the center server 30 collects information on restriction signs from the vehicle 12 and executes various processes for performing driving diagnosis on the driving information of the vehicle 12.
[0049] The regulation value information DB 160 stores information on regulation signs collected from the vehicles 12 and their reliability.
[0050] Specifically, the speed limit, stop signs, no parking, and no turns information for each location is stored together with its reliability.
[0051] The extracted image DB 170 stores captured images included in the vehicle information when a violation is diagnosed.
[0052] As shown in FIG. 7, in the center server 30 of this embodiment, a CPU 30A executes a processing program 150 to function as a collection unit 250, an acquisition unit 260, a preprocessing unit 270, a diagnosis unit 280, and an extraction unit 290.
[0053] The collection unit 250 has a function of acquiring information on restriction indications from the vehicle-mounted device 20 of the vehicle 12. The collection unit 250 stores the acquired information on restriction indications in the restriction value information DB 160 together with the reliability and location information indicating the digitized location.
[0054] The acquisition unit 260 has a function of acquiring vehicle information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12 .
[0055] If the reliability of the digitized regulatory display information is equal to or greater than a threshold, the pre-processing unit 270 sets the information as a regulatory value and updates the regulatory value at a certain time with one of the regulatory values in a certain interval including the time.
[0056] Specifically, when the converted traffic regulation indication is a sign related to a speed limit, the preprocessing unit 270 updates all traffic regulation values in the certain section with the maximum speed limit, which is the traffic regulation value in the certain section.
[0057] More specifically, if the reliability of a speed limit sign is equal to or greater than a threshold (e.g., 50%), the converted speed limit is recorded. Then, to avoid erroneous detection or non-detection of speed limit signs, a filter process is performed to replace the recorded speed limit with the maximum value for a certain section, which is one second before and after the speed limit, and all speed limits in the certain section are updated with the maximum value.
[0058] In addition, when the digitized regulatory display is a display related to a prohibited act, if the reliability of the digitized regulatory display information is equal to or greater than a threshold value (e.g., 50%), the pre-processing unit 270 sets a flag that is a regulatory value to on, and if a flag that is a regulatory value in a certain section is on, updates the flags that are all regulatory values in the certain section to on.
[0059] For example, if the reliability of a stop sign is 50% or higher, the flag for the stop sign is turned on (1) and recorded. Then, in order to avoid erroneous detection or non-detection of a stop sign, a filter process is performed to replace the recorded flag with the maximum value for a certain interval of 0.2 seconds before and after the sign, and all flags in the certain interval are updated with the maximum value.
[0060] In addition, when the reliability of a no parking sign is 50% or higher, the flag related to the no parking sign is turned on (1) and recorded. Then, in order to avoid erroneous detection or non-detection of the no parking sign, a filter process is performed to replace the recorded flag with the maximum value within a certain interval of 0.2 [sec] before and after the sign, and the maximum value is updated as all flags in the certain interval.
[0061] In addition, when the reliability of a no turn sign is 50% or higher, the flag related to the no turn sign is turned on (1) and recorded. Then, in order to avoid erroneous detection or non-detection of a no turn sign, a filter process is performed to replace the recorded flag with the maximum value for a certain interval of 0.2 seconds before and after the sign, and all flags in the certain interval are updated with the maximum value.
[0062] The diagnosis unit 280 diagnoses a violation when the driving of the vehicle based on the driving information included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 satisfies a violation condition based on the updated regulation value.
[0063] Specifically, the diagnosis unit 280 diagnoses whether the driving of the vehicle 12 is in violation, using a condition that the vehicle 12 is traveling at a speed significantly exceeding the speed limit around a speed limit sign as a violation condition.
[0064] For example, the violation condition is vehicle speed > speed limit + excessive speed. The excessive speed is 20km / h, 30km / h, or 40km / h.
[0065] Furthermore, the diagnosis unit 280 diagnoses whether the driving of the vehicle 12 violates a traffic violation condition of not stopping around a stop sign.
[0066] For example, if a flag related to a stop sign is on, a violation is provisionally diagnosed, but if the vehicle speed falls below a threshold value (e.g., 1 km / h), the provisional diagnosis of a violation is deleted and a diagnosis of no violation is made.
[0067] Furthermore, the diagnosis unit 280 diagnoses whether the driving of the vehicle 12 is in violation of a no parking sign, taking parking in the vicinity of a no parking sign as a violation condition.
[0068] For example, if the flag related to the no parking sign is on and the vehicle speed is below a threshold value (e.g., 1 km / h), a provisional diagnosis of a violation is made, and if the duration during which the vehicle speed is below the threshold value (1 km / h) is less than a threshold value (e.g., 180 seconds), the provisional diagnosis of a violation is removed and a diagnosis of no violation is made.
[0069] Furthermore, the diagnosis unit 280 diagnoses whether the driving of the vehicle 12 is in violation of a no turn sign, using the violation condition of making a turn around the no turn sign.
[0070] For example, if the flag related to the no turn sign is on, a violation is provisionally diagnosed, and if the maximum yaw angle of the vehicle 12 is less than a threshold value (e.g., 150°) and the maximum steering angle of the steering wheel of the vehicle 12 is less than a threshold value (e.g., 400°), the provisional diagnosis of a violation is removed and a diagnosis of no violation is made.
[0071] When the violation condition is satisfied, the extraction unit 290 extracts the captured image included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 and stores the extracted image in the extracted image DB 170.
[0072] In addition, when the violation condition is no longer satisfied, the extraction unit 290 stops extracting the captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12, and stores the extracted captured images in the extracted image DB 170.
[0073] Furthermore, the extraction unit 290 deletes the extracted captured image when a predetermined violation cancellation condition based on the driving information is satisfied.
[0074] Specifically, when the violation condition of driving significantly exceeding the speed limit around a speed limit sign is met, extraction of captured images contained in the vehicle information obtained from the vehicle-mounted device 20 of the vehicle 12 is started, and when the violation condition is no longer met, extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170.
[0075] When the flag related to the stop sign is on, extraction of captured images included in the vehicle information obtained from the vehicle-mounted device 20 of the vehicle 12 is started, and when a predetermined time (e.g., 10 seconds) has elapsed, extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170. A violation release condition is that the minimum vehicle speed is less than a threshold (e.g., 1 km / h), and when the violation release condition is satisfied, it is considered that the vehicle is temporarily stopped, and the extracted captured images are deleted.
[0076] In addition, when the flag related to the no parking sign is on and the vehicle speed is less than a threshold value (e.g., 1 km / h), extraction of captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 is started, and when the vehicle speed becomes equal to or greater than the threshold value (1 km / h), extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170. In addition, a violation release condition is that the duration of the state in which the vehicle speed is less than the threshold value (1 km / h) is less than a predetermined time (e.g., 180 seconds), and when the violation release condition is satisfied, it is deemed not to be parking, and the extracted captured images are deleted. A violation release condition is that the flag related to the no parking sign is not on in the section in which the vehicle speed is less than the threshold value (e.g., 1 km / h) and in the predetermined time (e.g., 5 seconds) before the section, and when the violation release condition is satisfied, the extracted captured images are deleted.
[0077] In addition, the violation cancellation condition is that someone gets in and out of the vehicle (passenger seat, back seat) within a section where the vehicle speed is below a threshold value (for example, 1 km / h), and if the violation cancellation condition is met, it is deemed not to be parking, and the extracted captured image is deleted.
[0078] In addition, when the flag related to the no turn sign is on, extraction of captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 is started, and when a predetermined time (e.g., 10 seconds) has elapsed, extraction of captured images is terminated and the extracted captured images are stored in the extracted image DB 170. A violation cancellation condition is that the maximum yaw angle of the vehicle 12 is less than a threshold value (e.g., 150°) and the maximum steering angle of the steering wheel of the vehicle 12 is less than a threshold value (e.g., 400°), and when the violation cancellation condition is satisfied, the extracted captured images are deleted.
[0079] (Flow of Control) The flow of processing executed in the driving diagnosis system 10 of this embodiment will be described with reference to the flowcharts of Fig. 8 to Fig. 11. The processing in the vehicle-mounted device 20 is executed by the CPU 20A of the vehicle-mounted device 20 functioning as an acquisition unit 200, a processing unit 210, and a transmission unit 220. The processing in the center server 30 is executed by the CPU 30A of the center server 30 functioning as a collection unit 250, an acquisition unit 260, a pre-processing unit 270, a diagnosis unit 280, and an extraction unit 290.
[0080] First, while the vehicle 12 is traveling, the CPU 20A of the vehicle-mounted device 20 executes a restriction display information transmission process shown in FIG.
[0081] In step S100 of FIG. 8, the processing unit 210 acquires a captured image of the outside of the vehicle 12 captured by the camera 24C.
[0082] In step S102, the processing unit 210 recognizes the restriction sign from the captured image captured by the camera 24C, converts the restriction sign into information, derives the reliability of the information of the restriction sign, and temporarily stores it in the restriction information DB 120.
[0083] In step S104, the processing unit 210 judges whether or not the restriction display has been recognized by the processing in step S102. If the restriction display has been recognized by the processing in step S102, the processing proceeds to step S106. On the other hand, if the restriction display has not been recognized by the processing in step S102, the processing for transmitting restriction display information is terminated.
[0084] In step S106, the transmission unit 220 transmits the information on the restriction display obtained as the recognition result in step S102 to the center server 30 together with the reliability and position information of the information on the restriction display, and ends the restriction display information transmission process.
[0085] The center server 30 stores the restriction display information received from the vehicle-mounted device 20 in the restriction value information DB 160 together with the reliability and the location information of the digitized location.
[0086] Furthermore, while the vehicle 12 is traveling, the vehicle-mounted device 20 transmits the acquired vehicle information to the center server 30. At this time, in the center server 30, the CPU 30A repeatedly executes the driving diagnosis process shown in Figs.
[0087] Specifically, CPU 30A repeatedly executes the driving diagnosis process shown in FIG. 9 for a speed limit sign as a regulatory indication.
[0088] In step S110 of FIG. 9, the acquisition unit 260 acquires the vehicle information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.
[0089] In step S112, the pre-processing unit 270 obtains, from the regulation value information DB 160, the speed limit, which is the regulation value corresponding to the position information included in the obtained vehicle information, and the reliability of the sign related to the speed limit.
[0090] In step S114, if the reliability of the speed limit sign obtained in step S112 is equal to or greater than a threshold (e.g., 50%), the pre-processing unit 270 records the converted speed limit. Then, the pre-processing unit 270 performs a filter process to replace the recorded speed limit with the maximum value in a certain section, which is one second before and after the speed limit, and updates all speed limits in the certain section with the maximum value.
[0091] In step S116, the diagnosis unit 280 determines whether or not a violation condition is met, where the vehicle 12 is traveling at a speed significantly exceeding the speed limit around a speed limit sign. If the violation condition is met, the diagnosis unit 280 diagnoses that the driving of the vehicle 12 is a violation, and the process proceeds to step S118. On the other hand, if the violation condition is not met, the process proceeds to step S120.
[0092] In step S118 , the extraction unit 290 starts extracting captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 .
[0093] In step S120, the extraction unit 290 determines whether or not a captured image is being extracted. If a captured image is being extracted, the process proceeds to step S122, where the extraction of the captured image is terminated, and the extracted captured image is stored in the extracted image DB 170. On the other hand, if a captured image is not being extracted, the driving diagnosis process is terminated.
[0094] Furthermore, for a stop sign as a regulatory display, the CPU 30A repeatedly executes the driving diagnosis process shown in Fig. 10. Note that the same processes as those in Fig. 9 will be described with the same reference numerals.
[0095] In step S110 of FIG. 10, the acquisition unit 260 acquires vehicle information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.
[0096] In step S112, the pre-processing unit 270 obtains, from the regulation value information DB 160, a flag related to a stop sign, which is a regulation value corresponding to the position information included in the obtained vehicle information, and a reliability of the stop sign.
[0097] In step S114, if the reliability of the stop sign acquired in step S112 is equal to or greater than a threshold (e.g., 50%), the pre-processing unit 270 turns on (1) the flag related to the stop sign and records it. Then, the pre-processing unit 270 performs a filter process to replace the recorded flag with the maximum value in a certain section, which is 0.2 seconds before and after the stop sign, and updates all flags in the certain section with the maximum value.
[0098] In step S200, the diagnosis unit 280 judges whether the flag related to the stop sign acquired in the above step S112 is on. If the flag related to the stop sign acquired in the above step S112 is not on, the driving diagnosis process is terminated. On the other hand, if the flag related to the stop sign acquired in the above step S112 is on, in step S202, the diagnosis unit 280 provisionally diagnoses a violation, and the extraction unit 290 starts extracting captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 when the flag related to the stop sign is on, and ends the extraction of captured images when a predetermined time (for example, 10 seconds) has elapsed, and stores the extracted captured images in the extracted image DB 170.
[0099] In step S204, the extraction unit 290 determines whether or not the violation release condition is satisfied, with the minimum vehicle speed being less than a threshold (for example, 1 km / h). If the violation release condition is not satisfied, the diagnosis of violation is confirmed and the driving diagnosis process is terminated. On the other hand, if the violation release condition is satisfied, it is considered that the vehicle is temporarily stopped, and the process proceeds to step S206.
[0100] In step S206, the extraction unit 290 deletes the provisional diagnosis of a violation and the extracted captured image, and ends the driving diagnosis process.
[0101] Furthermore, for a no turn sign as a regulatory display, the CPU 30A repeatedly executes the driving diagnosis process shown in FIG. 10, in the same manner as for a stop sign.
[0102] Here, in step S112, the pre-processing unit 270 obtains, from the regulation value information DB 160, a flag related to the no turn sign, which is a regulation value corresponding to the position information included in the obtained vehicle information, and the reliability of the no turn sign.
[0103] In step S114, if the reliability of the no-turn sign acquired in step S112 is equal to or greater than a threshold (e.g., 50%), the pre-processing unit 270 turns on (1) the flag related to the no-turn sign and records it. Then, the pre-processing unit 270 performs a filter process to replace the recorded flag with the maximum value in a certain section, which is 0.2 seconds before and after the sign, thereby updating all flags in the certain section with the maximum value.
[0104] In step S200, the diagnosis unit 280 judges whether the flag related to the no turn sign acquired in the above step S112 is on. If the flag related to the no turn sign acquired in the above step S112 is not on, the driving diagnosis process is terminated. On the other hand, if the flag related to the no turn sign acquired in the above step S112 is on, in step S202, the diagnosis unit 280 provisionally diagnoses a violation, and the extraction unit 290 starts extracting captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 when the flag related to the no turn sign is on, and when a predetermined time (for example, 10 seconds) has elapsed, ends the extraction of captured images, and stores the extracted captured images in the extracted image DB 170.
[0105] In step S204, the extraction unit 290 determines whether or not a violation release condition is satisfied, with the maximum yaw angle of the vehicle 12 being less than a threshold value (e.g., 150°) and the maximum steering angle of the steering wheel of the vehicle 12 being less than a threshold value (e.g., 400°). If the violation release condition is not satisfied, the diagnosis of a violation is confirmed and the driving diagnosis process is terminated. On the other hand, if the violation release condition is satisfied, it is determined that no turn has been made and the process proceeds to step S206.
[0106] In step S206, the extraction unit 290 deletes the provisional diagnosis of a violation and the extracted captured image, and ends the driving diagnosis process.
[0107] Furthermore, for a no parking sign as a regulatory display, the CPU 30A repeatedly executes the driving diagnosis process shown in Fig. 11. Note that the same processes as those in Fig. 9 will be described with the same reference numerals.
[0108] In step S110 of FIG. 11, the acquisition unit 260 acquires vehicle information of the vehicle 12 from the vehicle-mounted device 20 of the vehicle 12.
[0109] In step S112, the pre-processing unit 270 obtains, from the regulation value information DB 160, a flag related to a no parking sign, which is a regulation value corresponding to the position information included in the obtained vehicle information, and a reliability of the no parking sign.
[0110] In step S114, when the reliability of the no parking sign acquired in step S112 is equal to or greater than a threshold (e.g., 50%), the pre-processing unit 270 turns on (1) the flag related to the no parking sign and records it. Then, the pre-processing unit 270 performs a filter process to replace the recorded flag with the maximum value in a certain section, which is 0.2 seconds before and after the sign, and updates all flags in the certain section with the maximum value.
[0111] In step S116, the diagnosis unit 280 determines whether or not the violation condition is met, taking parking in the vicinity of a no parking sign as the violation condition. If the violation condition is met, the driving of the vehicle 12 is diagnosed as a violation, and the process proceeds to step S118. On the other hand, if the violation condition is not met, the process proceeds to step S120.
[0112] In step S118 , the extraction unit 290 starts extracting captured images included in the vehicle information acquired from the vehicle-mounted device 20 of the vehicle 12 .
[0113] In step S120, the extraction unit 290 determines whether or not a captured image is being extracted. If a captured image is being extracted, the process proceeds to step S122, where the extraction of the captured image is terminated, and the extracted captured image is stored in the extracted image DB 170. On the other hand, if a captured image is not being extracted, the driving diagnosis process is terminated.
[0114] In step S300, the extraction unit 290 determines whether or not the violation release condition is satisfied, with the duration of the state in which the vehicle speed is less than a threshold (e.g., 1 km / h) being less than a predetermined time (e.g., 180 seconds). If the violation release condition is not satisfied, the diagnosis of violation is confirmed and the driving diagnosis process is terminated. On the other hand, if the violation release condition is satisfied, it is determined that the vehicle is not parked, and the process proceeds to step S302.
[0115] In step S302, the extraction unit 290 deletes the provisional diagnosis of a violation and the extracted captured image, and ends the driving diagnosis process.
[0116] (Summary of the embodiment) When the reliability of the digitized information of the restriction display is equal to or higher than a threshold, the center server 30 of this embodiment sets the digitized information of the restriction display to a restriction value and updates the restriction value at a certain time with the maximum restriction value in a certain section including the time. Therefore, the center server 30 can perform a driving diagnosis without shaking based on the restriction display acquired from the captured image.
[0117] Furthermore, the center server 30 of the present embodiment extracts a captured image when a violation condition is satisfied. Therefore, the center server 30 can record a captured image when a violation is diagnosed.
[0118] In addition, the center server 30 of the present embodiment deletes the extracted captured image when a predetermined violation release condition based on the driving information is satisfied. Therefore, the center server 30 can delete the extracted captured image when a violation is once diagnosed, and then it is determined that there is no violation.
[0119] [remarks] In the above embodiment, all regulated values in a certain interval were updated with the maximum regulated value in the certain interval, but this is not limited thereto, and all regulated values in a certain interval may be updated with the frequently occurring value or the median value of the regulated value in the certain interval.
[0120] In addition, various processes executed by the CPU 20A and the CPU 30A by reading the software (programs) in the above embodiment may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. In addition, each of the above-mentioned processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0121] In the above embodiment, each program has been described as being prestored (installed) in a non-transitory computer-readable recording medium. For example, the control program 100 in the vehicle-mounted device 20 is prestored in the ROM 20B, and the processing program 150 in the center server 30 is prestored in the storage 30D. However, the present invention is not limited to this, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Also, the program may be downloaded from an external device via a network.
[0122] The process flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be changed, without departing from the spirit of the invention. [Explanation of symbols]
[0123] 12 Vehicles 20 Onboard equipment 30 Center server (information processing device) 30A CPU (processor) 30D Storage (Memory) 150 Processing Program (Program) 250 Collection Department 260 Acquisition Department 270 Pretreatment section 280 Diagnostic Department 290 Extraction part
Claims
1. an acquisition unit that acquires driving information related to the driving of a vehicle and information on a regulatory display that is digitized from an image of the regulatory display captured by a camera mounted on the vehicle; If the reliability of the information of the digitized regulatory indication is equal to or greater than a threshold, the information is set to a regulatory value; a pre-processing unit that updates a regulation value at a certain time with any one of the regulation values in a certain interval including the certain time; a diagnosis unit that diagnoses a violation when the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulation value; Including, The regulation sign is a sign related to a speed limit, The information on the regulation display is a speed limit, The preprocessing unit is an information processing device that updates all regulation values in the certain interval with a maximum regulation value in the certain interval.
2. The information processing apparatus according to claim 1 , further comprising an extraction unit that extracts the captured image when the violation condition is satisfied.
3. The information processing apparatus according to claim 2 , wherein the extraction section terminates the extraction of the captured image when the violation condition is no longer satisfied.
4. The information processing device according to claim 2 , wherein the extraction unit deletes the extracted captured image when a predetermined violation removal condition based on the driving information is satisfied.
5. Acquiring driving information related to the driving of a vehicle and information on a regulatory display that is digitized from an image of the regulatory display captured by a camera mounted on the vehicle; If the reliability of the information of the digitized regulatory indication is equal to or greater than a threshold, the information is set to a regulatory value; updating a regulation value at a certain time with any one of the regulation values in a certain interval including the time; When the driving of the vehicle based on the driving information satisfies a violation condition based on the updated regulation value, a violation is diagnosed. Including, The regulation sign is a sign related to a speed limit, The information on the regulation display is a speed limit, In the updating, all the regulation values in the certain section are updated with the maximum regulation value in the certain section. A driving diagnosis method in which processing is performed by a computer.
6. Acquiring driving information related to the driving of a vehicle and information on a regulatory display that is digitized from an image of the regulatory display captured by a camera mounted on the vehicle; If the reliability of the information of the digitized regulatory indication is equal to or greater than a threshold, the information is set to a regulatory value; updating a regulation value at a certain time with any one of the regulation values in a certain interval including the time; The traveling of the vehicle based on the traveling information satisfies a violation condition based on the updated regulation value. If this is the case, a violation will be diagnosed. Including, The regulation sign is a sign related to a speed limit, The information on the regulation display is a speed limit, In the updating, all the regulation values in the certain section are updated with the maximum regulation value in the certain section. A program that causes a computer to carry out processing.
Citation Information
Patent Citations
Cruise traveling control device of car
JP2006321354A
Safe driving degree storage system, on-vehicle equipment and center
JP2007172487A
Operation management device for vehicle and computer program
JP2015014988A
Vehicle drive support system
JP2021056575A
