System, program, machine learning method, machine learning model, and the like

Abstract

To provide a technique for detecting a target without depending only on an electromagnetic wave in a predetermined wavelength band such as a laser and a radar.SOLUTION: The system 100 includes a camera 21 that is mounted on a vehicle and captures an image of the outside of the vehicle to generate a captured image, detection units 22 and 33 that detect a speed limit violation control device from the captured image using a machine learning model, and control units 24 and 32 that perform control based on the detection results of the detection units 22 and 33.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system, a program, a machine learning method, a machine learning model, and the like. [Background technology]

[0002] Speed ​​enforcement devices for enforcing vehicle speed limits are sometimes installed at speed enforcement points on roads on which vehicles travel. Speed ​​enforcement devices estimate the vehicle's speed by irradiating a traveling vehicle with a laser or radar (hereinafter, these may be collectively referred to as "electromagnetic waves") in a specified wavelength band and receiving or sensing the reflected radio waves or light. If a vehicle is traveling at a speed exceeding the legal speed limit set for that road, the speed enforcement device detects the vehicle as speeding and photographs it with a camera.

[0003] Generally, speed enforcement devices are installed in places where drivers tend to unconsciously increase their vehicle speed, such as on expressways and public roads with good visibility. Therefore, if a vehicle can detect speed enforcement devices and alert the driver, it can help the driver to drive safely.

[0004] Conventionally, there have been provided detectors that are mounted on vehicles and that detect lasers or radars emitted from speed enforcement devices and notify the driver of the presence of the speed enforcement device. These detectors can notify the driver of the presence of the speed enforcement device and encourage safe driving. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-196066 Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply detecting electromagnetic waves can result in electromagnetic waves being received from vending machines or automatic doors, which may be mistakenly recognized as speed enforcement devices.

[0007] In view of the above background, one of the objects of the present invention is to provide a technology for detecting targets indicating speed control points without relying solely on electromagnetic waves in a specified wavelength band, such as laser or radar.

[0008] Another object of the present invention is to aggregate images of a predetermined target taken from a vehicle in a server.

[0009] Furthermore, an object of the present invention is to provide a technology related to generating a learning model for detecting targets that indicate speed control points.

[0010] Furthermore, the purpose of the present invention is not limited to these, and the applicant intends to obtain rights for configurations that aim to achieve the effects achieved by parts of the configuration disclosed in the specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses problems in which the phrase "can be" is read as "the problem is...." Each problem is described as an independent problem, and the applicant intends to obtain rights for configurations that solve each problem separately through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description in the specification, the applicant intends to include part of the configuration described in this specification in the scope of the patent claim through amendments or divisional applications. Furthermore, the applicant has disclosed configurations that solve problems that combine these independent problems, and the applicant intends to obtain rights for them. [Means for solving the problem]

[0011] (1) One embodiment of the system includes a detection unit that uses a machine learning model to detect targets indicating speed enforcement points from an image captured outside the vehicle, and a control unit that performs control based on the detection results from the detection unit. This makes it possible to detect targets indicating speed enforcement points from the captured image. For example, this reduces the possibility of misidentifying electromagnetic wave sources other than speed enforcement devices, such as vending machines and automatic doors, as speed enforcement devices.

[0012] (2) In one aspect of the system, in the system of (1), the detection unit may include a primary detection unit that detects the target using a first machine learning model, and a secondary detection unit that detects the target using a second machine learning model when the target is detected by the primary detection unit, and the control unit may perform control based on detection results from the primary detection unit and the secondary detection unit. In this way, the target is detected using a two-stage method using the first machine learning model and the second machine learning model, thereby enabling more accurate detection of the target.

[0013] (3) In one aspect of the system, in the system of (2), the primary detection unit may be provided in an on-board device mounted in the vehicle, and the secondary detection unit may be provided in a server that communicates with the on-board device via a communication network. This eliminates the need for the on-board device to have a second machine learning model, and the second machine learning model can be shared by, for example, multiple on-board devices. Furthermore, by transmitting captured images in which targets are detected by the on-board device's primary detection to the server, the processing load and communication load on the server can be reduced compared to transmitting all captured images.

[0014] (4) In one aspect of the system, in the system of (2) or (3), the primary detection unit detects the target from the image by performing pattern matching with binary data as the first machine learning model, and the secondary detection unit detects the target from the image using a deep learning model as the second machine learning model. In this way, it is expected that primary detection can be performed faster than secondary detection.

[0015] (5) In one embodiment, the system of any one of (2) to (4) further includes a detector for detecting electromagnetic waves in a predetermined wavelength band, and the primary detection unit detects the target based on the first machine learning model and the detection result of the detector. In this way, primary detection can be performed by using a detection method of a conventional speed enforcement device in combination.

[0016] (6) In one aspect of the system, in the system of (5), the control unit may perform the control when the target is detected from the image based on the first machine learning model during the period when the detector receives the electromagnetic waves. In this way, the control when a speed enforcement device is detected is performed when electromagnetic waves in a predetermined wavelength band used by a speed enforcement device are received, thereby increasing the possibility of performing the control when a speed enforcement device is actually present.

[0017] (7) In one aspect of the system, in any one of the systems (1) to (6), the control unit may include a first control unit that controls an alarm unit provided in the vehicle to output an alarm based on the detection result of the primary detection unit. In this way, an alarm can be issued to a person in the vehicle (e.g., the driver) based on the detection result of the primary detection.

[0018] (8) In one aspect of the system, in any one of the systems (2) to (6), the control unit may include a first control unit that controls an alarm unit provided in the vehicle to output an alarm based on the detection result of the secondary detection unit. In this way, an alarm can be issued to a person in the vehicle (e.g., the driver) based on the detection result of the secondary detection.

[0019] (9) In one aspect of the system, in the systems of (2) to (6), the control unit may include a first control unit that controls an alarm unit provided in the vehicle to output a first alarm based on the detection result of the primary detection unit, and controls an alarm unit provided in the vehicle to output a second alarm different from the first alarm based on the detection result of the secondary detection unit. In this way, a person in the vehicle (e.g., the driver) can recognize the alarm by distinguishing between the result of the primary detection and the result of the secondary detection.

[0020] (10) In one aspect of the system, in any one of the systems (7) to (9), the first control unit may perform control to output different warnings depending on the likelihood of the target detection result. In this way, different warnings are issued depending on the likelihood of the detection result of the speed enforcement device, so that a person in the vehicle (e.g., the driver) can grasp the possibility that a speed enforcement device actually exists.

[0021] (11) In one aspect, the system of any one of (1) to (10) further includes a storage unit that stores the position information of the target, and the control unit may include a second control unit that controls updating of the position information of the target in the storage unit based on detection results from a plurality of the detection units mounted on different vehicles. In this way, the position information of the position where the target is detected can be stored and updated according to the detection results of the target.

[0022] (12) In one aspect, the system of any one of (7) to (10) further includes a storage unit that stores position information of the target, and the control unit has a second control unit that controls updating of the position information of the target in the storage unit based on detection results from a plurality of the detection units mounted on different vehicles, and the first control unit controls outputting of the warning based on the position information of the target stored in the storage unit and the position of the vehicle. In this way, the position information of the position where the target is detected can be stored and updated according to the detection result of the target, and a warning can be issued to a person in the vehicle (e.g., the driver) according to the position of the vehicle relative to the target.

[0023] (13) In one aspect of the system, in the system of (11) or (12), the first control unit may be provided in an on-board device mounted in a vehicle, and the second control unit may be provided in a server that communicates with the on-board device via a communication network. In this way, the server acquires the results of target detection by the on-board device, so that the target detection results can be shared among multiple on-board devices.

[0024] (14) One aspect of the system includes an on-board device mounted in a vehicle and a server installed on a communication network. The on-board device includes a camera that captures images of the exterior of the vehicle and generates captured images, and a vehicle-side transmitter that transmits the captured images to the server via the communication network. The server includes a server-side receiver that receives the captured images from the on-board device via the communication network, a detection unit that detects targets from the captured images using a machine learning model, and a storage unit that stores the detection results. In this manner, the server can acquire captured images from the on-board device, detect targets, and store the detection results. In particular, the server can acquire captured images from multiple vehicles to obtain detection results for a large number of targets.

[0025] (15) In one aspect of the system, in the system of (14), the in-vehicle device may include a primary detection unit that detects the target from the captured image using a machine learning model with a lower detection standard than the machine learning model, the vehicle-side transmission unit may transmit the captured image in which the target is detected by the primary detection unit to the server, and the detection unit may function as a secondary detection unit that detects the target from the captured image received by the server-side reception unit. In this way, the captured image in which the target is detected by the primary detection of the in-vehicle device is transmitted to the server, thereby reducing the communication load and the load of secondary detection.

[0026] (16) In one aspect of the system, in the system of (14), the on-board device includes a location information acquisition unit that acquires location information of the vehicle, the vehicle-side transmission unit transmits the captured image together with the location information of the vehicle when the captured image was taken to the server, and the storage unit stores the detection result in association with the location information. In this way, the location information of the location where the target object was detected can be stored in the server.

[0027] (17) In one aspect of the system, in the system of (16), the on-board device may include a vehicle-side receiving unit that receives a notification based on the information stored in the storage unit, and an output unit that outputs information based on the notification. In this way, the vehicle can receive a notification regarding the presence of a target.

[0028] (18) One aspect of the system is a system including an in-vehicle device, the in-vehicle device including: a detection unit that detects a target object from an image captured outside the vehicle using a first machine learning model; a transmission unit that transmits the image to a server when the detection unit detects the target object; a reception unit that receives from the server a result of processing that detects the target object from the image using a second machine learning model; and a control unit that performs control based on the detection result by the detection unit and the received result of the processing. In this way, the in-vehicle device can obtain a target detection result using the second machine learning model even if it does not include the second machine learning model.

[0029] (19) In one embodiment, the system acquires multiple images of targets that indicate a speed enforcement zone, and performs learning using training data associated with the inclusion of the target in each of the acquired images. In this way, a learning model can be generated for detecting targets that indicate a speed enforcement zone from images.

[0030] (20) In one aspect of the system, supervised learning is performed using, as a positive example, an image taken of the outside of a vehicle by a camera mounted on the vehicle, the image taken when electromagnetic waves in a predetermined wavelength band are detected by a detector mounted on the vehicle. In this way, the image taken when electromagnetic waves are received is used as the positive example, making it possible to accurately obtain positive examples to be used in supervised learning for target detection.

[0031] (21) In one aspect, the system of (20) further includes a detection unit that detects targets indicating a vehicle speed enforcement point from the image, and performs supervised learning using the image when the target is detected and the detector detects electromagnetic waves in the predetermined wavelength band as a positive example. This reduces the possibility of learning, as a positive example, an image captured when no target is detected but the detector detects electromagnetic waves in the predetermined wavelength band, i.e., an image captured when the detector detects electromagnetic waves in the predetermined wavelength band from another electromagnetic wave source such as a vending machine.

[0032] (22) In one aspect, the system of (20) or (21) further includes a detection unit for detecting the target from the image, and the supervised learning is performed using an image in which the detector does not detect electromagnetic waves in the predetermined wavelength band but detects the target as a negative example. In this way, if the detector does not detect electromagnetic waves in the predetermined wavelength band but detects a speed enforcement device, it is considered that an object that can be confused with the target is reflected in the captured image, and in this way, a high-quality negative example can be obtained.

[0033] (23) In one aspect, the system of any one of (19) to (22) may accept submissions of the images taken when a person inside the vehicle visually confirms the target, and perform the supervised learning using the submitted images as positive examples. In this way, a higher quality machine learning model can be generated by performing supervised learning using photographed images that a user visually confirms contain the target.

[0034] (24) In one aspect, the system of (23) may accept, in addition to the image taken when a person in the vehicle visually confirms the target, a submission of the detection result of the electromagnetic waves in the predetermined wavelength band by the detector, and perform the supervised learning on the submitted image as a positive example or a negative example depending on the submitted detection result. In this way, by using the detection result of the electromagnetic waves in the predetermined wavelength band in addition to the captured image in which the user visually confirms that the target is included, a better machine learning model can be generated.

[0035] (25) In one aspect, the system of any one of (19) to (24) may receive a response input from a person in the vehicle when the target is detected from the image and an alarm is issued, and perform the supervised learning using the image as a positive or negative example depending on the result of the received response. In this way, when an alarm is issued, the person in the vehicle inputs a response indicating whether the alarm is a correct or false alarm, and the result of this response is reflected in the machine learning model, thereby generating a better machine learning model.

[0036] (26) In one aspect of the system, in the systems (1) to (25), the target may be a vehicle enforcement device. In this way, it is possible to provide a technology for detecting a vehicle enforcement device as a target.

[0037] (27) In one aspect, the program is a program for causing a computer to realize the functions realized by an on-board device, at least a part of the functions of any of the systems (1) to (26). In this way, a program for detecting a target from an image can be provided.

[0038] (28) In one embodiment of the machine learning method, a learning model for detecting a target indicating a speed control point from an image is generated by acquiring each of a plurality of images of a target indicating a speed control point, and learning is performed using training data associated with the inclusion of the target in each of the acquired images.

[0039] (29) One embodiment of the machine learning model includes an input layer to which an image is input, an output layer that outputs data corresponding to the likelihood that the image contains a target object that indicates a vehicle speed enforcement site, and at least one intermediate layer whose parameters are trained using training data that receives each of a plurality of images as input and outputs data corresponding to the likelihood that the image contains the target object. In this way, it is possible to provide a machine learning model for detecting targets that indicate a vehicle speed enforcement site from an image.

[0040] The inventions described in (1) to (29) above can be combined in any way. For example, it is desirable to combine all or part of the invention described in (1) with at least part of the structure of at least one of the inventions described in (2) and subsequent paragraphs. In particular, it is desirable to combine the invention described in (1) with at least part of the structure of at least one of the inventions described in (2) and subsequent paragraphs. Furthermore, it is also possible to extract any structure from the inventions described in (1) to (29) and combine the extracted structures. The applicant of this application intends to obtain rights to inventions including these structures. Furthermore, even if a description is made of "in the case of..." or "when...," it is not intended to describe a structure limited to that case or time. These are merely examples of better structures, and the applicant intends to obtain rights to structures other than these cases or times. Furthermore, any descriptions that specify an order are not limited to this order. The applicant also discloses structures in which some parts are deleted or the order is changed, and the applicant intends to obtain rights to such structures. [Effects of the Invention]

[0041] According to one aspect, a technology for detecting speed enforcement devices without relying solely on electromagnetic waves in a predetermined wavelength band can be provided. According to another aspect, images of predetermined targets captured by a vehicle can be aggregated in a server. Furthermore, according to yet another aspect, a technology for generating a learning model for detecting targets indicating speed enforcement points can be provided.

[0042] Furthermore, the effects of the present invention are not limited to these, and effects achieved by the components disclosed in the specification and drawings, etc. are also disclosed, and the applicant intends to obtain rights to the components that achieve these effects through divisional applications, amendments, etc. For example, in this specification, statements such as "can..." clearly state the effects that are achieved, and there are also parts that demonstrate the effects even without the statement "can...". Furthermore, there are effects that can be understood from the components even without such statements. [Brief explanation of the drawings]

[0043] [Figure 1] 1 is a diagram illustrating a configuration of a detection system according to an embodiment of the present invention. [Figure 2A] FIG. 10 is a diagram illustrating an example of an object detection process according to an embodiment of the present invention. [Figure 2B] FIG. 10 is a diagram illustrating an example of an object detection process according to an embodiment of the present invention. [Figure 2C] FIG. 10 is a diagram illustrating an example of an object detection process according to an embodiment of the present invention. [Figure 2D] FIG. 10 is a diagram illustrating an example of an object detection process according to an embodiment of the present invention. [Figure 3] 1 is a flowchart of a detection method according to an embodiment of the present invention. [Figure 4] 1 is a flowchart of a machine learning method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0044] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment described below shows an example of how the present invention can be implemented, and the present invention is not limited to the specific configuration described below. When implementing the present invention, a specific configuration corresponding to the embodiment may be appropriately adopted.

[0045] FIG. 1 is a diagram showing the configuration of a system 100 according to an embodiment of the present invention. The system 100 includes a detector 10, a drive recorder 20 as an in-vehicle device (also referred to as a vehicle-side device), and a server 30. The detector 10 and the drive recorder 20 are mounted on a vehicle. The vehicle may be, for example, a privately owned vehicle or a commercial vehicle, and may be a passenger car, a bus, a truck, a forklift, or other specialized vehicle. The server 30 is installed in a location separate from the vehicle. The drive recorder 20 and the server 30 can transmit and receive information to each other via a communication network. The communication network may be, for example, a public communication line such as LTE, 4G, 5G, or other system. The drive recorder 20 can be directly or indirectly connected to a communication network such as a mobile phone network via wireless communication. Note that the example of FIG. 1 shows the detector 10 and the drive recorder 20 mounted on one vehicle relative to the server 30, but the drive recorders 20 of multiple vehicles may be connected to one server 30. In this case, a plurality of drive recorders 20 are installed in different vehicles.

[0046] The detector 10 detects electromagnetic waves in a predetermined wavelength band emitted by a speed enforcement device. A speed enforcement device is a device installed at a vehicle speed enforcement point. A speed enforcement point is a location where vehicle speeds are enforced. A speed enforcement point may be a fixed location where a speed enforcement device is installed in advance, or a relatively lightweight and small speed enforcement device may be installed at an unfixed location. A speed enforcement device corresponds to a target indicating a vehicle speed enforcement point. The electromagnetic waves may be, for example, radio waves in a predetermined wavelength band, such as microwaves (radar radio waves), or light in a predetermined wavelength band, such as infrared light (e.g., 905 nm, 850 nm, 950 nm, 1900 nm), such as laser light (e.g., pulsed light). Speed ​​enforcement devices are classified into radar and laser types, and the detector 10 of this embodiment is compatible with both. For this purpose, the detector 10 includes a radar detection unit 11 and a laser detection unit 12. The radar detection unit 11 includes a receiving unit that receives radio waves in the radar wavelength band emitted from a radar-based speed enforcement device. The radar detection unit 11 may determine that the radio waves have been detected when, for example, the intensity of the radio waves in the radar wavelength band exceeds a threshold. The laser detection unit 12 includes a light receiving unit that receives light in the laser wavelength band emitted from the laser-based speed enforcement device. The laser detection unit 12 may determine that the light has been detected when, for example, the intensity of the light in the laser wavelength band exceeds a threshold. The detector 10 may further include an alarm unit 13. The alarm unit 13 issues an alarm when a predetermined condition is met. The alarm unit 13 may issue an alarm in response to a signal from the drive recorder 20, for example. The alarm may be issued using a method that can be perceived by humans (for example, sound, display, light, etc.). In addition to these, the detector 10 may also be equipped with an operation unit that accepts operation inputs, etc.

[0047] The detector 10 is installed in a position where it can receive electromagnetic waves from a speed enforcement device, such as on the dashboard of a vehicle or near the top of the windshield. When the radar detection unit 11 of the detector 10 detects radar, the alarm unit 13 issues an alarm indicating that the detector 10 has detected radar. When the detector 10 detects radar, the alarm unit 13 outputs a signal indicating that the detector 10 has detected radar to the control unit 24 of the drive recorder 20. When the laser detection unit 12 of the detector 10 detects laser, the alarm unit 13 issues an alarm indicating that the detector 10 has detected laser. When the detector 10 detects laser, the detector 10 outputs a signal indicating that the detector 10 has detected laser to the control unit 24 of the drive recorder 20. That is, the drive recorder 20 issues an alarm to a person in the vehicle when it detects a radar-based speed enforcement device and when it detects a laser-based speed enforcement device. The person in the vehicle is typically the driver. The following description will be given assuming that the person in the vehicle is the driver. Furthermore, the drive recorder 20 distinguishes between when a radar-type speed enforcement device is detected and when a laser-type speed enforcement device is detected, and reports (notifies) the same to the drive recorder 20 .

[0048] The drive recorder 20 includes a camera 21, a primary detection unit 22, a position information acquisition unit 23, a control unit 24, an alarm unit 25 as an output unit, a communication unit 26, and an operation input unit 27. The drive recorder 20 corresponds to a photographing device. The camera 21 is installed facing the front of the vehicle. The camera 21 continuously photographs the outside in front of the vehicle and generates photographed images in time series. The photographed images are still images unless otherwise specified. A video file may be generated by arranging multiple photographed images in time series. Speed ​​enforcement devices installed above or to the side of the road (on the shoulder or side of the road) are photographed by the camera 21. The camera 21 may be, for example, a so-called omnidirectional camera or a semi-spherical camera.

[0049] The primary detection unit 22 detects speed enforcement devices from the captured image by performing object detection processing on the captured image generated by the camera 21. In this embodiment, the detection standard for speed enforcement devices in the primary detection unit 22 is lower than the detection standard for speed enforcement devices in the secondary detection unit 33, which will be described later. In other words, the tolerance level for erroneous detection in the primary detection unit 22 (i.e., the tolerance level for erroneously detecting an object other than a speed enforcement device, which is the target, as a speed enforcement device) is higher than that in the secondary detection unit 33. This allows the primary detection unit 22 to detect speed enforcement devices as thoroughly as possible for captured images in which speed enforcement devices are detected in secondary detection.

[0050] Specifically, the primary detection unit 22 stores, as binary data, patterns of pre-prepared images (particularly images captured by an in-vehicle camera) of various speed enforcement devices. The binary data representing the image patterns of the speed enforcement devices corresponds to a first machine learning model. The primary detection unit 22 then calculates the degree of match between a rectangular partial image in the captured image and the binary data. This degree of match is expressed as a probability, or likelihood, that the object in the partial image is a speed enforcement device. If this probability exceeds a predetermined threshold, the primary detection unit 22 determines that the speed enforcement device is included (i.e., is captured) in the captured image. The primary detection unit 22 detects the speed enforcement device in the captured image by such pattern matching. If the primary detection unit 22 detects a speed enforcement device in the captured image, it outputs the captured image and the detection result to the control unit 24. Note that the primary detection unit 22 may perform object detection processing using a machine learning model by a method other than pattern matching. The primary detection unit 22 may extract image features from the captured image and perform object detection processing based on the image features and a predefined machine learning model. The image features may be, for example, SIFT features, SURF features, or other features. Thus, in this embodiment, the primary detection unit 22 performs object detection processing to detect targets based on the content of the captured image.

[0051] The location information acquisition unit 23 acquires location information of the vehicle. In this embodiment, the location information acquisition unit 23 acquires the location information by calculation. Specifically, the location information acquisition unit 23 includes a GPS receiver that receives GPS signals from multiple GPS satellites, and acquires the location information by performing calculations using a predetermined formula from the received multiple GPS signals. The location information acquisition unit 23 outputs the acquired location information to the control unit 24. Note that this location information may also include altitude information.

[0052] The control unit 24 controls each unit of the drive recorder 20 (corresponding to a first control unit). The control unit 24 is, for example, a microcomputer including an arithmetic processing circuit and a memory. The arithmetic processing circuit includes, for example, a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other arithmetic processing circuits. The memory includes, for example, a random access memory (RAM) or other volatile memory. The arithmetic processing circuit performs various controls by temporarily reading data into the memory and performing arithmetic processing. The control unit 24 mainly controls the alarm unit 25 and the communication unit 26. The control unit 24 obtains captured images in which a speed enforcement device has been detected from the primary detection unit 22, obtains vehicle position information from the position information acquisition unit 23, and further obtains radar or laser detection results from the detector 10. When the control unit 24 receives a result from the primary detection unit 22 that a speed enforcement device has been detected, the control unit 24 controls the communication unit 26 to transmit a set of data consisting of the corresponding captured image, the position information at the time the image was captured, and the radar or laser detection result from the communication unit 26 to the server 30. Note that when the control unit 24 receives a result from the primary detection unit 22 that a speed enforcement device has been detected, but has not received a detection result from the detector 10 that a radar or laser has been detected, the control unit 24 attaches information to the effect that radar and laser were not detected to the captured image and the position information, and assembles these into a set of data.

[0053] The alarm unit 25, which serves as an output unit, outputs an alarm under the control of the control unit 24 when the primary detection unit 22 detects a speed enforcement device. The alarm unit 25 may be a buzzer that outputs a buzzer sound. Alternatively, the alarm unit 25 may be an audio output device that outputs a predetermined alarm sound to notify the driver of the presence of a speed enforcement device. In this case, the alarm sound may be an electronic melody or a verbal message. The alarm unit 25 may also be equipped with a display panel such as a liquid crystal panel, and display an alarm image on the display panel. Furthermore, the alarm unit 25 may be a lamp that notifies the driver of the presence of a speed enforcement device by lighting or unlighting it or by its color. Furthermore, the alarm unit 25 is not limited to the above-mentioned alarms that appeal to the auditory or visual senses, but may also notify the driver of the presence of a speed enforcement device by appealing to other senses of the driver (e.g., tactile sense).

[0054] The alarm output unit 25 may output an alarm via the detector 10 instead of or in addition to a configuration for outputting an alarm in the drive recorder 20. In this case, the alarm output unit 25 has an interface for connecting to the detector 10 via a wired or wireless communication path. The control unit 24 outputs a signal for issuing an alarm via the detector 10 from the alarm unit 25 to the detector 10. In the detector 10, the alarm unit 13 issues an alarm in response to receiving this signal. The alarm from the alarm unit 13 may be issued in the manner described above. For example, if the detector 10 is a monitor-type device installed on a dashboard, issuing an alarm via a display on the detector 10 may make it easier for the driver to visually check the display related to the alarm.

[0055] In this embodiment, communication unit 26 is a wireless LAN module. Communication unit 26 is connected via wireless LAN to a mobile router or a smartphone with a tethering function, and is thereby connected to a mobile phone communication network, which is a wide-area communication network, via the mobile router or the smartphone with a tethering function. Alternatively, communication unit 26 may be a communication module that is directly connected to a mobile phone communication network. In this case, a SIM card is installed in this communication module, and communication unit 26 is connected to the wide-area communication network via a mobile phone communication network compatible with the SIM card.

[0056] In this embodiment, the communication unit 26 communicates particularly with the server 30 under the control of the control unit 24. Specifically, as described above, the communication unit 26 transmits to the server 30 a set of data including a captured image, location information, and a radar or laser detection result when a speed enforcement device is detected by the primary detection unit 22. Furthermore, when a speed enforcement device is detected in the secondary detection by the secondary detection unit 33 (described later), the communication unit 26 receives a report to that effect from the server 30.

[0057] The control unit 24 controls the alarm unit 25 to output an alarm when the primary detection unit 22 receives a report that a speed enforcement device has been detected from a captured image, when the detector 10 reports that a radar or laser has been detected, and when the communication unit 26 receives a report from the server 30 that a speed enforcement device has been detected in secondary detection. At this time, the alarm may be output with different content depending on the cause of the alarm output (i.e., whether the report is from the primary detection unit 22, the radar detection unit 11, the laser detection unit 12, or the detector 10, whether multiple of these apply simultaneously, or whether the report is from the server 30).

[0058] Furthermore, the content of the alarm may be set so that the alert level increases in the following order: detection of a speed enforcement device by only the primary detection unit 22, detection of a speed enforcement device by only the detector 10, detection of a speed enforcement device by both the primary detection unit 22 and the detector 10, and detection of a speed enforcement device by the server 30. For example, if the alarm is output as audio, the volume may be increased as the alert level increases. Also, for example, if the alarm is displayed as an image, the image may be displayed in a color corresponding to the alert level.

[0059] The control unit 24 may be configured not to output an alarm just when the primary detection unit 22 receives a report that a speed enforcement device has been detected from a captured image, or just when the detector 10 reports that a radar or laser has been detected, but may output an alarm when the primary detection unit 22 receives a report that a speed enforcement device has been detected from a captured image and also receives a report that a radar or laser has been detected from the detector 10. Even in this case, when the communication unit 26 receives a report from the server 30 that a speed enforcement device has been detected in secondary detection, the control unit 24 may output an alarm regardless of the reports from the primary detection unit 22 and the detector 10.

[0060] The operation input unit 27 receives input of operations performed by the driver. The operation input unit 27 may include, for example, a physical button of a press type or other type, a touch sensor or proximity sensor, or other operation device. The operation input unit 27 may also receive input of voice operations or gesture operations. The operation input unit 27 may include, for example, a microphone when receiving input of voice operations, and may include, for example, a camera or an optical sensor when receiving input of gesture operations. The operation input unit 27 may also receive input of an operation signal indicating an operation by the driver from a device external to the drive recorder 20 (for example, the detector 10 or a one-touch button assigned a predetermined function). In addition, the drive recorder 20 may have a terminal unit for connecting to an OBD2 connector provided in the vehicle to acquire information regarding the vehicle state (i.e., vehicle information). The drive recorder 20 may also include various sensors, such as an acceleration sensor and an illuminance sensor, as needed.

[0061] The server 30 includes a communication unit 31, a control unit 32, a secondary detection unit 33, a storage unit 34, and a learning unit 35. The communication unit 31 is a communication module that receives information transmitted from the drive recorder 20 and transmits the information to the drive recorder 20. The communication unit 31 particularly receives a set of data transmitted from the drive recorder 20, including captured images, location information, and information on the presence or absence of radar and laser detection. The communication unit 31 also transmits the detection results of the secondary detection unit 33 to the drive recorder 20.

[0062] The control unit 32 controls the communication unit 31, the secondary detection unit 33, the storage unit 34, and the learning unit 35 (corresponding to a second control unit). The control unit 32 is, for example, a microcomputer including an arithmetic processing circuit and a memory. When the communication unit 31 receives a set of data from the drive recorder 20, the control unit 32 causes the secondary detection unit 33 to perform secondary detection on the captured image. Furthermore, when a speed enforcement device is detected as a result of secondary detection by the secondary detection unit 33, the control unit 32 associates the result of secondary detection with the location information received together with the captured image and stores them in the storage unit 34, and also stores the captured image in the learning unit 35 as a positive example.

[0063] The secondary detection unit 33 detects speed enforcement devices from the captured image by performing object detection processing on the captured image. The secondary detection unit 33 performs object detection using an algorithm different from that of the primary detection unit 22. In this embodiment, the secondary detection unit 33 performs object detection using a machine learning model (corresponding to a second machine learning model) such as a convolutional neural network (CNN). Specifically, the secondary detection unit 33 uses a deep learning model as the machine learning model. Since the captured image transmitted from the drive recorder 20 is an image in which a speed enforcement device has already been detected by the primary detection unit 22 of the drive recorder 20, the secondary detection unit 33 detects speed enforcement devices with higher accuracy than the primary detection unit 22. In other words, the secondary detection unit 33 determines that an object mistakenly recognized as a speed enforcement device by the primary detection unit 22 is not a speed enforcement device.

[0064] The machine learning model used by the secondary detection unit 33 can also identify the type of speed enforcement device. There are several types of speed enforcement devices, such as H systems, LH systems, speed warning safety systems, portable, and semi-portable. The secondary detection unit 33 detects these speed enforcement devices from the captured image and also identifies their type. The secondary detection unit 33 uses a deep learning model to obtain the probability, or likelihood, that an object included in a partial image of the captured image is a speed enforcement device of each type. If this probability is the highest and exceeds a predetermined threshold, it is determined that the most likely speed enforcement device is present. If the probability of any type of speed enforcement device does not exceed the threshold, it is determined that no speed enforcement device is present in the partial image.

[0065] 2A to 2D are diagrams showing an example of object detection processing in secondary detection unit 33. In the examples of FIGS. 2A to 2D, different types (types 1 to 4) of speed enforcement devices are detected. Secondary detection unit 33 identifies an area in the captured image where a speed enforcement device has been detected and labels that area with the identified type. Secondary detection unit 33 detects speed enforcement devices installed on the side of the road as shown in FIGS. 2A to 2C, as well as speed enforcement devices installed on the road as shown in FIG. 2D.

[0066] The memory unit 34 is a database that stores, when the secondary detection unit 33 detects a speed enforcement device from a captured image, data that associates information indicating the presence of a speed enforcement device, information on the type of the speed enforcement device, and information on the vehicle's location at the time the captured image in which the speed enforcement device was detected was obtained, as shared enforcement data. The memory unit 34 also stores map information, and the information on the presence of a speed enforcement device and the information on its type are stored in a format in which they are arranged at points on a map corresponding to the associated location information (associated with the point information on the map). In other words, the memory unit 34 stores a distribution map of speed enforcement devices.

[0067] The learning unit 35 performs machine learning to generate a machine learning model to be used in the secondary detection unit 33. Note that the learning in the learning unit 35 may be performed asynchronously with the other components described above. Therefore, the server 30 stores or outputs only the learning data to be used in this machine learning, and the learning unit 35 may be provided as a learning device in a device other than the server 30. The system 100 that performs processing for machine learning centered on the learning unit 35 can be regarded as a machine learning system, and the server 30 can also be regarded as a learning device.

[0068] The learning unit 35 performs learning using multiple images of speed enforcement devices as training data, each of which is associated with the inclusion of a speed enforcement device in the image. The learning unit 35 generates training data by labeling each of multiple images of various types of speed enforcement devices as a positive example. Furthermore, the images used as training data may include, for example, at least one of a person, such as a police officer enforcing speed enforcement at a speed enforcement point, a vehicle, such as a police vehicle, that may be parked at a speed enforcement point, and a landscape (e.g., a cityscape) of a location that is likely to be set as a speed enforcement point. Because these are elements that are relatively likely to appear together with a speed enforcement device, using these images as training data is expected to result in the generation of a better machine learning model. The positive example training data is data in which a rectangular area of ​​a speed enforcement device and a label indicating the type of speed enforcement device are attached to the image.

[0069] Furthermore, captured images containing objects that appear similar to speed enforcement devices but are not speed enforcement devices can be used as negative example training data. Examples of such objects include gate-type structures (e.g., gate-shaped structures, see FIG. 2D). This is because such structures may appear similar to the H system. The negative example training data is data in which a rectangular area of ​​an object mistakenly identified as a speed enforcement device and a label indicating a non-speed enforcement device are attached to the captured image. The labels can be attached manually or automatically by, for example, an administrator managing the server 30. This approach is expected to enable the generation of better machine learning models from captured images.

[0070] The following means may be employed as a means for automating at least part of the task of labeling captured images: The learning unit 35 labels the captured images based on the results of secondary detection by the secondary detection unit 33 and the detection results by the detector 10 sent from the drive recorder 20, and generates training data as learning data for the learning unit 35.

[0071] Specifically, if a speed enforcement device is detected by the secondary detection unit 33 and electromagnetic waves in a predetermined wavelength band are also detected in the detection results by the detector 10, the learning unit 35 uses the captured image as positive example training data. The positive example training data is data in which a rectangular area of ​​the speed enforcement device and a label for the type of speed enforcement device are attached to the captured image. If a speed enforcement device is detected by the secondary detection unit 33 but electromagnetic waves in the predetermined wavelength band are not detected in the detection results by the detector 10, the learning unit 35 determines that an object that looks similar to a speed enforcement device but is not actually a speed enforcement device has been mistakenly detected as a speed enforcement device by the secondary detection unit 33, and therefore the captured image is used as negative example training data. Note that the negative example training data is data in which a rectangular area of ​​the object mistakenly identified as a speed enforcement device and a label for a non-speed enforcement device are attached to the captured image.

[0072] If the secondary detection unit 33 does not detect a speed enforcement device, the learning unit 35 does not use the captured image as learning data. This is because a captured image in which the secondary detection unit 33 does not detect a speed enforcement device contains an object that could be mistaken for a speed enforcement device in the primary detection, but the image has already been determined not to be a speed enforcement device in the secondary detection, and therefore is not a very good negative example. Note that even if the secondary detection unit 33 does not detect a speed enforcement device, if the detection result by the detector 10 detects electromagnetic waves in a predetermined wavelength band, there is a possibility that a speed enforcement device is actually present. However, there is also a possibility that the detector 10 detected electromagnetic waves from a source other than a speed enforcement device, such as a vending machine, and therefore the captured image is not used as learning data.

[0073] The multiple captured images used as learning data preferably include images of the speed enforcement device taken from multiple different directions. The multiple different directions preferably include a direction from which the front and at least one side can be captured, and particularly preferably a direction from which the speed enforcement device is viewed from a slightly diagonal forward position relative to the front. Furthermore, for mobile (e.g., portable and semi-portable) speed enforcement devices (see FIG. 2B), it is also preferable to include images taken from multiple different directions. This is because such mobile speed enforcement devices are placed on the roadside and are more likely to be photographed from a diagonal forward position than fixed speed enforcement devices. In this case, when a vehicle is relatively far from the speed enforcement device, the front side of the speed enforcement device is more likely to be photographed. However, when a vehicle is relatively close to the speed enforcement device, the front side of the speed enforcement device is less likely to be photographed, and the side (e.g., the front and part of the right side relative to the vehicle's direction of travel) is more likely to be photographed. Therefore, by including images of a speed enforcement device photographed from each of a plurality of directions, it is expected that a high-quality machine learning model can be generated that suppresses changes in the detection accuracy of the speed enforcement device due to the distance between the vehicle and the speed enforcement device. In this sense, by including multiple photographed images used as training data that include images of the speed enforcement device photographed at multiple different distances, it is also expected that a high-quality machine learning model can be generated that suppresses changes in the detection accuracy of the speed enforcement device due to the distance between the vehicle and the speed enforcement device. Note that the photographed images used as training data for positive examples may be as described above.

[0074] The learning unit 35 performs deep learning using the training data of positive examples and negative examples as described above as learning data, and generates a machine learning model that is a deep learning model. The secondary detection unit 33 performs secondary detection using the machine learning model generated by the learning unit 35, thereby improving the accuracy of secondary detection.

[0075] Fig. 3 is a flowchart of a detection method according to an embodiment of the present invention. While Fig. 1 illustrates an example in which the system includes detector 10, the detection method illustrated in Fig. 3 illustrates an example in which detector 10 is not used. In Fig. 3, the left column illustrates processing in drive recorder 20, and the right column illustrates processing in server 30.

[0076] First, the drive recorder 20 captures an image of the area ahead of the vehicle with the camera 21 (step S21). The primary detection unit 22 detects speed enforcement devices from the captured image obtained by the camera 21 using a machine learning model (for example, for pattern matching) (step S22). The control unit 24 determines whether a speed enforcement device has been detected in the primary detection by the primary detection unit 22 (step S23), and if no speed enforcement device has been detected (NO in step S23), causes the camera 21 to continue capturing images (step S21).

[0077] If a speed enforcement device is detected in the primary detection by primary detection unit 22 (YES in step S23), control unit 24 causes alarm unit 25 to output an alarm (first alarm) (step S24) and acquires location information from location information acquisition unit 23 (step S25). Then, control unit 24 controls communication unit 26 to transmit data including a captured image in which the speed enforcement device was detected and location information at that time to server 30 (step S26).

[0078] The communication unit 31 of the server 30 receives data transmitted from the drive recorder 20 (step S31). The control unit 32 controls the secondary detection unit 33 to perform object detection processing to detect speed enforcement devices from the received captured image (step S32). The secondary detection unit 33 detects speed enforcement devices from the captured image using a machine learning model, which is a deep learning model. The control unit 32 determines whether or not the secondary detection unit 33 has detected a speed enforcement device (step S33), and ends the processing if a speed enforcement device has not been detected (NO in step S33). As a result, the drive recorder 20 simply outputs an alarm based on the primary detection and ends the processing.

[0079] If a speed enforcement device is detected by secondary detection unit 33 (YES in step S33), memory unit 34 associates the type of speed enforcement device with the location information sent along with the captured image in which the speed enforcement device was detected and stores the information (step S34). In this case, control unit 32 also controls communication unit 31 to send to drive recorder 20 detection result data including information that a speed enforcement device was detected in the secondary detection and the type of speed enforcement device (step S35).

[0080] The communication unit 26 of the drive recorder 20 receives the data from the server 30 (step S27). In response to receiving this data, the control unit 24 controls the alarm unit 25 to output an alarm (second alarm) (step S28), and ends the processing. The second alarm has different content from the first alarm, and the driver of the vehicle can confirm that it is the second alarm.

[0081] FIG. 4 is a flowchart of a machine learning method according to an embodiment of the present invention. The following describes the operation of automating the process of labeling captured images. When the server 30, acting as a learning device, receives the captured image and the detection result from the detector 10 from the drive recorder 20 via the communication unit 31, the secondary detection unit 33 performs secondary detection (step S41). Based on the secondary detection result and the detection result, the learning unit 35 stores the secondary detection result as learning data. Specifically, if the secondary detection unit 33 detects a speed enforcement device (YES in step S42) and the detection result from the detector 10 also detects electromagnetic waves in a predetermined wavelength band (YES in step S43), the learning unit 35 stores the captured image as training data of a positive example (step S44).

[0082] If the secondary detection unit 33 detects a speed enforcement device (YES in step S42) but the detection result by the detector 10 does not detect electromagnetic waves in the predetermined wavelength band (NO in step S43), the learning unit 35 stores the captured image as training data for a negative example (step S45). If the secondary detection unit 33 does not detect a speed enforcement device, the learning unit 35 does not use the captured image as learning data (step S46).

[0083] The learning unit 35 determines whether the conditions for executing learning are met (step S47). The learning unit 35 determines that a predetermined number of pieces of learning data have been accumulated as a condition for executing learning. If the conditions are met (YES in step S47), the learning unit 35 executes learning to generate a machine learning model using the learning data including the accumulated training data of positive examples and negative examples (step S48).

[0084] The learning unit 35 may further adopt the following learning method. The system 100 may have a function of posting information about a speed enforcement device when the driver visually confirms the speed enforcement device. For example, when the driver visually confirms the speed enforcement device, the driver performs a predetermined operation (posting operation) using the operation input unit 27. In response to accepting the posting operation, the control unit 24 acquires from the camera 21 a captured image (e.g., the most recent captured image) taken at the time the posting operation was accepted. The control unit 24 transmits the acquired captured image together with data indicating that the posting operation was accepted from the communication unit 26 to the server 30. In the server 30, the communication unit 31 accepts the posting by receiving this data. The control unit 32 supplies the received data to the learning unit 35. The learning unit 35 performs supervised learning using the captured image as training data of positive examples associated with the inclusion of a speed enforcement device. When the driver posts information about the speed enforcement device, there is a high possibility that the captured image includes a speed enforcement device. Therefore, if supervised learning is performed using these captured images, a better deep learning model can be generated, which is expected to be effective in responding to, for example, the emergence of a new type of speed enforcement device. The drive recorder 20 may also be configured to allow the type of speed enforcement device and its location information to be input through a posting operation. In this case, the drive recorder 20 also transmits the input data to the server 30. The server 30 stores this data in the storage unit 34.

[0085] With regard to this function of posting information about speed enforcement devices, in response to receiving a posting operation, the control unit 24 may transmit a set of data from the communication unit 26 to the server 30, including a captured image (e.g., the most recent captured image) captured at the time the posting operation was received and radar or laser detection results at the time the operation was received (e.g., for at least one predetermined period before and after the posting operation). The control unit 32 then supplies this set of data to the learning unit 35. If the learning unit 35 determines, based on the detection results, that the captured image was captured when radar or laser was received, it may use the set of data as positive example training data associated with the inclusion of a speed enforcement device in the captured image, and perform supervised learning. Furthermore, if the learning unit 35 determines, based on the detection results, that the captured image was captured when radar or laser was not received (including, for example, when the intensity is below a threshold), it may use the set of data as negative example training data associated with the inclusion of a speed enforcement device in the captured image, and perform supervised learning. When a driver posts information about a speed enforcement device, if radar or laser is detected, there is a high possibility that the speed enforcement device is actually captured in the captured image. Conversely, if radar or laser is not detected, there is a relatively high possibility that the speed enforcement device is not actually captured in the captured image due to reasons such as the driver mistaking it. In this way, by performing learning using the captured image and the radar or laser detection results, a better quality deep learning model can be generated.

[0086] This function of posting information about speed enforcement devices may be limited to specific drivers. For example, system 100 may accept posts from specific drivers and use them for learning based on the number of posts made so far (e.g., the number of posts exceeds a threshold) or the reliability of previously posted information (e.g., the reliability exceeds a threshold). The reliability may be, for example, the number of drivers who have judged the posted information to be correct by other drivers and performed an action to that effect. In this way, a better deep learning model can be generated.

[0087] The system 100 may also have a function of acquiring a response input from the driver when an alarm is issued and learning based on the result of the response. When the alarm unit 25 outputs an alarm, if the driver can actually visually confirm the speed enforcement device, the driver operates the operation input unit 27 to input a response indicating that the alarm is correct. On the other hand, if the driver cannot visually confirm the speed enforcement device, the alarm may be a false alarm. If the driver determines that the alarm is a false alarm, the driver operates the operation input unit 27 to input a response indicating that the alarm is a false alarm. In response to receiving any of these operations, the control unit 24 acquires from the camera 21 an image captured at the time the operation was received (e.g., the most recent image). The control unit 24 associates the acquired image with the result of the driver's response and transmits the data, which is obtained by associating the acquired image with the result of the driver's response, to the server 30 from the communication unit 26. The communication unit 31 in the server 30 receives the data. The control unit 32 supplies the received data to the learning unit 35. The learning unit 35 performs supervised learning by using the captured image as training data for positive examples if the alarm is correct, or as training data for negative examples if the alarm is false, depending on the response result. In this way, the learning unit 35 can generate a better deep learning model by creating a deep learning model that reflects the driver's response. In particular, if the captured image at the time of a false alarm is used as training data for negative examples, it is expected that a deep learning model can be generated that reduces false alarms when a structure or the like that looks similar to a speed enforcement device is present and a false alarm is issued.

[0088] The warning may be as follows: In the drive recorder 20, the control unit 24 may prevent the alarm unit 25 from outputting an alarm when the vehicle speed is below a threshold (e.g., 30 km / h or less), even if a speed enforcement device is detected. For example, the control unit 24 may prevent the primary detection unit 22 from detecting a speed enforcement device when the vehicle speed is below the threshold. Instead of not outputting an alarm, the alarm unit 25 may lower the level of the alarm compared to when the vehicle speed is higher than the threshold. For example, when the vehicle speed is at the threshold, the vehicle is unlikely to be speeding, and it is presumed that the driver is being driven safely. Therefore, even if the drive recorder 20 does not output an alarm or lowers the level of the alarm, it is considered that there is little inconvenience to the driver. The threshold may be, for example, the legal speed of the road on which the vehicle is traveling. The drive recorder 20 may acquire the vehicle speed via an OBD2 connector provided in the vehicle or using a vehicle speed sensor.

[0089] The machine learning model generated in this manner may include an input layer to which an image is input, an output layer that outputs data corresponding to the likelihood that the image contains a speed enforcement device, and at least one intermediate layer in which parameters are trained using training data that receives each of a plurality of images as input and outputs data corresponding to the likelihood that the image contains a speed enforcement device. The parameters are trained by the training unit 35. This allows for the provision of a machine learning model for detecting speed enforcement devices from images. The data corresponding to the likelihood may be data that changes depending on the likelihood and may be a value indicating the probability that the image contains a speed enforcement device. In this case, the image is determined to contain a speed enforcement device if the probability exceeds a threshold. However, the data indicating the likelihood that the image contains a speed enforcement device may also be information indicating whether the image contains a speed enforcement device. The secondary detection unit 33, which performs secondary detection using such a deep learning model, inputs images captured by the drive recorder 20 to the input layer of the deep learning model, performs calculations in the intermediate layer, and determines whether a speed enforcement device has been detected based on the data output from the output layer.

[0090] As described above, according to the system 100 of the embodiment of the present invention, rather than detecting speed enforcement devices using only the detector 10 and outputting an alarm, the presence of a speed enforcement device is recognized by detecting the speed enforcement device from the image captured by the camera 21 using a machine learning model in the primary detection unit 22 and the secondary detection unit 33, thereby reducing the possibility of detecting electromagnetic wave sources other than speed enforcement devices, such as automatic doors, as speed enforcement devices and issuing an alarm (false alarm).

[0091] Furthermore, the system 100 according to the embodiment of the present invention includes a drive recorder 20 and a server 30. The drive recorder 20, which is an in-vehicle device, performs primary detection, and then the server 30 performs secondary detection on captured images in which a speed enforcement device is detected during primary detection. This reduces the detection processing load on the drive recorder 20, reduces the communication load associated with transmitting captured images from the drive recorder 20 to the server 30, and enables the server 30 to perform detection processing, which involves a relatively heavy processing load. Even if the drive recorder 20 lacks the capability to perform processing using a deep learning model due to a small microcomputer or other reasons, or if it is difficult to perform such processing (e.g., long processing time), the drive recorder 20 can obtain the results of the detection processing using the deep learning model. When a speed enforcement device is detected in a captured image during primary detection, the server 30 performs detection processing for the speed enforcement device using the deep learning model. This reduces the processing load compared to performing detection processing on all captured images. Furthermore, the drive recorder 20 does not need to be equipped with a deep learning model, and for example, the deep learning model can be shared by multiple drive recorders. In this way, even if a vehicle is driving through an area where a mobile speed control device is installed for the first time, the drive recorder 20 can output an alarm. This can support the driver's safe driving.

[0092] In addition, in the system 100 of the embodiment of the present invention, an alarm is output even if a speed enforcement device is detected in primary detection, which has a relatively low detection accuracy, and if a speed enforcement device is detected in secondary detection, which has a relatively high detection accuracy, an alarm different from that output in the case of primary detection is output, thereby providing the driver of the vehicle with an early alarm and a highly accurate alarm.

[0093] Furthermore, in the system 100 according to the embodiment of the present invention, if learning data is obtained using the results of detection using a machine learning model for a captured image and the results of detection by a detector, it is possible to automatically prepare learning data of positive and negative examples without determining whether a person is a positive or negative example. However, if learning data is generated by determining whether a person is a positive or negative example, the system 100 can generate a high-quality deep learning model.

[0094] The drive recorder 20 may be configured not to issue at least one of the warnings based on primary detection and the warnings based on secondary detection. If the drive recorder 20 does not issue a warning based on primary detection, the possibility of an alarm being issued even when a speed enforcement device is not actually present decreases. If the drive recorder 20 is configured not to issue a warning based on secondary detection, for example, when the communication time between the drive recorder 20 and the server 30 is long due to the capabilities of the drive recorder 20 or the quality of the communication network, or when the time from when the drive recorder 20 captures the image to when it obtains the results of secondary detection is long due to other reasons, the warning will not be issued at the wrong time.

[0095] The entities that realize the functions described in the above embodiments (entities that execute each process) can be modified in various ways. For example, the processes performed by the drive recorder 20 in the above embodiments may be performed by a detector 10, which is another in-vehicle device. For example, the control unit of the detector 10 may acquire captured images from the drive recorder 20, perform primary detection, and, if a speed enforcement device is detected, transmit the captured images to the server 30. Upon receiving data from the server 30, the control unit of the detector 10 may control the alarm unit 13 to issue an alarm (second alarm) in response to the data received. Furthermore, the in-vehicle device is not limited to a drive recorder and a detector, but may also be a car navigation device mounted on the vehicle, an information processing device such as a smartphone or a tablet computer (for example, an information processing device that functions as a car navigation device and is installed in the vehicle), or other devices mounted on the vehicle. Furthermore, the camera that captures the outside of the vehicle may be a digital video camera or a camera included in another imaging device.

[0096] The in-vehicle device may perform primary and secondary detection. In this case, if the in-vehicle device detects a speed enforcement device from a captured image during primary detection, the in-vehicle device performs detection processing for the speed enforcement device using a deep learning model. This reduces the processing load on the in-vehicle device compared to performing detection processing on all captured images. Furthermore, since there is no need to consider the time required for communication over a communication network between the in-vehicle device and the server, this is also desirable for issuing an alert based on secondary detection at an appropriate timing. Furthermore, if the in-vehicle device determines that a speed enforcement device has been detected through primary and secondary detection, it may transmit location information indicating the vehicle's location at the time of the detection to the server 30. Furthermore, rather than performing two-stage object detection processing, i.e., primary and secondary detection, a one-stage or three or more-stage object detection processing may be performed. In the case of a one-stage object detection processing, for example, a deep learning model may be used. Three or more stages of object detection processing may be performed using different algorithms (e.g., different machine learning models).

[0097] In the above embodiment, the system 100 includes the drive recorder 20 and the server 30. However, the system 100 may also be configured with a single in-vehicle device. In this case, only one detection unit may be provided, and this detection unit may perform highly accurate object detection using, for example, a deep learning model. In this case, the in-vehicle device may also perform primary and secondary detection, store location information of speed enforcement devices, and update this information. Furthermore, the in-vehicle device may transmit this location information to another in-vehicle device, for example, via vehicle-to-vehicle communication, so that the location information can be shared among multiple vehicles.

[0098] In this way, the system only needs to have a configuration that includes a detection unit that uses a machine learning model to detect targets that indicate speed enforcement points from images taken outside the vehicle, and a control unit that performs control based on the detection results of the detection unit.

[0099] Furthermore, as described above, the primary detection unit 22 and secondary detection unit 33 of the above embodiment can determine the probability of whether a speed enforcement device is present in a captured image. That is, the probability that an object within a rectangular frame in a captured image is a speed enforcement device can be determined. Therefore, the warning unit 25 may prepare warning levels according to this probability and output a warning according to the corresponding warning level. For example, if the warning is audio, the volume of the sound may be adjusted according to the warning level.

[0100] Furthermore, in the above embodiment, the system 100 detects speed enforcement devices as targets, and the server 30 generates a distribution map of speed enforcement devices, but the system 100 may perform similar processing to generate a distribution map for targets other than speed enforcement devices. For example, by using various landmarks as targets, the system 100 can generate a distribution map of POIs (Points of Interest).

[0101] The drive recorder 20 may control whether to output an alarm depending on the result of object detection processing on the captured image, depending on the environment in which the device (vehicle) is located. The control unit 24 of the drive recorder 20 may limit the output of an alarm (e.g., stop the output) depending on the result of the object detection processing, for example, when a sensor indicates that the vehicle is traveling in a relatively dark environment (e.g., when the illuminance measured by the illuminance sensor is at a threshold), or when the surrounding environment is dark during a predetermined time period, such as at night. In other words, the control unit 24 may limit the output of an alarm depending on the result of the object detection processing, for example, when a sensor detects that the vehicle is traveling in a relatively bright environment (e.g., when the illuminance measured by the illuminance sensor exceeds a threshold), or when the surrounding environment is bright during a predetermined time period, such as daytime. Furthermore, in the system 100, during the period in which the output of an alarm depending on the result of the object detection processing is limited, an alarm may be output if the detector 10 detects radar or laser. This alarm may be different from the alarm depending on the result of the object detection processing. In this way, the drive recorder 20 outputs an alarm according to the result of the object detection process when the vehicle is traveling in an environment where a speed enforcement device is easy to detect from the captured image, which is expected to reduce unnecessary processing or the possibility of outputting a false alarm. However, if the drive recorder 20 uses images captured by a night vision camera or other methods and the vehicle is traveling in a relatively dark environment, it is desirable to output an alarm according to the result of the object detection process even in such an environment.

[0102] In the above embodiment, the object indicating a speed enforcement point is a speed enforcement device. Alternatively, or in combination with this, the object indicating a speed enforcement point may include at least one of a person, such as a police officer enforcing the speed at the speed enforcement point, a vehicle, such as a police car, that may be parked at the speed enforcement point, and a landscape (e.g., a streetscape) of a point that is likely to be set as a speed enforcement point. In particular, because police officers often wear uniforms and police vehicles have a distinctive appearance, it is believed that these can be used as clues to detect the object using the above-described object detection process.

[0103] The scope of the present invention is not limited to the structures explicitly described in the specification, but also includes combinations of various aspects of the present invention disclosed herein. The structures of the present invention that are sought to be patented are specified in the appended claims, but it is the intention of the present inventors to claim structures disclosed in this specification in the future, even if they are not currently specified in the claims.

[0104] The present invention is not limited to the configurations described in the above-described embodiments. The components of each of the above-described embodiments and variations may be arbitrarily selected and combined. Furthermore, any component of each embodiment or variation may be arbitrarily combined with any component described in the Summary of the Invention or any component embodying any component described in the Summary of the Invention. The present invention intends to obtain rights to these configurations through amendments or divisional applications of this application. Furthermore, even if a description is made of "in the case of..." or "when...," this does not mean that the configuration is limited to that case or time. Configurations that are not limited to those cases or times are also disclosed, and the present invention intends to obtain rights to them. Furthermore, even if a description is made in an order, the order is not limited to this order. Configurations in which some parts are deleted or the order is changed are also disclosed, and the present invention intends to obtain rights to them.

[0105] Furthermore, we intend to obtain rights to the overall design or partial design by converting the application to a design registration. The drawings depict the entire device in solid lines, but they also include partial designs claimed for parts of the device. For example, a partial design can be a partial design for a part of the device, or a part of that part. A partial design can be a part of the device, or a part of that part. We intend to obtain rights not only for the overall design, but also for partial designs in which any part of the solid line portion of the drawing is drawn as a broken line. Furthermore, all of the modules, components, and parts inside the device's casing, shown in the drawings, are subject to independent commerce, and we intend to similarly obtain rights by converting the application to a design registration. [Explanation of symbols]

[0106] 10 Detector 11 Radar detection unit 12 Laser detection unit 13 Alarm section 20 Drive Recorder 21 Camera 22 Primary detection unit 23 Location information acquisition section 24 Control Unit 25 Alarm section 26 Communications Department 27 Operation input section 30 servers 31 Communications Department 32 Control Unit 33 Secondary detection unit 34 Storage section 35 Learning Department 100 systems

Claims

1. a detection unit that detects a target; a storage unit that stores position information of a target; a first control unit that controls outputting an alarm based on the detection result of the detection unit; a second control unit that controls updating of the position information of the target object in the storage unit based on detection results from the plurality of detection units mounted on different vehicles; and The first control unit controls the output of the warning based on the position information of the target object stored in the storage unit and the position of the vehicle. A system characterized by:

2. the first control unit is provided in an in-vehicle device mounted on a vehicle, the second control unit is provided in a server that can communicate with the in-vehicle device, The server is capable of sharing target detection results among multiple in-vehicle devices. The system of claim 1.

3. the second control unit stores in the storage unit a detection result regarding the photographed image obtained by the detection unit and position information received together with the photographed image in association with each other, and updates the position information of the target object based on the stored information.

3. The system according to claim 1 or 2.

4. The target is a speed enforcement device. A system according to any one of claims 1 to 3.

5. the detection unit detects the target by performing an object detection process on the captured image.

5. A system according to any one of claims 1 to 4.

6. A program for causing a computer to realize the functions of the system according to any one of claims 1 to 5.

Images