Device and system for violation detection

Abstract

Device (300) for detecting an infringement by a vehicle operator, the device comprising: one or more sensors (500) for detecting a vehicle and triggering one or more cameras (600) to take one or more pictures when the vehicle reaches a picture capture point; one or more flashing lights (400) for illuminating the vehicle or part of it with light in a narrow band; wherein the one or more cameras are designed to capture one or more images of at least one part of the vehicle operator, wherein at least one of the one or more cameras comprises a narrowband filter which transmits only or substantially only the wavelengths of light produced by the one or more flashlights and eliminates the majority of the ambient light and / or the light produced by the sun; one or more computer processors for automatically analyzing the one or more captured images to detect an infringement, wherein an automatic analysis includes detecting a phone or mobile device, comprising searching the one or more images for a phone or mobile device, the search comprising a neural network, and classifying, comprising determining the location of the driver in the one or more images, wherein, after the determination, the one or more images are cropped to produce an image with a standard width and height centered on a coordinate of the driver, wherein one or more cropped images are analyzed to determine the brightest and darkest pixels, and an offset is applied to all pixels, wherein the analysis of the one or more cropped images maximizes the contrast in an area of ​​interest, wherein the neural network comprises a pre-trained image classifier, the classifier receiving the one or more cropped images, and an output model of the neural network designed to classify the one or more cropped images into an appropriate class, the appropriate classes representing either images containing illegal use of phones or mobile devices (i.e., positive) or images not containing illegal use of phones or mobile devices (i.e., negative), and wherein a threshold is used to limit which images are classified as positive and which as negative, and wherein the automatic image analysis includes classifying an image as showing a driver using a telephone or mobile device, and / or detecting a telephone or mobile device in an image and reporting the location of the telephone or mobile device; and one or more outputs to provide the one or more captured images that include the identified infringement, in order to identify the infringement.

Description

AREA OF INVENTION

[0001] The present invention relates to a device and a system for violation detection. In one embodiment, this invention relates to a device and a system for violation detection for recognizing distracted operation of a vehicle. BACKGROUND OF THE INVENTION

[0002] Traffic violations, such as distracted driving and speeding, are a leading cause of death, injury, and financial costs. While autonomous vehicles have the potential to solve or at least reduce this problem, these vehicles are not yet readily available. Although device-based locking mechanisms and in-vehicle technology can reduce instances of distracted driving and speeding, they are not universally practical solutions, as vehicle operators must choose to implement them. U.S. Patent Publications No. 20110009107 and 20130271605 describe examples of in-vehicle locking devices. Accordingly, enforcement is currently the only credible short-term solution.

[0003] Video cameras and other types of cameras are known to be used for vehicle detection and recording evidence. However, this approach has shortcomings in terms of 24 / 7 operation and generating high-quality, legally admissible evidence.

[0004] Australian Patent No. 20020141618 discloses a system for monitoring and reporting traffic violations at a traffic point. The system comprises a digital camera system deployed at a traffic point. The camera system is remotely coupled to a data processing system. The data processing system includes an image processor for assembling vehicle and scene images generated by the digital camera system, a verification process for checking the validity of the vehicle images, an image processing system for identifying driver information from the vehicle images, and a notification process for transmitting potential violation information to one or more law enforcement agencies.

[0005] Australian patent no. 20120162432 describes a point-to-point speed system whose implementation uses an anonymized method for storing the images.

[0006] Australian patent no. 20120007983 describes a two-image process in which a first image is used to capture a trademark and a second image is used to capture evidence of the infringement.

[0007] Australian patent no. 20090046897 describes a secondary image speed verification where two photos are used to confirm speed based on images.

[0008] Australian patent no. 20090207046 discloses a method for recognizing vehicles by first recognizing the license plate.

[0009] Australian patent no. 20060047371 describes coupling an automatic license plate recognition (ANPR) camera with a loop detector.

[0010] Australian patent no. 20050073436 teaches how to alert an official upon a match of a license plate.

[0011] US patent 6,266,627 describes detecting the speed of a vehicle using radar and taking photographs when the vehicle exceeds a threshold.

[0012] Several technologies have been described that provide rudimentary mechanisms for detecting distracted driving.

[0013] US Patent Publication No. 2008 / 0036623, the publication of US Patent Application No. 11 / 678,489, describes a method and device for the automated detection of mobile phone use by vehicle drivers. The device includes a detection system comprising at least one mobile phone signal receiver, at least one image capture device, and at least one computer. The mobile phone signal receiver is used to detect a mobile phone signal transmitted by a vehicle. The at least one image capture device is used to capture at least one image of the vehicle. The at least one computer is used to store information associated with at least one of the mobile phone signals transmitted by the vehicle and at least one image of the vehicle in a storage device.The information stored in the memory device can be used to determine whether a person associated with the vehicle should be prosecuted for the illegal use of a mobile phone while driving. This document also teaches the use of viewing angles offset to the left or right of the driver to capture the hand holding a mobile phone, or a viewing angle sufficiently close and directly in front of the driver. The use of an artificial intelligence system to detect phone use by a passenger or other violations is also described.

[0014] U.S. Patent Publication No. 2010 / 00271497, the publication of U.S. Patent Application No. 12 / 769,41, describes a traffic monitoring system and methods for its use. This traffic monitoring system may be suitable for automatically monitoring vehicle traffic at desired locations, such as traffic lights, school zones, construction sites, remote locations, accident hotspots, and / or locations with frequent traffic violations. The traffic monitoring system may be used to detect traffic violations involving a vehicle (e.g., running a red light, speeding, and / or driving while using a mobile phone) and to collect information about the vehicle and / or the vehicle's driver.The traffic monitoring system may include one or more mobility features that make this system suitable for use and redeployment at any number of desired locations, for example, where traffic monitoring is desirable but no suitable permanent infrastructure is available on site or where such infrastructure is too costly.

[0015] International patent publication no. WO2015 / 017883, the publication of international patent application no. PCT / AU2014 / 00783, describes a potential traffic violation alert system comprising an image source that captures or receives an image of a driver or other person in the vehicle; a processor configured to analyze the image to determine whether the driver is using a mobile phone while driving or whether the driver or person is wearing a seatbelt; and an output that displays the result of the analysis. Driver localization is achieved by setting a range relative to the rest of the vehicle in which the driver is generally located. This localization varies depending on the vehicle type. The mobile phone can be detected by searching for a visual footprint and / or image features associated with the face.A subtest can analyze the image to determine whether a hand is visible in the image at a certain distance from the edge of the face, or whether the driver is looking downwards.

[0016] The reference to the state of the art in this description is not to be understood as an endorsement or any kind of indication thereof, and should not be interpreted as meaning that the state of the art is part of general technical knowledge. BRIEF SUMMARY OF THE INVENTION

[0017] In general, embodiments of the present invention relate to a device and a system for violation detection. In one embodiment, this invention relates to a device and a system for violation detection for detecting distracted operation of a vehicle.

[0018] In one embodiment, the invention comprises a system for detecting a violation by a vehicle operator, comprising: a computer for receiving one or more images of at least one part of a vehicle operator; a computer processor for automatically analyzing one or more received images in order to detect a violation; and wherein the computer or another computer provides the one or more received images that include the detected violation in order to detect the violation.

[0019] According to one embodiment, one or more images can be received through a computer network.

[0020] The system can also include vehicle recognition.

[0021] The system can further include capturing one or more images of at least part of the vehicle operator. The vehicle being operated can be the detected vehicle.

[0022] The other computer can be connected to the first computer and ready for operation. This connection can be established via a computer network.

[0023] In one embodiment, the invention comprises a computer program product, comprising: a computer-usable medium and a computer-readable program code located on the computer-usable medium for detecting a violation by a vehicle operator, wherein the computer-readable code comprises: Computer-readable program code devices that are (i) configured to cause the computer to receive one or more images of at least one part of the vehicle operator; computer-readable program code devices that (ii) are configured to cause the computer to automatically analyze the one or more received images in order to detect a violation; and computer-readable program code devices that (iii) are configured to cause the computer to provide the one or more received images that include the detected violation in order to detect the violation.

[0024] The computer program product may further include computer-readable program code devices that are (iv) configured to cause the computer to recognize the vehicle.

[0025] The computer program product may further comprise computer-readable program code devices that are (v) configured to cause the computer to take one or more images of at least a part of the vehicle operator. The operated vehicle may be the detected vehicle.

[0026] In one embodiment, the invention consists of a device for detecting a violation by a vehicle operator, wherein the device comprises: one or more sensors for detecting a vehicle; one or more cameras for recording one or more images of at least one part of the vehicle operator; one or more computer processors for automatically analyzing the one or more recorded images in order to detect a violation; and one or more outputs to provide the one or more recorded images that include the detected violation, in order to detect the violation.

[0027] The device may further include one or more flashing lights to illuminate the detected vehicle or part of it with light in a narrow band.

[0028] The one or more cameras according to one embodiment can include a narrowband filter that only allows the wavelengths of light generated by the one or more flashes to pass through.

[0029] In one embodiment, the invention consists of a computer system for detecting a violation by a vehicle operator, wherein the computer system comprises: one or more sensors for detecting a vehicle; one or more cameras for recording one or more images of at least one part of the vehicle operator; a computer processor for automatically analyzing one or more recorded images in order to detect a violation; and an output to provide the one or more recorded images that include the detected violation, in order to identify the violation.

[0030] The system may further include one or more flashing lights to illuminate the detected vehicle or part thereof with narrowband light. The recording may be a recording in which at least one of the one or more cameras includes a narrowband filter that only transmits the wavelengths of light generated by the one or more flashing lights.

[0031] In one embodiment, the invention comprises a computer program product, comprising: a computer-usable medium and a computer-readable program code located on the computer-usable medium for detecting a violation by a vehicle operator, wherein the computer-readable code comprises: Computer-readable program code devices that are (a) configured to cause the computer to detect a vehicle based on input from one or more sensors; computer-readable program code devices which (b) are configured to cause the computer to take one or more pictures of at least part of the vehicle operator of the detected vehicle using one or more cameras; computer-readable program code devices that are (c) configured to cause the computer to automatically analyze the one or more captured images in order to detect a violation; and computer-readable program code devices that (d) are configured to cause the computer to provide the one or more captured images that include the detected violation in order to detect the violation.

[0032] The computer program product may further include computer-readable program code devices configured to cause the computer to illuminate the detected vehicle or part thereof with one or more flashing lights emitting light in a narrowband.

[0033] The recording can be a recording in which at least one of the one or more cameras includes a narrowband filter that only allows through the wavelengths of light produced by the one or more flashes.

[0034] The one or more cameras of one of the above embodiments may comprise a 5 to 50 MP, 10 to 45 MP, or 20 to 35 MP camera. The one or more cameras may include a sensor selected to maximize light sensitivity and / or minimize noise. The camera sensor may enable the acquisition of high-quality images in low light. The sensor may have an excellent dynamic range. The dynamic range may be at least 66 dB or more than 70 dB. The one or more cameras may include a rolling shutter or a global shutter. In one embodiment, the one or more cameras include a global shutter. The one or more cameras may be monochrome. The one or more cameras may have a minimum exposure time of only 0.5 ms (1 / 2000th), 0.1 ms (1 / 10000th), or 0.2 ms (1 / 5000th).The exposure time can range from 0.05ms to 0.5ms; or from 0.1ms to 0.3ms.

[0035] The one or more cameras of one of the embodiments described above may include one or more filters. The one or more cameras may or may not include an infrared blocking filter (IR blocking filter). The one or more cameras may include a narrowband filter provided on the front or rear of the one or more lenses. The narrowband filter may transmit only the wavelengths of light produced by the respective flash of the one or more flash units. This narrowband filter may eliminate most of the ambient light and / or light produced by the sun. If the one or more flash units include an 850 nm flash unit, the one or more filters may transmit light only between 700 and 1000 nm; 750 and 950 nm; 800 and 900 nm; 820 and 890 nm; 830 and 880 nm; 850 and 870 nm; or 840 and 860 nm.The one or more filters can eliminate approximately 90%, 95%, or 97.5% of the light that is normally visible to the camera, allowing only light of the same wavelength as that emitted by one or more flash units to pass through.

[0036] The narrowband filter can block all light or substantially all light except for light at or around a certain wavelength.

[0037] The narrowband filter can cover a wavelength band of less than 5 nm; 5; 10; 15; 20; 25; 30; 35; 40; 45; 50; 55; 60; 65; 70; 75; 80; 85; 90; 95; 100; 110; 120; 130; 140; 150; 200; 250; 300; 350; 400; 450; or 500 nm. The narrowband filter can also cover a wavelength band of 5 nm or less; 10 nm or less; 15 nm or less; 20 nm or less; 25 nm or less; 30 nm or less; 35 nm or less; 40 nm or less; 45 nm or less; 50 nm or less; 55nm or less; 60nm or less; 65nm or less; 70nm or less; 75nm or less; 80nm or less; 85nm or less; 90nm or less; 95nm or less; or 100nm or less.

[0038] In one particular embodiment, the narrowband filter may include a Bi850 near-IR interference band bass filter.

[0039] In another specific embodiment, the narrowband filter can cover a useful range from 845 to 860 nm, that is, a range of 15 nm.

[0040] The one or more cameras of one of the above embodiments may include one or more lenses, such as a varifocal lens. The one or more cameras may include a C-mount lens or a larger format camera. The one or more cameras may include a fixed lens, such as an industrial prime lens. The prime lens may have a nominal resolution of 12 MP. In one embodiment, the nominal resolution of the one or more camera lenses is the same as or greater than that of the sensor to avoid a blurred image.

[0041] The one or more lenses according to any of the embodiments above may have a focal length of 10 to 100 mm; 20 to 80 mm; or 30 to 60 mm. For a mobile installation, the focal length may be 20 to 50 mm; 25 to 45 mm; or 30 to 40 mm. For a mobile installation, the focal length may be 35 mm. For a fixed installation, the focal length may be 35 to 65 mm; 30 to 60 mm; or 45 to 55 mm. For a fixed installation, the focal length may be 50 mm. The focal length may be selected to provide a strong zoom onto the vehicle for higher resolution and sufficient width and context to show one or more of: the overall width of the vehicle; a large portion of a roadway; and vehicle registration plates.

[0042] According to one of the embodiments above, the one or more cameras may or may not include a polarizer.

[0043] In one embodiment, the one or more cameras may include one or more flashlights for illuminating the detected vehicle or part thereof. The one or more flashlights may include one or more 760 nm and one 850 nm flashlights. The one or more flashlights may be capable of flashing 10,000 to 100,000; 20,000 to 80,000; or 30,000 to 50,000 times per day at high intensity and for a short duration. In one embodiment, the one or more flashlights are capable of flashing 40,000 times per day at high intensity and for a short duration. The one or more flashlights may include one or more light sources, which may include one or more LED light sources and / or one or more laser light sources. The one or more light sources may cover a narrow spectrum. The one or more light sources may have 10 to 1.The system comprises 000; 40 to 500; 300 to 400 light sources. Each of the one or more light sources can include an IR LED light source. The one or more light sources can be closely aligned using individual lenses. The individual lenses can be at 1 to 35 degrees; 15 to 30 degrees; or 20 to 25 degrees. In one embodiment, the individual lenses are at 22 degrees. The one or more flashlights can include one or more capacitor banks to store charge between flashlights. The one or more light sources, when triggered, can produce a high light intensity for a very short duration.

[0044] In one particular embodiment, the one or more cameras comprise a 12MP C mount camera with a Sony Pregius global shutter sensor.

[0045] According to one of the embodiments described above, the light source can comprise a tightly controlled wavelength. The tight control can encompass a narrow spectral band. The one or more camera filters can exclude all light that is not within the controlled spectral band. In one embodiment, the tightly controlled wavelength comprises a light source with a narrow spectrum. The light source with a narrow spectrum can have a full width at half maximum (FWHM) spectral bandwidth of 5, 10, 20, 25, 30, 35, 40, 45, or 50 nm.

[0046] In a specific embodiment, the one or more light sources may include an Oslon Black, Oslon Black Series 850 nm -80°, SFH 4715AS, available from Osram Opto Semiconductors.

[0047] According to one of the embodiments above, the spectral bandwidth can be determined at 50% Irel,max FWHM (Full Width at Half Maximum).

[0048] The one or more flashing lights may include a main flashing light and a separate, offset flashing light for license plates.

[0049] One or more camera settings can be changed between shots of individual images included in that single image or set of images. The setting can be changed quickly and / or automatically. The one or more camera settings can include exposure time and / or flash intensity.

[0050] According to one of the embodiments described above, an auxiliary camera may be included. The auxiliary camera may capture one or more images of a vehicle license plate.

[0051] In one embodiment, a high angle into the vehicle can be used for the camera to view the violation. The angle can be 30 to 90, 35 to 90, or 40 to 90 degrees vertically from a ground plane to the camera. In one embodiment, the angle can be 65 degrees. The angle can be 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71; The angles may include 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, or 90 degrees. In another embodiment, at least 40 degrees vertically from a ground plane to the camera are used. If the vehicle is a truck, smaller angles may be required than for a passenger car.

[0052] According to one of the embodiments described above, a camera mounting position is defined between 2 and 15 m; 3 and 12 m; or 4 and 10 m above the road surface. In one embodiment, the mounting position is at least 4 m and / or 7 to 10 m above the road surface.

[0053] In another embodiment, a camera mounting position may be included that encompasses a horizontal angle between -70 and 70 degrees; -45 and 45 degrees; and -30 and 30 degrees. In an embodiment for roadside use, the horizontal angle may encompass up to 45 degrees. In an embodiment for use above the roadway, the horizontal angle may encompass 0 degrees. The above-roadway use may include a fixed installation of a high-level camera.

[0054] In another embodiment, one or more cameras can be positioned on the passenger side of the vehicle.

[0055] The camera system can include one or more cameras to provide additional angles into the vehicle. These two or more cameras can provide depth information using stereoscopic means to increase detection accuracy.

[0056] In a further embodiment, a video camera is also included. The video camera can provide one or more additional contextual and license plate information; a wider field of view; sustained evidence of distraction over several seconds; and license plate information.

[0057] According to one of the embodiments described above, a processor may be included in a computer. The computer may be connected to the device at the installation site or via a network connection. The network connection may be an Ethernet connection. The computer may be located up to 100 m away. The computer may be mounted within 5 m. The computer may include one or more graphics cards to improve processing speed. The computer may include a Linux operating system.

[0058] Automatic image analysis can include classifying an image, such as showing a driver using a phone or mobile device, and / or detecting a phone or mobile device in an image, and reporting the location of the phone or mobile device.

[0059] The one or more images can comprise one or more standardized images. The one or more standardized images can comprise one or more cropped images with a standard width and height, centered or substantially centered on a driver coordinate. The width and height can be chosen to capture all details of the driver while excluding any passengers. The one or more cropped images can be filled with gray so that the driver coordinate remains centered and the one or more cropped images have a standard size. The one or more cropped images can be analyzed to determine the lightest and darkest pixels. An offset can be applied to all pixels in the one or more cropped images so that the darkest pixels are set to a value of zero (0).Scaling can be applied to all pixels in the one or more cropped images, setting the brightest pixels to a value of 255. All other pixels can be linearly scaled between 0 and 255 using a histogram adjustment operation. Analyzing the one or more cropped images can maximize contrast in the area of ​​interest. The one or more cropped images can be resized so that the resulting file adheres to a standard file size.

[0060] The one or more cropped images can be adjusted to include a standardized brightness and / or contrast. This image adjustment can be automatic, such as through computer processing.

[0061] Automated analysis may include detection. The detection of a phone or mobile device may involve searching one or more images for a phone or mobile device. The search may involve a neural network or artificial neural network, such as a deep neural network or a deep convolutional neural network. The search may encompass the entire image. If a phone or mobile device is detected, the one or more images may be sent for further review based on the confidence level. This further review may be performed by a person. One or more images may be excluded if a detected phone or mobile device is attributed to a passenger rather than a vehicle operator or driver.One or more images can be automatically excluded if the phone or mobile device is detected in a holder and not being held in a hand.

[0062] The captured and / or received image may include a view of the entire front of the vehicle and the roadway in which the vehicle is located.

[0063] The neural network can include an object recognition system. The object recognition system can use a real-time YOLO (You Only Look Once) object recognition architecture based on a neural network.

[0064] The neural network can include an image classifier. The image classifier can be a Visual Geometry Group (VGG) classifier based on a neural network. The classifier can receive one or more cropped images. The classifier can be pre-trained. Pre-training can involve training on a dataset such as the 1,000-class ImageNet set. The network's output model can be modified to identify only two classes: received cropped images that involve illegal use of a phone or mobile device (i.e., positive), and received cropped images that do not involve illegal use of a phone or mobile device (i.e., negative). The classifier can be trained by fine-tuning on a training set of sample images.The sample images can include more than one hundred, more than one thousand, more than one hundred thousand, or more than one million sample images, such as cropped images. The training set can include positively and negatively marked images. When the received cropped image of the driver is fed into the classification network, the analysis can determine a confidence level as output that the image is positive for the use of a phone or mobile device. For example, the determined confidence level can range from a 0% confidence output, indicating a very low probability of phone or mobile device use, to a 100% confidence output, indicating a very high probability of phone or mobile device use. A threshold can be used to limit which images are classified as positive and which as negative.The threshold can include a dynamically adjustable value, so that an upper margin of images is classified as positive and forwarded for manual review. This upper margin can comprise the top 5, 10, 15, 20, or 25% of received cropped images with the specified confidence level.

[0065] According to one of the embodiments described above, the analysis can include a real-time object detector. The real-time object detector can identify the driver based on the presence of a steering wheel and the person behind it. The training set can optionally include images from various angles, both vertical and horizontal. The training set can include images of many different vehicle types and optionally images of left-hand drive and right-hand drive vehicles. The analysis can include positive identification of the driver and ignore passengers. The real-time object detector can output the driver's position in the image and optionally a confidence score that the driver has been found.

[0066] According to one of the above implementation methods, the received image can be cropped if the driver has been located in the received image.

[0067] The standardized one or more cropped images can be fed to an image classifier. The image classifier can receive the standardized one or more cropped images. The image classifier may have been trained on a training set that includes several examples of cropped images of the driver. The training set includes tagged images of drivers illegally touching phones or mobile devices, drivers not illegally touching phones or mobile devices, and drivers not touching phones or mobile devices at all. When the driver images are received by the image classifier, output can be provided that includes a most-probable category and a confidence score for making that determination.

[0068] Classification can include determining the driver's position within one or more images. Position determination can involve steering wheel detection and / or person detection. Once determined, the one or more images can be cropped to generate an image showing only the driver and their immediate surroundings. The immediate surroundings can be defined as the driver's span. The cropped image can then be provided for further review. This further review can be performed by a person.

[0069] According to one of the embodiments described above, the detection may include a confidence factor or threshold. In one embodiment, only the one or more captured images above a threshold are provided. The confidence threshold can be adjusted. The threshold can range from 0% missed violations, 100% manually processed images to 100% missed violations, 0% manually processed images. In one embodiment, the threshold includes 5% missed violations for 10% manually processed images.

[0070] Further testing can be performed by one person and may involve a multi-stage process. Further testing can be performed when a high level of confidence is determined. The multi-stage process may include one or more of the following: Generating a cropped image of a driver from one or more images; Uploading the cropped image to a server; Logging into an "image verification" website; Presentation of the cropped image; Select one of four options: 1) On phone or mobile device; 2) Not on phone or mobile device; 3) Unsafe; and 4) Other; If 1), 3) or 4) are selected, a message can be sent to the camera system; and The next cropped image can then be presented for review or further examination.

[0071] The server can be a cloud server. The server can be an AWS SQS (Amazon Web Services Simple Queue Service). Logging in can involve entering credentials. The message can be sent from the server.

[0072] According to one of the embodiments above, the image may be provided for further review or confirmation. This further review may be performed by a human. The provision of the image may be by a computer or a telecommunications network. The further review or confirmation may include evaluating whether the image shows a driver clearly using a telephone or mobile device; a driver clearly not using a telephone or mobile device; or an uncertain classification. The classification may provide a confidence score for each categorization. If the confidence score for telephone or mobile device use exceeds a threshold and / or the confidence score for no telephone or mobile device use is lower than a threshold, the image may be sent for further review.

[0073] In one embodiment, the images can be provided to a nearby interceptor vehicle. The interceptor vehicle may be a police vehicle. The interceptor vehicle can receive the provided one or more captured images after further examination. The one or more images can be provided to the nearby interceptor vehicle if a high probability of telephone or mobile device use, or of a violation, is determined. The further examination can be performed remotely or by a dedicated person deployed on-site. The one or more images can be provided to the interceptor vehicle on a personal computing device such as a mobile phone or other mobile device.

[0074] Deployment can involve transferring the incident file or one or more images to a server. The incident file or one or more images can be transferred to a dedicated folder on the server.

[0075] The incident file or one or more images can be downloaded from the server for further analysis. This analysis may include decrypting and / or extracting the incident file. The incident file can be processed like a red light violation, speeding ticket, or other traffic violation.

[0076] According to one of the embodiments above, a registration search can be performed and a violation notification can be sent to a registered owner of the vehicle.

[0077] In another embodiment, one or more sensors, cameras, and / or processors are mounted on a fixed installation; on a tripod; on a vehicle; or on a trailer. The fixed installation may include scaffolding, a bridge, or other structures. The tripod may be temporarily deployed at a desired location. The trailer may include a work light trailer. The fixed installation and trailer mounting may involve a high-level mounting. In another embodiment, the processor is located at a remote location.

[0078] According to one of the embodiments above, the one or more violations include distracted driving or distracted operation of the vehicle and / or the use of a mobile phone or mobile device. The violation may include the use of a mobile device such as a tablet computer, a laptop computer, a smartwatch, a gaming device, or any device with a display screen. The violation may also include the failure to wear a seatbelt and / or improper seating or restraint of the driver or one or more passengers.

[0079] According to one of the embodiments described above, the detection and / or sensor may include a radar. The radar may be a 3D or 4D tracking radar for speed control. The radar may be used to trigger the camera to capture one or more images.

[0080] According to one of the embodiments above, the vehicle can be a motor vehicle such as a car, a wagon, a van, a truck, a moped, a motorcycle or a bus.

[0081] According to one of the embodiments above, the one or more recorded images can be provided to a road authority or the police.

[0082] In one embodiment, the capture of one or more images is triggered by detection and / or one or more sensors. The triggered capture enables one or both of the following: capture at a precise position; and sufficient illumination. The capture can occur when the vehicle reaches an image capture point.

[0083] The one or more images can be provided with additional data. The additional data can include one or more pieces of information such as vehicle speed, average speed, location, and timestamps. The one or more images and / or the additional data can be contained in an incident file. The incident file can be encrypted using one or more encryption methods. The incident file can be an SQLite database file. The encryption can include a randomly generated symmetric AES key. The AES key can then be encrypted with a public RSA key and packaged with the incident file. The file can only be decrypted with the private RSA key to access the AES key. The one or more images and the additional data can be hashed and encrypted with a private key.The hash can be verified using the public key.

[0084] The one or more images may include at least one image that includes a license plate and at least one image that includes the violation. The one or more images may include images from more than one angle.

[0085] The one or more images that include the license plate can be captured with a reduced exposure setting. The reduced exposure setting can include lower gain and / or exposure time. The one or more images that include the violation can be captured with an increased exposure setting. The increased exposure setting can include higher gain and / or exposure time. The license plate image can be exposed at a value of 1% to 50% relative to the size of the violation. In one embodiment, the license plate image is exposed at a value of 5%.

[0086] In another embodiment, a real-time display of offenses on a variable message sign can show images of the violations.

[0087] In a further embodiment, a survey service can be provided that shows a predominance of a violation according to time of day, vehicle type, location, or another parameter or variable.

[0088] Further aspects and / or features of the present invention will become apparent from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] To ensure that the invention is easily understood and can be implemented in practice, embodiments of the present invention are now described with reference to the accompanying drawings, where identical reference numerals refer to identical elements. The drawings serve only as examples, wherein: Fig. 1. A flowchart is a diagram that shows a process. Fig. 2A and Fig. 2B are schematic diagrams showing an embodiment of a calculating device and a computer system according to the invention. Fig. Figures 3A to 3H show exemplary images taken during the day using a tripod and without flash, according to an embodiment of the invention. Fig. Figures 4A to 4F show exemplary images taken with flash during the day using a vehicle, according to another embodiment of the invention. Fig. Figures 5A to 5D show exemplary images taken with a vehicle during the night using a flash, according to another embodiment of the invention. Fig. Figure 6A shows a suitable flash light according to an embodiment of the invention. Fig. 6B a device mounted on a vehicle according to an embodiment of the invention and which transmits the flashing light of Fig. 6A contains, shows. Fig. 6C a device mounted on a scaffold according to an embodiment of the invention and which transmits the flash of Fig. 6A contains, shows. Fig. 7A shows a trailer or mobile device according to an embodiment of the invention. Fig. Figure 7B shows a fixed device according to another embodiment of the invention, which in this example was used on a bridge. Fig. Figure 7C shows another fixed device according to another embodiment of the invention, which in this example was used on a scaffold for a variable message sign (VMS). Fig. Figure 8A shows an exemplary image taken during a successful use of the device according to an embodiment of the invention.

[0090] Expert readers will recognize that elements in the drawings are presented for the sake of simplicity and clarity and are not necessarily drawn to scale. For example, the relative dimensions of some elements in the drawings may be distorted to improve understanding of the embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0091] Embodiments of the present invention relate to a device and a system for violation detection.

[0092] The invention is based, at least in part, on the inventor's unexpected discovery that, in one embodiment, a violation can be detected by avoiding glare and by providing sufficient light and resolution to see through the windshield and observe the offending behavior. Furthermore, the inventor has discovered in the field of image recognition that efficiency can be achieved by reducing the search area and minimizing the number of images that users need to review.

[0093] Surprisingly, the inventor has found that in one embodiment of the invention, removing glare from windshields by using a narrow-spectrum flash and a narrow-spectrum filter on the camera can advantageously remove light from other sources such as the sun and provide only artificial lighting for image capture.

[0094] Although this is described as an infringement with reference to the use of mobile phones, the invention is not limited to this. For example, the use of other mobile devices such as tablet and laptop computers, smartwatches, and gaming devices can also be recognized as an infringement.

[0095] In one embodiment, the present invention captures legally admissible evidence that persons are using their telephone or mobile device while operating a motor vehicle. The system can operate semi-autonomously to capture photographic evidence and can automatically detect the use of the telephone or mobile device.

[0096] As in Fig. As shown in Figure 1, the procedure comprises four main steps: Detecting 110 a vehicle; Capturing 120 one or more images; Analyzing 130 for computing device use; and Providing 140 the one or more images to detect the infringement.

[0097] The invention can be used with four different installation platforms. The first is with a device mounted on a fixed installation such as scaffolding, a bridge, or other structure. This is similar to the installation of a speed monitoring system (average speed). The second is with a device mounted on a tripod and used temporarily with battery power. The third is with a device attached to a vehicle for mobile use. This is very similar to a mobile radar camera. The fourth is with a device attached to a trailer, similar to a work light trailer, with the cameras mounted very high to replicate the height of a fixed installation.

[0098] The one or more captured images can be processed according to one or two data usage scenarios. In the first scenario, evidence is captured and then transferred to a back-office system. This corresponds to the functionality of radar and red-light cameras. In the second scenario, the one or more images are transmitted to a nearby police interceptor vehicle. The evidence may or may not have been reviewed by a human operator.

[0099] To capture vehicles traveling at any speed, photograph them at highly precise positions along the road, and ensure each image is adequately illuminated, a triggered image capture system can be used instead of a video system. Radar can be used to detect a vehicle, and when the vehicle reaches the correct capture point, one or more cameras take one or more pictures. These one or more images, along with other data such as vehicle speed, location, and precise timestamps, can be compiled into an incident file. This incident file can then be encrypted using a combination of encryption methods.

[0100] An incident file is a digitally signed package of images and metadata, such as one or more detection data points, location, vehicle details, phone usage detection values, etc. Typically, the incident file is encrypted.

[0101] In one embodiment, the radar includes a 3D or 4D tracking radar for speed control. The radar accurately tracks the position and speed of all vehicles and informs one or more cameras when to take pictures.

[0102] Fig. Figure 6A shows an embodiment of a flashlight 400, which is suitable for use in a device 300 mounted on a vehicle according to an embodiment of the invention.

[0103] Fig. Figure 6B shows an embodiment of a camera 600, which is included in device 300 according to an embodiment of the invention. Camera 600 comprises a 12MP C-mount camera with a Sony Pregius global shutter sensor.

[0104] Fig. Figure 6C shows an embodiment of a device 300 mounted on a scaffold, in which the flash 400, the radar 500 and the camera 600 are easily visible.

[0105] The selected sensor type advantageously maximizes light sensitivity and minimizes noise, enabling high-quality images even in low-light conditions. The sensor can exhibit an excellent dynamic range, effectively overcoming problems associated with "license plate burn," where the back of the license plate is overexposed due to the flash energy, overexposing the lettering and rendering the license plate illegible. The sensor used may be quite unique in its ability to correct this issue. Excellent dynamic range refers to a range encompassing at least 66 dB or more than 70 dB.

[0106] Most sensors are rolling shutter sensors, which exhibit image distortion with fast-moving objects. When using a global shutter camera, all pixels can be exposed simultaneously and therefore do not show motion-induced distortion.

[0107] In one embodiment, the IR blocking filter of one or more cameras is removed, making them sensitive to infrared light.

[0108] In the embodiment used to capture the images shown in the figures, the one or more cameras were monochrome to improve light sensitivity and image sharpness. In other embodiments, a color camera is used, resulting in reduced image quality.

[0109] To avoid motion blur, the exposure time of one or more cameras should be at least 0.5 ms (1 / 2000th of a second) and ideally set to 0.1 ms (1 / 1000th of a second). The exposure time can range from 0.05 ms to 0.5 ms, or typically from 0.1 to 0.3 ms. In practice, 0.2 ms (1 / 5000th of a second) is typically used. These settings ensure sharp images for vehicles traveling at 100 km / h and still provide acceptable images for vehicles traveling at speeds up to 300 km / h.

[0110] The resolution of one or more cameras can be high enough to provide sufficiently accurate evidence to clearly establish that the object in the person's hand is a telephone or mobile device and not some other object.

[0111] The inventor determined that the video camera's resolution (2 MP) was insufficient. Tests with 9 MP and 12 MP showed that at least 5 MP was necessary. Higher resolutions, e.g., 20 MP+, have other problems such as increased sensor noise, lower sensitivity, or high costs.

[0112] By taking multiple pictures, the license plate can be captured in an earlier photo and evidence of distracted driving in a later photo. To capture the license plate, the camera's exposure settings can be reduced (e.g., lower gain and exposure time), while to capture the distracted driving behavior, the camera's exposure settings should be increased, with higher gain and exposure time. For example, a license plate image can be exposed at a relative intensity of 1% to 50% of the violation setting. In one particular embodiment, the license plate image is exposed at 5%.

[0113] The camera settings can be quickly and automatically changed between shots to facilitate this. If this doesn't happen, images from phones or mobile devices will be too dark and lack detail, and license plates will be too bright, overexposed by the flash, and potentially illegible. Other options can be used to address this issue, such as changing the flash intensity between shots, using a separate offset flash for illumination and thus license plate recognition, or using a different camera for license plates.

[0114] In one embodiment, a C-mount lens is used, which provides good depth of field so that all objects are in focus, even at the very large apertures required for the application. In other embodiments, a larger format camera can be used. This is because it uses a narrower lens than larger format lenses. The selected lens can be a prime lens, such as an industrial prime lens with a resolution of 12 MP. The resolution can be equal to or higher than that of the sensor to avoid blurry images. In other embodiments, a varifocal lens can be used.

[0115] In the Fig. In the embodiment shown in Figure 6B, the narrowband filter is placed in front of the lens, allowing only the wavelengths produced by the one or more flash units to pass through. In other embodiments, the narrowband filter is placed behind the lens. This filter eliminates most of the light produced by the sun and / or ambient light, and thus also most of the glare effects.

[0116] The narrowband filter can block all or substantially all light except for light at or around a certain wavelength. An example of a narrowband filter that has been implemented is the Bi850 near-IR interference band bass filter, which, according to its description, has a usable range of 845 to 860 nm, that is, 15 nm. This narrowband filter is available from Midwest Optical Systems, Inc., 322 Woodwork Lane, Palatine, IL 60067, USA. Based on the teaching provided, a person skilled in the art can readily select a suitable narrowband filter.

[0117] Two flash options can be provided: one at 760 nm and a second at 850 nm. One or both of these flash options can be included.

[0118] The filter used with the 850 nm flash can transmit only light, or essentially only light, between 840 and 860 nm, thereby eliminating 98% of the available sunlight that would otherwise be detected by the sensor. In other embodiments, the light transmission can be between 700 and 1000 nm, 750 and 950 nm, 800 and 900 nm, 820 and 890 nm, 820 and 880 nm, or 850 and 870 nm. In the Fig. In the embodiment shown in Figure 6B, the one or more filters eliminate 97.5% of the light normally visible to the camera and allow only light with the same wavelength as the one or more flashes to pass through. In other embodiments, the one or more filters eliminate approximately 90% or approximately 95% of the light normally visible to the camera and allow only light with the same wavelength as the one or more flashes to pass through.

[0119] The narrowband filter can cover a wavelength band of less than 5 nm; 5; 10; 15; 20; 25; 30; 35; 40; 45; 50; 55; 60; 65; 70; 75; 80; 85; 90; 95; 100; 110; 120; 130; 140; 150; 200; 250; 300; 350; 400; 450; or 500 nm. The narrowband filter can also cover a wavelength band of 5 nm or less; 10 nm or less; 15 nm or less; 20 nm or less; 25 nm or less; 30 nm or less; 35 nm or less; 40 nm or less; 45 nm or less; 50 nm or less; 55nm or less; 60nm or less; 65nm or less; 70nm or less; 75nm or less; 80nm or less; 85nm or less; 90nm or less; 95nm or less; or 100nm or less.

[0120] The lenses used can have a focal length of 35 mm for mobile installations with a lower mounting height and 50 mm for fixed installations with a higher mounting height. The focal length can be selected to allow a tight zoom on the vehicle for higher resolution evidence. Sufficient width and context can be provided to show one or more sections of the vehicle's full width, most of the roadway, and the license plate.

[0121] A polarizer can be used or not to further reduce glare from the sun.

[0122] Advantageously, one or more flashing lights can be capable of flashing 40,000 times per day at high intensity but for a short duration without interruption. In other embodiments, one or more flashing lights can be capable of flashing 10,000 to 100,000; 20,000 to 80,000; or 30,000 to 50,000 times per day at high intensity and for a short duration. The one or more flashing lights may need to flash two or more times for each vehicle. Conventional xenon flashing lights are not capable of such high repetition rates, so LED or laser technology can be used. Xenon flashing lights also operate in the broad spectrum, while LEDs and lasers can operate in the narrow spectrum.

[0123] Fig. 6A and Fig. Figure 6B shows an embodiment of a customer design of one or more flashlights 400 comprising 384 IR LEDs, which are closely directional using individual lenses at 22 degrees, in a vehicle-mounted device 300 according to an embodiment of the invention. Large capacitor banks can store charge between flashlights, and the LEDs, when triggered, can generate a high luminous intensity for a very short time.

[0124] The one or more flashes can comprise one or more light sources. The one or more light sources can comprise one or more LED light sources and / or one or more laser light sources. The one or more light sources can cover a narrow spectrum. The one or more flashes can comprise one or more light sources. The one or more flashes can comprise 10 to 1,000; 40 to 500; or 300 to 400 light sources. Each of the one or more light sources can include an IR LED light source.

[0125] In other embodiments, the individual lenses can be at 1 to 35; 15 to 30; or 20 to 25 degrees.

[0126] The LED wavelength can be tightly controlled to ensure that light is provided within a specific spectral band, and a camera filter can be used to exclude all light that is not within that band.

[0127] The narrowly controlled wavelength can encompass a narrow spectrum light source that transmits the majority of its emitted light within 5, 10, 20, 25, 30, 35, 40, 45, or 50 nm of its central wavelength.

[0128] In a specific embodiment, the one or more light sources may include an Oslon Black, Oslon Black Series 850 nm -80°, SFH 4715AS, available from Osram Opto Semiconductors.

[0129] The spectral bandwidth can be determined at 50% Irel,max FWHM (Full Width at Half Maximum).

[0130] Both 760 nm and 850 nm flash variants were used and tested. At 760 nm, the camera is more sensitive, but the available LEDs are not as bright as those at 850 nm. Typical vehicle windshields allow more light to pass through at 760 nm than at 850 nm. A slight red glow is visible with any 760 nm flash. 850 nm is a more common IR wavelength and is less visible to drivers. The filtering available at 850 nm is narrower than at 760 nm, blocking more sunlight and thus reducing glare. Both approaches are viable, and with other LED suppliers, almost any wavelength between 730 nm and 950 nm can be used, provided it is narrow and not wide.

[0131] The inventor unexpectedly discovered that high viewing angles into the vehicle may be necessary to see a phone or mobile device held low behind the steering wheel. Angles of up to 80 degrees vertically from the ground plane to the camera can be used. A minimum of 40 degrees may be required. Typically, trucks may require lower angles, and passenger cars may require higher angles. These high viewing angles may necessitate high mounting positions, at least 4 meters and typically 7 to 10 meters above the road surface.

[0132] The angle can be 30 to 90, 35 to 90, or 40 to 90 degrees vertically from a base plane to the camera. In one embodiment, the angle can be 65 degrees. The angle can be 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89 or 90 degrees vertically from the ground.

[0133] When using a lower mount or a side-mounted mount, the horizontal angle can compensate for a lack of vertical angle. A horizontal angle of up to 45 degrees allows a view into the interior without obstruction from the A-pillar.

[0134] A camera mounting position encompassing a horizontal angle between -70 and 70 degrees, -45 and 45 degrees, and -30 and 30 degrees can be used. When used at the roadside, the horizontal angle can be up to 45 degrees. When used above the roadway, the horizontal angle can be 0 degrees. In the case of above-road use, a fixed, high-level camera installation may be required.

[0135] The inventor has determined that the one or more cameras can be mounted more readily on the passenger side of the vehicle than on the driver's side. This is because tests have shown that people prefer to use their phone or mobile device on the side with more available space. In a right-hand drive vehicle, this is the left side.

[0136] By taking multiple pictures of each vehicle, a single camera can be used to provide evidence from different angles, thereby increasing the chance of obtaining evidence of the use of a phone or mobile device.

[0137] A video camera can also be provided to offer additional context, a wider field of view, and sustained evidence of distractions lasting several seconds. License plates can also be extracted from the video.

[0138] The incriminating data, comprising one or more images, can be packaged into an SQLite database file and then encrypted with a randomly generated symmetric AES key. The AES key can then be encrypted with a public RSA key and packaged along with the incriminating file. The file can only be decrypted with the corresponding private RSA key, which is necessary to access the AES key. The data in the file can be hashed and encrypted with a private key. The hash can be verified against the public key. If the evidence file has been tampered with, the hash will not match.

[0139] A computer, such as computer device 201, can be connected to the device at the site. The connection can be an Ethernet connection. The computer can be located up to 100 m away or mounted within a 5 m radius. The computer can use one or more graphics cards to improve processing speed depending on the application. A Linux operating system can be used.

[0140] Advantageously, automation can be used to reduce the camera's image load, i.e., to ignore images of operators who are not using a phone or mobile device. Radio frequency surveillance (RF) approaches cannot be used because they are not suitable for detecting activities related to data transmission or the passive (non-transmitting) use of a mobile device, such as viewing content stored on the phone or mobile device. In one embodiment, image analysis is used to identify those vehicle operators who are using a phone or mobile device.

[0141] Here are two examples of automation procedures. The first classifies an image as showing a driver using a phone or mobile device, and the second detects a phone or mobile device in an image and reports its location.

[0142] In the first exemplary automation procedure, the position of the vehicle operator is determined within one or more images. This can be achieved through steering wheel detection and / or person detection. The one or more images can then be cropped to generate an image showing only the vehicle operator and the operator's immediate surroundings, which can be defined by a span. This can include the operator and the area the operator can reasonably reach with their hands. The cropped images can then be presented to a classifier. The classifier can then assess whether the image clearly shows an operator using a phone or mobile device, clearly shows an operator not using a phone or mobile device, or is an ambiguous classification. The classification can provide a confidence score for each categorization.If confidence in the use of a phone or mobile device exceeds a threshold and / or confidence in the non-use of a phone or mobile device falls below a threshold, the images may be sent for further review.

[0143] In the second automation method, the entirety of one or more images can be searched for a phone or mobile device using a neural network, such as a deep convolutional neural network. If a phone or mobile device is identified based on the confidence score, the one or more images can be sent for further examination. Some images can be automatically excluded if the phone or mobile device is associated with a passenger rather than the operator. Some images can be automatically excluded if the phone or mobile device is detected in a holder rather than being handheld.

[0144] Depending on any automation procedure, the one or more cropped images can be standardized. The cropped images can be created using a standard width and height centered or substantially centered on the driver's coordinates. The width and height can be chosen to capture all details of the driver while excluding any passengers. In cases where the driver's coordinates are near the edge of the camera's field of view, the cropped image can be filled with gray so that the driver's coordinates remain centered and the cropped image maintains a standard size. Each cropped image can then be analyzed to determine its lightest and darkest pixels. An offset can be applied to all pixels in the cropped area so that the darkest pixels are set to zero (0), i.e., black.A scaling adjustment can then be applied to all pixels in the cropped area, setting the brightest pixels to a value of 255, i.e., white. All other pixels can be linearly scaled between 0 and 255 using a histogram adjustment operation. The overall effect can be to maximize contrast in the area of ​​interest. A final processing step can be applied, in which the cropped image is resized so that the resulting electronic file, e.g., JPEG, has a standard file size.

[0145] Additionally, one or more cropped images can be adjusted using any automation method to achieve standardized brightness and / or contrast. This image adjustment can be automatic, such as through computer processing.

[0146] The automated detection process cannot operate with 100% accuracy. To capture all violations, images that do not contain a phone or mobile device are also flagged. In a fully automated system, some or many violations will be missed. Therefore, further review may include a manual check. To avoid excessive data bandwidth, this further review may be conducted through a multi-stage process.

[0147] The automated analysis may include detection. In one embodiment, the detection of a phone or mobile device involves searching one or more images for a phone or mobile device. The search may involve a neural network or artificial neural network, such as a deep neural network or a deep convolutional neural network. The search may encompass the entire image. If a phone or mobile device is detected based on the confidence score, the one or more images may be sent for further examination. One or more images may be excluded if a detected phone or mobile device is associated with a passenger rather than a vehicle operator or driver.On the other hand, one or more images can be automatically excluded if the phone or mobile device is detected in a holder and is not being held by hand.

[0148] The captured and / or received images may include a view of the entire front of the vehicle and the roadway in which the vehicle is located.

[0149] The neural network can include an object detection system. The object detection system can use a real-time YOLO (You Only Look Once) object detection architecture based on a neural network, as described at https: / / pjreddie.com / darknet / yolo / .

[0150] The neural network can include an image classifier. The image classifier can be a VGG (Visual Geometry Group) classifier based on a neural network. The classifier can receive one or more cropped images. The classifier can be pre-trained, for example, by training on a dataset such as the 1,000-class ImageNet set. The model can be modified to identify only two classes: received cropped images that involve illegal use of a phone or mobile device (i.e., positive), and received cropped images that do not involve illegal use of a phone or mobile device (i.e., negative). The classifier can be trained by fine-tuning on a training set of sample images.The sample images can include more than one hundred, more than one thousand, more than one hundred thousand, or more than one million sample images, such as cropped images. The training set can include both positively and negatively labeled images. When the received cropped image of the driver is fed into the classifier network, the analysis can determine a confidence level as an output that the image is positive for the use of a phone or mobile device. The determined confidence level can range, for example, from a 0% confidence output indicating that the use of a phone or mobile device is very unlikely, to a 100% confidence output indicating that the use of a phone or mobile device is very likely. A threshold can be used to limit which images are considered positive and which are considered negative.The threshold can include a dynamically adjustable threshold, so that an upper margin of images is considered positive and sent for manual review.

[0151] The upper margin can comprise the top 5%, 10%, 15%, 20%, or 25% of received cropped images with respect to the specified confidence level.

[0152] The analysis can include a real-time object detector. This detector can identify the driver by the presence of a steering wheel and the person behind it. The training set can include images from various angles, optionally both vertical and horizontal variations, ensuring the solution can be used in any application and still detect the driver. The training set can include images of many different vehicle types and, optionally, images of left-hand drive and right-hand drive vehicles. The analysis can include positive driver identification while ignoring passengers. The real-time object detector can output the driver's position within the image and, optionally, a confidence score indicating that the driver has been found.

[0153] According to one of the above forms, the received image can be cropped if the driver has been located in the received image.

[0154] The single standardized cropped image or multiple standardized cropped images can be fed to the image classifier. The image classifier can classify the single standardized image or multiple standardized cropped images. The image classifier may have been trained on a training set containing multiple examples of cropped driver images. The training set includes tagged images of drivers illegally touching phones or mobile devices, drivers not illegally touching phones or mobile devices, and drivers not touching phones or mobile devices at all. When the driver's image is received by the image classifier, an output can be provided that includes a most-probable category and a confidence level for making that determination.The most likely category can be selected from: a) on the phone or mobile device or b) not on the phone or mobile device.

[0155] Classification can include determining the driver's position within one or more images. Position determination can involve steering wheel detection and / or person detection. Once determined, the one or more images can be cropped to generate an image showing only the driver and their immediate surroundings. The immediate surroundings can encompass the driver's span. The cropped image can then be provided for further review.

[0156] If one or more images are detected with a high degree of confidence indicating the use of a phone or mobile device, data transfer and the review system are activated. This may include: cropping the operator generated by the camera system; uploading the one or more cropped images to a server; logging a user into an 'image review' website; the user may be presented with an image and asked to choose one of four options: 1) On phone or mobile device; 2) Not on phone or mobile device; 3) Uncertain; and 4) Other; if 1), 3), or 4) is selected, a message is sent to the camera system; the next cropped image may be presented to the user for review.The one or more camera systems can initiate a transfer of the fully encrypted violation file, which can be sorted according to the selected option, for example, transfer to the folder 'On phone or mobile device'; the customer can download the violation file from cloud storage; and the customer can decrypt and extract the violation file data, processing this data as if it were a red light or speeding violation.

[0157] The server may include a cloud server. Currently, AWS SQS is used. Logging in may involve entering credentials.

[0158] A typical driver package is around 100kB, while a violation package can range from 2MB to 10MB (depending on image and video options). Uploading violations that demonstrably contain data is the only way to save a significant amount of bandwidth.

[0159] Another advantage of the invention is that a confidence threshold for initiating a data review can be adjusted based on the customer's needs. For example, a setting can be chosen on the spectrum from 0% missed violations, 100% manually processed images to 100% missed violations, 0% manually processed images. A typical setting might be, for example, 5% missed violations with 10% manually processed images.

[0160] There are several examples of its use. In the case of use by a road authority, the device and system of an enforcement camera operate similarly to a red-light camera system. The one or more cameras are permanently mounted and operate 24 / 7. They can record evidence of the illegal use of a phone or mobile device, and this evidence can be submitted to a central processing unit for handling violations. This unit can then conduct a registration search and send a penalty notice by mail to the registered owner.

[0161] In the case of use by a mobile traffic enforcement agency, the one or more cameras, instead of being stationary, can be mounted on a vehicle or trailer. The one or more cameras can be driven to a position and temporarily activated. The one or more cameras can be rotated through different locations according to a plan to provide wider coverage. This model is very similar to how mobile speed enforcement operations are conducted. The data processing is nearly identical to that of a stationary setup.

[0162] Another scenario involves police use. In such cases, the device and system are used in conjunction with a manned police operation. This operation may be focused on addressing distracted driving or may involve other tasks such as checking for drunk driving, vehicle registration checks, and responding to traffic violations.

[0163] Typically, a camera system mounted on a mobile vehicle or trailer can be deployed in the same manner as in the case of use by the mobile traffic authority, although the device and system can also be deployed using a portable tripod solution. The device and system can capture one or more images of each passing vehicle. If the device and system detect a high probability of the use of a mobile computing device, the images can be transmitted to the police. The one or more images can be reviewed by a person, either via an internet-connected remote review service or by a dedicated person stationed with the camera system to reduce the workload of the police.

[0164] The police can view one or more images using conventional methods, such as on a phone or mobile device like a smartphone, tablet, or other mobile device. The image data will show the offender, the vehicle type, and the license plate number. The police can then wait for the target vehicle to approach and stop it to deal with the crime on the spot.

[0165] The device and system can also be used to enforce a speed or average speed; use more than one camera to provide additional angles into the interior to improve law enforcement; use more than one camera to provide depth information by means of stereoscopic techniques to improve detection accuracy; provide a real-time display of offenses to a variable message sign showing images of the offending driver; provide survey services showing the prevalence of behavior by time of day, vehicle type, location, or any other parameter or variable.

[0166] An embodiment of a computer system 200 and a computer device 201, which is suitable for use in the present invention, is described in Fig. 2A and Fig. Figure 2B shows the computer system 200. In the embodiment shown, the computer system 200 comprises a computer device 201, including input devices such as a keyboard 202, a mouse pointer device 203, a scanner 226, an external hard disk 227, and a microphone 280; and output devices, including a printer 215, a display device 214, and a loudspeaker 217. In some embodiments, the video display 214 may include a touchscreen.

[0167] A modulator-demodulator (modem) transceiver device 216 can be used by the computer device 201 to communicate with a communication network 220 via a connection 221. The network 220 can be a wide area network (WAN), such as the Internet, a mobile network, or a private WAN. The computer device 201 can be connected to other similar personal devices 290 or server computers 291 via the network 220. If the connection 221 is a telephone line, the modem 216 can be a conventional dial-up modem. Alternatively, the modem 216 can be a broadband modem if the connection 221 is a high-capacity connection (e.g., cable). A wireless modem can also be used for wireless communication with the network 220.

[0168] The computer device 201 typically includes at least one processor 205 and memory 206, which may consist, for example, of semiconductor random access memory (RAM) and semiconductor read-only memory (ROM). The device 201 also includes a number of input / output interfaces (I / O interfaces), including: an audio-video interface 207, which is coupled to the video display 214, the speakers 217, and the microphone 280; an I / O interface 213 for the keyboard 202, the mouse 203, the scanner 226, and the external hard disk 227; and an interface 208 for the external modem 216 and the printer 215. In some implementations, the modem 216 may be integrated into the computer device 201, for example, within the interface 208.The computer device 201 also has a local network interface 211, which enables the computer system 200 of the computer device 201 to be coupled to a local computer network 222, known as a local area network (LAN), via a connection 223.

[0169] As also illustrated, the local area network 222 can also be coupled to a wide area network 220 via a connection 224, which typically includes a so-called "firewall" device or a device with similar functionality. The interface 211 can be provided by an Ethernet circuit board, a Bluetooth wireless arrangement, an IEEE 802.11 wireless arrangement, or another suitable interface.

[0170] The I / O interfaces 208 and 213 can provide one or both of serial and parallel connectivity, with the former typically being implemented according to Universal Serial Bus (USB) standards and featuring corresponding USB connectors (not illustrated).

[0171] Storage devices 209 are provided and typically include a hard disk drive (HDD) 210. Other storage devices, such as an external HD 227, a disk drive (not shown), and a magnetic tape drive (not shown), may also be used. An image disk drive 212 is typically provided to serve as a non-volatile data source. Portable storage devices, such as image disks (e.g., CD-ROM, DVD, Blu-ray Disc), USB RAM, external hard disks, and floppy disks, may be used as suitable data sources for the personal device 200. Another data source for the personal device 200 is provided by the one or more server computers 291 via network 220.

[0172] The components 205 to 213 of the computer device 201 typically communicate via a networked bus 204 in a manner that results in a conventional operating mode of a personal device 200. In the embodiment described in Fig. 2A and Fig. As shown in Figure 2B, processor 205 is coupled to system bus 204 by connections 218. Similarly, memory 206 and video disk drive 212 are coupled to system bus 204 by connections 219. Examples of personal devices 200 on which the described arrangements can be implemented include IBM PCs and compatible devices, Sun Sparc stations, Apple computers, smartphones, tablet computers, or a similar device comprising a computer device 201 similar to a computer module. It is clear that if the personal device 200 comprises a smartphone or tablet computer, a display device 214 may include a touchscreen, and other input and output devices, such as a mouse pointer device 203, a keyboard 202, a scanner 226, and a printer 215, may not be included.

[0173] Fig. Figure 2B is a detailed schematic block diagram of processor 205 and memory 234. Memory 234 represents a logical assembly of all memory modules, including memory device 209 and semiconductor memory 206, to which computer device 201 is connected. Fig. 2A can be accessed.

[0174] The methods described above can be implemented using a computer device 200, wherein the methods can be implemented as one or more software application programs 233 that are executable in the computer device 201. In particular, the steps of the methods can be implemented by instructions 231 in the software that is executed in the computer device 201.

[0175] The software instructions 231 can be formed as one or more code modules, each of which serves to perform one or more specific tasks. The software 233 can also be divided into two separate parts, wherein a first part and the corresponding code modules carry out the procedure, and a second part and the corresponding code modules manage a graphical user interface between the first part and the user.

[0176] The software 233 can be stored on a computer-readable medium contained in a storage device of the type described herein. The software is loaded from the computer-readable medium or via network 221 or 223 into the personal device 200 and then executed by the personal device 200. In an example, the software 233 is stored on storage medium 225, which is read by the disk drive 212. Software 233 is typically stored in the HDD 210 or the memory 206.

[0177] A computer-readable medium containing such software 233 or such a recorded computer program is a computer program product. The use of the computer program product in the personal device 200 preferably implements a device or arrangement for implementing the methods described above.

[0178] In some cases, the software application programs 233 can be provided to the user encoded on one or more disk storage media 225, such as a CD-ROM, DVD, or Blu-ray Disc, and read via a suitable drive 212, or alternatively, read by the user from the network 220 or 222. Furthermore, the software can also be loaded into the personal device 200 from other computer-readable media. Computer-readable storage media refers to any non-transient, tangible storage medium that provides recorded instructions and / or data to the computer device 201 or personal device 200 for execution and / or processing.Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray Disc, a hard disk drive, a ROM or integrated circuit, USB storage devices, a magneto-optical disk, or a computer-readable card, such as a PCMCIA card, and the like, whether such devices are internal or external to the computer device 201. Examples of transitory or non-tangible computer-readable transmission media, which may also be involved in providing software application programs 233, instructions 231, and / or data to the computer device 201, include radio or infrared transmission channels, as well as a network connection 221, 223, 334 to another computer or networked device 290, 291 and the Internet or an intranet containing email transmissions and information recorded on websites and the like.

[0179] The second part of the application programs 233 and the corresponding code modules mentioned above can be executed to implement one or more graphical user interfaces (GUIs) to be displayed on the screen 214 or represented in some other way. By manipulating, typically, the keyboard 202, mouse 203, and / or screen 214 (if a touchscreen is included), a user of a personal device 200 and the methods described above can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces can also be implemented, such as an audio interface that uses voice prompts output via loudspeaker 217 and user voice commands entered via microphone 280.The manipulations include mouse clicks, screen touches, voice prompts and / or user voice commands that can be transmitted over network 220 or 222.

[0180] When the computer device 201 is initially powered on, a power-on self-test (POST) program 250 can be executed. The POST program 250 is typically stored in a ROM 249 of the semiconductor memory 206. A hardware device such as the ROM 249 is sometimes referred to as firmware. The POST program 250 examines hardware in the computer device 201 to ensure proper functioning and typically checks the processor 205, memory 234 (209, 206), and a BIOS (Basic Input-Output System) software module 251, which is also typically stored in the ROM 249, for correct operation. Once the POST program 250 has run successfully, BIOS 251 activates the hard disk drive 210. Activating the hard disk drive 210 causes a bootstrap loader program 252, located on the hard disk drive 210, to be executed via processor 205.This loads an operating system 253 into RAM 206, after which operating system 253 begins to run. Operating system 253 is a system-level application executable by processor 205 to perform various high-level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface.

[0181] Operating system 253 manages memory 234 (209, 206) to ensure that each process or application running on computer device 201 has sufficient memory to execute without conflicting with memory allocated to another process. Furthermore, the various types of memory available in personal device 200 must be used correctly so that each process can run effectively. Therefore, the compiled memory 234 is not intended to illustrate how specific segments of memory are allocated, but rather to provide a general overview of the memory that computer device 201 can access and how it is used.

[0182] The processor 205 contains a number of functional modules, including a control unit 239, an arithmetic logic unit (ALU) 240, and a local or internal memory 248, sometimes referred to as cache memory. The cache memory 248 typically contains a number of memory registers 244, 245, 246 in a register section that stores data 247. One or more internal buses 241 connect these functional modules. The processor 205 also typically has one or more interfaces 242 for communicating with external devices via the system bus 204 using a connection 218. The memory 234 is connected to the bus 204 by connection 219.

[0183] Application program 233 contains a sequence of instructions 231, which may include conditional branching and loop statements. Program 233 may also contain data 232, which is used in the execution of program 233. The instructions 231 and the data 232 are stored at memory locations 228, 229, 230 and 235, 236, 237, respectively. Depending on the relative size of the instructions 231 and the memory locations 228-230, a particular instruction may be stored in a single memory location, represented by the instruction shown in memory location 230. Alternatively, an instruction may be segmented into a number of parts, which are stored in separate memory locations, as represented by the instruction segments shown in memory locations 228 and 229.

[0184] In general, processor 205 receives a set of instructions 243, which it executes. Processor 205 then waits for a subsequent input, to which processor 205 responds by executing another set of instructions. Each input can be provided by one or more of a number of sources, containing data generated by one or more of the input devices 202, 203, or 214 (if they include a touchscreen), data received from an external source via one of the networks 220 or 222, data retrieved from one of the storage devices 206 or 209, or data retrieved from a storage medium 225 inserted into the appropriate reader 212. The execution of a set of instructions may, in some cases, result in the output of data. Execution may also involve storing data or variables in memory 234.

[0185] The disclosed arrangements use input variable 254, which is stored in memory 234 at corresponding memory locations 255, 256, 257, and 258. The described arrangements generate output variable 261, which is stored in memory 234 at corresponding memory locations 262, 263, 264, and 265. Intermediate variable 268 can be stored at memory locations 259, 260, 266, and 267.

[0186] The register sections 244, 245, 246, the arithmetic logic unit (ALU) 240, and the control unit 239 of the processor 205 work together to perform sequences of microoperations necessary to execute "fetch, decode, and execute" cycles for each instruction in the instruction set that constitutes the program 233. Each fetch, decode, and execute cycle comprises: (a) a retrieval operation that retrieves or reads an instruction 231 from memory location 228, 229, 230; (b) a decoding operation in which the control unit 239 determines which instruction was retrieved; and (c) an execution operation in which the control unit 239 and / or the ALU 240 execute the instruction.

[0187] After that, another retrieval, decoding, and execution cycle can be performed for the next instruction. Similarly, a memory cycle can be performed in which the control unit 239 stores or writes a value to a memory location 232.

[0188] Each step or subprocess in the above described procedures can be linked to one or more segments of program 233 and can be executed by the register section 244-246, the ALU 240 and the control unit 239 in the processor 205, which work together to perform the fetch, decode and execute cycles for each instruction in the instruction set for the specified segments of program 233.

[0189] One or more other computers 290 can be connected to the communication network 220, as shown in Fig. 2A is recognizable. Each such computer 290 can have a similar configuration to the computer device 201 and corresponding peripherals.

[0190] One or more other server computers 291 can be connected to the communication network 220. These server computers 291 respond to requests from the personal device or other server computers to provide information.

[0191] Method 100 can alternatively be implemented in dedicated hardware, such as one or more integrated circuits that perform the functions or partial functions of the described methods. Such dedicated hardware can include graphics processors, digital signal processors, or one or more microprocessors and associated memory.

[0192] It is clear that to implement the methods as described above, it is not necessary for the processors and / or the memory of the processing machine to be physically located at the same geographical location. That is, each of the processors and memory used in the invention can be located at geographically different locations and can be connected to each other in a suitable manner to communicate. Additionally, it is clear that each of the processors and / or memory can consist of different physical device parts. Therefore, it is not necessary for a processor to be a single device part at one location and for the memory to be a separate single device part at another location. That is to say, it is considered that the processor can consist of two device parts at two different physical locations. The two different device parts can be connected in any suitable manner.Additionally, the storage can contain two or more parts of storage in two or more physical locations.

[0193] For further clarification: The processing described above is performed by different components and different storage devices. However, it is understood that the processing described above, which is performed by two different components, can also be performed by a single component according to a further embodiment of the invention. Furthermore, the processing performed by a specific component as described above can be performed by two different components. Similarly, the storage performed by two different storage areas as described above can be performed by a single storage area according to a further embodiment of the invention. Furthermore, the storage performed by a specific storage area as described above can be performed by two storage areas.

[0194] Furthermore, various technologies can be used to provide communication between the different processors and / or memories and to enable the processors and / or memories of the invention to communicate with other units, i.e., to receive further instructions or to access and use remote memories. Technologies that can be used to provide such communication could include, for example, a network, the Internet, an intranet, an extranet, a LAN, Ethernet, a telecommunications network (e.g., a mobile or wireless network), or any client-server system that provides communication. Such communication technologies can use any suitable protocol, for example, TCP / IP, UDP, or OSI.

[0195] The following non-limiting examples illustrate the invention. These examples are not to be understood as limiting: The examples serve only for illustration. The examples are to be understood as an explanation of the invention. Examples of tripod use during the day - no flash

[0196] Fig. Figures 3A to 3E show images taken according to the invention using a tripod, with the images being taken during the day and without flash. Fig. 3A and Fig. 3B allowed detection of mobile phone use; whereby Fig. 3B zooms in on an image. Fig. 3C shows that capturing and reading the license plate is possible. Fig. 3D enabled detection of mobile phone use and driving at a speed ten percent higher than the speed limit. Fig. 3E allowed detection of a hand outside the vehicle and not on the steering wheel.

[0197] Fig. 3F; Fig. 3G; and 3H show other exemplary images demonstrating the use of a mobile phone. DAYTIME VEHICLE USE - FLASHING LIGHT

[0198] Fig. 4A and Fig. 4B; Fig. 4C and Fig. Images 4D, 4E, and 4F are pairs of images taken using a vehicle according to the invention. The images were taken during the day using a flash. Fig. 4B, Fig. 4D and Fig. 4F are zoomed in to show the images from Fig. 4A, Fig. 4C and Fig. 4E to better illustrate the use of a mobile phone. VEHICLE OPERATION AT NIGHT - FLASHING LIGHT

[0199] Fig. 5A and Fig. Images 5B, 5C, and 5D are pairs of images taken using a vehicle according to the invention. The images were taken at night using a flash. Fig. 5B and Fig. 5D images are zoomed in to improve the use of the images from Fig. 5A and Fig. 5C to be displayed better on mobile phones. Fig. 5C and Fig. Figure 5D shows that the invention is advantageously able to detect the violation on a distant roadway. EXAMPLES OF INITIAL OPERATIONS

[0200] The device of the invention has been successfully used in a mobile embodiment, for example on a trailer, see Fig. 7A, which shows its use in the Southern Highlands of New South Wales, Australia; and in various solid embodiments, see Fig. 7B, which shows the deployment on an overpass above the M4 motorway in New South Wales, Australia; and Fig. 7C, showing its use on a scaffold for a variable message sign (VMS) along Anzac Parade in Sydney, Australia.

[0201] These sample deployments resulted in successful image captures showing drivers using mobile phones. An example image is in Fig. 8A is shown.

[0202] In early 2019, six operations were conducted at fixed locations in Australia, running continuously for 90 days. During these operations, 8,066,292 vehicle transits were recorded and analyzed. Of these 8,066,292 transits, 95,445 drivers were clearly identified as illegal users of a mobile phone. The operations took place on roads with speed limits between 70 km / h and 100 km / h.

[0203] In mid-2019, six operations involving trailers were conducted in Australia, lasting a total of 42 days. During these operations, 446,367 vehicle transits were recorded, with 8,438 drivers identified as illegal mobile phone users. The operations took place on roads with speed limits between 60 km / h and 90 km / h.

[0204] In another trailer-based deployment in Tasmania, Australia, the device of the invention was used continuously for 36 hours to monitor a single lane. The device of the invention captured evidence of 446 drivers illegally using a mobile phone, 173 drivers exceeding the speed limit of 80 km / h by more than 7 km / h, and identified 51 vehicles that had been out of service for more than 30 days. The device recorded a total of 15,984 vehicles during this period.

[0205] In this description, the terms "includes", "comprehensive" or similar terms mean non-exclusive inclusion, so that an establishment that includes a list of items may contain not only those items but also other items that are not listed.

[0206] Throughout this description, care has been taken to describe the invention without limiting it to a specific embodiment or a particular combination of features. Those skilled in the field can recognize variations from specific embodiments that nevertheless fall within the scope of the invention. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] AU 20020141618

[0004] AU 20120162432

[0005] AU 20120007983

[0006] AU 20090046897

[0007] AU 20090207046

[0008] AU 20060047371

[0009] AU 20050073436

[0010] US 6,266,627

[0011] US 2008 / 0036623

[0013] US 11 / 678,489

[0013] US 2010 / 00271497

[0014] US 12 / 769,41

[0014] WO 2015 / 017883

[0015] AU 2014 / 00783

[0015] Cited non-patent literature

[0000] Midwest Optical Systems, Inc., 322 Woodwork Lane, Palatine, IL, 60067, USA

[0116] https: / / pjreddie.com / darknet / yolo /

[0149]

Claims

[1] Device (300) for detecting an infringement by a vehicle operator, the device comprising: one or more sensors (500) for detecting a vehicle and triggering one or more cameras (600) to take one or more pictures when the vehicle reaches a picture capture point; one or more flashing lights (400) for illuminating the vehicle or part of it with light in a narrow band; wherein the one or more cameras are designed to capture one or more images of at least one part of the vehicle operator, wherein at least one of the one or more cameras comprises a narrowband filter which transmits only or substantially only the wavelengths of light produced by the one or more flashlights and eliminates the majority of the ambient light and / or the light produced by the sun; one or more computer processors for automatically analyzing the one or more captured images to detect an infringement, wherein an automatic analysis includes detecting a phone or mobile device, comprising searching the one or more images for a phone or mobile device, the search comprising a neural network, and classifying, comprising determining the location of the driver in the one or more images, wherein, after the determination, the one or more images are cropped to produce an image with a standard width and height centered on a coordinate of the driver, wherein one or more cropped images are analyzed to determine the brightest and darkest pixels, and an offset is applied to all pixels, wherein the analysis of the one or more cropped images maximizes the contrast in an area of ​​interest, wherein the neural network comprises a pre-trained image classifier, the classifier receiving the one or more cropped images, and an output model of the neural network designed to classify the one or more cropped images into an appropriate class, the appropriate classes representing either images containing illegal use of phones or mobile devices (i.e., positive) or images not containing illegal use of phones or mobile devices (i.e., negative), and wherein a threshold is used to limit which images are classified as positive and which as negative, and wherein the automatic image analysis includes classifying an image as showing a driver using a telephone or mobile device, and / or detecting a telephone or mobile device in an image and reporting the location of the telephone or mobile device; and one or more outputs to provide the one or more captured images that include the identified infringement, in order to identify the infringement. [2] Computer system (201) for detecting an infringement of the law by a driver, the computer system comprising: the device according to claim 1, wherein the provision for further verification or confirmation is carried out via a computer or telecommunications network. [3] A computer program product (233) comprising the following: a computer-readable medium and a computer-readable program code stored on that computer-readable medium for detecting an infringement by a driver, the computer-readable code comprising the following: Computer-readable program code devices (i) configured to cause the computer to detect a vehicle based on input from one or more sensors and to trigger one or more cameras (600) to take one or more pictures when the vehicle reaches a picture capture point; (ii) computer-readable program code devices configured to cause the computer to illuminate the detected vehicle or part thereof with one or more flashes of light in a narrowband; computer-readable program code devices (iii) configured to cause the computer to take one or more pictures of at least part of the driver using one or more cameras, wherein at least one of the one or more cameras includes a narrowband filter which transmits only the wavelengths of light produced by the one or more flashes and eliminates most of the ambient light and / or the light produced by the sun; computer-readable program code devices (iv) configured to cause the computer to automatically analyze one or more captured images in order to detect an infringement, wherein the automatic analysis includes a phone or mobile device detection which involves searching the one or more images for a phone or mobile device, wherein the search includes a neural network and a classification which involves determining the location of the driver in the one or more images, where, after determination, one or more images are cropped to create an image that has a standard width and height centered on a driver coordinate, wherein one or more cropped images are analyzed to determine the brightest and darkest pixels, and an offset is applied to all pixels, wherein the analysis of the one or more cropped images maximizes the contrast in an area of ​​interest, wherein the neural network comprises a pre-trained image classifier, wherein the classifier receives the one or more cropped images, and an output model of the neural network is designed to classify the one or more cropped images into an appropriate class, wherein the appropriate classes represent either images containing illegal use of telephones or mobile devices (i.e., positive) or images not containing illegal use of telephones or mobile devices (i.e., negative), and wherein a threshold is used to limit which images are classified as positive and which as negative, and wherein the automatic image analysis includes classifying an image as showing a driver using a telephone or mobile device, and / or detecting a telephone or mobile device in an image and reporting the location of the telephone or mobile device; and Computer-readable program code devices (v) configured to cause the computer to provide the one or more recorded images comprising the detected infringement in order to detect the infringement. [4] Device, system or product according to any one of claims 1 to 3, wherein the one or more cameras (600) comprise a camera with 5 to 50 MP, 10 to 45 MP or 20 to 35 MP. [5] Device, system or product according to any one of claims 1 to 4, wherein the one or more cameras (600) comprise a global shutter sensor. [6] Device, system or product according to any one of claims 1 to 5, wherein the narrowband filter transmits light between 700 and 1000 nm; 750 and 950 nm; 800 and 900 nm; 820 and 890 nm; 830 and 880 nm; 850 and 870 nm or 840 and 860 nm. [7] Device, system or product according to any one of claims 1 to 6, wherein the filter(s) comprise an infrared (IR) blocking filter. [8] Device, system or product according to any one of claims 1 to 7, wherein the one or the flashlights (400) comprise one or more light sources and wherein the one or the flashlights optionally comprise one or more capacitor banks for storing charge between flashes. [9] Device, system or product according to any one of claims 1 to 8, wherein the one or more flashes (400) have a strictly controlled wavelength. [10] Device, system or product according to any one of claims 1 to 9, wherein the one or more flashlights (400) comprise a main flash and a separate offset flash for marking. [11] Device, system or product according to any one of claims 1 to 10, wherein the detection and / or sensor (500) comprises a radar. [12] Device, system or product according to any one of claims 1 to 11, wherein the one or more flashlights (400) are capable of being triggered 10,000 to 100,000 times per day with high intensity and short duration. [13] Device, system or product according to any one of claims 1 to 12, wherein the neural network comprises an object recognition system.

Images