IMAGING SYSTEM, PROCESSING APPARATUS, AND COMPUTER-IMPLEMENTED METHOD IN AN IMAGING SYSTEM - Patent application

Abstract

This imaging system comprises: a sensor that measures the speed of a moving object; a camera that photographs the object, the camera being different from the sensor; and a processing circuit that controls operation of the sensor and the camera. The processing circuit causes the sensor to measure the speed of the moving object and generate speed information about the object and measurement timing information about the speed. The processing circuit generates control data including a photography timing of the camera on the basis of (a) location information about the object or information about the distance to the object during speed measurement, (b) the speed information, and (c) the measurement timing information. The processing circuit causes the camera to output image data including image information about the object on the basis of the control data.

Description

[Technical Field]

[0001] The present disclosure relates to imaging systems, processing devices, and computer-implemented methods in imaging systems. [Background technology]

[0002] ETC (Electronic Toll Collection) is a system that allows drivers to pay tolls on toll roads such as expressways without having to stop their vehicles at toll booths. Vehicle classification is one of the factors that determine the toll fee structure. Vehicle classification is determined by, for example, vehicle attribute information such as size and number of axles. When a vehicle passes through an ETC lane, communication takes place between the onboard unit installed in the vehicle and the roadside antenna installed in the lane, and data necessary for calculating the toll, such as information on the entrance toll booth and vehicle classification, is exchanged. This is how the toll is calculated.

[0003] In ETC, vehicle detectors individually recognize vehicles, and based on the recognition results, processes such as starting and ending intercommunication, switching roadside displays, and opening and closing gates are performed. Most vehicle detectors are optical, with multiple detectors arranged to determine various conditions such as the vehicle's length and direction of travel. Furthermore, ETC determines whether a vehicle with a towing structure is being towed and counts the number of axles on large vehicles to calculate tolls. Currently, footplate-shaped sensors grounded to the ground are used as axle sensors to count the number of axles.

[0004] Current ETC systems are equipped with gates, and vehicles are required to slow down to a certain speed near the gates. In the future, in order to alleviate traffic congestion, it is expected that there will be an increasing demand for ETC systems that allow vehicles to pass through at high speeds without slowing down. For such ETC systems, an imaging system that can capture high-speed vehicles without them dropping out of the frame, i.e., without missing any objects from the captured image, is desirable.

[0005] Patent Document 1 discloses an automatic imaging device that outputs a command to an imaging device when the speed of a vehicle measured by radar exceeds a certain value. Patent Document 2 discloses a device that measures the traveling speed of a vehicle using radio waves from a mobile phone and takes an image of a specific vehicle based on the measurement result. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-3495 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-56000 Summary of the Invention [Problem to be solved by the invention]

[0007] The present disclosure provides an imaging system that can capture images of not only slow-moving objects but also fast-moving objects without causing the objects to go out of frame. [Means for solving the problem]

[0008] An imaging system according to one aspect of the present disclosure includes a sensor that measures the speed of a moving object, a camera that is different from the sensor and that images the object, and a processing circuit that controls the operation of the sensor and the camera, wherein the processing circuit causes the sensor to measure the speed of the object and generate speed information of the object and information on the timing at which the speed is measured, generates control data including shooting timing information for the camera based on (a) position information of the object or distance information to the object at the time of speed measurement, (b) the speed information, and (c) the measurement timing information, and causes the camera to output image data including image information of the object based on the control data.

[0009] A general or specific aspect of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable recording disk, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. The computer-readable recording medium may include a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory). An apparatus may be composed of one or more devices. When an apparatus is composed of two or more devices, the two or more devices may be located in a single device or may be located separately in two or more separate devices. In this specification and claims, the term "apparatus" may refer not only to a single device but also to a system consisting of multiple devices. [Effects of the Invention]

[0010] According to the technology of the present disclosure, it is possible to realize an imaging system that can capture not only slow-moving objects but also fast-moving objects without causing the objects to go out of frame. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram schematically illustrating the configuration of an imaging system according to a first exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. [Figure 2A] FIG. 2A is a block diagram schematically illustrating an example of the configuration of a sensor that is an FMCW-LiDAR device. [Figure 2B] FIG. 2B is a flow chart that generally illustrates an example of operations performed by processing circuitry included in the sensor. [Figure 3] FIG. 3 is a diagram schematically showing the time variation of the frequencies of the reference light and the reflected light when the vehicle is traveling. [Figure 4] FIG. 4 is a flowchart schematically illustrating an example of an operation performed by the processing circuitry in the first embodiment. [Figure 5]FIG. 5 is a diagram schematically illustrating the configuration of an imaging system according to a second exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. [Figure 6] FIG. 6 is a flowchart schematically illustrating an example of an operation performed by the processing circuitry in the second embodiment. [Figure 7A] FIG. 7A is a diagram showing the positional relationship between the captured image and the ROI when the speed of the vehicle 10 is v1. [Figure 7B] FIG. 7B is a diagram showing the positional relationship between the captured image and the ROI when the speed of the vehicle 10 is v2. [Figure 8] FIG. 8 is a diagram schematically illustrating the configuration of an imaging system according to a third exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. [Figure 9] FIG. 9 is a flowchart schematically illustrating an example of an operation performed by the processing circuitry in the third embodiment. [Figure 10] FIG. 10 is a diagram schematically illustrating the configuration of an imaging system according to a fourth exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. [Figure 11] FIG. 11 is a flowchart schematically illustrating an example of an operation performed by the processing circuitry in the fourth embodiment. [Figure 12] FIG. 12 is a diagram schematically illustrating the configuration of an imaging system according to the fifth exemplary embodiment of the present disclosure. [Figure 13A] FIG. 13A is a perspective view schematically showing how the body of a vehicle is illuminated with illumination light in the imaging system according to the sixth embodiment. [Figure 13B] FIG. 13B is a perspective view that schematically shows how the wheels of a vehicle are illuminated with illumination light in the imaging system according to the sixth embodiment. [Figure 13C] FIG. 13C is a graph showing the change in measurement speed over time. [Figure 14] FIG. 14 is a diagram showing a schematic configuration example of an ETC that combines the fifth and sixth embodiments. [Figure 15]FIG. 15 is a flowchart that schematically illustrates an example of the operations performed by the processing circuitry in the ETC shown in FIG. [Figure 16A] FIG. 16A is a diagram schematically showing how the speed of a vehicle is measured at a first measurement time in the imaging system according to the seventh embodiment. [Figure 16B] FIG. 16B is a diagram schematically showing how the speed of a vehicle is measured at a second measurement time in the imaging system according to the seventh embodiment. [Figure 16C] FIG. 16C is a graph showing the change in vehicle speed over time. [Figure 17] FIG. 17 is a diagram schematically illustrating the configuration of an imaging system according to an eighth exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. [Figure 18] FIG. 18 is a diagram schematically illustrating the configuration of an imaging system according to a ninth exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a cardboard box. DETAILED DESCRIPTION OF THE INVENTION

[0012] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection configurations, steps, and step orders shown in the following embodiments are merely examples and are not intended to limit the technology of the present disclosure. Among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components. Each figure is a schematic diagram and is not necessarily an exact illustration. Furthermore, in each figure, substantially identical or similar components are assigned the same reference numerals. Duplicate descriptions may be omitted or simplified.

[0013] In this disclosure, all or part of a circuit, unit, device, component, or part, or all or part of a functional block in a block diagram, may be implemented by one or more electronic circuits, including, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). An LSI or IC may be integrated on a single chip or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated on a single chip. While the terms LSI and IC are used here, the term may be changed depending on the degree of integration, and may be referred to as a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA), which is programmable after the LSI is manufactured, or a reconfigurable logic device, which can reconfigure connections within the LSI or set up circuit partitions within the LSI, may also be used for the same purpose.

[0014] Furthermore, all or part of the functions or operations of a circuit, unit, device, component, or section can be implemented by software processing. In this case, the software is recorded on one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are performed by the processor and peripheral devices. A system or device may include one or more non-transitory recording media on which software is recorded, a processor, and necessary hardware devices, such as interfaces.

[0015] In this disclosure, "light" refers to electromagnetic waves including not only visible light (wavelength of about 400 nm to about 700 nm), but also ultraviolet light (wavelength of about 10 nm to about 400 nm) and infrared light (wavelength of about 700 nm to about 1 mm).

[0016] First, the findings that form the basis of this disclosure will be explained. Patent Document 1 discloses outputting a shooting command to an imaging device to capture an image of a moving vehicle, but does not disclose a method for determining the timing to capture an image of a moving vehicle, i.e., the timing to operate the camera shutter. Patent Document 2 discloses capturing an image of a vehicle when it reaches a predetermined shooting area, but does not disclose a method for determining the timing to reach the predetermined shooting area.

[0017] When a camera's angle of view is set to wide and a vehicle is photographed from a sufficiently far distance, it is possible to capture the moving vehicle without it being out of the frame, even if the timing of the photograph is not precisely determined. However, in this case, the license plate and / or the driver's face appear small in the image, resulting in reduced resolution. As a result, there is a problem in that it is difficult to accurately obtain information about the vehicle model and / or driver. In particular, when photographing at night, the amount of ambient light decreases, lowering the image's S / N ratio, making it even more difficult to obtain information about the subject. Furthermore, when photographing a vehicle with a short distance to the preceding vehicle, there is also the problem that the license plate of the vehicle being photographed is hidden by the body of the preceding vehicle.

[0018] The present inventor has discovered an imaging system that solves the above-mentioned problems. In an imaging system according to an embodiment of the present disclosure, a sensor measures the speed of a moving object, and a camera captures the object at an appropriate timing based on the measurement result. As a result, it is possible to capture an image of the moving object without it going out of frame. The imaging system according to an embodiment of the present disclosure will be described below.

[0019] The imaging system according to the first aspect includes a sensor that measures the speed of a moving object, a camera that captures an image of the object and is different from the sensor, and a processing circuit that controls the operation of the sensor and the camera. The processing circuit causes the sensor to measure the speed of the object and generate speed information of the object and timing information for measuring the speed, generates control data including timing information for capturing an image of the camera based on (a) position information of the object or distance information to the object at the time of speed measurement, (b) the speed information, and (c) the measurement timing information, and causes the camera to output image data including image information of the object based on the control data.

[0020] This imaging system can capture not only slow-moving objects but also fast-moving objects without causing them to go out of frame.

[0021] An imaging system according to a second aspect is the imaging system according to the first aspect, wherein the processing circuit causes the camera to capture an image of the object based on the image capture timing information of the camera.

[0022] This imaging system can photograph an object and output image data of the object.

[0023] The imaging system according to the third item is the imaging system according to the first item, wherein the processing circuit causes the camera to capture a video of the object and outputs the image data from the video captured by the camera.

[0024] This imaging system can output image data of an object from a moving image.

[0025] The imaging system according to the fourth item is the imaging system according to any one of the first to third items, wherein the processing circuit causes the sensor to measure the position or distance of the object at the time of speed measurement and the speed at the same time.

[0026] This imaging system makes it easy to calculate the timing of imaging.

[0027] The imaging system according to the fifth item is an imaging system according to any one of the first to fourth items, wherein the control data further includes at least one of information for determining a ROI (Region of Interest) to be included in an image captured by the camera, information for determining the focus position of the camera, and information for determining the position of the camera.

[0028] In this imaging system, the operation of the camera can be controlled based on the above control data.

[0029] The imaging system according to the sixth item is an imaging system according to any one of the first to fifth items, wherein the control data includes at least one of information determining the rotation angle of the pan rotation and / or tilt rotation of the camera and information determining the zoom magnification of the camera.

[0030] In this imaging system, the operation of the camera can be controlled based on the above control data.

[0031] An imaging system according to a seventh item is the imaging system according to any one of the first to sixth items, wherein the sensor is an FMCW-LiDAR device.

[0032] This imaging system can simultaneously and accurately obtain information on the position of an object or information on the distance to the object, and information on the velocity of the object.

[0033] In the imaging system according to the eighth aspect, in the imaging system according to the fifth aspect, the control data further includes information for determining an ROI to be included in an image captured by the camera, and the processing circuit causes the camera to extract the ROI from the captured image.

[0034] This imaging system can output ROI image data.

[0035] An imaging system according to a ninth item is the imaging system according to any one of the first to eighth items, wherein the processing circuit generates classification data including classification information of the object based on the image data.

[0036] This imaging system can generate classification information for the object.

[0037] In a tenth aspect of the present invention, the imaging system is the imaging system according to the fifth aspect, wherein the camera includes an actuator for translating the position of the camera. The control data further includes information for determining the position of the camera. The processing circuit controls the camera to translate the position of the camera using the actuator.

[0038] In this imaging system, the position of the camera can be adjusted.

[0039] An imaging system according to an eleventh aspect is the imaging system according to the sixth aspect, wherein the camera includes an actuator that changes the orientation of the camera. The control data further includes information that determines the orientation of the camera. The processing circuit controls the camera to change the orientation of the camera using the actuator.

[0040] In this imaging system, the direction of the camera can be adjusted.

[0041] In a twelfth aspect of the present invention, the imaging system is the imaging system according to the sixth aspect, wherein the camera includes an actuator for changing a zoom magnification. The control data includes information for determining the zoom magnification of the camera. The processing circuit controls the camera to change the zoom magnification using the actuator.

[0042] In this imaging system, the zoom magnification of the camera can be adjusted.

[0043] The imaging system according to the thirteenth item is an imaging system according to any one of the first to twelfth items, wherein the control data further includes at least one of information determining the exposure time of the camera and information determining the opening and closing degree of the aperture of the camera.

[0044] In this imaging system, the operation of the camera can be controlled based on the above control data.

[0045] An imaging system according to a fourteenth aspect is the imaging system according to any one of the first to thirteenth aspects, wherein the object is a vehicle, and the image represented by the image data includes an image of a license plate of the vehicle.

[0046] This imaging system can capture images of license plates of moving vehicles.

[0047] An imaging system according to a fifteenth aspect is the imaging system according to any one of the first to thirteenth aspects, wherein the object is a vehicle, and the image represented by the image data includes an image of a driver or a passenger of the vehicle.

[0048] This imaging system can capture an image of the driver or passengers of a moving vehicle.

[0049] An imaging system according to a sixteenth aspect is the imaging system according to any one of the first to fifteenth aspects, wherein the object is a vehicle. The vehicle includes wheels. The sensor measures the traveling speed of the vehicle and the rotational speed of the wheels. The speed information is information relating to the traveling speed of the vehicle and the rotational speed of the wheels.

[0050] This imaging system can obtain information about the vehicle's traveling speed and the wheel rotation speed.

[0051] An imaging system according to a seventeenth item is the imaging system according to the sixteenth item, wherein the processing circuit generates axle number data including information about the number of axles of the vehicle based on the speed information.

[0052] This imaging system can generate information about the number of axles on a vehicle.

[0053] An imaging system according to an eighteenth item is the imaging system according to the seventeenth item, wherein the processing circuit generates vehicle type data including vehicle type information of the vehicle based on the image information and the axle number information.

[0054] This imaging system can generate vehicle model information.

[0055] An imaging system according to a 19th item is the imaging system according to any one of the first to eighteenth items, wherein the processing circuit causes the sensor to measure the velocity multiple times at different times. The velocity information is information about the velocity measured multiple times. The measurement timing information is information about the timing at which the velocity was measured multiple times.

[0056] In this imaging system, the imaging timing can be determined more accurately based on the acceleration or deceleration tendency of the object.

[0057] An imaging system according to a twentieth item is the imaging system according to any one of the first to nineteenth items, further comprising another sensor that measures the velocity of the object from a direction different from that of the sensor. The processing circuit causes the other sensor to measure the velocity of the object and generate other velocity information of the object, and determines the velocity of the object based on the velocity information and the other velocity information.

[0058] With this imaging system, the direction of movement of the object can be determined accurately, and the timing of imaging can be determined more accurately.

[0059] A processing device according to a twenty-first item includes a processor and a memory storing a computer program executed by the processor. The computer program causes the processor to: cause a sensor to measure the speed of an object and generate speed information of the object and timing information for measuring the speed; generate control data including timing information for capturing an image of a camera different from the sensor based on (a) position information of the object or distance information to the object at the time of speed measurement, (b) the speed information, and (c) the timing information for measuring the speed; and cause the camera to output image data including image information of the object based on the control data.

[0060] This processing device makes it possible to photograph not only slow-moving objects but also fast-moving objects without causing them to go out of frame.

[0061] A method according to a twenty-second item is a computer-implemented method in an imaging system, which includes: causing a sensor to measure the speed of an object and generating speed information of the object and information on timing of measuring the speed; generating control data including image capture timing information of a camera different from the sensor based on (a) position information of the object or information on the distance to the object at the time of speed measurement, (b) the speed information, and (c) the measurement timing information; and causing the camera to output image data including image information of the object based on the control data.

[0062] This method allows not only slow-moving objects but also fast-moving objects to be photographed without dropping out of the frame.

[0063] (Embodiment 1) First, an example configuration of an imaging system according to a first embodiment of the present disclosure will be described with reference to FIG. 1. FIG. 1 is a diagram schematically illustrating the configuration of an imaging system according to the first exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. For ease of explanation, FIG. 1 illustrates mutually orthogonal X-, Y-, and Z-axes. However, these axes do not limit the orientation of the vehicle and imaging system, and the orientation of the vehicle and imaging system is arbitrary. The +X direction is the direction of the arrow on the X-axis, and the -X direction is the opposite direction. The same applies to the ±Y and ±Z directions.

[0064] A vehicle 10 shown in Fig. 1 travels in the +X direction on a road surface parallel to the XY plane. The traveling speed of the vehicle 10 may be, for example, the legal speed limit on a highway. The imaging system 100A shown in Fig. 1 includes a sensor 20, a camera 30, and a processing circuit 40. The subject to be imaged in the first embodiment is the license plate of the vehicle 10.

[0065] In the imaging system 100A, the processing circuit 40 causes the sensor 20 to measure the speed v, which is the traveling speed of the vehicle 10. Based on the measurement result, the processing circuit 40 causes the camera 30 to capture an image of the license plate of the vehicle 10 at an appropriate timing, and to generate and output captured image data of the license plate. As a result, it becomes possible to capture the license plate of the vehicle 10 traveling at high speed without it being framed out of the captured image 32 shown in the enlarged view of FIG. 1.

[0066] The components included in the vehicle 10 and the imaging system 100A will be described below.

[0067] The vehicle 10 is a standard automobile including four wheels, but may also be, for example, a large truck including four or more wheels, or a motorcycle including two wheels.

[0068] The sensor 20 detects the speed v and the measured position P of the vehicle 10. v The speed measurement position P of the vehicle 10 is measured. vis the position of the vehicle 10 at the time of speed measurement, i.e., at the time of speed measurement, and more specifically, is the three-dimensional position of the point on the vehicle 10 where the sensor 20 measures the speed v. In the example shown in FIG. 1, this point is the front part of the vehicle 10. The speed v of the vehicle 10 and the speed measurement position P v The method for measuring this will be described in detail later.

[0069] It is desirable that the sensor 20 measures the speed from a location remote from the vehicle 10. In this case, the sensor 20, the camera 30, and the processing circuit 40 can be integrated, and the imaging system 100A can be made smaller overall. The sensor 20 measures the speed v and the speed measurement position P of the vehicle 10. v It is desirable to measure the speed v and the speed measurement position P of the vehicle 10 at the same time. In this case, it becomes easy to calculate the timing of taking pictures, and it is possible to simplify the imaging system. It is desirable that the sensor 20 includes an oscillation source that periodically modulates the frequency, and measures the speed and speed measurement position of the vehicle by causing interference between the reflected wave from the vehicle and the reference wave (i.e., measurement by the FMCW method). In this case, the speed v and speed measurement position P of the vehicle 10 are v It is possible to measure the vehicle 10 more precisely at the same time. The sensor 20 is preferably an FMCW (Frequency Modulated Contuous Wave)-LiDAR (Light Detecting And Ranging) device. In this case, by irradiating the vehicle 10 with a laser beam focused in a point shape, the measurement direction can be determined accurately with higher resolution, and the speed v and relative distance d of the vehicle 10 can be measured at the same time from a location away from the vehicle 10. In addition, since the light receiving area of ​​the reflected light can be limited by a lens, it is possible to reduce the possibility of errors in distance and speed due to multipath. The configuration of the FMCW-LiDAR device will be described in detail later.

[0070] The sensor 20 may be a Doppler radar device that uses a Doppler radar to measure the speed v of the vehicle 10. Alternatively, the sensor 20 may be a device that estimates the speed v of the vehicle 10 from multiple frames of images captured by a separate camera.

[0071] The camera 30 captures an image of at least a portion of the vehicle 10 within a range of an angle of view ψ. The camera 30 may be, for example, an RGB camera or a monochrome camera. The camera 30 is a device separate from the sensor 20.

[0072] The processing circuit 40 controls the operations of the sensor 20 and the camera 30, and processes the signals output from the sensor 20 and the camera 30. The operation of the processing circuit 40 will be described in detail later.

[0073] The computer program executed by the processing circuit 40 is stored in a memory 42 such as a ROM or RAM (Random Access Memory). Thus, the imaging system 100A includes a processing device including the processing circuit 40 and the memory 42. The processing circuit 40 and the memory 42 may be integrated into a single IC or LSI, may be integrated on a single circuit board, or may be provided on separate circuit boards. The functions of the processing circuit 40 may be distributed across multiple circuits. The processing device may be installed in a remote location away from the other components and control the operations of the sensor 20 and the camera 30 via a wired or wireless communication network.

[0074] It should be noted that if the processing circuit 40 executes the processing by a combination of electronic circuits (or logic circuits), the memory 42 for storing the computer program is not necessary.

[0075] 1, processing circuitry 40 may be mounted on sensor 20 or camera 30. Alternatively, part of processing circuitry 40 may be mounted on sensor 20 and the remaining part may be mounted on camera 30.

[0076] Next, an example configuration and principle of an FMCW-LiDAR device will be described with reference to Figs. 2A to 3. Fig. 2A is a block diagram that schematically shows an example configuration of a sensor 20, which is an FMCW-LiDAR device. The sensor 20 shown in Fig. 2A includes a light source 22, an interference optical system 24, a photodetector 26, and a processing circuit 28. The interference optical system 24 includes a splitter 24a and a mirror 24b. The thick arrows in Fig. 2A indicate the flow of light.

[0077] The light source 22 emits a laser beam 20L0 to illuminate the vehicle 10. The light source 22 may emit the laser beam 20L0 continuously or may emit the laser beam 20L0 intermittently at a repetition frequency of several tens of Hz to several hundreds of Hz. Alternatively, the light source 22 may emit the laser beam 20L0 after another sensor detects that the vehicle 10 is approaching.

[0078] The frequency of the laser light 20L0 can be modulated like a triangular wave. The frequency modulation period can be, for example, 10 nanoseconds or more and 10 milliseconds or less. The frequency modulation amplitude can be, for example, 100 MHz or more and 1 THz or less. The wavelength of the laser light 20L0 can be, for example, within the near-infrared wavelength range of 700 nm or more and 2000 nm or less. Since the amount of near-infrared light in sunlight is less than the amount of visible light, using near-infrared light as the laser light 20L0 can reduce the influence of sunlight as noise. Alternatively, the wavelength of the laser light 20L0 can be within the visible wavelength range of 400 nm or more and 700 nm or less, or within the ultraviolet wavelength range.

[0079] The interference optical system 24 splits the laser light 20L0 emitted from the light source 22 into reference light 20L1 and irradiation light 20L2 using a splitter 24a, emits the reference light 20L1 toward a mirror 24b, and emits the irradiation light 20L2 toward the vehicle 10. The interference optical system 24 causes the reference light 20L1, which is reflected back by the mirror 24b, and the reflected light 20L3, which is the irradiation light 20L2 reflected back by the vehicle 10, to interfere with each other using the splitter 24a, to generate interference light 20L4, and emits the interference light 20L4 toward the photodetector 26. The photodetector 26 detects the interference light 20L4 and generates and outputs a detection signal.

[0080] 2B is a flowchart that schematically illustrates an example of the operations performed by the processing circuit 28 included in the sensor 20. The processing circuit 28 performs the operations of steps S01 to S04 shown in FIG. 2B.

[0081] <Step S01> The processing circuit 28 causes the light source 22 to emit a laser beam.

[0082] <Step S02> The processing circuit 28 causes the photodetector 26 to detect the interference light and generate and output a detection signal.

[0083] <Step S03> The processing circuit 28 acquires the detection signal.

[0084] <Step S04> The processing circuit 28 generates and outputs measurement data based on the detection signal. The measurement data is output at measurement time t s , speed information regarding the speed v of the vehicle 10, and the speed measurement position P v Includes speed measurement position information for measurement time t s is the time at which the processing circuit 28 acquires the detection signal.

[0085] It should be noted that the sensor 20 does not need to include the processing circuit 28, and the processing circuit 40 included in the imaging system 100A may execute the operations of steps S01 to S04.

[0086] FIG. 3 is a diagram showing the time variations in the frequencies of the reference light and the reflected light when the vehicle 10 is moving. The solid line represents the reference light, and the dashed line represents the reflected light beam. The frequency of the reference light shown in FIG. 3 repeats a time variation of a triangular wave. That is, the frequency of the reference light increases linearly over one period and then decreases linearly by the same amount. The frequency of the reflected light beam is shifted in the positive direction along the time axis compared to the frequency of the reference light by the amount of time it takes for the output light to be emitted from the sensor 20, reflected by the vehicle 10, and returned as reflected light. Furthermore, when the relative distance d between the moving vehicle 10 and the sensor 20 becomes shorter, the frequency of the reflected light shifts in the positive direction along the frequency axis due to Doppler shift compared to when the vehicle 10 is stationary.

[0087] The interference light resulting from the superposition of the reference light and the reflected light has a beat frequency corresponding to the difference between the frequency of the reflected light and the frequency of the reference light. The beat frequency differs depending on whether the frequencies of the reference light and the reflected light beam increase linearly or decrease linearly. In the example shown in Figure 3, the beat frequency f1 when both frequencies decrease linearly is higher than the beat frequency f2 when both frequencies increase linearly.

[0088] The modulation frequency, which is the reciprocal of the time period of the reference light frequency, is f FMCW , where Δf is the frequency width, which is the difference between the maximum and minimum values ​​of the frequency of the reference light, c is the speed of light in a vacuum, and λ is the wavelength of the reference light, the relative distance d and relative speed v between the sensor 20 and the vehicle 10 are s are expressed by the following equations (1) and (2), respectively.

number

number

[0089] Here, the beat frequencies f1 and f2 are measured in a time domain where the frequencies of the reference light and the reflected light are close to a constant increase or decrease. FIG. 3 is a diagram showing a schematic diagram of the time variation of the frequencies of the reference light and the reflected light and the measurement time domain of the beat frequency when the vehicle 10 is traveling. The beat frequencies f1 and f2 are measured within the range of the time domain t1 and t2, respectively. Measuring the beat frequencies in such time domains t1 and t2 reduces the fluctuation of the values ​​of the frequencies f1 and f2 within the measurement time domain, and the relative distance d and the relative velocity v s The measurement accuracy is improved. Furthermore, from equations (1), (2) and Figure 3, the relative distance d and relative velocity v s are calculated from beat signals acquired in the same measurement time domain. This means that, in principle, when the sensor 20 is an FMCW type, it is possible to measure the speed v and the measured position Pv of the vehicle 10 more precisely at the same time. When sensor 20 is a Doppler radar, the measurement time domain for relative distance is the time from transmitting a pulse to receiving it. On the other hand, the measurement time domain for relative velocity is the time the pulse is irradiated onto the target (i.e., the pulse width), which is strictly different from the measurement time domain for relative distance. Therefore, it is more preferable to use an FMCW sensor for sensor 20, as it can measure relative distance and relative velocity more precisely at the same time. As shown in FIG. 1, the angle between the traveling direction of the vehicle 10 and the direction opposite to the measurement direction is θ, and the sensor position, which is the three-dimensional position of the sensor 20, is P s (x s , y s , z s ), then the speed measurement position P of the vehicle 10 v (x v , y v , z v ) and velocity v are expressed by the following equations (3) and (4), respectively. The measurement direction is the traveling direction of the irradiated light 20L2 emitted from the sensor 20. The sensor position P s is located at the center of the light-detecting surface of the sensor 20.

number

number

[0090] Here, the measurement direction is assumed to be parallel to the road surface. Even if the measurement direction intersects with the road surface, if the angle between the measurement direction and the road surface is known, the speed measurement position P v As described above, it is possible to calculate the velocity measurement position P v and velocity v are the relative distance d and relative velocity v, respectively. s The determination can be made based on the following:

[0091] The speed measurement position information included in the measurement data output from the sensor 20 is the speed measurement position P v Even if the relative distance is d, the velocity measurement position P v Similarly, the velocity information included in the measurement data output from the sensor 20 may be information on the velocity v or information on the relative velocity v s The relative velocity v s can be converted to velocity v using equation (4).

[0092] Since the vehicle 10 has a certain size, when the sensor 20 continuously measures the speed, multiple pieces of speed information may be output while the laser light continues to irradiate the vehicle 10. In order to select one valid piece of speed information from the multiple pieces of speed information, the speed information may be compared with the speed measurement position information measured at the same time. For example, when the front part of the vehicle 10 is irradiated with the laser light, the speed measurement position P v changes in the X and Y directions, and when the side of the vehicle 10 is illuminated, the speed measurement position P v By examining the change in the laser beam intensity, it is possible to determine which part of the vehicle is being irradiated with the laser beam.

[0093] In addition, when the measurement direction and the running line of the vehicle 10 are always constant, the speed measurement position P vis the sensor position P s The position where the measurement direction starting from the point P intersects with the running line of the vehicle 10 can be uniquely determined. Therefore, the speed measurement position P v In this case, the measurement data does not need to include the speed measurement position information. However, the speed measurement position P v is calculated to obtain a more accurate velocity measurement position P v is obtained.

[0094] Next, an example of the operation performed by the processing circuit 40 in the first embodiment for the camera 30 to capture an image of the license plate of the vehicle 10 at an appropriate timing will be described in detail with reference to Fig. 4. Fig. 4 is a flowchart that schematically shows an example of the operation performed by the processing circuit 40 in the first embodiment. The processing circuit 40 performs the operations of steps S101 to S104 shown in Fig. 4.

[0095] <Step S101> The processing circuit 40 transmits the speed v and the distance d or the measured speed position P of the vehicle 10 to the sensor 20. v The measurement data includes the speed information of the vehicle 10, the speed measurement position information, and the image capture timing information. The method for generating the measurement data is as described with reference to FIG. 2B.

[0096] <Step S102> The processing circuit 40 acquires the measurement data.

[0097] <Step S103> The processing circuit 40 determines the photographing time t based on the following information: p The control data is generated based on (a) the position information of the vehicle 10 or the distance information to the vehicle 10 at the time of speed measurement, (b) the speed information of the vehicle 10, and (c) the measurement timing information.

[0098] The processing circuit 40 detects the speed measurement position P v and vehicle 10's shooting position Pp The distance between the vehicle 10 and the speed measurement position P is determined as the travel distance. v may be determined based on the speed measurement position information included in the measurement data, or may be determined as the position where the measurement direction intersects with the driving line of the vehicle 10, as described above. p is the three-dimensional position at which the driver or passenger can be photographed at the center of the angle of view ψ of the camera 30. When the photographing direction and the driving line of the vehicle 10 are always constant, the photographing position P p is the camera position P c The camera position P is the position where the shooting direction starting from the point P intersects with the driving line of the vehicle 10. c is located at the center of the imaging plane of the camera 30. The imaging direction is the normal direction to the imaging plane of the camera 30.

[0099] Shooting time t p is calculated as follows: L is the distance traveled by the vehicle 10 from the time of speed measurement to the time of photographing, and P is the photographing position of the vehicle 10. p (x p , y p , z p )=P p (x v +L, y v , z v ) The shooting time t p is expressed by the following equation (5).

number

[0100] <Step S104> The processing circuit 40 causes the camera 30 to perform an image capturing operation based on the control data. Specifically, the processing circuit 40 transmits the control data to the camera 30 to cause the camera 30 to capture the license plate of the vehicle 10 at the image capturing time t p The photographed image data is generated and output from the photographed image data at the photographed time t pThe captured image data includes captured image information about a captured image 32 of the license plate of the vehicle 10. The image represented by the captured image data is the captured image 32 captured by the camera 30. The captured image 32 is an image whose capture range is determined by the angle of view ψ of the camera 30. The image includes an image of the license plate of the vehicle 10.

[0101] The captured image 32 is captured at time t p Alternatively, the captured image 32 may be obtained by capturing a moving image and selecting a frame at a capture time t from a plurality of frames included in the moving image. p That is, the processing circuit 40 causes the camera 30 to take a video of the vehicle 10 and output captured image data from the video taken by the camera 30. Specifically, the processing circuit 40 transmits control data including image capture timing information to the camera 30, and causes the camera 30 to select an image at the image capture time t p The frame is selected and acquired from the video. The frame is a still image. p If the video does not contain a still image with an exact time that matches the shooting time t p Such an operation may be performed by the camera 30 by selecting the still image closest to the capture time t p This is essentially the same as the operation of having the camera take a still image, generate captured image data, and output the data.

[0102] According to the imaging system 100A of the first embodiment, the above-described operation of the processing circuit 40 makes it possible to capture an image of the license plate of the vehicle 10 traveling at high speed without it going out of frame.

[0103] (Embodiment 2) Next, an example configuration of an imaging system according to a second embodiment of the present disclosure will be described with reference to Fig. 5. In the following embodiments, differences from the first embodiment will be mainly described. Fig. 5 is a diagram schematically illustrating the configuration of an imaging system according to an exemplary second embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. The components of imaging system 100B shown in Fig. 5 are the same as the components of imaging system 100A shown in Fig. 1. The subject to be photographed in the second embodiment is the driver or passenger of vehicle 10.

[0104] In the imaging system 100B, the processing circuit 40 causes the sensor 20 to measure the speed v of the vehicle 10. Based on the measurement result, the processing circuit 40 causes the camera 30 to capture an image of the vehicle 10 at an appropriate timing and generate captured image data representing a captured image 32. The processing circuit 40 causes the camera 30 to extract an ROI (Region of Interest), which is a portion including an image of the driver or passenger, from the captured image, and generate and output ROI image data. As a result, it becomes possible to capture an image of the driver or passenger of the vehicle 10 traveling at high speed without the driver or passenger being framed out of the ROI image shown in the enlarged view of FIG. 5. The enlarged view of FIG. 5 shows the captured image 32. The area surrounded by a dashed line in the captured image 32 represents the ROI.

[0105] Next, an example of the operation performed by the processing circuitry 40 in the second embodiment for the camera 30 to capture an image of the driver or passenger of the vehicle 10 at an appropriate timing will be described in detail with reference to Fig. 6. Fig. 6 is a flowchart that schematically shows an example of the operation performed by the processing circuitry 40 in the second embodiment. The processing circuitry 40 performs the operations of steps S201 to S204 shown in Fig. 6.

[0106] <Steps S201 and S202> The operations in steps S201 and S202 are the same as the operations in steps S101 and S102 shown in Fig. 4. However, the speed measurement position P v is the side of the vehicle 10, not the front.

[0107] <Step S203> The processing circuit 40 determines the photographing time t based on the following information: p In addition to the photographing timing information regarding the vehicle 10, control data is generated that includes information for determining the ROI included in the photographed image 32. The information on which the control data is based is (a) position information of the vehicle 10 or distance information to the vehicle 10 at the time of speed measurement, (b) speed information of the vehicle 10, (c) measurement timing information, and (d) information regarding the length of the ROI in each of the X direction and Z direction, which will be described later.

[0108] Shooting position P p is the position at the center of the field of view of the camera 30 where the driver or passenger can be photographed. When the vehicle 10 is traveling in the +X direction, the measurement time t s From a given time t c The distance traveled by vehicle 10 after the time has elapsed is L=v·t c The shooting time is expressed as t p =t s +t c is.

[0109] Here, the camera 30 captures the image at time t p When capturing an image 32, the speed of the vehicle 10 at which the image of the driver or passenger is positioned in the center of the captured image 32 is defined as the reference speed v0. Furthermore, the speed v of the vehicle 10 differs from the reference speed v0 by Δv. That is, v = v0 + Δv. In this case, the travel distance L of the vehicle 10 is L = (v0 + Δv) t c The deviation ΔL of the travel distance compared to when the speed of the vehicle 10 is v0 is ΔL=Δv t c is expressed by

[0110] Therefore, compared to when the speed of the vehicle 10 is v0, the amount of deviation ΔR of the image of the driver or passenger in the captured image 32 is ΔR=α·ΔL. Here, α is a coefficient corresponding to the angle of view of the camera 30. The smaller the angle of view of the camera 30, the larger the value of the coefficient α. If the number of pixels in the X direction of the captured image 32 is I x In the example shown in the enlarged view of FIG. 5A, the central coordinate R in the X direction of the ROI is x is expressed by the following equation (6).

number

[0111] The length of the ROI in the X direction may be, for example, half the length of the captured image 32 in the X direction. x The coordinate range of the ROI in the X direction can be determined from the distance and length. The coordinate range of the ROI in the Z direction can be, for example, the range of the upper half of the captured image 32 in the Z direction.

[0112] <Step S204> The processing circuit 40 transmits control data to the camera 30 to cause the camera 30 to capture the driver or passenger of the vehicle 10 at the capture time t p The processing circuit 40 causes the camera 30 to take an image, generate ROI image data, and output the ROI image data. The ROI image data includes ROI image information related to the image of the ROI. Specifically, the processing circuit 40 causes the camera 30 to generate captured image data about the driver or passenger, extract the ROI from the captured image 32 indicated by the captured image data, and generate and output ROI image data. The image indicated by the ROI image data is an image of the extracted ROI. The image includes an image of the driver or passenger of the vehicle 10.

[0113] Next, with reference to Figures 7A and 7B, it will be described how the positional relationship between the captured image 32 and the ROI depends on the speed v of the vehicle 10. Figures 7A and 7B are diagrams showing the positional relationship between the captured image 32 and the ROI when the speed v of the vehicle 10 is v1 and v2, respectively. The speed v1 is lower than the reference speed v0, and the speed v2 is higher than the reference speed v0.

[0114] As shown in FIG. 7A, when the speed v1 is lower than the reference speed v0, the image of the driver is positioned to the left in the photographed image 32. On the other hand, as shown in FIG. 7B, when the speed v2 is higher than the reference speed v0, the image of the driver is positioned to the right in the photographed image 32. The center coordinate in the X direction of the ROI for the speed v1 is R x1The center coordinate of the ROI in the X direction for velocity v2 is R x2 Then, v1 <v2であればR x1 <R x2 That is, the higher the speed v of the vehicle 10, the more the ROI is positioned to the right of the captured image 32.

[0115] According to the imaging system 100B of the second embodiment, the above-described operation of the processing circuit 40 makes it possible to capture an image of the driver or passenger of the vehicle 10 traveling at high speed without causing them to be framed out of the image of the ROI.

[0116] (Embodiment 3) Next, a configuration example of an imaging system according to a third embodiment of the present disclosure will be described with reference to Fig. 8. Fig. 8 is a diagram schematically illustrating the configuration of an imaging system according to the third exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. The components of imaging system 100C shown in Fig. 8 are the same as the components of imaging system 100A shown in Fig. 1. However, camera 30 includes an optical system (not shown) that is capable of adjusting the focus position. The subject to be photographed in the third embodiment is the license plate of vehicle 10.

[0117] In the imaging system 100C, the processing circuit 40 causes the sensor 20 to measure the speed v of the vehicle 10, and determines the focus position f of the camera 30 based on the measurement result. p The processing circuit 40 transmits the determined focus position f p The focus position is adjusted based on the above, and then the license plate of the vehicle 10 is photographed at an appropriate timing, and photographed image data is generated and output. As a result, it becomes possible to photograph the license plate of the vehicle 10 traveling at high speed more clearly without it going out of frame.

[0118] Next, an example of the operation performed by the processing circuitry 40 in the third embodiment for the camera 30 to capture an image of the license plate of the vehicle 10 at an appropriate timing will be described in detail with reference to Fig. 9. Fig. 9 is a flowchart that schematically shows an example of the operation performed by the processing circuitry 40 in the third embodiment. The processing circuitry 40 performs the operations of steps S301 to S304 shown in Fig. 9.

[0119] <Steps S301 and S302> The operations in steps S301 and S302 are the same as the operations in steps S101 and S102 shown in FIG.

[0120] <Step S303> The processing circuit 40 determines the photographing time t based on the following information: p In addition to the photographing timing information regarding the focus position f of the camera 30, p The control data is generated including information for determining the following: (a) position information of the vehicle 10 or distance information to the vehicle 10 at the time of speed measurement, (b) speed information of the vehicle 10, and (c) measurement timing information.

[0121] Shooting position P p is the position at which the license plate can be photographed at the center of the angle of view of the camera 30. When the vehicle 10 is traveling in the +X direction, at the measurement time t s From a given time t c The distance traveled by vehicle 10 after the time has elapsed is L=v·t c The shooting time is expressed as t p =t s +t c is.

[0122] Here, the camera 30 captures the image at time t p When capturing an image 32, the speed of the vehicle 10 at which the image of the license plate is located in the center of the captured image 32 is defined as the reference speed v0. Furthermore, the speed v of the vehicle 10 differs from the reference speed v0 by Δv. That is, v = v0 + Δv. In this case, the travel distance L of the vehicle 10 is L = (v0 + Δv) t cThe deviation ΔL of the travel distance compared to when the speed of the vehicle 10 is v0 is ΔL=Δv t c is expressed by

[0123] Therefore, compared to when the speed of the vehicle 10 is v0, the shift amount of the focus position Δf p is Δf p =ΔL·cosφ, where φ is the angle between the traveling direction of the vehicle 10 and the direction opposite to the shooting direction of the camera 30. When the speed of the vehicle 10 is v0, the focus distance is f p0 Then, in the example shown in FIG. 8, the focus position f p is expressed by the following equation (7): p0 is the camera position P c and shooting position P p It is expressed by the distance between

number

[0124] <Step S304> The processing circuit 40 transmits control data to the camera 30 to instruct the camera 30 that the focus position is f p Then, the license plate of the vehicle 10 is photographed at the photographing time t p The license plate of the vehicle 10 is located at the focus position of the camera 30.

[0125] According to the imaging system 100C of the third embodiment, the above-described operation of the processing circuit 40 makes it possible to capture a clearer image of the license plate of a vehicle 10 traveling at high speed without the license plate going out of frame.

[0126] The camera 30 may be equipped with an optical system capable of adjusting the aperture. The control data in step S303 may further include information for determining the aperture of the camera 30. The processing circuit 40 transmits the control data to the camera 30, causing the camera 30 to change the aperture. The imaging system 100C according to the third embodiment determines the focus position more accurately, so that the image of the license plate does not become unclear even if the aperture of the camera 30 is opened and the focal depth of the optical system becomes shallow. As a result, an image with a high S / N ratio can be acquired even in low ambient light conditions, such as at night.

[0127] The camera 30 may adjust the exposure time according to the opening / closing degree of the aperture. The control data in step S303 may further include information for determining the exposure time of the camera 30. The processing circuit 40 transmits the control data to the camera 30 to cause the camera 30 to adjust the exposure time. The exposure time is adjusted so as to obtain a desired amount of light according to the opening / closing degree of the aperture. Alternatively, the control data may not include information for determining the exposure time, and the camera 30 itself may adjust the exposure time according to the opening / closing degree of the aperture.

[0128] The control data in step S303 includes the focus position f of the camera 30 in addition to the photographing timing information. p The information further includes at least one of information for determining the aperture of the camera 30, information for determining the opening and closing degree of the aperture of the camera 30, and information for determining the exposure time of the camera 30.

[0129] (Embodiment 4) Next, with reference to FIG. 10, an example configuration of an imaging system according to a fourth embodiment of the present disclosure will be described. FIG. 10 is a diagram schematically illustrating the configuration of an imaging system according to the fourth exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. An imaging system 100D illustrated in FIG. 10 differs from the imaging system 100A illustrated in FIG. 1 in that the camera 30 further includes an actuator 34 that translates its position in the X direction. The actuator 34 may translate its position in a direction other than the X direction. The subject to be imaged in the fourth embodiment is the driver or passenger of the vehicle 10.

[0130] In the imaging system 100D, the processing circuit 40 causes the sensor 20 to measure the speed v of the vehicle 10. Based on the measurement result, the processing circuit 40 causes the camera 30 to translate the position of the camera 30 in the X direction using the actuator 34, and then causes the camera 30 to capture an image of the vehicle 10 at an appropriate timing, and generate and output captured image data. As a result, it becomes possible to more reliably capture an image of the driver or passengers of the vehicle 10 traveling at high speed without them being out of the frame.

[0131] Next, an example of the operation performed by the processing circuitry 40 in the fourth embodiment for the camera 30 to capture an image of the driver or passenger of the vehicle 10 will be described in detail with reference to Fig. 11. Fig. 11 is a flowchart that schematically shows an example of the operation performed by the processing circuitry 40 in the fourth embodiment. The processing circuitry 40 performs the operations of steps S401 to S404 shown in Fig. 11.

[0132] <Steps S401 and S402> The operations in steps S401 and S402 are the same as the operations in steps S101 and S102 shown in Fig. 4. However, the speed measurement position P v is the side of the vehicle 10, not the front.

[0133] <Step S403> The processing circuit 40 determines the photographing time t based on the following information: p In addition to the photographing timing information regarding the camera position P c The control data is based on (a) the position information of the vehicle 10 or the distance information to the vehicle 10 at the time of speed measurement, (b) the speed information of the vehicle 10, (c) the measurement timing information, and (d) information about the movable range of the camera 30, which will be described later.

[0134] Shooting position P pis a position where the camera 30 can capture an image of the driver or passenger at the center of the angle of view when the camera 30 is in the center of its movable range. When the vehicle 10 is traveling in the +X direction, the measurement time t s From a given time t c The distance traveled by vehicle 10 after the time has elapsed is L=v·t c The shooting time is expressed as t p =t s +t c is.

[0135] Here, the camera 30 captures the image at time t p When capturing an image 32, the speed of the vehicle 10 at which the image of the driver or passenger is positioned in the center of the captured image 32 is defined as the reference speed v0. Furthermore, the speed v of the vehicle 10 differs from the reference speed v0 by Δv. That is, v = v0 + Δv. In this case, the travel distance L of the vehicle 10 is L = (v0 + Δv) t c The deviation ΔL of the travel distance compared to when the speed of the vehicle 10 is v0 is ΔL=Δv t c Since the vehicle 10 and the actuator 34 are moving in the same direction, the amount of deviation in the position of the actuator 34 is ΔL compared to when the velocity of the vehicle 10 is v0. The camera position P c is expressed by the following equation (8).

number

[0136] where M x1 and M x2 are the coordinates of the left and right ends of the camera's movable range, respectively.

[0137] <Step S404> The processing circuit 40 transmits control data to the camera 30 to cause the camera 30 to position the camera 30 at the above-mentioned camera position P c Then, the driver or passenger of the vehicle 10 is moved in parallel to the photographing time t p The camera takes a photograph, generates photographed image data, and outputs the data.

[0138] According to the imaging system 100D of the fourth embodiment, the above-described operation of the processing circuit 40 makes it possible to more reliably capture an image of the license plate of a vehicle 10 traveling at high speed without the license plate going out of frame.

[0139] The camera 30 may be equipped with an actuator that changes the orientation of the camera 30 by panning and / or tilting. Panning is rotation about the Z axis, and tilting is rotation about the X axis. The control data in step S403 may further include information that determines the angle of panning and / or tilting of the camera 30. The processing circuit 40 transmits the control data to the camera 30 to cause the camera 30 to change the orientation of the camera 30 by the actuator.

[0140] Furthermore, the camera 30 may be provided with an actuator for changing the zoom magnification of the camera 30. The control data in step S403 may further include information for determining the zoom magnification of the camera 30. The processing circuit 40 transmits the control data to the camera 30 to cause the camera 30 to change the zoom magnification of the camera 30 by the actuator. When the camera 30 is positioned at the shooting position P of the vehicle 10, p If the zoom magnification is decreased as the distance increases, the driver or passengers of the vehicle 10 traveling at high speed can be photographed without being out of frame.

[0141] The control data in step S403 includes the camera position P of the camera 30 in addition to the photographing timing information. c The information further includes at least one of information for determining the rotation angle of the pan and / or tilt rotation of the camera 30, information for determining the zoom magnification of the camera 30.

[0142] (Embodiment 5) Next, with reference to FIG. 12, an exemplary configuration of an imaging system according to a fifth embodiment of the present disclosure will be described. FIG. 12 is a diagram schematically illustrating the configuration of an imaging system according to an exemplary fifth embodiment of the present disclosure. The components of the imaging system 100E illustrated in FIG. 12 are the same as the components of the imaging system 100A illustrated in FIG. 1. The imaging system 100E illustrated in FIG. 12 acquires captured image data from the camera 30, analyzes the captured image data, and generates and outputs classification data including classification information of the vehicle 10. The analysis of the captured image data may involve, for example, reading characters on the license plate. The classification information of the vehicle 10 may be, for example, the vehicle type, vehicle classification, and toll fee. The imaging system 100E according to the fifth embodiment can accurately acquire information about the vehicle 10. Note that objects other than the vehicle 10 may also be captured.

[0143] (Embodiment 6) Next, with reference to FIGS. 13A to 13C , an example of speed measurement in an imaging system according to embodiment 6 of the present disclosure will be described. The components of the imaging system according to embodiment 6 of the present disclosure are the same as the components according to any of embodiments 1 to 5. In embodiments 1 to 5, the irradiation position of the irradiated light 20L2 is designed to be higher than the highest position of the wheels of the vehicle 10 and lower than the highest position of the body of the vehicle 10, relative to the road surface. Therefore, the body of the vehicle 10 is irradiated with the irradiated light 20L2, but the wheels are not irradiated with the irradiated light 20L2. In contrast, in embodiment 6, the irradiation position of the irradiated light 20L2 is designed to be lower than the highest position of the wheels of the vehicle 10 and higher than the lowest position of the body of the vehicle 10, relative to the road surface. Therefore, not only the body but also the wheels of the vehicle 10 can be irradiated with the irradiated light 20L2.

[0144] 13A and 13B are perspective views each showing a schematic view of the vehicle body and wheels of a vehicle 10 illuminated with illumination light 20L2 in the imaging system according to the sixth embodiment. The vehicle 10 shown in FIGS. 13A and 13B includes a vehicle body 10a and four wheels 10b. As shown in FIG. 13A, by illuminating the vehicle body 10a of the vehicle 10 with illumination light 20L2, the velocity v of the vehicle body 10a can be obtained. A The velocity v of the vehicle body 10a can be measured. A is the traveling speed of the vehicle 10. As shown in FIG. 13B, by illuminating the wheel 10b of the vehicle 10 with the illumination light 20L2, the speed v obtained by superimposing the speed of the vehicle body 10a and the rotation speed of the wheel 10b is obtained. B The speed information included in the measurement data output from the sensor 20 is information relating to the speed of the vehicle body 10a and the rotation speed of the wheels 10b.

[0145] FIG. 13C is a graph showing the change in measured speed over time. During time periods when the measured speed is non-zero, the vehicle 10 is illuminated by the illumination light 20L2. The non-zero measured speed is not constant but indicates two different speeds. During time periods when the measured speed indicates a relatively low speed, the body 10a of the vehicle 10 is illuminated by the illumination light 20L2. During time periods when the measured speed indicates a relatively high speed, the wheels 10b of the vehicle 10 are illuminated by the illumination light 20L2. By analyzing the change in measured speed over time, information regarding the length of the vehicle 10 and the number of axles of the vehicle can be obtained, making it possible to accurately determine the vehicle type and / or vehicle classification of the vehicle 10. In the example shown in FIG. 13C, the vehicle length can be calculated by multiplying the speed v of the vehicle 10 by the width of the time periods when the measured speed is non-zero. The number of axles of the vehicle 10 can be determined from the number of time periods when the measured speed indicates a relatively high speed. In the example shown in FIG. 13C, the vehicle 10 has two axles.

[0146] An example of the operation of the processing circuit 40 in the sixth embodiment is as follows. The processing circuit 40 acquires measurement data and generates axle count data including axle count information of the vehicle 10 based on the speed information included in the measurement data. The processing circuit 40 further generates and outputs vehicle model data including vehicle model information of the vehicle 10 based on the axle count information included in the axle count data and the captured image information included in the captured image data. The vehicle model information is information related to the vehicle model of the vehicle 10.

[0147] (ETC combining embodiments 5 and 6) In the future, it is expected that ETC will require the following operations to supplement information that cannot be obtained by vehicle detectors and axle sensors and to prevent fraudulent driving. This operation involves photographing vehicle 10 using camera 30, obtaining information about the vehicle type or driver based on the license plate or the driver's face image contained in the photograph, and linking this information with information obtained by vehicle detectors and axle sensors. By combining the fifth and sixth embodiments, it is possible to realize an ETC that can meet such expectations.

[0148] Next, with reference to FIG. 14, an example configuration of an ETC that combines the fifth and sixth embodiments will be described. FIG. 14 is a diagram schematically illustrating an example configuration of an ETC that combines the fifth and sixth embodiments. The ETC 200 shown in FIG. 14 includes a sensor 20, a first camera 30a, a second camera 30b, a first wireless device 50a, a second wireless device 50b, and a display device 60. The ETC 200 includes a first gate 70a, a second gate 70b, and a third gate 70c, which are positioned in this order along the traveling direction of the vehicle 10. The first gate 70a supports the sensor 20 and the first wireless device 50a. The second gate 70b supports the first camera 30a and the second camera 30b. The third gate 70c supports the second wireless device 50b. The ETC 200 also includes a processing circuit 40, a memory 42, and a storage device 44, which are located away from the above-mentioned components.

[0149] The sensor 20 detects the speed v and the measured position P of the vehicle 10. vThe sensor 20 can acquire information for determining the length and number of axles of the vehicle. The first camera 30a photographs the license plate of the vehicle 10, and the second camera 30b photographs the driver of the vehicle 10. Note that instead of the first and second cameras 30a and 30b, a single camera whose orientation can be changed may photograph the license plate and driver of the vehicle 10 at different times. The first wireless device 50a and the second wireless device 50b communicate with an ETC on-board unit of the vehicle 10. The ETC on-board unit stores data including driving section information and toll classification information. The display device 60 displays toll information such as highway tolls to the driver of the vehicle 10.

[0150] The processing circuit 40 controls the operations of the sensor 20, the first camera 30a, the second camera 30b, the first wireless device 50a, the second wireless device 50b, and the display device 60. The processing circuit 40 processes data output from the sensor 20, the first camera 30a, and the second camera 30b. Based on the processing results, the processing circuit 40 stores information about the vehicle 10 suspected of illegal driving in the storage device 44.

[0151] Next, an example of the operation performed by the processing circuit 40 in the ETC 200 shown in Fig. 14 will be described in detail. Fig. 15 is a flowchart that schematically shows an example of the operation performed by the processing circuit 40 in the ETC 200 shown in Fig. 14. The processing circuit 40 performs the operations of steps S501 to S514 shown in Fig. 15.

[0152] <Step S501> The processing circuit 40 causes the first wireless device 50a to communicate with the ETC on-board unit mounted on the vehicle 10 and acquire data including travel section information and toll category information of the vehicle 10 stored in the ETC on-board unit. For convenience, this toll category information is referred to as toll category information (A).

[0153] <Step S502> The processing circuit 40 transmits to the sensor 20 the speed v and the measured position P of the vehicle 10. v The processing circuit 40 measures the measured value, generates the measured value, and outputs the measured value.

[0154] <Step S503> The processing circuit 40 causes the first camera 30a to capture an image of the license plate of the vehicle 10, and generates and outputs captured image data of the license plate. The processing circuit 40 acquires the captured image data.

[0155] <Step S504> The processing circuit 40 causes the second camera 30b to capture an image of the driver of the vehicle 10, and generates and outputs captured image data about the driver. The processing circuit 40 acquires the captured image data.

[0156] <Step S505> The processing circuit 40 determines the length and number of axles of the vehicle 10 based on the measurement data output from the sensors 20 .

[0157] <Step S506> The processing circuit 40 determines whether the determined length and number of axles of the vehicle 10 are sufficiently reliable. For example, environmental factors such as rain or thick fog may cause the S / N ratio of the light reflected by the vehicle 10 to fall below a predetermined threshold, resulting in at least a portion of the measurement data output from the sensor 20 being missing. In such a case, the processing circuit 40 determines that the reliability is insufficient. As another example, the processing circuit 40 also determines that the reliability is insufficient when the determined length of the vehicle 10 is an unrealistic length, such as 10 m or more, or when the number of axles is an unrealistic number, such as 10 or more. If the determination is Yes, the processing circuit 40 executes the operation of step S507. If the determination is No, the processing circuit 40 executes the operation of step S508.

[0158] <Step S507> The processing circuit 40 determines fare category information based on the vehicle length and axle count information, which for convenience will be referred to as fare category information (B).

[0159] <Step S508> The processing circuit 40 determines the fee category information (B) by using both the information contained in the measurement data output from the sensor 20 and the information contained in the photographed image data of the license plate. The information contained in the photographed image data may be, for example, text information on the license plate.

[0160] <Step S509> The processing circuit 40 compares the fee category information (A) with the fee category information (B) to determine whether they match. If the determination is Yes, the processing circuit 40 performs the operation of step S510. If the determination is No, the processing circuit 40 performs the operation of step S512.

[0161] <Step S510> The processing circuit 40 causes the display device 60 to display fare information based on the fare category information and the travel section information.

[0162] <Step S511> The processing circuit 40 causes the second wireless device 50b to transmit data including the toll information to the ETC vehicle-mounted unit, which then notifies the driver of the toll information.

[0163] <Step S512> If the fee category information (A) and the fee category information (B) do not match each other, the processing circuit 40 causes the display device 60 to display fee information and warning information.

[0164] <Step S513> The processing circuit 40 causes the second wireless device 50b to transmit data including the toll information and the warning information to the ETC vehicle-mounted unit, which then notifies the driver of the toll information and issues a warning to the driver.

[0165] <Step S514> If the fee category information (A) and the fee category information (B) do not match, the vehicle 10 is suspected of illegal driving. The processing circuit 40 stores in the storage device 44 the photographed image data of the license plate and the driver, and the vehicle data including the vehicle information of the vehicle 10, in association with each other. The vehicle information is information about the vehicle 10, such as the length of the vehicle and the number of axles.

[0166] As described above, ETC200, which combines embodiments 5 and 6, can not only accurately determine the toll category for vehicle 10, but also accurately detect whether vehicle 10 is suspected of illegal driving and reliably store data linked to vehicle 10 in memory device 44.

[0167] (Embodiment 7) 16A to 16C, an example of speed measurement in the imaging system according to the seventh embodiment of the present disclosure will be described. In the above-described embodiments, the speed of the vehicle 10 is measured only once. In contrast, in the imaging system according to the seventh embodiment, the speed of the vehicle 10 is measured multiple times.

[0168] 16A and 16B are diagrams each showing a schematic diagram of how the speed of the vehicle 10 is measured at a first measurement time and a second measurement time in the imaging system according to the seventh embodiment. As shown in FIG. 16A, the processing circuit 40 causes the sensor 20 to measure the speed v1 of the vehicle 10 at the first measurement time t s1 16B, the velocity v2 of the vehicle 10 is measured at the second measurement time t s2 The velocity measurement position P shown in FIGS. v The processing circuit 40 causes the sensor 20 to generate and output measurement data. The measurement data includes speed information regarding the speeds v1 and v2, the speed measurement position P v The speed and location information for the measurement time t s1 and t s2 Includes measurement timing information for

[0169] FIG. 16C is a graph showing the change in the speed of the vehicle 10 over time. As shown in FIG. 16C, the processing circuit 40 can acquire information on the change in speed of the vehicle 10. When the vehicle 10 is accelerating or decelerating, the processing circuit 40 can more accurately determine the image capture time of the vehicle 10 based on multiple pieces of speed information at different times. For example, when the vehicle 10 is decelerating as shown in FIG. 16C, the image capture timing can be delayed compared to when the vehicle is traveling at a constant speed, allowing the license plate of the vehicle 10 or the driver or passengers to be captured at the center of the angle of view. When the vehicle 10 is accelerating, the image capture timing is earlier than when the vehicle is traveling at a constant speed.

[0170] An example of the operation of the processing circuit 40 in the seventh embodiment is as follows. The processing circuit 40 causes the sensor 20 to measure the speed of the vehicle 10 multiple times at different times, and generates and outputs measurement data. The speed information included in the measurement data is information related to the speed of the vehicle 10 measured multiple times. The measurement timing information included in the measurement data is information related to the timing at which the speed of the vehicle 10 was measured multiple times.

[0171] (Embodiment 8) Next, an example of speed measurement in an imaging system according to an eighth embodiment of the present disclosure will be described with reference to Fig. 17. In the above-described embodiment, the speed of the vehicle 10 is measured by a single sensor 20. In the imaging system according to the eighth embodiment, the speed of the vehicle 10 is measured by multiple sensors from different measurement directions.

[0172] 17 is a diagram schematically illustrating the configuration of an imaging system according to an eighth exemplary embodiment of the present disclosure, and the positional relationship between the imaging system and a vehicle. The imaging system 100F illustrated in FIG. 17 differs from the imaging system 100A illustrated in FIG. 1 in that the imaging system 100F includes a first sensor 20a and a second sensor 20b instead of a single sensor 20. The first sensor 20a and the second sensor 20b are located at different locations and measure the speed v of the vehicle 10 from different measurement directions. The subject to be photographed in the eighth embodiment is the license plate of the vehicle 10.

[0173] The processing circuit 40 measures the same measurement time t s At the same speed measurement position P v Specifically, the processing circuit 40 causes the first sensor 20a and the second sensor 20b to measure the relative speed of the vehicle 10. sa The first measurement data is generated and output. The first measurement data is the relative velocity v sa 1st speed information regarding speed measurement position P v The first speed measurement position information and measurement time t s Similarly, the processing circuit 40 transmits the second sensor 20b the first image capturing timing information relating to the relative velocity v of the vehicle 10. sb The second measurement data is generated and output. The second measurement data is the relative velocity v sb 2nd speed information regarding speed measurement position P v Second speed measurement position information and measurement time t s The second image capturing timing information includes second image capturing timing information related to the second image capturing timing information.

[0174] The processing circuit 40 acquires the first and second measurement data and calculates the relative velocity v as follows: sa and v sb The speed v of the vehicle 10 is determined from the above equation. The angle between the traveling direction of the vehicle 10 and the direction opposite to the measurement direction of the first sensor 20a is defined as θ a The angle between the traveling direction of the vehicle 10 and the direction opposite to the measurement direction of the second sensor 20b is θ b The processing circuit 40 calculates v sa / cosθ a =v sb / cosθ b Angle θ that satisfies the relationship a and θ b Determine the determined angle θ a and θ b Thus, the traveling direction of the vehicle 10 and the speed v of the vehicle 10 are sa / cosθ a =v sb / cosθ bTherefore, as shown in FIG. 17, even if the traveling direction of the vehicle 10 is not parallel to the X direction, the processing circuit 40 can determine the photographing position P p As a result, it becomes possible to photograph the license plate of the vehicle 10 traveling at high speed in any direction on the road more reliably without the license plate going out of the frame.

[0175] An example of the operation of the processing circuit 40 in the eighth embodiment is as follows. The processing circuit 40 causes the first sensor 20a to measure the speed of the vehicle 10, and generates and outputs first measurement data including first speed information of the vehicle 10. Similarly, the processing circuit 40 causes the second sensor 20b to measure the speed of the vehicle 10, and generates and outputs second measurement data including second speed information of the vehicle 10. The processing circuit 40 further acquires the second measurement data in addition to the first measurement data, and determines the speed v of the vehicle 10 based on the first speed information and the second speed information.

[0176] (Embodiment 9) In the above-described embodiment, the object is a vehicle. The object may be any object moving at high or low speed. Next, with reference to FIG. 18 , an example will be described in which the speed of each of multiple cardboard boxes transported by a belt conveyor in a factory is measured. The transport speed of the cardboard boxes is not as fast as the traveling speed of the vehicles. FIG. 18 is a diagram schematically illustrating the configuration of an imaging system according to an exemplary embodiment 9 of the present disclosure and the positional relationship between the imaging system and the cardboard boxes. The components of the imaging system 100G according to embodiment 9 are the same as the components of the imaging system 100A according to embodiment 1. FIG. 18 illustrates two belt conveyors, a first belt conveyor 90a and a second belt conveyor 90b, located at different heights, and a platform 90c with a slope connecting them. The multiple cardboard boxes 80 shown in FIG. 18 are transported in the order of the first belt conveyor 90a, the platform 90c, and the second belt conveyor 90b. The open arrows in FIG. 18 indicate the direction in which the multiple cardboard boxes 80 are transported. The multiple cardboard boxes 80 may also be transported by other means. Barcodes are attached to the surfaces of the plurality of cardboard boxes 80. The object to be photographed in the ninth embodiment is the barcode on each of the plurality of cardboard boxes 80.

[0177] The processing circuit 40 executes operations similar to those shown in FIG. 4, causing the camera 30 to generate and output captured image data for each of the multiple cardboard boxes 80 sliding down the platform 90c. The captured image data includes captured image information regarding the captured image 32 of each of the multiple cardboard boxes 80. The speeds of the multiple cardboard boxes 80 sliding down the platform 90c may not be the same due to differences in air resistance, for example. Even in such cases, the imaging system 100G according to the ninth embodiment makes it possible to capture the barcodes of the cardboard boxes 80 being conveyed without them being out of the frame. Note that instead of the cardboard boxes 80 sliding down the platform 90c, it is also possible to capture the cardboard boxes 80 being conveyed while they are positioned on a conveyor belt.

[0178] The imaging system 100G according to the ninth embodiment is also effective when the transport speed of the belt conveyor changes over time or when the moving speeds of multiple cardboard boxes 80 differ from one another. Note that the information acquired from the surface of the cardboard box may not be the bar code, but may instead be, for example, the contents of the shipping slip, the contents and number of items in the cardboard box, and the expiration date.

[0179] The components and operations of the first to ninth embodiments described above may be combined in any manner as long as no contradiction occurs. [Industrial Applicability]

[0180] The imaging system according to the present disclosure can be used, for example, as a monitoring device in ETC or an inspection device in a factory. [Explanation of symbols]

[0181] 10 vehicles 10a Body 10b wheels 20 sensors 20L0 laser light 20L1 reference light 20L2 irradiation light 20L3 reflected light 20L4 interference light 20a First sensor 20b Second sensor 22 Light source 24 Interference optical system 24a turnout 24b Mirror 26 Photodetector 28 Processing Circuit 30 Camera 30a 1st camera 30b Second Camera 32 captured images 34 Actuator 40 Processing circuit 42 memory 44 Storage device 50a First radio device 50b 2nd radio device 60 Display device 70a Gate 1 70b 2nd Gate 70c Gate 3 80 cardboard boxes 90a First conveyor belt 90b Second conveyor belt 90c units 100A~100G Imaging System 200 ETC

Claims

1. a sensor for measuring the speed of a moving object; a camera that photographs the object, different from the sensor; processing circuitry for controlling the operation of the sensor and the camera; Equipped with The processing circuitry causing the sensor to measure the velocity of the object and generate velocity information of the object and velocity measurement timing information; generating control data including image capture timing information for the camera based on (a) position information of the object or distance information to the object measured at the same time as the speed measurement, (b) the speed information, and (c) the measurement timing information; causing the camera to output image data including image information of the object based on the control data; Imaging system.

2. the processing circuit causes the camera to capture an image of the object based on the image capture timing information of the camera; The imaging system according to claim 1 .

3. the processing circuit causes the camera to capture a video of the object and to output the image data from the video captured by the camera; The imaging system according to claim 1 .

4. The control data is Information for determining a region of interest (ROI) included in an image captured by the camera; information for determining the focus position of the camera; and and further comprising at least one piece of information determining the position of the camera. The imaging system according to claim 1 .

5. The control data is information for determining the rotation angle of the pan and / or tilt rotation of the camera; and at least one of information determining the zoom factor of the camera; The imaging system according to claim 1 .

6. The sensor includes an oscillation source that periodically modulates frequency, and measures the position of the object or the distance to the object, and the velocity of the object by causing a reflected wave from the object to interfere with a reference wave. The imaging system according to claim 1 .

7. The sensor is an FMCW-LiDAR device. The imaging system according to claim 6 .

8. the control data further includes information for determining an ROI to be included in an image captured by the camera; the processing circuit causes the camera to extract the ROI from the captured image; The imaging system according to claim 5 .

9. The imaging system according to claim 1 , wherein the processing circuit generates classification data including classification information of the object based on the image data.

10. the camera includes an actuator that translates the position of the camera; the control data further includes information for determining the position of the camera; the processing circuit causes the camera to translate the position of the camera using the actuator; The imaging system according to claim 5 .

11. the camera includes an actuator that changes the orientation of the camera; the control data further includes information for determining an orientation of the camera; the processing circuit causes the camera to change the orientation of the camera using the actuator; The imaging system according to claim 6 .

12. the camera includes an actuator for changing a zoom magnification; the control data includes information for determining a zoom magnification of the camera; the processing circuit causes the camera to change the zoom magnification by the actuator; The imaging system according to claim 6 .

13. The control data is information for determining an exposure time for the camera; and Further, the information includes at least one of information for determining the opening and closing degree of the aperture of the camera. The imaging system according to claim 1 .

14. the object is a vehicle, The image represented by the image data includes an image of the license plate of the vehicle. The imaging system according to any one of claims 1 to 13.

15. the object is a vehicle, The image represented by the image data includes an image of a driver or a passenger of the vehicle. The imaging system according to any one of claims 1 to 13.

16. the object is a vehicle, the vehicle having wheels; the sensor measures the traveling speed of the vehicle and the rotational speed of the wheels; The speed information is information about the traveling speed of the vehicle and the rotation speed of the wheels.

16. The imaging system according to claim 1.

17. the processing circuitry generates axle count data including axle count information for the vehicle based on the speed information. The imaging system of claim 16.

18. the processing circuit generates vehicle type data including vehicle type information of the vehicle based on the image information and the axle number information. The imaging system of claim 17.

19. the processing circuitry causes the sensor to measure the velocity multiple times at different times; the speed information is information about the speed measured multiple times, the measurement timing information is information regarding timings at which the velocity is measured multiple times; 19. The imaging system according to claim 1.

20. Further, another sensor is provided to measure the velocity of the object from a direction different from that of the sensor; The processing circuitry causing the other sensor to measure the velocity of the object to generate other velocity information of the object; determining the velocity of the object based on the velocity information and the other velocity information; 20. The imaging system according to claim 1.

21. a processor; a memory storing a computer program to be executed by the processor; Equipped with The computer program causes the processor to: causing a sensor to measure a velocity of an object and generate velocity information of the object and timing information of the measurement of the velocity; (a) generating control data including image capture timing information of a camera different from the sensor based on position information of the object or distance information to the object at the time of speed measurement, (b) the speed information, and (c) the measurement timing information; causing the camera to output image data including image information of the object based on the control data; Execute Processing equipment.

22. 1. A computer-implemented method in an imaging system, comprising: The method comprises: causing a sensor to measure a velocity of an object and generate velocity information of the object and timing information of the measurement of the velocity; (a) generating control data including image capture timing information of a camera different from the sensor based on position information of the object or distance information to the object at the time of speed measurement, (b) the speed information, and (c) the measurement timing information; causing the camera to output image data including image information of the object based on the control data; A method comprising:

Images