Light source recognition system

Abstract

To provide a light source recognition system capable of recognizing a target light source with high accuracy.SOLUTION: A light source recognition system comprises: a mark light source 21a and a flash control device 21b that are installed on an object 91; a camera 32 that is mounted on a mobility 92 separate from the object 91; and a computer 42 that detects the mark light source 21a from an imaging result of the camera 32. The flash control device 21b makes the mark light source 21a flash in accordance with predetermined rules. When the computer 42 detected a plurality of light sources from an imaging result of the camera 32, it calculates a frequency of the flashing of each light source on the basis of time series data associated with the light source, and selects a mark light source 21a from the plurality of light sources on the basis of the frequency of flashing corresponding to the predetermined rules.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a light source recognition system. [Background technology]

[0002] Systems have been developed that recognize the light source of a vehicle ahead and recognize the vehicle ahead. For example, Japanese Patent Application Laid-Open Publication No. 2006-232009 discloses a technology in which self-luminous markers (near-infrared LEDs) are mounted on the side mirrors of motorcycles and the self-luminous markers are made to flash regularly so that other vehicles can recognize the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-232009 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above document does not describe a specific detection method for a following vehicle to detect the regular blinking emitted by the light source of the preceding vehicle. In particular, it does not describe a technology for distinguishing between disturbance light and the light source of the preceding vehicle. Examples of disturbance light include reflected light from a puddle and sunlight irregularly blocked by swaying leaves. For this reason, it is necessary to accurately distinguish between disturbance light and the target light source. An object of the present invention is to provide a light source recognition system that can accurately recognize the target light source. [Means for solving the problem]

[0005] The light source recognition system of the present invention includes a light source unit having a mark light source that emits light of a predetermined wavelength and is installed on an object, and a blinking control device that controls on / off of the mark light source, a camera mounted on a mobility separate from the object, and a computer that detects the mark light source from the image capture result of the camera. The blinking control device blinks the mark light source according to a predetermined rule. When the computer detects multiple light sources from the image capture result of the camera, it calculates the blinking frequency of each of the light sources based on time series data for each of the light sources, and selects the mark light source from the multiple light sources based on the blinking frequency corresponding to the predetermined rule. Effect of the Invention

[0006] According to the present invention, the blinking frequency of each light source is calculated based on the time series data of each light source. The blinking frequency of the mark light source corresponds to a predetermined rule, whereas disturbance light is often an irregular blinking or stationary light. Therefore, the computer can identify the mark light source that blinks at a frequency corresponding to the predetermined rule from among multiple light sources including disturbance light based on the frequency of each light source calculated from the time series data. According to the present invention, the target light source can be recognized with high accuracy. [Brief description of the drawings]

[0007] [Figure 1] 1 is a configuration diagram of a light source recognition system according to an embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram for explaining binarization according to the present embodiment. [Diagram 3] 11A and 11B are diagrams for explaining the binarization exclusion process of the present embodiment. [Figure 4] 4A to 4C are diagrams for explaining a frequency matching process according to the present embodiment. [Diagram 5] 5 is a flowchart illustrating an example of light source recognition control according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, as a mode for carrying out the present invention, a light source recognition system 1 which is one embodiment of the present invention will be described in detail with reference to the drawings. In addition to the following examples, the present invention can be carried out in various forms with various modifications and improvements based on the knowledge of those skilled in the art.

[0009] In the light source recognition system 1 of this embodiment, multiple mobility vehicles are used. The mobility vehicles are, for example, vehicles on which a user rides. As an example, each mobility vehicle in this embodiment is a walking area EV that has three wheels, one front wheel and two rear wheels, and is driven by an electric motor (for example, an in-wheel motor). For example, the rear mobility vehicle recognizes the position of the front mobility vehicle, and the front and rear mobility vehicles can perform automatic platooning (automatic following driving by the rear mobility vehicle). By using the light source recognition system 1, the rear mobility vehicle can recognize the position (distance and direction) of the mobility vehicle to be followed (forward).

[0010] As shown in FIG. 1, the first mobility 91 includes a light source unit 21, a camera 31, a computer 41, a boarding unit 51, an operation unit 61, a light source installation unit 71, and an operation control device 81. The light source unit 21 includes two mark light sources 21a and a blinking control device 21b. Each mark light source 21a is a light source installed on an object (here, the first mobility 91) and is configured to emit light of a predetermined wavelength. In this embodiment, each mark light source 21a is a near-infrared LED that emits near-infrared light. The predetermined wavelength is preferably a wavelength equal to or less than near-infrared light (near-infrared light or visible light). The blinking control device 21b controls the on / off (blinking) of each mark light source 21a.

[0011] The camera 31 is configured to capture an image of the area in front of the first mobility 91. The camera 31 is installed on the upper part of the first mobility 91 (around the operation unit 61) and faces downward at a predetermined angle. The camera 31 captures an image of the area in front of the first mobility 91 at predetermined time intervals (e.g., every few ms). The camera 31 transmits the captured image to the computer 41. The camera 31 is, for example, a near-infrared camera.

[0012] The computer 41 detects a mark light source (a mark light source 22a of another mobility described later) from the imaging results of the camera 31. The computer 41 calculates the distance between the detected mark light source 22a and itself (the first mobility 91) and the direction in which the mark light source 22a appears from itself (the first mobility 91). The computer 41 is configured to execute light source recognition control to recognize the mark light source (22a) of the other mobility. The computer 41 may function as the blinking control device 21b. In other words, the blinking control device 21b and the computer 41 may be configured as one common computer.

[0013] The boarding section 51 is a base on which the user stands. The operation section 61 is a section that extends upward from the front part of the boarding section 51, and is configured to allow the user to operate the steering wheel, the accelerator, the brakes, etc. while standing. The light source installation section 71 is provided at the rear part of the boarding section 51. Two marker light sources 2b are installed in the light source installation section 71 so as to emit light backward. The operation control device 81 is a computer that controls the driving, braking, etc. of the first mobility 91. The operation control device 81 controls the electric motor, etc. based on the detection results of various sensors mounted on the first mobility 91.

[0014] The second mobility 92 has the same configuration as the first mobility 91, and includes a light source unit 22, a camera 32, a computer 42, a boarding unit 52, an operation unit 62, a light source installation unit 72, and an operation control device 82. The light source unit 22 includes two marker light sources 22a and a blinking control device 22b. Since the corresponding components of the mobilities 91 and 92 have the same functions, a description of each component of the second mobility 92 will be omitted.

[0015] (Light source recognition control) The light source recognition control executable by each computer 41, 42 will be described using the light source recognition control of the computer 42 as an example. In this case, the light source recognition system 1 includes the light source unit 21 of the first mobility 91, the camera 32 of the second mobility 92, and the computer 42 of the second mobility 92. In the first mobility 91, the blinking control device 21b blinks the two mark light sources 21a according to a predetermined rule in response to a user's operation or a predetermined program. The predetermined rule is set, for example, as "blinking at M hertz with a duty ratio of N%." The blinking of the two mark light sources 21a is synchronized. The mark light source 2a of the first mobility 91 blinks backward. The camera 32 mounted on the second mobility 92, which is a mobility separate from the first mobility 91, captures the first mobility 91 including the two mark light sources 21a in front.

[0016] When the computer 42 detects multiple (here, three or more) light sources from the imaging results of the camera 32 in response to a user operation or a predetermined program, the computer 42 calculates the blinking frequency of each light source based on the time series data for each light source, and selects the mark light source 21a from the multiple light sources based on the blinking frequency corresponding to a predetermined rule. That is, the computer 42 executes light source recognition control in response to a user operation or a predetermined program. Note that, when the computer 42 detects only two light sources from the imaging results of the camera 32, the computer 42 may determine whether or not each of the light sources is the mark light source 21a by the light source recognition control. That is, it can be said that the computer 42 determines whether or not the detected light source is the mark light source 21a. Note that, hereinafter, the imaging results are also referred to as imaging data.

[0017] As a specific example, the computer 42 reads the position of each light source on the imaging data, i.e., the two-dimensional coordinates (X, Y) of each light source, from the imaging data of the camera 32 continuously acquired for a predetermined period. The computer 42 assigns an individual light source number to each light source at a different position to identify each light source. The computer 42 executes various calculations on the premise that the first mobility 91 and the second mobility 92 are not moving, i.e., the mark light source 21a is not moving. The computer 42 recognizes the position (coordinates) of each light source and creates time series data on the intensity (or brightness) for each light source. As an example, if the camera 32 captures images every 10 ms and the predetermined period is 640 ms, the computer 42 will acquire 64 pieces of imaging data within the predetermined period. The computer 42 creates time series data for each light source based on the 64 pieces of imaging data.

[0018] The computer 42 binarizes the blinking of each light source based on the time series data for a predetermined period, calculates the sum of the binarized values ​​for each light source, and determines the presence or absence of a light source that is not the marker light source 21a (hereinafter also referred to as a "non-marker light source") based on the sum of the values ​​for each light source and a predetermined rule, and if a non-marker light source is detected, excludes the non-marker light source from the selection targets. This process is also referred to as a "binarization exclusion process." The binarization exclusion process reduces the execution load of the light source recognition control and enables high-speed processing.

[0019] As a specific example of the binarization exclusion process, as shown in Fig. 2, the computer 42 binarizes and expresses blinking in the time series data of each light source by using a predetermined intensity as a threshold, and when the intensity is equal to or greater than the threshold, it represents the light as ON, and when the intensity is less than the threshold, it represents the light as OFF. The computer 42 calculates the sum of the binarized values ​​within a predetermined period, for example, by representing ON as +1 and OFF as -1. The binarized data of multiple light sources is shown in Fig. 2.

[0020] In the blinking control device 21b of the present embodiment, the predetermined rule is set to blink the mark light source 21a at 10 Hz and a duty ratio of 50%, for example. The computer 42 stores the predetermined rule. The predetermined rule is common to both mobilities 91 and 92. The computer 42 stores a selection target range (upper and lower thresholds) for the sum. The computer 42 leaves light sources whose sum falls within the selection target range as selection targets, and excludes light sources whose sum does not fall within the selection target range from the selection targets. If the duty ratio of the mark light source 21a is 50% as in this example, it can be estimated that the same or similar numbers of on and off will appear within a predetermined period. Therefore, the selection target range is set, for example, to a range of 0 ± noise tolerance (for example, about 10% of the number of imaging data). For example, as shown in FIG. 3, if 64 imaging data are to be calculated, the selection target range may be set to -7 or more and 7 or less (lower threshold = -7, upper threshold = 7). In this example, 21 light sources are excluded from the 25 detected light sources, leaving only 4 light sources to be selected (subject to frequency analysis).

[0021] As a process for excluding light sources from the selection targets (thinning process), the computer 42 may execute a "time exclusion process" in addition to the binarization exclusion process. The computer 42 stores a threshold time set to a time longer than the lighting time of the mark light source 21a in one cycle. As the time exclusion process, the computer 42 determines a light source that has been lit for a threshold time or longer as a non-marker light source based on the time series data of each light source, and excludes the light source 21a from the selection targets. The time exclusion process also reduces the load of the light source recognition control, enabling high-speed processing.

[0022] In the frequency analysis result showing the relationship between the frequency and intensity for each light source, the computer 42 stores a peak threshold, which is the intensity threshold corresponding to the peak region of the frequency of the target light source 21a, and a non-peak threshold, which is the intensity threshold for frequencies smaller than the peak region including the minimum peak frequency. The computer 42 compares the peak threshold and the non-peak threshold with the frequency analysis results of each light source among the selection targets, and selects the target light source 21a from among the selection targets. This process is also referred to as "frequency matching process". The light source recognition control of the present embodiment includes a binarization exclusion process and a frequency matching process. The computer 42 may execute the frequency matching process after executing the binarization exclusion process and / or the time exclusion process.

[0023] As shown in FIG. 4, in the frequency analysis result of the target light source 21a (horizontal axis: frequency, vertical axis: intensity), a peak appears corresponding to the frequency of the target light source 21a (here, 10 Hz). When the frequency of the target light source 21a is 10 Hz, peaks appear at around 10 Hz, 30 Hz, and 50 Hz, respectively. The computer 42 stores a predetermined range including the peak frequency of the target light source 21a as a peak region. For the computer 42 of the present embodiment, peak thresholds L1, L2, and L3 are set for each peak region A1, A2, and A3, and a non-peak threshold L4 is set for a frequency region B smaller than the lower limit value of the peak region A1 including the minimum frequency (minimum peak frequency: 10 Hz) among the peak frequencies. The non-peak threshold L4 is smaller than the peak threshold L1 (L4 < L1). Also, the peak thresholds L2 and L3 are smaller than the peak threshold L1 (L2 < L1, L3 < L1).

[0024] The computer 42 of this embodiment selects, as the mark light source 21a, a light source whose intensity in the peak region A1 is equal to or greater than the peak threshold L1, whose intensity in the frequency region B is less than the non-peak threshold L4, whose intensity in the peak region A2 is equal to or greater than the peak threshold L2, and whose intensity in the peak region A3 is equal to or greater than the peak threshold L3. The computer 42 may be set to select, as the mark light source 21a, a light source whose intensity in the peak region A1 is equal to or greater than the peak threshold L1, and whose intensity in the frequency region B is less than the non-peak threshold L4. As shown in FIG. 4, the computer 42 can accurately distinguish between the reflected light of the water surface and the mark light source 21a. In this way, the computer 42 may select the mark light source 21a based on the peak threshold for one peak region A and the non-peak threshold for the frequency region B.

[0025] An example of the light source recognition control will be described with reference to FIG. 5. In response to a user's operation (e.g., a button operation) on the second mobility 92, the camera 32 starts capturing images, and the computer 42 starts analyzing the captured image data (S1). In response to a user's operation (e.g., a button operation) on the first mobility 91, the blinking control device 21b blinks the two mark light sources 21a according to a predetermined rule (S2). The computer 42 executes a binarization exclusion process (S3). In addition to the binarization exclusion process, a time exclusion process may also be executed. Thereafter, the computer 42 executes a frequency matching process on the thinned-out light sources to be selected (S4). If there are only two light sources selected in the frequency matching process (S5: Yes), the computer 42 determines that they are the mark light source 21a, and ends the light source recognition control. In this way, according to the light source recognition system 1 of this embodiment, the following vehicle can easily recognize the mark light source 21a (following target) by the user's operation.

[0026] On the other hand, if the number of light sources selected in the frequency matching process is three or more or one (S5: No), the computer 42 executes a first recheck process or a second recheck process (S6). In the first recheck process, the computer 42 executes a binarization exclusion process (S3) and a frequency matching process (S4) for a second predetermined period (e.g., 10 to 650 ms) in which the start and end of the predetermined period (e.g., 0 to 640 ms) are delayed by a predetermined time (the time when new imaging data is input) from the previous time.

[0027] In the second recheck process, the computer 42 executes a waveform similarity check. In the waveform similarity check, the computer 42 extracts two light sources from the three or more light sources remaining as selection targets, compares the waveforms of their time-series data, and calculates the similarity between the two light sources. The calculation of the similarity is performed by a known method, for example, a cosine similarity calculation. In the cosine similarity calculation, a binarized value in the time-series data is used, and the waveform of the time-series data of each light source is considered as a vector to calculate a cosine. Theoretically, a pair with a similarity of 1 is the mark light source 21a. If the similarity is equal to or greater than a threshold, the computer 42 determines that the two light sources are the mark light source 21a.

[0028] For example, the computer 42 performs a waveform similarity check on all combinations (pairs) of two light sources until it finds a combination whose similarity is equal to or greater than a threshold. Alternatively, the computer 42 may perform a waveform similarity check on all combinations (pairs) of two light sources, and determine the combination whose similarity is equal to or greater than a threshold and has the highest similarity as the mark light source 21a.

[0029] In this way, in the case where the light source unit 21 has two or more marker light sources 21a arranged at different positions and a blinking control device 21b that blinks the two or more marker light sources 21a according to the same predetermined rule, the computer 42 may calculate the similarity of the time series data of the light sources at the two different positions and select two light sources whose similarity is equal to or greater than a predetermined threshold as the marker light source. This also makes it possible to accurately select the marker light source 21a from a selection target including three or more light sources.

[0030] The computer 42 may be configured to execute, for example, a first recheck process the first time (initial time) as the recheck process (S6), and if the number of light sources to be selected is still not two, execute a second recheck process the second time. The computer 42 can execute the first recheck process and the second recheck process in a predetermined order. After the recheck process, it is determined whether the number of light sources to be selected is two or not (S7). If the computer 42 cannot detect the mark light source 21a even after continuing the light source recognition control for a predetermined execution time, the computer 42 is configured to inform the user that the mark light source 21a has not been detected by a notification means, and to end the light source recognition control.

[0031] The computer 41 of the first mobility 91 can also execute the light source recognition control similar to the above. For example, when the first mobility 91 and the second mobility 92 are arranged in the reversed front-to-back position, the computer 41 of the first mobility 91 executes the light source recognition control based on the imaging data of the camera 31 of the first mobility 91, and selects the marker light source 22a of the second mobility 92 from the multiple light sources. In this case, the light source recognition system 1 is configured to include the light source unit 22 of the second mobility 92, the camera 31 of the first mobility 91, and the computer 41 of the first mobility 91. In this way, each of the two or more mobility can execute the light source recognition control.

[0032] As an example of operation, when a user riding on the second mobility 92 performs a predetermined operation (e.g., a button push) on the second mobility 92, the camera 32 operates and the computer 42 executes light source recognition control (frequency matching processing, etc.) until a predetermined detection time has elapsed. Then, after the user operation on the second mobility 92, when a user riding on the first mobility 91 performs a predetermined operation (e.g., a button push) on the first mobility 91, the blinking control device 21b (which may be the computer 41) blinks the mark light source 21a for a predetermined blinking time. The predetermined detection time and the predetermined blinking time can be set arbitrarily, but for example, the predetermined blinking time is set to a time longer than the predetermined detection time. In this way, the computer 42 can detect the mark light source 21a of the first mobility 91 and calculate the position (distance and direction) of the first mobility 91 by a simple operation. Note that a user operation of the first mobility 91 (a vehicle to be followed) may be performed first by setting various times.

[0033] As described above, according to this embodiment, the blinking frequency of each light source is calculated based on the time series data of each light source. The blinking frequency of the mark light source 21a (22a) corresponds to a predetermined rule, whereas the disturbance light is an irregular blinking or stationary light. Therefore, the computer 42 (41) can identify the mark light source 21a (22a) that blinks at a frequency corresponding to the predetermined rule from among a plurality of light sources including the disturbance light, based on the frequency of each light source calculated from the time series data. According to this embodiment, the target light source can be recognized with high accuracy.

[0034] According to this embodiment, various exclusion processes can reduce the processing load of the computers 41 and 42 and perform high-speed processing. Also, according to this embodiment, a following vehicle can easily recognize a target mobility by a user operation, for example, a button operation. Also, according to this embodiment, for example, for four mobility vehicles, one group can be easily formed in response to a user operation, so that the four vehicles can travel in a tandem (platooning), or two groups can be formed in which two vehicles in each group travel in a tandem.

[0035] (others) The present invention is not limited to the above embodiment. For example, when the light source recognition system 1 is applied to mobility at night or indoors where there is little light, the marker light source may be visible light. The mobility may be a transport vehicle, a general vehicle, heavy machinery, or an aircraft. Each mobility, for example, the first mobility 91 and the second mobility 92, may be different types of mobility. The object may not be a mobility (moving object) and may not move. The "computer" is a concept that includes a processor and a memory and includes an ECU. The computer may be a remote computer installed in a different location from the mobility. For example, imaging data may be transmitted from a camera of a following vehicle to a remote computer by wireless communication, the light source recognition control may be executed by the remote computer, and the selection result (recognition result) of the light source may be transmitted to the on-board computer of the following mobility. However, from the viewpoint of time loss, it is preferable that the computer that executes the light source recognition control is an on-board computer. The computer may execute frequency analysis based on time series data without executing various exclusion processes. The computer may execute the frequency matching process based on the time series data of each light source binarized as described above, or may execute the frequency matching process without binarization. The number of the mark light source 21a (22a) of the light source unit 21 (22) may be one or may be two or more. When there are multiple mark light sources 21a (22a), the blinking control device 21b (22b) does not need to synchronize the blinking of the multiple mark light sources 21a (22a). In the description, the "intensity" of light can be replaced with "brightness." [Explanation of symbols]

[0036] 1...light source recognition system, 21, 22...light source unit, 21a, 22a...marker light source, 21b, 22b...flashing control device, 31, 32...camera, 41, 42...computer, 91...first mobility (object), 92...second mobility (mobility).

Claims

1. a light source unit having a mark light source that is installed on an object and emits light of a predetermined wavelength, and a blinking control device that controls the on / off of the mark light source; A camera mounted on a mobility separate from the object; A computer that detects the mark light source from the imaging result of the camera; A light source recognition system comprising: The blinking control device blinks the marker light source according to a predetermined rule, The computer includes: When a plurality of light sources are detected from the imaging result of the camera, a blinking frequency of each of the light sources is calculated based on time series data for each of the light sources; selecting the mark light source from the plurality of light sources based on a blinking frequency corresponding to the predetermined rule; Light source recognition system.

2. The computer includes: In the frequency analysis result showing the relationship between the frequency and the intensity for each light source, A peak threshold value, which is an intensity threshold value corresponding to a peak region of the frequency of the mark light source, and a non-peak threshold value, which is an intensity threshold value for frequencies smaller than the peak region including a minimum peak frequency, are stored; comparing the peak threshold value and the non-peak threshold value with a frequency analysis result of each of the light sources in the selection target, and selecting the mark light source from the selection target; The light source recognition system of claim 1 .

3. The computer includes: binarizing the blinking of each of the light sources based on the time series data for a predetermined period of time, and calculating a sum of the binarized values ​​for each of the light sources; determining whether or not there is a non-marker light source, which is a light source that is not the marker light source, based on the value of the sum for each of the light sources and the predetermined rule; When the non-marker light source is detected, the non-marker light source is excluded from selection targets. The light source recognition system according to claim 1 or 2.

4. The computer stores a threshold time set to a time longer than a lighting time in one cycle of the mark light source, The light source that has been turned on for the threshold time or longer is excluded from targets for selection of the mark light source. The light source recognition system according to claim 1 or 2.

5. The light source unit includes two or more of the mark light sources arranged at different positions from each other, and the blinking control device that blinks the two or more mark light sources according to the same predetermined rule, The computer includes: Calculating a similarity between time series data of the light source at two different positions; selecting two of the light sources whose similarity is equal to or greater than a predetermined threshold as the landmark light sources; The light source recognition system according to claim 1 or 2.

Images