Marker and marker identification device, and programs therefor

Abstract

To provide a marker which is suppressed in influence of external light and furthermore enables a plurality of identifiers to be robustly determined.SOLUTION: A marker 1 comprises: a light-emitting unit 14 composed of a plurality of light-emitting elements 15; a light emission pattern table storage unit 10 in which a light emission pattern stored with a light emission pattern is preset, wherein in the light emission pattern, any of the light-emitting elements 15 do not continue a light-off state and each of the light-emitting elements 15 does not continue a light-on state, throughout two adjacent switchover timing; a light emission pattern 12 referring to the light emission pattern table and determining a light emission pattern that corresponds to an identifier input from the outside; and a control unit 13 controlling the light-on state and light-off states of each of the light-emitting elements 15 on the basis of the light emission pattern determined by the light emission pattern determination unit 12.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a marker, a marker identification device, and a program therefor. [Background technology]

[0002] Strong calibration is a technique for estimating camera parameters such as the position, orientation, and angle of view of a camera using an object whose three-dimensional shape and size are known, or multiple markers whose three-dimensional positional relationships are known.

[0003] For example, it is possible to estimate camera parameters using four or more points with known positional relationships that exist on the same plane, and this is called the Perspective 4-Point Problem (P4P problem). In strong calibration that uses a solution to this P4P problem, it is necessary to set which image each point corresponds to on the image captured by the camera.

[0004] In this strong calibration, a method is known for distinguishing between light-emitting markers, which are point light sources such as light-emitting diodes (LEDs), and planar non-light-emitting markers, which are printed with rectangular patterns of known shape and size (hereinafter, the first conventional method). In this first conventional method, a printed identification pattern may be used in addition to the light-emitting diodes and rectangular patterns.

[0005] A conventional method for calibrating a camera using a pattern in the vicinity of a known three-dimensional position is known (Non-Patent Document 1). In the method described in Non-Patent Document 1, visual feature points existing in the vicinity of a control point are extracted, and the image coordinates of the control point are estimated based on the arrangement and appearance of these points.

[0006] When a spacecraft such as an asteroid probe lands, a marker (target marker) made of a retroreflective material is dropped near the landing target point, and the marker is used as a guide for guidance. At this time, a method is known in which a blinking light is irradiated near the marker, images are taken with a camera mounted on the spacecraft both when the light is irradiated and when it is not, and the difference between the images taken at both times is calculated to highlight and extract the marker (Non-Patent Document 2).

[0007] Lighthouses used for nighttime navigation are installed at known locations with known latitude and longitude, and have properties similar to strongly calibrated markers. These lighthouses have a specific way of shining called light quality, and it is possible to identify which lighthouse it is by its light color, light cycle, time pattern (steady light, light and dark pattern, flash, sudden flash, equal light and dark light, group flash, Morse code light, etc.), and time and color combination pattern (alternate light, alternating flash, etc.) (hereinafter, conventional second method). [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Toshie Misu, Masato Fujii, Masahiro Shibata, "Camera Parameter Estimation Method Using SIFT Features Near Calibrated Control Points", Proceedings of the 8th Forum on Information Science and Technology (FIT 2009), no.3, H-049, pp.207-212 (2009) [Non-Patent Document 2] Fuyuto Terui, Yuichi Tsuda, Junko Ogawa, Yuya Mimasu, "Automatic and autonomous functions in navigation and guidance control of the asteroid probe "Hayabusa 2"", Artificial Intelligence, vol. 29, no. 4, pp. 327-334 (July 2014) Summary of the Invention [Problem to be solved by the invention]

[0009] In the above-mentioned conventional method 1, if the non-luminous marker is not illuminated, it is difficult to use the non-luminous marker for strong calibration. Also, in the conventional method 1, if a part of the printed identification pattern is hidden by another subject or if a shadow falls on a part of the identification pattern, uniform illumination cannot be obtained, which may result in erroneous recognition.

[0010] In the technology described in Non-Patent Document 1, if an object with a similar pattern is present within the field of view of the camera to be calibrated, the accuracy and robustness of the calibration may be reduced due to misidentification. Making the pattern more complex reduces the possibility of a similar pattern being present in the surroundings, but this may result in a decrease in the accuracy of identifying the pattern and a decrease in the robustness of the calibration.

[0011] The technology described in Non-Patent Document 2 does not take into consideration the identification of multiple target markers. With the second conventional method, if there are bright light-emitting or flashing objects near the lighthouse, or if the intensity or color of the ambient light around the viewer is significant, there is a risk of misrecognizing the brightness or color of the light. As described above, no prior art has been proposed that can suppress the effects of the color and pattern of the subject and external light and robustly determine multiple identifiers.

[0012] Therefore, an object of the present invention is to provide a marker and a marker identification device, as well as a program for carrying out the same, that can robustly identify a plurality of identifiers while suppressing the effects of the color and pattern of the subject and external light. [Means for solving the problem]

[0013] In order to solve the above problems, the marker according to the present invention is a marker that emits light in a light emission pattern corresponding to a unique identifier by turning on or off a light emitting element at a predetermined switching timing, and emits light with different spectral characteristics. The first light-emitting element, the second light-emitting element, and the third light-emitting element The configuration includes a light emitting section formed of a light emitting element, a light emitting pattern table storage section, a light emitting pattern determination section, and a control section.

[0014] According to this configuration, the light emission pattern table storage unit is preset with a light emission pattern table that stores, for each identifier, light emission patterns in which all light-emitting elements do not continue to be turned off and each light-emitting element does not continue to be turned on across two adjacent switching timings. Moreover, the light emission pattern determination unit refers to the light emission pattern table and determines a light emission pattern corresponding to the identifier inputted from outside. The control unit controls turning on or off each light-emitting element at the switching timing based on the light-emitting pattern determined by the light-emitting pattern determination unit. The light emitting section can emit light in a first emission color or a second emission color by combining the on or off states of the first light emitting element, the second light emitting element, and the third light emitting element. The light emission pattern table storage unit has a light emission pattern table preset therein, in which the light emitting unit emits light in the first light emission color or the second light emission color.

[0015] In this way, the marker emits light in a light emission pattern corresponding to a unique identifier over two adjacent switching timings, enabling robust identifier determination by the marker identification device. At this time, all light-emitting elements of the marker do not continue to be turned off, and each light-emitting element does not continue to be turned on, so that the influence of external light can be suppressed when the light emission pattern is detected by the marker identification device.

[0016] In addition, in order to solve the above-mentioned problems, the marker identification device of the present invention is a marker identification device that detects the emission pattern of a marker and determines a unique identifier corresponding to the emission pattern, and is configured to include a light receiving unit composed of a plurality of light receiving elements that receive light from the marker with different spectral characteristics, a difference calculation unit, a threshold processing unit, and an identifier determination unit.

[0017] According to this configuration, the difference calculation section calculates the difference value of the detection result of each light receiving element between two adjacent calculation timings. In addition, the threshold processing unit performs threshold processing on the difference value calculated by the difference calculation unit using positive and negative thresholds, and classifies the results of the threshold processing into three values ​​indicating a change from a lit state to an off state, a change from an off state to a lit state, and a continuation of the off state or the lit state. The identifier determination unit then determines the identifier by associating the three values ​​classified by the ternarization unit with each digit of the ternary number.

[0018] In this way, the marker identification device calculates the difference in the light receiving level between two adjacent calculation timings, and therefore can robustly determine multiple identifiers. At this time, since all the light emitting elements do not continue to be turned off and each light emitting element does not continue to be turned on, the marker identification device can suppress the influence of external light when detecting the light emission pattern.

[0019] The present invention can also be realized by a program for causing a computer to function as the above-mentioned marker. Moreover, the present invention can also be realized by a program for causing a computer to function as the above-mentioned marker identification device. Effect of the Invention

[0020] According to the present invention, the influence of the color and pattern of the subject and external light can be suppressed, and multiple identifiers can be robustly determined. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic configuration diagram of a marker identification system according to each embodiment. [Diagram 2] 1 is a block diagram showing a configuration of a marker according to a first embodiment. FIG. [Diagram 3] FIG. 2 is an explanatory diagram illustrating the luminous color of a light-emitting unit in the first embodiment. [Figure 4] FIG. 2 is an explanatory diagram for explaining a light emission pattern table in the first embodiment. [Diagram 5] FIG. 3 is an explanatory diagram illustrating an example of a synchronization signal in the first embodiment. [Figure 6] FIG. 3 is an explanatory diagram illustrating an example of a synchronization signal in the first embodiment. [Figure 7] FIG. 3 is an explanatory diagram illustrating an example of a synchronization signal in the first embodiment. [Figure 8] 5 is a flowchart showing the operation of a control unit in the first embodiment. [Figure 9] FIG. 11 is a block diagram showing the configuration of a marker identification device according to a second embodiment. [Figure 10] 10 is a flowchart showing the operation of the marker identification device according to the second embodiment. [Figure 11] FIG. 13 is a block diagram showing the configuration of a marker according to the third embodiment. [Figure 12] FIG. 13 is an explanatory diagram for explaining a light emission pattern table in the third embodiment. [Figure 13] FIG. 13 is a block diagram showing a configuration of a marker identification device according to a fourth embodiment. [Figure 14] FIG. 13 is a schematic configuration diagram of a marker identification system according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. However, each embodiment described below is intended to embody the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following. In addition, the same symbols are used for the same means, and the description may be omitted.

[0023] [Outline of the marker identification system] An overview of a marker identification system 100 according to each embodiment will be described with reference to FIG. For example, the marker identification system 100 is installed in a photography studio, and measures the three-dimensional position of each marker 1 attached to a measurement target α, which is a subject, and determines an identifier unique to each marker 1. Then, the marker identification system 100 performs desired image processing (e.g., camera calibration and generation of a three-dimensional model) based on the three-dimensional position and identifier of each marker 1. As shown in Fig. 1, the marker identification system 100 includes a marker 1, a marker identification device 2, an image processing device 3, and a photographing camera C.

[0024] The marker 1 emits light in a light emission pattern corresponding to a unique identifier by turning on or off the light emitting element at a predetermined switching timing. This light emission pattern refers to the time change (pattern) of the color emitted by the marker 1. In other words, a unique identifier is assigned to the marker 1 in advance, and the marker emits light in a light emission pattern indicating the identifier. In this embodiment, three markers are attached to the head and both knees of a person who is the measurement target α, and emit light in different light emission patterns. Each marker 1 has the same configuration, and details will be described later.

[0025] The marker identification device 2 detects the light emission pattern of each marker 1 and determines a unique identifier corresponding to the light emission pattern. In this embodiment, the marker identification device 2 is built into the photographing camera C. Here, the photographing camera C associates a photographed image of the measurement target α with the coordinates and identifier of each marker 1 in the photographed image, and outputs the result to the image processing device 3. Details of the marker identification device 2 will be described later.

[0026] The image processing device 3 performs arbitrary image processing using the captured image of the measurement object α input from the photographing camera C and the coordinates and identifiers of each marker 1 in the captured image. For example, the image processing device 3 performs, as image processing, a calibration process for the photographing camera C or a three-dimensional model generation process for the measurement object α. The camera C is similar to a typical camera used for shooting broadcast programs, except that it has the above-mentioned marker identification device 2 built in.

[0027] It goes without saying that the configuration and uses of the marker identification system 100 are not limited to the example shown in FIG. Moreover, when the marker 1 according to the first embodiment is used, the marker identification device 2 according to the second embodiment is used. On the other hand, when the marker 1A according to the third embodiment is used, the marker identification device 2A according to the fourth embodiment is used.

[0028] (First embodiment) [Marker configuration] The configuration of the marker 1 according to the first embodiment will be described with reference to FIG. As shown in FIG. 2, the marker 1 includes a light emission pattern table storage unit 10, a light emission pattern determination unit 12, a control unit 13, and a light emitter .

[0029] The light emission pattern table storage unit 10 is a storage device such as a memory that stores a light emission pattern table 11 (FIG. 4). This light emission pattern table 11 stores, for each identifier, a light emission pattern in which all the light emitting elements 15 do not continue to be turned off and each light emitting element 15 does not continue to be turned on over two adjacent switching timings (synchronization signals). For example, the light emission pattern table 11 is set in advance by an administrator of the marker identification system 100, and will be described in detail later.

[0030] The light emission pattern determination unit 12 determines a light emission pattern corresponding to an externally input identifier by referring to the light emission pattern table 11 stored in the light emission pattern table storage unit 10. Then, the light emission pattern determination unit 12 outputs the determined light emission pattern to the control unit 13.

[0031] The control unit 13 controls the turning on or off of each light-emitting element 15 based on the light-emitting pattern determined by the light-emitting pattern determination unit 12. That is, the control unit 13 switches the light-emitting color of the light-emitting unit 14 over time. Here, the switching over time may be repeated at a preset cycle, or may be switched automatically or manually at any timing. In this embodiment, this switching over time is performed based on a synchronization signal that indicates the switching timing of turning on or off each light-emitting element 15.

[0032] The light-emitting unit 14 is composed of a plurality of light-emitting elements 15 that emit light with different spectral characteristics. That is, the light-emitting unit 14 can emit visible light or other electromagnetic waves (hereinafter, "turn on") and stop emitting electromagnetic waves (hereinafter, "turn off"). In this embodiment, the light-emitting unit 14 is composed of three light-emitting elements 15: a first light-emitting element 151, a second light-emitting element 152, and a third light-emitting element 153.

[0033] The light emitting element 15 emits light in a unique and different spectrum (hereinafter referred to as "primary color") and has two states, a light-on state and a light-off state. For example, the light emitting element 15 is a full-color light emitting diode (LED). Specifically, the first light emitting element 151, the second light emitting element 152, and the third light emitting element 153 are a red LED, a green LED, and a blue LED built into the full-color LED, respectively. In this case, the light emitting element 15 can emit light in eight colors, namely black (K), red (R), green (G), yellow (Y), blue (B), magenta (M), blue-green (C), and white (W) (see FIG. 3).

[0034] <Marker illumination> The emission of light by the marker 1 will now be described in detail. Hereinafter, the light-emitting state of the light-emitting unit 14 will be referred to as the light-emitting color (including the off state). The light-emitting color of the light-emitting unit 14 is expressed by a vector V (1) ,V (2) That is, the light-emitting unit 14 can emit a first emission color V by combining the on and off states of the three light-emitting elements 15. (1) or second emission color V (2) The light emitting unit 14 can emit light in the first emission color V (1) and second emission color V (2) and are repeated alternately.

[0035] First emission color V (1) is expressed by the primary color V1 of the first light-emitting element 151 as shown in the following formula (1). (1), the primary color V2 of the second light-emitting element 152 (1) , and the primary color V3 of the third light-emitting element 153 (1) The vector is expressed as a vector containing the three primary color components:

[0036]

number

[0037] Similarly, the second emission color V (2) is expressed by the primary color V1 of the first light-emitting element 151 as shown in the following formula (2). (2) , the primary color V2 of the second light-emitting element 152 (2) , and the primary color V3 of the third light-emitting element 153 (2) The first emission color V (1) and second emission color V (2) Each component of represents a "0" as off and a "1" as on.

[0038]

number

[0039] Let the primary color V1 represent red, the primary color V2 represent green, and the primary color V3 represent blue. As shown in Figure 3, the luminous color V=[0,0,0] T represents black (K) because the primary color V1 of the first light emitting element 151, the primary color V2 of the second light emitting element 142, and the primary color V3 of the third light emitting element 153 are all turned off. T represents red (R) because only the primary color V1 of the first light-emitting element 151 is lit. T represents green (G) because only the primary color V2 of the second light emitting element 152 is lit. T represents yellow (Y) because the primary color V1 of the first light-emitting element 151 and the primary color V2 of the second light-emitting element 152 are lit. T represents blue (B) because only the primary color V3 of the third light-emitting element 153 is lit. Trepresents reddish purple (M) because the primary color V1 of the first light-emitting element 151 and the primary color V3 of the third light-emitting element 153 are lit. T represents blue-green (C) because the primary color V2 of the second light-emitting element 152 and the primary color V3 of the third light-emitting element 153 are lit. T represents white (W) because the primary color V1 of the first light emitting element 151, the primary color V2 of the second light emitting element 142, and the primary color V3 of the third light emitting element 153 are all lit.

[0040] As shown in FIG. 4, the light emission pattern table storage unit 10 stores a first light emission color V (1) or second emission color V (2) In other words, the light emission pattern table 11 is set in advance so that the first light emission color V of the light emission unit 14 is set in accordance with an identifier input from the outside. (1) and second emission color V (2) In the light emission pattern table 11, the first light emission color V corresponding to each identifier ID is determined. (1) and second emission color V (2) shall satisfy the following two conditions:

[0041] (Condition 1) First emission color V (1) and second emission color V (2) Both of the V (1) ≠0 or V (2) ≠0) (Condition 2) First emission color V (1) and second emission color V (2) In this case, the same primary colors are not both turned on (i.e., V (1) ·V (2) =0)

[0042] By satisfying the above two conditions, the first emission color V (1) and second emission color V (2) The difference vector V (2) -V (1) is always a non-zero vector value and is unique for each identifier.

[0043] For example, consider a case where the light emitting element 15 is a full-color light emitting diode having red, green, and blue as primary colors. In this case, as shown in FIG. 4, the light emitting pattern table 11 includes a first light emitting color V (1) and second emission color V (2) By distinguishing the order of the illumination patterns, 26 illumination patterns can be set. Here, the identifier is a number uniquely assigned to each illumination pattern, and in this embodiment, it is an integer value of "+1 to +13" and "-1 to -13" (but other than "0").

[0044] In the light emission pattern table 11 of FIG. 4, the light emission pattern numbered +k and the light emission pattern numbered −k are mutually the first light emission color V (1) and second emission color V (2) In addition, in order to simplify the explanation, the light emission pattern table 11 is illustrated as being divided into two in Fig. 4, but in reality it is one table.

[0045] Since the marker identification device 2 described later uses a synchronization signal, the first emission color V (1) and second emission color V (2) Therefore, the marker 1 can use the light emission pattern table 11 with numbers having the same integer value but different positive and negative signs, and can express 26 different light emission patterns.

[0046] Here, the first emission color V (1) and second emission color V (2) The case where the sequence of the first emission color V is distinguishable is when the timing of the photographing camera C is synchronized with the timing of the blinking and color change of the light emitting element 15, and the first emission color V corresponds to the odd-numbered and even-numbered photographed images. (1) and second emission color V (2) For example, an identification signal indicating an odd-numbered or even-numbered image (field or frame) in a video signal is extracted from a video synchronization signal, and the first emission color V is switched in synchronization with this identification signal. (1) and second emission color V (2)Therefore, marker 1 is suitable for an environment where it is easy to obtain a video synchronization signal.

[0047] When an identifier is input from the outside, the light emission pattern determination unit 12 selects a light emission pattern (first light emission color V) corresponding to the identifier (number) from the light emission pattern table 11. (1) and second emission color V (2) Then, the light emission pattern determination unit 12 outputs the obtained light emission pattern to the control unit 13.

[0048] The control unit 13 determines the light emission pattern (V (1) ,V (2) In other words, the control unit 13 receives a synchronization signal S(t) indicating the switching timing, and controls the light-emitting unit 14 to alternately take the light-emitting state corresponding to the light-emitting pattern (V (1) ,V (2) ) to control turning on or off of each light-emitting element 15 in synchronization with the synchronization signal S(t).

[0049] For example, when the synchronization signal S(t) has a binary level (0, 1), the control unit 13 can control the light emission state L(t) of the light-emitting unit 14 as shown in the following formula (3) (t is time). Here, when the synchronization signal S(t) has a level of "0", the control unit 13 controls the light-emitting unit 14 to emit the first light emission color V (1) On the other hand, when the level of the synchronization signal S(t) is “1”, the light-emitting unit 14 emits light of the second light color V (2) to emit light.

[0050]

number

[0051] The synchronization signal S(t) may be a signal that alternates between binary (0, 1) levels at a fixed time period, as shown in Fig. 5. For example, the synchronization signal S(t) may be generated by the synchronization separation circuit 4 from a video synchronization signal such as a black burst signal of the NTSC (National Television Committee). For example, the synchronization separation circuit 4 outputs an identification signal indicating an odd-numbered or even-numbered image generated from the NTSC black burst signal to the control unit 13 as the synchronization signal S(t).

[0052] Furthermore, the synchronization signal S(t) may be a signal that alternates between binary levels "0, 1" at any timing, as shown in FIG. 7, the synchronization signal S(t) may be a signal that alternates between binary levels of "0, 1" triggered by a trigger signal T(t) generated each time an image is taken by the photographing camera C. For example, the trigger signal T(t) may be a release signal (contact signal for an electromagnetic release), a sync contact signal (an X contact or an FP contact used to trigger light emission such as a flash), or a motor drive control signal for film winding.

[0053] The light-emitting unit 14 turns on or off each of the light-emitting elements 15 so as to achieve the light-emitting state L(t) input from the control unit 13. Here, the light-emitting unit 14 turns on or off the first light-emitting element 151, the second light-emitting element 152, and the third light-emitting element 153 according to the light-emitting state L(t) expressed by the following formula (4).

[0054]

number

[0055] Here, first light-emitting element 151 is turned off when the first component L1(t) of formula (4) is "0" and turned on when the first component L1(t) of formula (4) is "1". Second light-emitting element 152 is turned off when the second component L2(t) of formula (4) is "0" and turned on when the second component L2(t) of formula (4) is "1". Third light-emitting element 153 is turned off when the third component L3(t) of formula (4) is "0" and turned on when the third component L3(t) of formula (4) is "1".

[0056] [Control unit operation] The operation of the control unit 13 will be described with reference to FIG. The control unit 13 receives the light emission pattern (V (1) ,V (2) ) is assumed to have been entered.

[0057] As shown in FIG. 8, in step S1, the control unit 13 substitutes the synchronization signal level at the immediately previous switching timing into a variable S(t-1). In step S2, the control unit 13 substitutes the synchronization signal level at the current switching timing into a variable S(t).

[0058] In step S3, the control unit 13 determines whether or not the synchronization signal S(t) is equal to the synchronization signal S(t-1). That is, the control unit 13 determines whether or not the levels of two synchronization signals having adjacent switching timings have changed.

[0059] If the two synchronization signals S(t) and the synchronization signal S(t-1) are equal (Yes in step S3), the control unit 13 returns to the process of step S2. If the two synchronization signals S(t) and the synchronization signal S(t-1) are different (No in step S3), the control unit 13 proceeds to the process of step S4.

[0060] In step S4, the control unit 13 determines whether the synchronization signal S(t) is “0.” In other words, the control unit 13 sets the light-emitting unit 14 to the first light-emitting color V (1) It is determined whether or not to emit light.

[0061] If the synchronization signal S(t) is "0" (Yes in step S4), the control unit 13 proceeds to the process of step S5. If the synchronization signal S(t) is not "0" (No in step S4), the control unit 13 proceeds to the process of step S6.

[0062] In step S5, the control unit 13 changes the light emission state L(t) of the light-emitting unit 14 to the first light emission color V(1) Set to. In step S6, the control unit 13 changes the light emission state L(t) of the light-emitting unit 14 to the second light emission color V (2) Set to. In step S7, the control unit 13 causes the light emitting unit 14 to emit light in the light emitting state L(t) set in step S5 or step S6. In step S8, the control unit 13 substitutes the synchronization signal S(t) for the synchronization signal S(t-1), and then returns to the process of step S2.

[0063] [Actions and Effects] As described above, the marker 1 emits light in a light emission pattern corresponding to a unique identifier over two adjacent switching timings, enabling robust identifier determination by the marker identification device 2. At this time, all of the light-emitting elements 15 of the marker 1 do not continue to be turned off, and each of the light-emitting elements 15 does not continue to be turned on, so that when the light emission pattern is detected by the marker identification device 2, the effects of the color and pattern of the measurement object α and external light can be suppressed.

[0064] Furthermore, the marker 1 emits a first light color V (1) and second emission color V (2)) Since the order of the markers can be distinguished, the total number of identifiers that can be expressed by the light emission pattern can be doubled. At this time, the marker 1 can use a conventional video synchronization signal such as the NTSC black burst signal, which improves convenience.

[0065] Second embodiment [Marker identification device configuration] The configuration of a marker identification device 2 according to the second embodiment will be described with reference to FIG. As shown in FIG. 9, the marker identification device 2 includes a light receiving unit 20, a difference calculation unit 22, and an identification unit 25.

[0066] The light receiving unit 20 is composed of a plurality of light receiving elements 21 that receive light from the marker 1 (FIG. 2) with different spectral characteristics. That is, the light receiving unit 20 receives light from the marker 1 and outputs detection results (received light levels) in a plurality of spectral distributions to the difference calculation unit 22. In this embodiment, the light receiving unit 20 is composed of three light receiving elements 21: a first light receiving element 211, a second light receiving element 212, and a third light receiving element 213.

[0067] The light receiving element 21 is a color light receiving element that receives the three primary colors and is provided in the photographing camera C. For example, the light receiving element 21 may be a three-plate image sensor or a single-plate image sensor. The light receiving element 21 may have its sensitivity characteristics to the wavelength of light adjusted by an optical filter such as a colored filter or an interference filter, or a prism such as a spectroscopic prism or a dichroic prism.

[0068] The spectral sensitivity characteristics of the first light receiving element 211, the second light receiving element 212, and the third light receiving element 213 are such that they can directly or indirectly separate the light emitted by the first light emitting element 151, the second light emitting element 152, and the third light emitting element 153. Note that indirect separation refers to separation by linear or nonlinear conversion.

[0069] In the following description, the first light receiving element 211, the second light receiving element 212, and the third light receiving element 213 will be described as corresponding to the first light emitting element 151, the second light emitting element 152, and the third light emitting element 153 in Fig. 2, respectively. That is, the first light receiving element 211 detects the light emitted by the first light emitting element 151, the second light emitting element 212 detects the light emitted by the second light emitting element 152, and the third light receiving element 213 detects the light emitted by the third light emitting element 153.

[0070] The difference calculation unit 22 calculates a difference value of the detection result of each light receiving element 21 between two adjacent calculation timings. In this embodiment, a synchronization signal indicating the calculation timing is input to the difference calculation unit 22, and the difference value of the light reception level of each light receiving element 21 is calculated between the two calculation timings indicated by the synchronization signal. Here, the light reception level is a multi-value luminance value, and is expressed in the range of "0 to 255", for example.

[0071] 9, the difference calculation unit 22 includes a delay unit 23 and a subtraction unit 24. In this embodiment, the difference calculation unit 22 includes three delay units 231-233 and three subtraction units 241-243 corresponding to the first light receiving element 211, the second light receiving element 212 and the third light receiving element 213.

[0072] Delay section 23 k In response to the sync signal input from the sync separator circuit 5, the kth light receiving element 21 k For example, the sync separation circuit 5 generates an identification signal from a black burst signal of NTSC as a sync signal and outputs it to the delay unit 23 as a sync signal. k 2. The sync separation circuit 5 may be integrated with the sync separation circuit 4 in FIG.

[0073] Subtraction section 24 k is expressed by the following formula (5): k The received light level J input from k (t) and delay unit 23 k The delayed light receiving level J k Difference value D from (t-1) k Calculate (t).

[0074]

number

[0075] The discrimination unit 25 detects the difference value D k Based on (t), the identification unit 25 recognizes an identifier unique to the detected light emission pattern, and includes a ternary value conversion unit 26 and an identifier determination unit 27. In this embodiment, the identification unit 25 includes three ternary value conversion units 261 to 263 corresponding to the first light receiving element 211, the second light receiving element 212 and the third light receiving element 213.

[0076] Ternary conversion section 26 k is the difference value D of the received light level input from the difference calculation unit 22k (t) is the positive / negative threshold p k ,q k In this embodiment, the threshold processing is expressed by the following equation (6). k is the positive threshold p k and a negative threshold q k Using the input difference value D k (t) is classified into three values, "0", "1", and "2", and the three-value classification E k (t) to the identifier determination unit 27. At this time, when the detection result of the light receiving element 21 indicates the light receiving level, the positive threshold p k Set the negative threshold q to "128". k Set it in advance to something like "-128".

[0077]

number

[0078] In addition, the three-value classification E k indicates a change from a light-on state to a light-off state, a change from a light-off state to a light-on state, and continuation of the light-off state or the light-on state. For example, the three-value classification E k In this example, "0" indicates a change from a light-on state to an off state, "1" indicates continuation of the off state or light-on state, and "2" indicates a change from an off state to a light-on state.

[0079] The identifier determination unit 27 is a ternarization unit 26 k The identifier determination unit 27 determines an identifier by associating the three values ​​classified by the unit 27 with each digit of the ternary number. The identifier determination unit 27 then outputs the determined identifier to an external device (for example, the image processing device 3).

[0080] Specifically, the identifier determination unit 27 obtains an integer value F(t) by regarding the ternary classifications E1(t) to E3(t) input from the ternarization units 261 to 263 as each digit of a ternary number, as shown in the following formula (7). That is, in this ternary number, the first digit represents the ternary classification E1(t), the second digit represents the ternary classification E2(t), and the third digit represents the ternary classification E3(t). As a result, the emission patterns indicated by the numbers F(t) and (26-F(t)) have their odd-numbered and even-numbered emission colors swapped with each other.

[0081]

number

[0082] Next, the identifier determination unit 27 estimates the identifier ID^ based on a function G that takes an integer value F(t) as an argument, as shown in the following equation (8).

[0083]

number

[0084] In this embodiment, the calculation timing of the difference calculation unit 22 is synchronized with the switching timing of the light-emitting unit 14, and it is possible to distinguish whether the light-emitting state of the light-emitting unit 14 is odd-numbered or even-numbered, so the function G(F(t)) may be any bijection. Furthermore, the function G(F(t)) may be a bijection, as shown in the following formula (9).

[0085]

number

[0086] In addition, when ID^(t)=0, it means "no identifier." In this way, the identifier determination unit 27 can distinguish whether the light emitting state of the light emitting unit 14 is an odd number or an even number, and determine 26 identifiers by corresponding to the light emitting pattern table 11 in FIG.

[0087] [Marker identification device operation] The operation of the marker identification device 2 will be described with reference to FIG. As shown in FIG. 10, in step S10, the marker identification device 2 initializes a variable k to 1. In step S11, the difference calculation unit 22 synchronizes with the synchronization signal and calculates the difference between the k-th light receiving element 21 k The light receiving level is variable J k Substitute into (t).

[0088] In step S12, the subtraction unit 24 k As shown in equation (5), the received light level J k (t),J k (t-1) difference value D k Calculate (t). In step S13, the delay unit 23 k is the light receiving level J k (t) is delayed by one period, and the received light level J k Let's call it (t-1).

[0089] In step S14, the ternarization unit 26 k is the difference value D k (t) is a negative threshold q k It is determined whether it is less than. Difference value D k (t) is a negative threshold q k If it is less than that (Yes in step S14), the marker identification device 2 proceeds to the process in step S16. Difference value D k (t) is a negative threshold q k If it is not less than this (No in step S14), the marker identification device 2 proceeds to the process of step S15.

[0090] In step S15, the ternarization unit 26 k is the difference value D k (t) is a positive threshold p k It is determined whether it exceeds the limit. Difference value D k (t) is a positive threshold p k If it exceeds (Yes in step S15), the marker identification device 2 proceeds to the process of step S17. Difference value Dk (t) is a positive threshold p k If it does not exceed (No in step S15), the marker identification device 2 proceeds to the process of step S18.

[0091] In step S16, the ternarization unit 26 k is a ternary classification E k Let (t) be “0”. In step S17, the ternarization unit 26 k is a ternary classification E k Let (t) be “2”. In step S18, the ternarization unit 26 k is a ternary classification E k Let (t) be “1”.

[0092] In step S19, the marker identification device 2 determines whether the variable k is "3" or not. If the variable k is "3" (Yes in step S19), the marker identification device 2 proceeds to the process of step S20. If the variable k is not "3" (No in step S19), the marker identification device 2 proceeds to the process of step S23.

[0093] In step S20, the identifier determination unit 27 performs a ternary classification E k Calculate the integer value F(t) by treating (t) as each digit of a ternary number. In step S21, the identifier determination unit 27 estimates the identifier ID^ based on a function G that takes an integer value F(t) as an argument, as shown in equation (8). In step S22, the identifier determination unit 27 outputs the estimated identifier ID^ to the outside, and then returns to the process of step S10.

[0094] In step S23, the marker identification device 2 increments the variable k, and then returns to the process of step S11.

[0095] [Actions and Effects] As described above, the marker identification device 2 calculates the difference in the light receiving level between two adjacent calculation timings, and therefore can robustly determine multiple identifiers. At this time, since all the light emitting elements 15 do not continue to be turned off, and each light emitting element 15 does not continue to be turned on, the marker identification device 2 can suppress the influence of the color and pattern of the measurement object α and external light when detecting the light emission pattern.

[0096] Furthermore, the marker identification device 2 emits a first emission color V (1) and second emission color V (2)) Since the order of the markers can be distinguished, the total number of identifiers that can be expressed by the light emission pattern can be doubled. At this time, the marker identification device 2 can use a conventional video synchronization signal such as the NTSC black burst signal, thereby improving convenience.

[0097] Third embodiment [Marker configuration] With reference to FIG. 11, the configuration of a marker 1A according to the third embodiment will be described with respect to differences from the first embodiment. The marker 1A differs from the first embodiment in that a clock signal is used instead of a synchronization signal. As shown in FIG. 11, the marker 1A includes a light emission pattern table storage unit 10A, a light emission pattern determination unit 12, a control unit 13A, a light emission unit 14, and a clock generation unit 16. Incidentally, the light emission pattern determination unit 12 and the light emission unit 14 are similar to those in the first embodiment, and therefore the description thereof will be omitted.

[0098] The light emission pattern table storage unit 10A is a storage device such as a memory that stores a light emission pattern table 11A (FIG. 12). (1) and second emission color V (2) Since the order of the first emission color V is indistinguishable, the emission pattern table 11A cannot contain numbers with the same integer value but different positive and negative signs. For this reason, the emission pattern table 11A contains the first emission color V (1) and second emission color V (2)Without distinguishing the order of the light emission patterns, 13 different light emission patterns can be set for the light emitting elements 15. In this embodiment, the identifiers are integer values ​​from "+1 to +13".

[0099] The control unit 13A controls the light emission pattern (V (1) ,V (2) ) to control turning on or off of each light-emitting element 15 in synchronization with a clock signal S(t) input from a clock generating unit 16. Note that the control unit 13A is similar to that of the first embodiment except that it uses the clock signal S(t) as a signal indicating the timing of switching between turning on and off each light-emitting element 15, and therefore further explanation will be omitted.

[0100] The clock generating unit 16 generates a clock signal S(t) that indicates an arbitrary switching timing. For example, a general oscillator circuit is used as the clock generating unit 16. The clock generating unit 16 outputs the generated clock signal S(t) to the control unit 13A.

[0101] Here, the first emission color V (1) and second emission color V (2) Here is a supplementary explanation of the case where the order of the images cannot be distinguished. The case where the order of the images cannot be distinguished is when the shooting timing of the shooting camera C and the timing of the blinking / color change of the light emitting element 15 are not synchronized, or when the video synchronization signal cannot distinguish between odd-numbered and even-numbered shot images even if both timings are synchronized. Therefore, the marker 1A is suitable for an environment where it is difficult to obtain a video synchronization signal.

[0102] [Actions and Effects] As described above, the marker 1A can robustly determine a plurality of identifiers by suppressing the influence of the color and pattern of the measurement object α and external light, similarly to the first embodiment. Furthermore, the marker 1A can be used even in an environment where it is difficult to obtain a video synchronization signal, thereby improving versatility.

[0103] (Fourth embodiment) With reference to FIG. 13, the configuration of a marker identification device 2A according to the fourth embodiment will be described with respect to differences from the second embodiment. The marker identification device 2A differs from the second embodiment in that a clock signal is used instead of a synchronization signal. As shown in FIG. 13, the marker identification device 2A includes a light receiving unit 20, a difference calculation unit 22A, a recognition unit 25A, and a clock generation unit . The means other than the delay unit 23A of the difference calculation unit 22A, the identifier determination unit 27A of the identification unit 25A, and the clock generation unit 28 are similar to those in the second embodiment, and therefore description thereof will be omitted.

[0104] Delay section 23A k The kth light receiving element 21 is driven in accordance with the clock signal input from the clock generating unit 28. k The delay unit 23A delays the received light level input from the k Since this is the same as the second embodiment except that a clock signal is used as a signal indicating the timing of calculating the difference value, further explanation will be omitted.

[0105] The identifier determination unit 27A is a ternarization unit 26 k The identifier is determined by associating the three values ​​for each of the spectral characteristics classified by V with each digit of the ternary number. (1) and second emission color V (2) Since the order of is indistinguishable, the function G(F(t)) is surjective and line-symmetric with respect to F=13. For example, the identifier determination unit 27A uses the function G(F(t)) expressed by the following formula (10).

[0106]

number

[0107] In addition, when ID^(t)=0, it means "no identifier." In this way, the identifier determination unit 27A can determine 13 identifiers in correspondence with the light emission pattern table 11A in FIG. 12 without distinguishing whether the light emission state of the light-emitting unit 14 is odd-numbered or even-numbered.

[0108] The clock generating section 28 generates a clock signal indicating an arbitrary calculation timing. The clock generating section 28 then transmits the generated clock signal to the delay section 23A. k 11. In this case, the frequency of the clock signal generated by clock generating unit 28 is equal to or higher than the frequency of the clock signal generated by clock generating unit 16 in FIG. 11. In other words, the calculation timing of difference calculating unit 22A is equal to or lower than the minimum value of the timing for switching the light emission state of marker 1A. Also, clock generating unit 28 may be common to clock generating unit 16 in FIG.

[0109] [Actions and Effects] As described above, the marker identification device 2A can robustly determine a plurality of identifiers by suppressing the influence of the color and pattern of the measurement object α and external light, similarly to the second embodiment. Furthermore, the marker identification device 2A can be used even in an environment where it is difficult to obtain a video synchronization signal, thereby improving versatility.

[0110] (Modification) Although each embodiment of the present invention has been described in detail above, the present invention is not limited to the above-described embodiments, and includes design modifications and the like within the scope of the gist of the present invention.

[0111] In the above-described embodiments, the marker identification device is described as being built into the photographing camera, but this is not limiting, and the marker identification device may be an independent device. In the above-described embodiments, the number of markers has been described as three, but the number of markers may be four or more. Furthermore, it goes without saying that the markers may be attached to measurement targets other than people.

[0112] In the above-described embodiments, the detection result of the light receiving element is described as a reception level (luminance value), but this is not limited to this. For example, the detection result of the light receiving element may be a binary value indicating whether or not light emission has been detected. Specifically, the detection result of the light receiving element is "0" when light emission cannot be detected, and "1" when light emission is detected. In this case, the positive threshold pk Set the negative threshold q to 0.5. k can be preset to "-0.5". Furthermore, the ternarization unit can eliminate the need for threshold processing by using the following equation (11) instead of equation (6).

[0113]

number

[0114] In the above-described embodiments, the markers are attached to the subject, but the present invention is not limited to this. As shown in Fig. 14, in a marker identification system 100B, a plurality of markers 1 may be attached to a wall 90 or a base 91 of a studio, and the position, attitude, and angle of view of a camera C may be calculated based on the detection results of each marker 1. In the marker identification system 100B in Fig. 14, the marker identification device 2 is independent from the camera C, but may be built into the camera C.

[0115] In the above-described embodiments, the marker and the marker identification device are described as hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, memory, and hard disk of a computer to operate as the marker and the marker identification device. These programs may be distributed via a communication line, or may be written on a recording medium such as a CD-ROM or a flash memory and distributed. [Explanation of symbols]

[0116] 1,1A marker 2,2A Marker Identification Device 3. Image Processing Device 4,5 Synchronous separation circuit 10,10A Light emission pattern table storage section 11,11A Light Emitting Pattern Table 12 Light emission pattern determination section 13, 13A Control section 14 Light emitting part 15 Light emitting element 16 Clock Generation Section 20 Light receiving section 21 Photodetector 22,22A Difference calculation section 23, 23A Delay section 24 Subtraction section 25,25A identification part 26 Trinarization section 27,27A Identifier judgment unit 28 Clock Generation Section 100 Marker Identification System C. Shooting camera α Measurement target

Claims

1. A marker that emits light in a light emission pattern corresponding to a unique identifier by turning on or off a light emitting element at a predetermined switching timing, a light-emitting unit including three light-emitting elements, a first light-emitting element, a second light-emitting element, and a third light-emitting element, each of which emits light with different spectral characteristics; a light emission pattern table storage unit in which a light emission pattern table is previously set, the light emission pattern being stored for each identifier, in which all the light emitting elements do not continue to be turned off and each light emitting element does not continue to be turned on over two adjacent switching timings; a light emission pattern determination unit that refers to the light emission pattern table and determines the light emission pattern corresponding to the identifier input from outside; a control unit that controls turning on or off each of the light-emitting elements at the switching timing based on the light-emitting pattern determined by the light-emitting pattern determination unit; Equipped with the light-emitting unit is capable of emitting light in a first emission color or a second emission color by combining a light-on state or a light-off state of the first light-emitting element, the second light-emitting element, and the third light-emitting element, the light emission pattern table storage unit is configured to store in advance the light emission pattern table in which the light emitting unit emits light in the first light emission color or the second light emission color; When the first light emitting element, the second light emitting element, and the third light emitting element are all turned off, the luminous color is K, When the first light emitting element is in a lit state and the second light emitting element and the third light emitting element are in an unlit state, the light emitting color is R, When the second light emitting element is in a lit state and the first light emitting element and the third light emitting element are in an unlit state, the luminous color is G. When the first light emitting element and the second light emitting element are in a light-on state and the third light emitting element is in a light-off state, the light emitting color is Y, When the first light emitting element and the second light emitting element are in an extinguished state and the third light emitting element is in an lit state, the light emitting color is B, When the first light emitting element and the third light emitting element are in a light-on state and the second light emitting element is in a light-off state, the light emitting color is M, When the second light emitting element and the third light emitting element are in a light-on state and the first light emitting element is in a light-off state, the light emitting color is C, When the first light emitting element, the second light emitting element, and the third light emitting element are all in a lit state, the luminous color is W, The light emission pattern table includes: For the identifier +1, the first luminous color is the luminous color K and the second luminous color is the luminous color R; For the identifier +2, the first luminous color is the luminous color R and the second luminous color is the luminous color G; For the identifier +3, the first luminous color is the luminous color K and the second luminous color is the luminous color G; For the identifier +4, the first luminous color is the luminous color K and the second luminous color is the luminous color Y; For the identifier +5, the first luminous color is the luminous color Y and the second luminous color is the luminous color B; For the identifier +6, the first luminous color is the luminous color G and the second luminous color is the luminous color B; For the identifier +7, the first luminous color is the luminous color G and the second luminous color is the luminous color M; For the identifier +8, the first luminous color is the luminous color R and the second luminous color is the luminous color B; For the identifier +9, the first luminous color is the luminous color K and the second luminous color is the luminous color B; For the identifier +10, the first luminous color is the luminous color K and the second luminous color is the luminous color M; For the identifier +11, the first luminous color is the luminous color R and the second luminous color is the luminous color C; For the identifier +12, the first luminous color is the luminous color K and the second luminous color is the luminous color C; A marker characterized in that the first light emitting color is preset to the light emitting color K and the second light emitting color is preset to the light emitting color W for the identifier +13.

2. The marker according to claim 1, characterized in that the control unit receives a synchronization signal indicating the switching timing, and controls the turning on or off of each of the light-emitting elements in synchronization with the synchronization signal based on the lighting pattern determined by the lighting pattern determination unit.

3. the synchronization signal is a video synchronization signal indicating an odd-numbered or even-numbered field or frame in a video signal, The marker according to claim 2, characterized in that the control unit receives the video synchronization signal and controls the turning on or off of each of the light-emitting elements in synchronization with the video synchronization signal based on the lighting pattern determined by the lighting pattern determination unit.

4. A clock generating unit that generates a clock signal indicating the switching timing, The marker according to claim 1 , wherein the control unit controls turning on or off of each of the light-emitting elements in synchronization with the clock signal based on the light-emitting pattern determined by the light-emitting pattern determination unit.

5. 1. A marker identification device that detects an illumination pattern of a marker and determines a unique identifier corresponding to the illumination pattern, comprising: a light receiving unit including a plurality of light receiving elements that receive light from the marker with different spectral characteristics; a difference calculation unit that calculates a difference value of the detection results of each light receiving element between two adjacent calculation timings; a ternarization unit that performs threshold processing on the difference value calculated by the difference calculation unit using positive and negative thresholds, and classifies the result of the threshold processing into three values ​​indicating a change from a light-on state to an extinction state, a change from the extinction state to the light-on state, and continuation of the extinction state or the light-on state; an identifier determination unit that determines the identifier by associating the three values ​​classified by the ternary value conversion unit with each digit of a ternary number; A marker identification device comprising:

6. The marker identification device according to claim 5, characterized in that the difference calculation unit receives a synchronization signal indicating the calculation timing and calculates a difference value of the light receiving level received by each of the light receiving elements between the two calculation timings indicated by the synchronization signal.

7. the synchronization signal is a video synchronization signal indicating an odd-numbered or even-numbered field or frame in a video signal, The marker identification device according to claim 6, characterized in that the difference calculation unit receives the video synchronization signal and calculates a difference value of the light receiving level received by each of the light receiving elements between the two calculation timings indicated by the video synchronization signal.

8. A clock generating unit that generates a clock signal indicating the calculation timing, 6. The marker identification device according to claim 5, wherein the difference calculation unit calculates a difference value of the light reception level received by each of the light receiving elements between the two calculation timings indicated by the clock signal.

9. 9. The marker identification device according to claim 5, wherein the light receiving section is a color light receiving element that receives light of three primary colors.

10. A program for causing a computer to function as the marker according to any one of claims 1 to 4.

11. A program for causing a computer to function as the marker identification device according to any one of claims 5 to 9.

Images