Image recognition system, and image recognition method
The image recognition system uses markers with a constant and blinking light-emitting section to efficiently identify and track objects, addressing cost and detection issues in existing systems, and improving tracking accuracy.
Patent Information
- Application Number
- JP2023077479
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-21
AI Technical Summary
Existing image recognition systems using luminescent markers with blinking patterns are costly due to the need for light-emitting devices that emit multiple colors, and blue light is difficult to detect, leading to potential tracking errors.
An image recognition system with markers that include a first light-emitting section always on and a second light-emitting section blinking in a chronological pattern, using a camera to capture images and a processor to detect and track marker positions, storing and linking this information with a storage device to identify and track objects.
The system allows for accurate identification and tracking of objects using markers with reduced costs by employing markers with a single color light-emitting section and a blinking section, enhancing detection and reducing installation constraints.
Smart Images

Figure 2024162177000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image recognition system and an image recognition method. [Background technology]
[0002] In order to track the location of people, objects, and other targets in sites where people and things move around, such as factories and power plants, a luminous marker that displays ID information for identifying the target with a blinking pattern has been proposed. By placing a luminous marker above a person's head or on an object, and taking an image containing the luminous marker with a camera installed on the ceiling to detect the blinking pattern of the luminous marker and its position, the target can be identified and its position tracked.
[0003] In order to detect the blinking pattern of a luminous marker, it is necessary to track the luminous marker for a certain period of time and detect the time series pattern of its on and off. However, when the luminous marker is off, it is not possible to detect the luminous marker, which may result in erroneous tracking of the luminous marker.
[0004] Thus, there is a technology in Patent Document 1 that shows ID information by changing colors instead of blinking lights. Patent Document 1 describes a technology that has a light-emitting body that emits at least three colors, periodically changes the colors emitted by the light-emitting body, switches the periodic color changes based on a predetermined ID, and shows the ID of the light-emitting device or the ID of the object to which the light-emitting device is attached by switching the periodic color changes. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2010-50720 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, the technology of Patent Document 1 requires the light-emitting device to be equipped with light-emitting bodies that emit at least three colors, which causes problems of cost.In addition, Patent Document 1 uses RGB (red, green, and blue) as the three colors, but B (blue) of RGB is dark and difficult to detect, making it unsuitable for a marker.
[0007] An object of the present invention is to provide a technique capable of identifying an object and tracking its position. [Means for solving the problem]
[0008] In order to solve the above-mentioned problems, a representative image recognition system of the present invention includes at least one marker installed on at least one object, and an image recognition device that detects the position of the marker, in which the marker includes a first light-emitting unit that is always on and a second light-emitting unit that flashes in a time series pattern of on and off set for each marker, and the image recognition device includes a camera that captures an image including the first light-emitting unit and the second light-emitting unit, a processor that detects the position of light included in the captured image as a marker detection position, and a storage device, and the processor groups the detected marker detection positions into marker detection groups consisting of one or two detected marker detection positions based on each position, and links the marker detection positions to information of the marker detection position group stored in the storage device based on the positions of the marker detection positions belonging to the marker detection position group, and stores the information of the marker detection position group in the storage device. Effect of the Invention
[0009] According to the present invention, it is possible to identify an object and track its location.
[0010] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]
[0011] [Figure 1]FIG. 1 illustrates an example of a configuration of an image recognition system according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of a hardware configuration of an image recognition apparatus according to a first embodiment. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of a marker according to the first embodiment. FIG. [Figure 4] 4 is a flowchart showing an example of the operation of the image recognition device according to the first embodiment. [Diagram 5] FIG. 2 illustrates an example of a marker tracking list stored in a primary storage device. [Figure 6] 13A and 13B are diagrams illustrating a process of extracting a matching marker detection position group. [Figure 7] 13 is a diagram showing an example of a marker ID list stored in a primary storage device. FIG. [Figure 8] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 9] 11 is a flowchart showing an example of the operation of the image recognition device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, the embodiments will be described with reference to the drawings.
[0013] Note that the embodiments described below do not limit the invention according to the claims, and not all of the elements and combinations described in the embodiments are necessarily essential to the solution of the invention. Also, illustrations and descriptions of well-known configurations that are essential to the configuration of the invention may be omitted. EXAMPLES
[0014] FIG. 1 is a diagram illustrating an example of a configuration of an image recognition system according to a first embodiment.
[0015] In FIG. 1, the image recognition system includes an image recognition device 100 and a marker 200.
[0016] The marker 200 has a first light-emitting unit 210 and a second light-emitting unit 211 on its upper surface. The marker 200 is installed, for example, above the head of the person 1 or on the top of the object 2, with the first light-emitting unit 210 and the second light-emitting unit 211 facing toward the ceiling.
[0017] The image recognition device 100 includes a camera 110 , an information processing device 130 , and a display 120 .
[0018] The camera 110 is installed on the ceiling of the room 3 so as to capture an image of the floor, and captures an image including a first light-emitting unit 210 and a second light-emitting unit 211 provided on the upper surface of a marker 200 located on the floor of the room 3.
[0019] The information processing device 130 is connected to the camera 110, acquires an image captured by the camera 110, and executes processing described below to detect the position of the marker 200 and a marker ID that identifies the marker 200. The display 120 is connected to the information processing device 130, and displays information output by the information processing device 130.
[0020] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the image recognition device 100 according to the first embodiment.
[0021] The image recognition device 100 includes a processor 101, a primary storage device 102, and a secondary storage device 103 inside an information processing device 130.
[0022] The processor 101 executes a program stored in the secondary storage device 103 to realize the processes described below.
[0023] The primary storage device 102 is a main storage device used when the processor 101 executes processing, and is composed of a volatile storage element such as a RAM (Random Access Memory).
[0024] The secondary storage device 103 is an auxiliary storage device for storing input data provided to the processor 101 and output data output from the processor 101, and is composed of a non-volatile storage element such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).
[0025] Programs and setting information for the operation of the processor 101 are stored in the secondary storage device 103, but are read into the primary storage device 102 during execution and are referred to by the processor 101.
[0026] FIG. 3 is a block diagram illustrating an example of a hardware configuration of the marker 200 according to the first embodiment.
[0027] The marker 200 includes a processor 201 , a primary memory device 202 , a secondary memory device 203 , a first light-emitting unit 210 , and a second light-emitting unit 211 .
[0028] The processor 201 realizes processing by executing a program stored in the secondary storage device 203 .
[0029] The primary storage device 202 is a main storage device used when the processor 201 executes processes, and is composed of a volatile storage element such as a RAM (Random Access Memory).
[0030] The secondary storage device 203 is an auxiliary storage device for storing input data provided to the processor 201 and output data output from the processor 201, and is composed of a non-volatile storage element such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).
[0031] Programs and setting information for the operation of the processor 201 are stored in the secondary storage device 203, but are read into the primary storage device 202 during execution and are referred to by the processor 201.
[0032] The first light-emitting section 210 and the second light-emitting section 211 are configured, for example, by LEDs (Light Emitting Diodes).
[0033] Here, the first light-emitting unit 210 is always on when the power is on, and the second light-emitting unit 211 blinks in a predetermined blinking pattern under the control of the processor 201. Since the first light-emitting unit is always on, the position of the camera 110 of the image recognition device 100 can be easily tracked even if the camera 110 has a frame rate of about 30 Hz.
[0034] The blinking pattern of the second light-emitting unit 211 differs depending on each marker 200. For example, the blinking pattern of the second light-emitting unit 211 differs between the marker 200 placed above the head of the person 1 and the marker 200 placed on the top of the object 2. By reading this blinking pattern, the image recognition device 100 can identify which marker 200 it is and identify the person or object whose position is being tracked.
[0035] In addition, in the first embodiment, the first light-emitting unit 210 and the second light-emitting unit 211 have the same color. In order to facilitate detection by the image recognition device 100, it is preferable to adopt a bright color other than blue, such as green or red, for the color of the first light-emitting unit 210 and the second light-emitting unit 211.
[0036] Furthermore, camera 110 is not limited to an RGB camera, and colors suitable for camera 110 may be selected for first light-emitting portion 210 and second light-emitting portion 211 .
[0037] The first light-emitting unit 210 and the second light-emitting unit 211 are spaced apart by a predetermined distance so that they can be distinguished on an image captured by the camera 110 of the image recognition device 100. This predetermined distance can be determined using general three-dimensional geometry, given the number of pixels and the angle of view of the camera 110 of the image recognition device 100, and the maximum distance from the marker 200.
[0038] FIG. 4 is a flowchart showing an example of the operation of the image recognition device 100 according to the first embodiment.
[0039] 4 shows a process in which the processor 101 of the image recognition device 100 captures the lighting positions of the first light-emitting unit 210 and the second light-emitting unit 211 and tracks the position of the marker 200. This process is performed by the processor 101 of the image recognition device 100 using the primary storage device 102 and the secondary storage device 103.
[0040] In step S300, the processor 101 performs binarization processing on the input frame acquired by the camera 110.
[0041] At this time, it is desirable to perform binarization processing according to the colors of the first light-emitting unit 210 and the second light-emitting unit. For example, if the color of the first light-emitting unit 210 and the second light-emitting unit is white, binarization processing is performed using the norm of RGB. Also, if the color of the first light-emitting unit 210 and the second light-emitting unit is G in RGB, it is advisable to extract only the G channel in RGB of the input frame and perform binarization processing.
[0042] The binarization process extracts a portion of the input frame in which either the first light-emitting unit 210 or the second light-emitting unit 211 that is lit up is captured. The processor 101 extracts the extracted portion as a marker detection position.
[0043] In step S301, the processor 101 groups the marker detection positions of the first light-emitting unit 210 and the second light-emitting unit 211 that belong to the same marker 200 from the marker detection positions extracted in step S300.
[0044] If there is another marker detection position within the first grouping predetermined range from the marker detection position extracted in step S300, those marker detection positions are grouped into a marker detection position group. On the other hand, if there is no another marker detection position within the first grouping predetermined range from the marker detection position extracted in step S300, only that marker detection position is made into a marker detection position group.
[0045] In an input frame captured when the blinking second light-emitting unit 211 is turned on, the marker detection position group includes two marker detection positions. On the other hand, in an input frame captured when the second light-emitting unit is turned off, the marker detection position group includes one marker detection position.
[0046] Here, the first predetermined grouping range is a range set for grouping the marker detection positions of the first light-emitting unit and the second light-emitting unit belonging to the same marker 200. If two marker detection positions are within the first predetermined grouping range, the two marker detection positions are determined to be the marker detection positions of the first light-emitting unit and the second light-emitting unit belonging to one marker 200.
[0047] The first predetermined range for grouping may be set to the same value as the maximum distance on the screen between the first light-emitting unit 210 and the second light-emitting unit 211 that belong to the same marker 200. This value can be calculated using general three-dimensional geometry once the angle of view of the camera 110 and the minimum distance between the first light-emitting unit 210 and the second light-emitting unit 211 and the camera 110 of the image recognition device 100 are determined.
[0048] What should be noted here are the minimum and maximum values of the distance from the camera 110 to the marker 200.
[0049] For example, if the camera 110 is mounted on the top of the wall of the room 3 so as to capture images in a diagonally downward direction, the marker 200 closer to the camera 110 will be captured large in the input frame, and the marker 200 farther from the camera 110 will be captured small in the input frame.
[0050] When the marker 200 and the camera 110 are very close, as described above, if the maximum distance on the screen between the first light-emitting unit 210 and the second light-emitting unit 211 belonging to the same marker 200 is taken as the first specified range for grouping, the first specified range for grouping will become very large, and will include most of the input frame.
[0051] As a result, the marker detection positions of multiple adjacent markers 200 fall within the first specified range for grouping, and the purpose of grouping the first light-emitting unit 210 and the second light-emitting unit 211 belonging to the same marker 200 cannot be achieved.
[0052] Furthermore, if the marker 200 and the camera 110 are very far away, the marker detection positions of multiple nearby markers 200 will fall within the first specified range for grouping, and the purpose of grouping the first light-emitting unit 210 and the second light-emitting unit 211 belonging to the same marker 200 cannot be achieved.
[0053] For this reason, it is necessary to devise a position for installing the camera 110 so that the minimum value of the distance from the camera 110 to the marker 200 is not too small and the maximum value is not too large.
[0054] For example, by installing camera 110 on the ceiling with the depression angle facing the floor, and installing marker 200 on person 1 walking on the floor or object 2 moving close to the floor, the minimum and maximum distances from camera 110 to marker 200 can be set to appropriate values. Also, if there is a wall close to camera 110, it is not necessary to greatly face the depression angle of camera 110 toward the wall in the direction of the wall.
[0055] This makes it possible to achieve the purpose of grouping the first light-emitting unit 210 and the second light-emitting unit 211 that belong to the same marker 200 in step S301.
[0056] The marker detection position groups grouped in step S301 and information indicating whether the second light-emitting unit is turned on or off are stored in the primary storage device 102 as a marker tracking list 400 shown in FIG.
[0057] FIG. 5 is a diagram showing an example of a marker tracking list stored in the primary storage device 102. As shown in FIG.
[0058] The marker tracking list 400 has a plurality of marker tracking records 410 distinguished by an index 401. Each marker tracking record 410 has a marker ID 402, a marker detection position group history 420, and a marker blink history 430.
[0059] In the marker detection position group history 420, the position information of the marker detection positions included in each marker detection position group in each input frame is stored in different columns.
[0060] For example, if the frame number of an input frame acquired by camera 110 is t and the frame number of an input frame n frames before frame t is tn, then the marker detection position group history 420 for frame t, index 0 stores position information (30, 130) of one marker detection position. This indicates that since the second light-emitting unit 211 was in an off state in frame t, one piece of marker position information was grouped as a marker detection position group in step 301 of Fig. 4.
[0061] Furthermore, the position information of two marker detection positions, (10, 110) and (10, 111), is stored in the marker detection position group history 420 of frame t-2, index 0. This indicates that since the second light-emitting unit 211 was in a lit state in frame t-2, the two marker detection positions were grouped into a marker detection position group in step 301 of Fig. 4.
[0062] Information indicating whether the second light-emitting unit 211 belonging to each marker detection position group in each input frame is turned on or off is stored in different columns in the marker blink history 430. For example, the marker blink history 430 for frame t, index 0 records "off," and the marker blink history 430 for frame t-2, index 0 records "on."
[0063] The marker ID 402 is determined based on the time-series pattern of lighting and extinguishing (blinking pattern) of the marker blinking history 430 in a process described below.
[0064] 4, in step S302, for each marker detection position group history 420 of the latest frame in each marker tracking record 410, a marker detection position group located within a predetermined range for tracking is extracted as a matching marker detection position group, and the two are linked together. In addition, marker detection position groups other than the matching marker detection position group are set as non-matching marker detection position groups.
[0065] The predetermined range for tracking is set in order to link the marker detection position group of the input frame with the marker detection position group history 420 of the same marker 200 in the marker tracking record 410 in order to track the position of the same marker 200. The predetermined range for tracking is set based on the distance that the person 1 or object 2 is expected to move during one frame in the input frames.
[0066] FIG. 6 is a diagram for explaining the process of extracting the matching marker detected position group in step S302.
[0067] 6 , the position of the marker detection position group history 420 of the latest frame of each marker tracking record 410 is 503. Moreover, the marker detection position group located within a predetermined range for tracking from the position 503 of the marker detection position group history 420 of the latest frame is a matching marker detection position group 501. Moreover, the marker detection position groups other than the matching marker detection position group 501 are non-matching marker detection position groups 502.
[0068] The marker detection position group and the marker detection position group history of the latest frame may include two marker detection positions corresponding to the first light-emitting unit 210 and the second light-emitting unit 211, respectively. In that case, in order to calculate the distance between the marker detection position group to be compared with the predetermined range for tracking and the marker detection position group history of the latest frame, multiple values are obtained depending on which of the two marker detection positions is used for the calculation. When multiple values are obtained as the distance between the marker detection position group and the marker detection position group history of the latest frame, any of the minimum, maximum, or average value of those values may be used. In addition, the predetermined range for tracking is set to an appropriate value depending on the value to be used.
[0069] Returning to FIG. 4, in step S303, information on the matching marker detection position group is written to the marker tracking record 410 to which the associated marker detection position group history 420 of the latest frame belongs.
[0070] Specifically, the marker detection positions included in the matching marker detection position group are written into the marker detection position group history 420 of frame t of the marker tracking record 410 to which the marker detection position group history 420 of the latest frame linked to the matching marker detection position group belongs. In addition, if there are two marker detection positions, "on" is written into the marker blink history 430 of frame t, and if there is one marker detection position, "off" is written into the marker blink history 430 of frame t.
[0071] In step S304, a new marker tracking record 410 is added to the marker tracking list 400, and information on the non-matching marker detection position group is written to the added marker tracking record 410.
[0072] Specifically, for the added marker tracking record 410, the marker detection positions included in the non-matching marker detection position group are written into the marker detection position group history 420 of frame t. Also, if there are two marker detection positions, "on" is written into the marker blink history 430 of frame t, and if there is one marker detection position, "off" is written into the marker blink history 430 of frame t.
[0073] In step S305, if there is a marker tracking record 410 for which a marker ID 402 has not been determined, the corresponding marker blink history 430 is referenced and a marker ID candidate is calculated based on the time series pattern of lighting and extinguishing (blink pattern).
[0074] To calculate the ID from the time series pattern of on and off (blinking pattern), for example, the method of Matsushita et al. (Nobuyuki Matsushita, Daisuke Hihara, Teruyuki Ushiro, Shinichi Yoshimura, Junichi Rekimoto, “ID Cam: A smart camera capable of simultaneously acquiring the scene and ID,” Transactions of the Information Processing Society of Japan, vol. 43, no. 12, pp. 3664-3674, 2002.) can be used. The method of Matsushita et al. uses a camera with a frame rate of 200 Hz, but it can also be applied to the frame rate of 30 Hz assumed in the present invention.
[0075] Next, the processes from step S350 to step S351 are repeated until the processes are completed for all the marker ID candidates.
[0076] In step S306, it is determined whether or not there is a marker tracking record (hereinafter referred to as a "marker tracking record with the same marker ID") other than the marker tracking record from which the marker ID candidate was calculated (hereinafter referred to as the "original marker tracking record") that has the same marker ID 402 as the marker ID candidate. The original marker tracking record and the marker tracking record with the same marker ID have been tracking the same marker 200.
[0077] If it is determined in step S306 that a marker tracking record with the same marker ID is found, the process proceeds to step S307.
[0078] In step S307, the original marker tracking record is merged with a marker tracking record having the same marker ID. Merging means adding the marker detection position group history 420 and marker blink history 430 written in the original marker tracking record to the marker detection position group history 420 and marker blink history 430 of the marker tracking record having the same marker ID.
[0079] For example, if the marker detection position group history 420 and marker blink history 430 of the original marker tracking record were written from frame t to frame tn, then in the marker detection position group history 420 and marker blink history 430 of the marker tracking record with the same marker ID, frames t to tn are blank, so they are added there.
[0080] On the other hand, if there is no marker tracking record with the same marker ID in step S306, the process proceeds to step S308. In step S308, it is determined whether the calculated marker ID candidate matches any of the assigned IDs 602 in the marker ID list 600 stored in the primary storage device 102.
[0081] FIG. 7 is a diagram showing an example of a marker ID list stored in the primary storage device 102. As shown in FIG.
[0082] The marker ID list 600 is a list in which IDs assigned to the markers 200 are registered as assigned IDs 602. Each assigned ID 602 is distinguished by an index 601. Each assigned ID 602 is linked to a unique blinking pattern of the second light-emitting unit 211.
[0083] By writing the allocation ID 602 to the secondary storage device 203 of the marker 200 as setting information, the processor 201 of the marker 200 controls the second light-emitting unit 211 to emit light in a blinking pattern linked to the allocation ID 602.
[0084] In step S308, if the calculated marker ID candidate matches any of the assigned IDs 602 in the marker ID list 600, the process proceeds to step S309. In step S309, the value of the marker ID candidate is written as the marker ID 402 of the original marker tracking record.
[0085] On the other hand, if the calculated marker ID candidate does not match any of the assigned IDs 602 in the marker ID list 600 in step S308, the process proceeds to step S310. In step S310, since some error occurred in the creation of the original marker tracking record, the original marker tracking record is erased and an error message is displayed on the display 120 of the image recognition device 100.
[0086] If the processes from step S350 to step S351 have been completed for all marker ID candidates, the process proceeds to step S352.
[0087] In step S352, the processor 101 of the image recognition device 100 displays the position of the marker 200 detected by the image recognition device 100 and its marker ID 402 on the display 120 based on the marker tracking list 400 stored in the primary memory device 102, and then ends this processing.
[0088] FIG. 8 is a diagram showing an example of a screen displayed on display 120. As shown in FIG.
[0089] FIG. 8 shows an example of a screen 800 displayed on the display 120, in which a position 801 of a marker 200 whose marker ID 402 is 5 and a position 802 of a marker 200 whose marker ID 402 is 8 are displayed.
[0090] The processor 101 displays the position of the marker 200 in the input frame acquired by the camera 110 and the marker ID 402 of the marker 200 on the screen 800 based on the marker detection position group history 420 of the latest frame of the marker tracking list 400 and the marker ID 402 of the marker 200.
[0091] The name of the person 1 (e.g., "A") who places the marker 200 and the name of the object 2 (e.g., "Drill 5") may be linked to the marker ID 402 and pre-registered in the primary storage device 102, so that the name of the person 1 or the name of the object 2 whose position has been detected can be displayed on the screen 800.
[0092] According to the first embodiment, the marker 200 includes a first light-emitting unit 210 that is always on and a second light-emitting unit 211 that blinks in a predetermined blinking pattern, and the image recognition device 100 extracts the marker detection positions of the first light-emitting unit 210 and the second light-emitting unit 211, and calculates a marker ID candidate based on the blinking pattern of the second light-emitting unit 211. This makes it possible to identify the target object and track its position. In addition, since the first light-emitting unit is always on, its position can be easily tracked.
[0093] In addition, since the first light-emitting unit 210 and the second light-emitting unit 211 emit light in the same color, costs can be reduced. Furthermore, by adopting bright colors for the first light-emitting unit 210 and the second light-emitting unit 211, the marker 200 can be easily detected. EXAMPLES
[0094] Example 2 differs from Example 1 in that the first light-emitting unit 210 and the second light-emitting unit 211 of the marker 200 emit light in different colors. Therefore, in Example 2, even if the first light-emitting unit 210 and the second light-emitting unit 211 are installed close to each other on the upper part of the marker 200, the image recognition device 100 can detect the light of the first light-emitting unit 210 and the second light-emitting unit 211 as different lights. The first light-emitting unit 210 and the second light-emitting unit 211 may be implemented by, for example, a single element having two colors.
[0095] In order to facilitate detection by image recognition device 100, it is preferable to adopt a bright color other than blue, such as green or red, for the colors of first light-emitting unit 210 and second light-emitting unit 211.
[0096] Furthermore, camera 110 is not limited to an RGB camera, and colors suitable for camera 110 may be selected for first light-emitting portion 210 and second light-emitting portion 211 .
[0097] Moreover, invisible light may be used as the light emitted by the blinking second light-emitting portion 211. This can reduce discomfort caused by the blinking light.
[0098] FIG. 9 is a flowchart illustrating an example of the operation of the image recognition device 100 according to the second embodiment.
[0099] In Fig. 9, the processes in steps S700 to S702 are different from those in Fig. 4. The same steps as those in Fig. 4 are given the same reference numerals and their explanation will be omitted.
[0100] In step S700, the processor 101 performs binarization processing on the input frame acquired by the camera 110 according to the color of the first light-emitting unit 210 in order to extract the position of the first light-emitting unit 210.
[0101] For example, if the color of the first light-emitting unit 210 is G in RGB, only the G channel in RGB of the input frame is extracted and binarized. Also, if the color of the first light-emitting unit 210 is a color in which GB in RGB is combined, binarization is performed using the norm of the GB channel of the input frame.
[0102] The binarization process extracts a portion in the input frame in which the first light-emitting unit 210 is captured. The processor 101 extracts the extracted portion as a marker detection position based on the first light-emitting unit.
[0103] Similarly, in step S701, the processor 101 performs binarization processing on the input frame acquired by the camera 110 according to the color of the second light-emitting unit 211 in order to extract the position of the second light-emitting unit 211.
[0104] The binarization process extracts a portion of the input frame in which a lit-up portion of second light-emitting unit 211 is captured. Processor 101 extracts the extracted portion as a marker detection position based on the second light-emitting unit.
[0105] In step S702, the processor 101 groups the marker detection positions of the first light-emitting unit 210 and the second light-emitting unit 211 that belong to the same marker 200 from the marker detection positions extracted in step S300.
[0106] If there is a marker detection position based on the second light-emitting unit 211 extracted in step S701 within the second grouping predetermined range from the marker detection position based on the first light-emitting unit 210 extracted in step S700, those marker detection positions are grouped to form a marker detection position group. On the other hand, if there is not a marker detection position based on the second light-emitting unit 211 extracted in step S701 within the second grouping predetermined range from the marker detection position based on the first light-emitting unit 210 extracted in step S700, only that marker detection position is formed into a marker detection position group.
[0107] In an input frame captured when the blinking second light-emitting unit 211 is turned on, the marker detection position group includes two marker detection positions. On the other hand, in an input frame captured when the second light-emitting unit is turned off, the marker detection position group includes one marker detection position.
[0108] Here, the second predetermined grouping range is a range set for grouping the marker detection positions of the first light-emitting unit and the second light-emitting unit belonging to the same marker 200. If two marker detection positions are within the second predetermined grouping range, the two marker detection positions are determined to be the marker detection positions of the first light-emitting unit and the second light-emitting unit belonging to one marker 200.
[0109] The second predetermined range for grouping may be set to the same value as the maximum distance on the screen between the first light-emitting unit 210 and the second light-emitting unit 211 that belong to the same marker 200. This value can be calculated using general three-dimensional geometry once the angle of view of the camera 110 and the minimum distance between the first light-emitting unit 210 and the second light-emitting unit 211 and the camera 110 of the image recognition device 100 are determined.
[0110] Here, in the second embodiment, the first light-emitting unit 210 and the second light-emitting unit 211 emit light in different colors, so that even if the first light-emitting unit 210 and the second light-emitting unit 211 are installed close to each other on the top of the marker 200, the image recognition device 100 can detect the light of the first light-emitting unit 210 and the second light-emitting unit 211 as different light. Therefore, compared to the first grouping predetermined range in the first embodiment, the second grouping predetermined range can be set very small.
[0111] For this reason, the second embodiment has the following advantages over the first embodiment: In the first embodiment, when the distance from the image recognition device 100 to the marker 200 becomes small, the first grouping predetermined range occupies most of the input frame.
[0112] In the second embodiment, such an event is unlikely to occur because the second grouping predetermined range is smaller. In other words, in the second embodiment, the distance from the image recognition device 100 to the marker 200 can be made shorter than in the first embodiment, and the degree of freedom in the installation position of the camera 110 can be improved.
[0113] In addition, if the first light-emitting section 210 and the second light-emitting section 211 are implemented as a single element with two colors, the first light-emitting section 210 and the second light-emitting section 211 will be in the same position, so the grouping process of step S702 is not necessary.
[0114] After the process of step S702, the processes from step S302 onward in FIG. 4 are executed.
[0115] According to the second embodiment, the marker 200 includes a first light-emitting unit 210 that is always on and a second light-emitting unit 211 that blinks in a predetermined blinking pattern, and the image recognition device 100 extracts the marker detection positions of the first light-emitting unit 210 and the second light-emitting unit 211, and calculates a marker ID candidate based on the blinking pattern of the second light-emitting unit 211. This makes it possible to identify an object and track its position. In addition, since the first light-emitting unit is always on, its position can be easily tracked.
[0116] Furthermore, since the first light-emitting unit 210 and the second light-emitting unit 211 emit light in different colors, the distance from the image recognition device 100 to the marker 200 can be made shorter, thereby improving the freedom of the installation position of the camera 110.
[0117] Furthermore, by adopting bright colors for the first light-emitting unit 210 and the second light-emitting unit 211, it is possible to easily detect the marker 200. Furthermore, by adopting invisible light for the light emitted by the second light-emitting unit 211, it is possible to reduce discomfort caused by flashing light.
[0118] The present invention is not limited to the above-described embodiment, but includes various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]
[0119] 100 Image Recognition Device 101 Processor 102 Primary storage 110 Camera 200 Markers 210 First light emitting part 211 Second light emitting part
Claims
1. An image recognition system including at least one marker installed on at least one object and an image recognition device that detects the position of the marker, The marker is a first light-emitting portion that is always lit; a second light-emitting unit that blinks in a time-series pattern of lighting and extinguishing set for each of the markers; The image recognition device includes: a camera that captures an image including the first light-emitting unit and the second light-emitting unit; A processor that detects a position of light included in the captured image as a marker detection position; A storage device, The processor, grouping the detected marker detection locations into marker detection groups each consisting of one or two of the detected marker detection locations based on their respective positions; An image recognition system that stores information about the marker detection position group in the storage device, linking the information to the marker detection position group stored in the storage device based on the positions of the marker detection positions belonging to the marker detection position group.
2. 2. The image recognition system according to claim 1, The processor, When there is one marker detection position belonging to the marker detection position group, information indicating that the second light-emitting unit is turned off is stored in the storage device; When there are two marker detection positions belonging to the marker detection position group, information indicating that the second light-emitting unit is lit is stored in the storage device.
3. 3. The image recognition system according to claim 2, The processor, An image recognition system that calculates an ID that identifies the marker based on a time series pattern of information indicating the lighting of the second light-emitting unit and information indicating the extinguishing of the second light-emitting unit, which is stored in the storage device.
4. 4. The image recognition system according to claim 3, An image recognition system in which the first light-emitting unit and the second light-emitting unit emit light of different colors.
5. 5. The image recognition system according to claim 4, The second light-emitting unit is an image recognition system that emits invisible light.
6. An image recognition method for an image recognition system including at least one marker installed on at least one object and an image recognition device that detects the position of the marker, comprising: The marker is a first light-emitting portion that is always lit; a second light emitting unit that blinks in a time series pattern of lighting and extinguishing set for each of the markers, The image recognition device includes: a camera that captures an image including the first light-emitting unit and the second light-emitting unit; A processor that detects a position of light included in the captured image as a marker detection position; A storage device, The processor, grouping the detected marker detection locations into marker detection groups each consisting of one or two of the detected marker detection locations based on their respective positions; An image recognition method that links information of the marker detection position group stored in the storage device based on the positions of marker detection positions belonging to the marker detection position group, and stores information of the marker detection position group in the storage device.
7. 7. The image recognition method according to claim 6, The processor, When there is one marker detection position belonging to the marker detection position group, information indicating that the second light-emitting unit is turned off is stored in the storage device; When there are two marker detection positions belonging to the marker detection position group, information indicating that the second light-emitting unit is turned on is stored in the storage device.
8. 8. The image recognition method according to claim 7, The processor, An image recognition method for calculating an ID that identifies the marker based on a time series pattern of information indicating the lighting of the second light-emitting unit and information indicating the extinguishing of the second light-emitting unit, which is stored in the storage device.
9. 9. The image recognition method according to claim 8, The image recognition method, wherein the first light-emitting unit and the second light-emitting unit emit light of different colors.
10. 10. The image recognition method according to claim 9, The second light-emitting unit emits invisible light.
Citation Information
Patent Citations
Light emitting device and method of tracking object
JP2010050720A
