Marker, information processing device, and program

By superimposing markers with different encoding types and wavelength bands, the information processing device efficiently detects multiple information types from a single marker, addressing the challenge of increased marker numbers and maintaining accuracy in mobile object control.

JP7584124B2Active Publication Date: 2024-11-15TOHOKU UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2020169115
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-10-06
Publication Date
2024-11-15
Estimated Expiration
2040-10-06

AI Technical Summary

Technical Problem

Existing information processing devices face challenges in reading large amounts of information from markers without increasing the number of markers, which can lead to reduced detection accuracy and increased area occupation, especially when multiple markers of the same encoding type are superimposed in different hues.

Method used

The use of markers that are superimposed with different encoding types and reflect electromagnetic waves in distinct wavelength bands, allowing for the detection of multiple information types without increasing the number of markers, by employing an information processing device with a control unit that can differentiate and combine the information from these markers.

Benefits of technology

This approach enables accurate detection of multiple information types from a single marker, reducing the number of markers needed and maintaining high detection rates, thus enabling precise control of mobile objects like drones or robots.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584124000001
    Figure 0007584124000001
  • Figure 0007584124000002
    Figure 0007584124000002
  • Figure 0007584124000003
    Figure 0007584124000003
Patent Text Reader

Abstract

To provide a marker that enables precise detection of more pieces of information from a picked-up image in which the marker is imaged while suppressing an increase in the number of the markers.SOLUTION: The marker includes a first marker representing first information and a second marker representing second information, which are superimposed on each other. The first marker reflects electromagnetic waves with a predetermined first wavelength band, and represents the first information by a predetermined first kind of encoding. The second marker reflects electromagnetic waves with a predetermined second wavelength band different from the first wavelength band, and represents the second information by a second kind of encoding different from the first kind.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a marker, an information processing device, and a program. [Background technology]

[0002] Research and development is being conducted on information processing devices that read information indicated by markers included in an image captured by an imaging unit included in a moving object and control the moving object. The moving object may be, for example, a drone or a robot.

[0003] In this regard, a QR code (registered trademark) is known as a marker that indicates information read by an information processing device (see Patent Document 1). Another such marker is the ArUco marker (see Non-Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 07-254037 [Non-patent literature]

[0005] [Non-Patent Document 1] S. Garrido-Jurado, et al., "Generation of fiducial marker dictionaries using mixed integer linear programming," Pattern Recognition, Volume 51, March 2016, Pages 481-491. Summary of the Invention [Problem to be solved by the invention]

[0006] Here, the more complex the operation of the moving object, the more the amount of information that the information processing device reads from the marker tends to increase. And when the amount of information that the information processing device is to read is large, one marker may not be able to show all of the information. To solve this, there is a method of having the information processing device read information indicated by each of two or more markers. However, with this method, the more information that the information processing device is to read increases, the more the number of markers increases. This leads to an increase in the area occupied by the markers, which is not desirable. As a method for suppressing the increase in the number of markers, a method is also known in which two or more markers having the same type of encoding are superimposed with different hues to form one marker. However, with this method, the detection rate of information indicated by any of the two or more markers superimposed as one marker may decrease. As a result, there are cases in which the information processing device is unable to operate the moving object with high accuracy.

[0007] The present invention has been made in consideration of the above circumstances, and has an objective of providing a marker, an information processing device, and a program that can accurately detect more information from an image in which the markers are captured, while suppressing an increase in the number of markers. [Means for solving the problem]

[0008] The present invention includes the following aspects. [1] A marker in which a first marker indicating first information and a second marker indicating second information are superimposed, the first marker reflecting electromagnetic waves in a predetermined first wavelength band and indicating the first information by a predetermined first type of encoding, and the second marker reflecting electromagnetic waves in a second wavelength band different from the first wavelength band and indicating the second information by a second type of encoding different from the first type. [2] At least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light, and a hue of the first marker is different from a hue of the second marker. [3] One of the hue of the first marker and the hue of the second marker is a hue located at a first vertex of three vertices of an equilateral triangle inscribed in a specified hue circle, and the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex of the three vertices, which is different from the first vertex, in a specified hue circle. [4] The first marker is an ArUco marker. [5] The second marker is a QR code. [6] In addition to the first marker and the second marker, one or more markers are superimposed. [7] An information processing device comprising: a control unit that detects the first information and the second information based on a captured image in which the marker described above is captured. [8] The control unit detects the first information indicated by the first marker based on the captured image, and detects a contour of the first marker, and detects the second information indicated by the second marker based on the detected contour of the first marker and the captured image. [9] An information processing device comprising: a control unit that detects the first information and the second information based on a captured image in which the marker described above is captured.

[10] The control unit detects the first information indicated by the first marker based on the captured image, and detects a contour of the first marker, and detects the second information indicated by the second marker based on the detected contour of the first marker and the captured image.

[11] The information processing device further includes an imaging unit having a lens to which a filter that transmits at least one of electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, and the control unit detects the first information and the second information based on the captured image captured by the imaging unit.

[12] The information processing device further includes a communication unit that communicates with a moving body, the moving body including an imaging unit having a lens to which a filter that transmits at least one of electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, the first information being information indicating a relative position and attitude between the imaging unit and the marker, and the second information being information indicating a position and attitude of the marker in a predetermined coordinate system, the control unit detecting the first information and the second information based on the captured image captured by the imaging unit, calculating a position and attitude of the moving body in the predetermined coordinate system based on the detected first information and second information, and controlling the moving body via the communication unit based on the calculated position and attitude of the moving body.

[13] The moving object is a drone or a robot.

[14] After detecting the first information and the second information, the control unit stores in the memory unit history information that corresponds information indicating the first marker from which the first information was detected and the detected second information.

[15] A program for causing a computer to execute a detection step of detecting the first information and the second information based on an image in which the marker described above is captured. Effect of the Invention

[0009] According to the present invention, it is possible to accurately detect more information from a captured image in which the markers are captured, while suppressing an increase in the number of markers. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of a configuration of a mobile object control system 1 according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of a marker MK. [Diagram 3] 13A and 13B are diagrams showing other examples of the configuration of the marker MK. [Figure 4] 13 is a diagram showing yet another example of the configuration of the marker MK. FIG. [Diagram 5] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device 20. [Figure 6] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device 20. [Figure 7] 1 is a diagram showing an example of a process flow in which the information processing device 20 controls the moving object 10. FIG. [Figure 8] 1 is a diagram showing an example of the relationship between a trajectory along which a moving object 10 is moved and a position calculated by an information processing device 20. FIG. [Figure 9] FIG. 9 is a diagram showing an example of the relationship in the height direction, out of the relationships shown in FIG. 8. [Figure 10] FIG. 1 is a diagram showing an example of an information processing device 20 including an imaging section 11 having a filter FT attached to the lens. [Figure 11] 13 is a diagram showing an example of a captured image of a marker MKA captured by the imaging section 11 when a filter FT is not attached to the imaging section 11. FIG. [Figure 12] 13 is a diagram showing an example of a captured image of a marker MKA captured by the imaging unit 11 when a filter FT that transmits only light in the red wavelength band is attached to the imaging unit 11. FIG. [Figure 13] 13 is a diagram showing an example of a captured image of a marker MKA captured by the imaging unit 11 when a filter FT that transmits only light in the blue wavelength band is attached to the imaging unit 11. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <Embodiment> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the embodiment, the marker is a figure, a mark, etc. that indicates various information by a predetermined type of encoding. The marker is, for example, an ArUco marker, a QR code, a barcode, a concentric circle marker, etc., but is not limited to these. Here, the ArUco marker, the QR code, the barcode, and the concentric circle marker are markers with different types of encoding. In the embodiment, a plurality of markers with different types of encoding indicate various information by different types of encoding. In the embodiment, a plurality of markers with the same type of encoding indicate various information by the same type of encoding. For example, the ArUco marker and the QR code are markers with different types of encoding. In addition, for example, two ArUco markers are markers with the same type of encoding.

[0012] <Configuration of the mobile control system> Hereinafter, the configuration of a mobile object control system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a mobile object control system 1 according to an embodiment.

[0013] The mobile object control system 1 includes a mobile object 10 and an information processing device 20. The mobile object 10 includes an imaging unit 11. The mobile object control system 1 may include other devices in addition to the mobile object 10 and the information processing device 20. In the mobile object control system 1, the mobile object 10 may be integrated with the information processing device 20. In the mobile object control system 1, the mobile object 10 may have some of the functions of the information processing device 20 described below.

[0014] The moving object control system 1 controls the moving object 10 by an information processing device 20. For example, the moving object control system 1 moves the moving object 10 along a predetermined trajectory under the control of the information processing device 20.

[0015] The moving body 10 may be any object as long as it is possible to move a part or the whole of the moving body 10 under the control of the information processing device 20. In the example shown in Fig. 1, the moving body 10 is a drone. Note that the moving body 10 may be a robot or the like instead of a drone.

[0016] The mobile object 10 is wirelessly connected to the information processing device 20 so as to be able to communicate with it. Note that the mobile object 10 may be configured to be wired and communicatively connected to the information processing device 20.

[0017] The imaging unit 11 is a camera equipped with an imaging element that converts focused electromagnetic waves into an electrical signal. In the following, as an example, a case will be described in which the imaging unit 11 is a camera equipped with a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like as an imaging element that converts focused visible light into an electrical signal. The imaging unit 11 is connected to the information processing device 20 wirelessly so as to be able to communicate with it. Note that the imaging unit 11 may be configured to be connected to the information processing device 20 wirelessly so as to be able to communicate with it.

[0018] The imaging unit 11 can capture an image of a range according to the position and attitude of the moving object 10. The imaging unit 11 captures an image of a range that the imaging unit 11 can capture under the control of the information processing device 20. The imaging unit 11 transmits the captured image to the information processing device 20.

[0019] The information processing device 20 is, for example, a multi-function mobile phone terminal (smartphone). Note that the information processing device 20 may be other information processing devices such as a tablet PC (Personal Computer), a notebook PC, a PDA (Personal Digital Assistant), a mobile phone terminal, a desktop PC, a workstation, etc., instead of a multi-function mobile phone terminal.

[0020] The information processing device 20 causes the imaging unit 11 to capture an image of a range including the marker MK. The marker MK is, for example, a marker that indicates information indicating the relative position and orientation of the imaging unit 11 and the marker MK, and information indicating the position and orientation of the marker MK in a predetermined coordinate system. The predetermined coordinate system is, for example, a world coordinate system. Note that the predetermined coordinate system may be another coordinate system that is stationary with respect to the marker MK.

[0021] The information processing device 20 detects information indicated by the marker MK based on an image including the marker MK among the images captured by the imaging unit 11, for example. The information processing device 20 calculates the position and attitude of the moving body 10 in a predetermined coordinate system based on the detected information. The information processing device 20 moves the moving body 10 along the trajectory based on the calculated position and attitude of the moving body 10 and trajectory information stored in advance. The trajectory information is information indicating the above-mentioned predetermined trajectory. Note that the information processing device 20 may be configured to make the moving body 10 perform another operation based on the captured image including the marker MK. In this case, the information processing device 20 may be configured to calculate the position and attitude of the moving body 10 in the predetermined coordinate system, or may be configured not to calculate the position and attitude of the moving body 10 in the predetermined coordinate system.

[0022] Here, the more complex the operation that the moving object 10 is made to perform based on the information indicated by the marker included in the captured image, the more the amount of information that the information processing device 20 reads from the marker tends to increase. And, when the amount of information that the information processing device 20 is made to read is large, one marker may not be able to indicate all of the information. To solve this, there is a method of making the information processing device 20 read information indicated by each of two or more markers. However, with this method, the number of markers increases as the amount of information that the information processing device 20 is made to read increases. As a method for suppressing the increase in the number of markers, a method is also known in which two or more markers having the same encoding type are superimposed with different hues to form one marker. However, with this method, the detection rate of information indicated by any of the two or more markers superimposed as one marker may decrease. In this case, the information processing device 20 may not be able to operate the moving object 10 with high accuracy.

[0023] Therefore, the marker MK is a marker on which markers indicating two or more pieces of information are superimposed. However, the two or more markers superimposed on the marker MK include at least markers indicating information by two different types of encoding. Therefore, for example, the two or more markers superimposed on the marker MK may be markers with different types of encoding. Also, for example, the two or more markers superimposed on the marker MK may include one or more markers indicating information by one of the two different types of encoding and one or more markers indicating information by the other of the two different types of encoding. Also, each of the two or more markers superimposed on the marker MK is an electromagnetic wave of a different wavelength band and is a marker that reflects an electromagnetic wave of a single predetermined wavelength band. Therefore, in the embodiment, it is not assumed that a color code is used as the two or more markers. In the following, as an example, a case will be described where the marker MK is a marker on which a first marker MK1 and a second marker MK2 are superimposed. The first marker MK1 is a marker that reflects an electromagnetic wave of a predetermined first wavelength band and indicates a predetermined first information by a predetermined first type of encoding. The second marker MK2 is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates a predetermined second information by a second type of encoding different from the first type. As a result, the marker MK can indicate each of the first information and the second information as one marker without decreasing the detection rate of each of the two pieces of information, the first information and the second information. As a result, the marker MK can accurately detect more information from a captured image in which the marker MK is captured while suppressing an increase in the number of markers. In other words, the information processing device 20 can accurately detect more information from a captured image in which the marker MK is captured while suppressing an increase in the number of markers. In this example, the first information is information indicating the relative position and attitude of the imaging unit 11 and the marker MK. In addition, in this example, the second information is information indicating the position and attitude of the marker MK in a predetermined coordinate system.In this case, the markers MK can move the moving body 10 along a predetermined trajectory with high accuracy while suppressing an increase in the number of markers. That is, in this case, the information processing device 20 can move the moving body 10 along a predetermined trajectory with high accuracy while suppressing an increase in the number of markers.

[0024] The first marker MK1 may be a marker indicating information other than the information indicating the relative position and orientation of the imaging unit 11 and the marker MK as the first information. The second marker MK2 may be a marker indicating information other than the information indicating the position and orientation of the marker MK in a predetermined coordinate system as the second information. The marker MK may be configured such that one or more other markers different from the first marker MK1 and the second marker MK2 are superimposed on the marker MK together with the first marker MK1 and the second marker MK2. Some or all of the one or more other markers may be markers indicating various information by a type of encoding different from the type of encoding of the first marker MK1 and the second marker MK2, or may be markers indicating various information by a type of encoding the same as the type of encoding of the first marker MK1 and the second marker MK2. Some or all of the one or more markers may be markers having different types of encoding from each other, or may be markers having the same type of encoding from each other.

[0025] Moreover, the first marker MK1 has a first marker portion that reflects electromagnetic waves in a predetermined first wavelength band. The first marker MK1 may have a configuration having a first background portion that becomes the background of the first marker MK1 in addition to the first marker portion, or may have no first background portion. The first background portion may be configured to reflect electromagnetic waves in a wavelength band different from the second wavelength band, or may be a portion that hardly reflects electromagnetic waves. The first marker MK1 may have a configuration having an additional first additional portion different from each of the first marker portion and the first background portion in addition to the first marker portion, or may have no first additional portion. However, this portion reflects electromagnetic waves in a wavelength band different from each of the first wavelength band and the second wavelength band. The second marker MK2 has a second marker portion that reflects electromagnetic waves in a second wavelength band different from the first wavelength band. The second marker MK2 may have a configuration having a second background portion that becomes the background of the second marker MK2 in addition to the second marker portion, or may have no second background portion. The second background portion may be configured to reflect electromagnetic waves in a wavelength band different from the first wavelength band, or may be a portion that hardly reflects electromagnetic waves. The second marker MK2 may be configured to have, in addition to the second marker portion, an additional second additional portion different from each of the second marker portion and the second background portion, or may be configured to not have the second additional portion. However, the second additional portion reflects electromagnetic waves in a wavelength band different from each of the first wavelength band and the second wavelength band.

[0026] In the following, as an example, a case where the first waveband and the second waveband are visible light wavebands will be described. In this case, the electromagnetic waves of the first waveband and the electromagnetic waves of the second waveband are visible light. The fact that the first waveband and the second waveband are different wavebands and that the first waveband and the second waveband are visible light wavebands means that the hue of the first marker is different from the hue of the second marker. Here, the hue of the first marker refers to the hue of the first marker portion. The hue of the second marker refers to the hue of the second marker portion. At least one of the first waveband and the second waveband may be a waveband different from the visible light waveband. For example, at least one of the first waveband and the second waveband may be an infrared waveband. In addition, the second waveband may be configured to overlap a part of the first waveband, or may not overlap the entire first waveband.

[0027] In the following, the configuration of such a marker MK, the configuration of the information processing device 20, and the process of controlling the moving object 10 by the information processing device 20 will be described in detail.

[0028] <Marker MK configuration>

[0029] The configuration of the marker MK will be described below. Fig. 2 is a diagram showing an example of the configuration of the marker MK. In Fig. 2, a marker MKA is shown as an example of the marker MK. The marker MKA is a marker in which a first marker MK1A, which is an example of the first marker MK1, and a second marker MK2A, which is an example of the second marker MK2, are superimposed. In Fig. 2, differences in hue are indicated by differences in hatching.

[0030] The first marker MK1A is an ArUco marker. That is, the first marker MK1A is an example of the first marker MK1 having a first marker portion and not having a first background portion and a first additional portion. The first marker MK1A is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1A is red. Note that the first marker MK1A may be any marker that indicates the first information by a type of encoding different from the second type, instead of the ArUco marker. Also, the first marker MK1A does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, and may be interpreted as being removed when superimposed on the second marker MK2A.

[0031] The second marker MK2A is a QR code. That is, the second marker MK2A is an example of the second marker MK2 having a second marker portion and a second background portion, and not having a second additional portion. The second marker MK2A is a marker with a green hue. That is, the hue of the second marker portion of the second marker MK2A is green. In the example shown in FIG. 2, the hue of the second background portion is black, which hardly reflects electromagnetic waves (in FIG. 2, the second background portion is not hatched to prevent the drawing from becoming complicated). The second marker MK2A may be any marker, instead of a QR code, as long as it indicates the second information by a type of encoding different from the first type.

[0032] As shown in FIG. 2, in the marker MKA, the hue of the first marker MK1A is different from the hue of the second marker MK2A. That is, in the marker MKA, the hue of the first marker portion of the first marker MK1A is different from the hue of the second marker portion and the second background portion of the second marker MK2A. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1A and the second information indicated by the second marker MK2A based on the captured image in which the marker MKA is captured. Here, in the marker MKA, the wider the width of the thinnest line among the lines constituting the first marker portion, the higher the detection rate of the first information from the first marker MK1A by the information processing device 20. Also, in the marker MKA, the wider the width of the thinnest line among the lines constituting the second marker portion, the higher the detection rate of the second information from the second marker MK2A by the information processing device 20. The width L1A indicated by the arrow shown in FIG. 2 is an example of the width of the thinnest line among the lines constituting the first marker portion. Moreover, the width L2A indicated by the arrow in Fig. 2 is an example of the width of the thinnest line among the lines constituting the second marker portion. In the example shown in Fig. 2, the width L1A is thicker than the width L2A. That is, in this example, the detection rate of the first information from the first marker MK1A by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2A by the information processing device 20.

[0033] Fig. 3 is a diagram showing another example of the configuration of the marker MK. In Fig. 3, a marker MKB is shown as an example of the marker MK. The marker MKB is a marker in which a first marker MK1B, which is an example of the first marker MK1, and a second marker MK2B, which is an example of the second marker MK2, are superimposed. In Fig. 3, differences in hue are indicated by differences in hatching.

[0034] The first marker MK1B is a concentric marker. That is, the first marker MK1B is an example of the first marker MK1 having a first marker portion and not having a first background portion and a first additional portion. The first marker MK1B is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1B is red. Note that the first marker MK1B may be any marker that indicates the first information by a type of encoding different from the second type, instead of a concentric marker. Note that the first marker MK1B does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, and may be interpreted as being removed when superimposed on the second marker MK2B.

[0035] The second marker MK2B is a barcode. That is, the second marker MK2B is an example of the second marker MK2 having a second marker portion and a second background portion, and not having a second additional portion. The second marker MK2B is a marker with a blue hue. That is, the hue of the second marker portion of the second marker MK2B is blue. In the example shown in FIG. 3, the hue of the second background portion is purple (in FIG. 3, the second background portion is not hatched to prevent the drawing from becoming complicated). The second marker MK2B may be any marker that indicates the second information by a type of encoding different from the first type, instead of a barcode.

[0036] As shown in FIG. 3, in the marker MKB, the hue of the first marker MK1B is different from the hue of the second marker MK2B. That is, in the marker MKB, the hue of the first marker portion of the first marker MK1B is different from the hue of the second marker portion and the second background portion of the second marker MK2B. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1B and the second information indicated by the second marker MK2B based on the captured image in which the marker MKB is captured. Here, the width L1B indicated by the arrow shown in FIG. 3 is an example of the width of the thinnest line among the lines constituting the first marker portion. Also, the width L2B indicated by the arrow shown in FIG. 3 is an example of the width of the thinnest line among the lines constituting the second marker portion. In the example shown in FIG. 3, the width L1B is thicker than the width L2B. That is, in this example, the detection rate of the first information from the first marker MK1B by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2B by the information processing device 20.

[0037] FIG. 4 is a diagram showing yet another example of the configuration of the marker MK. In FIG. 4, a marker MKC is shown as an example of the marker MK. The marker MKC is a marker in which a first marker MK1C, which is an example of the first marker MK1, a second marker MK2C, which is an example of the second marker MK2, and a third marker MK3C, which is a third marker, are superimposed. In the following, for convenience of explanation, the type of encoding of the third marker MK3C will be referred to as a third type. In the following, for convenience of explanation, the information indicated by the third marker MK3C will be referred to as third information. In FIG. 4, differences in hue are indicated by differences in hatching.

[0038] The first marker MK1C is an ArUco marker. That is, the first marker MK1C is an example of the first marker MK1 having a first marker portion and not having a first background portion and a first additional portion. The first marker MK1C is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1C is red. Note that the first marker MK1C may be any marker that indicates the first information by a type of encoding different from each of the second type and the third type, instead of the ArUco marker. Also, the first marker MK1C does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, and may be interpreted as being removed when superimposed on the second marker MK2C.

[0039] The second marker MK2C is a QR code. That is, the second marker MK2C is an example of the second marker MK2 having a second marker portion and a second background portion, and not having a second additional portion. The second marker MK2C is a marker with a blue hue. That is, the hue of the second marker portion of the second marker MK2C is blue. In the example shown in FIG. 4, the hue of the second background portion is black, which hardly reflects electromagnetic waves (in FIG. 4, the second background portion is not hatched to prevent the drawing from becoming complicated). The second marker MK2C may be any marker, instead of a QR code, as long as it indicates the second information by a type of encoding different from each of the first and third types.

[0040] The third marker MK3C is a character code. The third marker MK3C has a character portion and does not have a background portion. The third marker MK3C is a marker with a green hue. That is, the hue of the character portion of the third marker MK3C is green. The third marker MK3C may be any marker that indicates the third information by a type of encoding different from each of the first and second types instead of a character code. The third marker MK3C does not necessarily have no background portion, and the portion may be interpreted as being transparent, or the portion may be interpreted as being removed when superimposed on the first marker MK1C and the second marker MK2C.

[0041] As shown in FIG. 4, in the marker MKC, the hue of the first marker MK1C is different from the hue of the second marker MK2C, and the hue of the third marker MK3C. That is, in the marker MKC, the hue of the first marker portion of the first marker MK1B is different from the hue of each of the second marker portion and the second background portion of the second marker MK2B, and the hue of the character portion of the third marker MK3C. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1C, the second information indicated by the second marker MK2C, and the third information indicated by the third marker MK3C based on the captured image in which the marker MKC is captured. Here, the width L1B indicated by the arrow shown in FIG. 4 is an example of the width of the thinnest line among the lines constituting the first marker portion. Also, the width L2C indicated by the arrow shown in FIG. 4 is an example of the width of the thinnest line among the lines constituting the second marker portion. Moreover, the width L3C indicated by the arrow in FIG. 4 is an example of the width of the thinnest line among the lines constituting the character portion of the third marker MK3C. In the example shown in FIG. 4, the width L1C is wider than the width L2C and the width L3C. Moreover, the width L2C is wider than the width L3C. That is, in this example, the detection rate of the first information from the first marker MK1C by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2C by the information processing device 20. Moreover, in this example, the detection rate of the second information from the second marker MK2C by the information processing device 20 is higher than the detection rate of the third information from the third marker MK3C by the information processing device 20.

[0042] In this way, the marker MK is a marker in which two or more markers with different types of encoding are superimposed. As a result, the marker MK can indicate two or more pieces of information. As a result, the information processing device 20 can detect two or more pieces of information from one marker MK. This leads to a reduction in the area and volume occupied by the marker. As a result, the marker MK can indicate two or more pieces of information and can increase the degree of freedom in installation. In addition, when markers indicating various pieces of information by existing types of encoding are used as the first marker MK1 and the second marker MK2 in the marker MK, existing algorithms and libraries can be used as algorithms and libraries for detecting the marker MK. In addition, for example, when a third marker such as the marker MKC is superimposed on the first marker MK1 and the second marker MK2, information indicating a detection method of the first marker MK1 can be made to be displayed on the third marker. In this case, instead of a marker indicating various pieces of information by existing types of encoding, a marker indicating various pieces of information by a newly developed type of encoding can be easily used as the first marker MK1. Furthermore, the information processing device 20 that detects the first information and the second information from the marker MK can improve the detection success rate of the information indicated by the marker that is detected later, either the first marker MK1 or the second marker MK2, based on the contour of the marker that is detected earlier, either the first marker MK1 or the second marker MK2, as described later. Note that the number of markers superimposed as the marker MK may be four or more.

[0043] <Hardware configuration of information processing device 20> Hereinafter, the hardware configuration of the information processing device 20 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the hardware configuration of the information processing device 20.

[0044] The information processing device 20 includes, for example, a CPU (Central Processing Unit) 21, a storage unit 22, an input receiving unit 23, a communication unit 24, and a display unit 25. These components are connected to each other via a bus so that they can communicate with each other. The information processing device 20 also communicates with each of the moving object 10 and the imaging unit 11 via the communication unit 24.

[0045] The CPU 21 is, for example, a processor that controls the entire information processing device 20. The CPU 21 may be another processor such as an FPGA (Field-Programmable Gate Array). The CPU 21 executes various programs stored in the storage unit 22.

[0046] The storage unit 22 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), etc. Note that the storage unit 22 may be an external storage device connected via a digital input / output port such as a universal serial bus (USB) instead of being built into the information processing device 20. The storage unit 22 stores various information, various images, various programs, etc. that are processed by the information processing device 20.

[0047] The input receiving unit 23 is an input device such as a keyboard, a mouse, a touch pad, etc. The input receiving unit 23 may be a touch panel that is integrated with the display unit 25.

[0048] The communication unit 24 includes, for example, an antenna, a digital input / output port such as a USB, an Ethernet (registered trademark) port, and the like.

[0049] The display unit 25 is a display device including, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel.

[0050] <Functional configuration of information processing device 20> The functional configuration of the information processing device 20 will be described below with reference to Fig. 6. Fig. 6 is a diagram showing an example of the functional configuration of the information processing device 20.

[0051] The information processing device 20 includes a storage unit 22, an input receiving unit 23, a communication unit 24, a display unit 25, and a control unit 26.

[0052] The control unit 26 controls the entire information processing device 20. The control unit 26 includes, for example, an imaging control unit 261, an image processing unit 262, a position and orientation calculation unit 263, and a moving body control unit 264. These functional units included in the control unit 26 are realized, for example, by the CPU 21 executing various programs stored in the storage unit 22. Furthermore, some or all of the functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

[0053] The imaging control unit 261 causes the imaging unit 11 to capture an image of the range that the imaging unit 11 can capture.

[0054] The image processing unit 262 acquires the captured image captured by the imaging unit 11 from the imaging unit 11. The image processing unit 262 performs various image processing based on the acquired captured image. For example, the image processing unit 262 detects each of the first marker MK1 and the second marker MK2 included in the captured image based on the acquired captured image. Also, for example, the image processing unit 262 detects first information indicated by the detected first marker MK1. Also, for example, the image processing unit 262 detects second information indicated by the detected second marker MK2.

[0055] The position and orientation calculation section 263 calculates the position and orientation of the moving object 10 in a predetermined coordinate system based on the result of the image processing by the image processing section 262 .

[0056] The moving object control unit 264 controls the moving object 10 based on the position and orientation of the moving object 10 calculated by the position and orientation calculation unit 263 .

[0057] <Processing by the information processing device 20 to control the moving object 10> Hereinafter, a process in which the information processing device 20 controls the moving body 10 will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a flow of a process in which the information processing device 20 controls the moving body 10. In the following, as an example, a case where the above-mentioned trajectory information is stored in the storage unit 22 at a timing before the processing of step S110 shown in FIG. 7 is performed will be described. In addition, in the following, as an example, a case where the position and attitude of the moving body 10 in a predetermined coordinate system at that timing coincide with a predetermined start position and a predetermined start attitude will be described. The predetermined start position is a position where the position of the moving body 10 coincides at a timing when the control of the moving body 10 by the information processing device 20 starts. The predetermined start attitude is a attitude where the attitude of the moving body 10 coincides at a timing when the control of the moving body 10 by the information processing device 20 starts. The position of the moving body 10 is represented, for example, by the position of the origin of a local coordinate system associated with the moving body 10 so as to move together with the moving body 10, among positions in a world coordinate system. The attitude of the moving body 10 is represented, for example, by the direction in which each coordinate axis of the local coordinate system faces among the directions in the world coordinate system. The position of the moving body 10 may be represented by another position according to the moving body 10. The attitude of the moving body 10 may be represented by another direction according to the moving body 10. In the following, as an example, a case where the marker MKA shown in FIG. 2 is used as the marker MK will be described. In the following, as an example, a case where the position and attitude of the moving body 10 match a predetermined start position and a predetermined start attitude, and the marker MKA is included in a range where the imaging unit 11 can capture images will be described.

[0058] The position and orientation calculation unit 263 reads out, from the storage unit 22, the trajectory information that is stored in advance in the storage unit 22 (step S110).

[0059] Next, the imaging control unit 261 causes the imaging unit 11 to capture an image of the range that the imaging unit 11 can capture (step S120). As a result, the imaging unit 11 captures an image of the range including the marker MKA. Then, the image processing unit 262 receives the captured image captured by the imaging unit 11 from the imaging unit 11.

[0060] Next, the image processing unit 262 performs image separation based on the captured image received from the imaging unit 11 in step S120 (step S130). Here, image separation refers to image processing that separates the hue components of the first marker MK1A and the hue components of the second marker MK2A from the captured image to generate a first image including the first marker MK1A and a second image including the second marker MK2A.

[0061] Next, the image processing unit 262 detects the first information indicated by the first marker MK1A included in the first image based on the first image generated by image separation in step S130 (step S140). In this example, the first information is information indicating the relative position and orientation between the imaging unit 11 and the marker MKA as described above. In this example, the imaging unit 11 is provided in the moving body 10. Therefore, the position and orientation of the imaging unit 11 can be approximately treated as the position and orientation of the moving body 10. That is, in step S140, the image processing unit 262 detects information indicating the relative position and orientation between the moving body 10 and the marker MKA as the first information. Note that the method of detecting the first information indicated by the first marker MK1A may be a known method or a method to be developed in the future.

[0062] Next, the image processor 262 detects the outline of the first marker MK1A included in the first image based on the first image (step S150). Note that the method for detecting the outline of the first marker MK1A may be a known method or a method to be developed in the future.

[0063] Next, the image processing unit 262 performs projective transformation of the image of the second marker MK2A included in the second image generated in step S130 based on the contour detected in step S150 (step S160). Here, the processing of step S160 will be described. As described above, the second marker MK2A has a lower detection rate compared to the first marker MK1A. Therefore, when the marker MKA is captured in the captured image from an oblique angle, the second marker MK2A may not be detected. Therefore, the image processing unit 262 performs projective transformation of the image of the second marker MK2A included in the second image into an image of the second marker MK2A viewed from the front based on the contour. Note that the method of performing such projective transformation may be a known method or a method to be developed in the future.

[0064] Next, the image processing unit 262 detects the second information indicated by the second marker MK2A based on the second image after the projective transformation in step S160 (step S170). In this example, the second information is information indicating the position and orientation of the marker MKA in a predetermined coordinate system, as described above. That is, in step S170, the image processing unit 262 detects information indicating the position and orientation of the marker MKA in a predetermined coordinate system as the second information. Note that the method of detecting the second information may be a known method or a method to be developed in the future.

[0065] Next, the position and orientation calculation section 263 calculates the position and orientation of the moving object 10 in a predetermined coordinate system based on the first information detected in step S140 and the second information detected in step S170 (step S180).

[0066] Next, based on the position and posture of the moving body 10 calculated in step S180 and the trajectory information read from the memory unit 22 in step S110, the moving body control unit 264 moves the moving body 10 along the trajectory indicated by the trajectory information (step S190), and terminates the processing.

[0067] As described above, the information processing device 20 detects the first information and the second information from the markers MKA, and controls the moving body 10 based on the detected first information and second information. This allows the information processing device 20 to accurately detect more information from the captured image in which the markers MKA are captured, while suppressing an increase in the number of markers.

[0068] Here, FIG. 8 is a diagram showing an example of the relationship between the trajectory along which the moving body 10 has been moved and the position calculated by the information processing device 20. The horizontal axis of the graph shown in FIG. 8 indicates the position in the X-axis direction in a predetermined coordinate system. The vertical axis of the graph shown in FIG. 8 indicates the position in the Y-axis direction in a predetermined coordinate system. The solid line in the graph indicates the trajectory along which the moving body 10 has been moved. The plots in the graph are positions calculated by the information processing device 20 based on the processing of the flowchart shown in FIG. 7, and indicate the position of the moving body 10 at each time while the moving body 10 is being moved along the trajectory. As shown in FIG. 8, the information processing device 20 can accurately calculate the position of the moving body 10 on the XY plane of the predetermined coordinate axes based on the captured image in which the marker MK is captured.

[0069] FIG. 9 is a diagram showing an example of the relationship in the height direction among the relationships shown in FIG. 8. The horizontal axis of the graph shown in FIG. 9 indicates the distance from the origin of a predetermined coordinate axis in the horizontal direction. The vertical axis of the graph indicates the position in the Z-axis direction in a predetermined coordinate system. The solid line in the graph indicates the trajectory of the moving body 10. The plots in the graph are positions calculated by the information processing device 20 based on the processing of the flowchart shown in FIG. 7, and indicate the position of the moving body 10 at each time while the moving body 10 is moving along the trajectory. As shown in FIG. 9, the information processing device 20 can accurately calculate the position of the moving body 10 in the Z-axis direction of the predetermined coordinate axis based on the captured image in which the marker MK is captured.

[0070] <First Modification of the Embodiment> The information processing device 20 described above may be configured integrally with the imaging unit 11. In this case, the information processing device 20 may be configured to control the moving body 10, or may not be configured to control the moving body 10. In this case, the imaging unit 11 has, for example, a lens to which a filter FT (e.g., a bandpass filter or the like) that transmits electromagnetic waves in at least one of a first wavelength band and a second wavelength band through the lens can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal. Here, FIG. 10 is a diagram showing an example of an information processing device 20 including the imaging unit 11 in which a filter FT is attached to the lens.

[0071] When such a filter FT is attached to the imaging section 11, the information processing device 20 can capture each of the first and second images described above without performing the image separation described above.

[0072] For example, Fig. 11 is a diagram showing an example of a captured image in which the marker MKA is captured by the imaging unit 11 when the filter FT is not attached to the imaging unit 11. The image P1 shown in Fig. 11 is an example of the captured image. As shown in Fig. 11, in this case, the first marker MK1A and the second marker MK2A superimposed as the marker MKA can each be seen in Fig. 11. That is, in this case, the information processing device 20 must perform image separation when detecting the first information and the second information.

[0073] On the other hand, Fig. 12 is a diagram showing an example of a captured image in which the marker MKA is captured by the imaging unit 11 when a filter FT that transmits only the red wavelength band is attached to the imaging unit 11. An image P2 shown in Fig. 12 is an example of the captured image. In this case, in Fig. 12, the second marker MK2A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA can be seen, and the first marker MK1A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA cannot be seen. That is, in this case, the information processing device 20 can detect the second information based on the captured image without performing image separation.

[0074] 13 is a diagram showing an example of a captured image of the marker MKA captured by the imaging unit 11 when a filter FT that transmits only the blue wavelength band is attached to the imaging unit 11. An image P3 shown in FIG. 13 is an example of the captured image. In this case, in FIG. 13, the first marker MK1A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA can be seen, and the second marker MK2A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA cannot be seen. That is, in this case, the information processing device 20 can detect the first information based on the captured image without performing image separation.

[0075] Here, if the wavelength band of light transmitted by the filter FT can be changed, the information processing device 20 can detect the first information and the second information from the marker MKA without performing image separation, as described above. This means that the information processing device 20 can detect two or more pieces of information from the marker MK without installing a new application program, since existing application programs capable of detecting each of the first marker MK1A and the second marker MK2A are installed. Therefore, when acquiring information using the marker MK, the user of the information processing device 20 can select between a method of attaching the filter FT to the information processing device 20 and performing detection using an existing application program, and a method of performing detection using a new application program capable of performing image separation. This leads to improved user convenience, and is preferable.

[0076] <Modification 2 of the embodiment> The filter FT described above may be configured to be attached to the imaging unit 11 provided in the moving object 10. In this case, the information processing device 20 can omit the process of step S130 in the flowchart shown in FIG. 7. However, in this case, in step S120, the information processing device 20 causes the imaging unit 11 to capture a range that the imaging unit 11 can capture, and receives each of the first image and the second image from the imaging unit 11, every time the information processing device 20 changes the wavelength band of light transmitted by the filter FT. As a result, even if the information processing device 20 is configured not to perform image separation, it can accurately detect more information from the captured image in which the markers MK are captured, while suppressing an increase in the number of markers.

[0077] <Modification 3 of the embodiment> The information processing device 20 described above may be configured to detect the first information and the second information, and then store in the storage unit 22 history information that associates the information indicating the first marker MK1 in which the first information is detected with the detected second information. As a result, the information processing device 20 can read out the second information associated with the information indicating the first marker MK1 in which the first information is detected from the storage unit 22 after detecting the first information at the second detection of the marker MK. That is, in this case, the information processing device 20 does not need to perform a process of detecting the second information. As a result, the information processing device 20 can reduce the load of the process of the flowchart shown in FIG. 7 and can speed up the process.

[0078] <Selection of the hue of the first and second markers and detection accuracy> In the above-described information processing device 20, when both the first wavelength band and the second wavelength band are visible light wavelength bands, the greater the difference between the hue of the first marker MK1 and the hue of the second marker MK2, the more accurately the information processing device 20 can detect the first information and the second information. Therefore, by selecting these two hues as described below, it is possible to prevent the information processing device 20 from detecting the first information and the second information with reduced accuracy.

[0079] First, one of the hues of the first marker MK1 and the second marker MK2 is assigned to an arbitrary hue in a predetermined hue circle. In the following, for convenience of explanation, the assigned hue is referred to as the first hue. Then, an equilateral triangle inscribed in the hue circle is considered. This triangle is inscribed in the hue circle so that one of the vertices of the triangle is located in the first hue. Then, the hue in which one of the two vertices of the triangle other than the vertex located in the first hue is located is assigned as the other of the hue of the first marker MK1 and the hue of the second marker MK2. In this way, by selecting the hue of the first marker MK1 and the hue of the second marker MK2, the information processing device 20 can suppress a decrease in the detection accuracy of the first information and the second information by the information processing device 20. Note that the hue of the first marker MK1 and the hue of the second marker MK2 may be selected by another method.

[0080] The information processing device 20 described above may be configured to control a device that does not have a moving part under the control of the information processing device 20, instead of the moving body 10. In this case, the device may be configured, for example, to have the imaging unit 11 attached to the device so that the imageable range can be changed, or to have the imaging unit 11 attached to the device so that the imageable range cannot be changed.

[0081] The marker MK described above may be attached to the moving object 10. In this case, the imaging unit 11 is installed at a position where it can capture an image of a range including the range in which the moving object 10 moves. One of the first marker MK1 and the second marker MK2 indicates the first marker MK1 indicating individual information of the moving object 10. The other of the first marker MK1 and the second marker MK2 indicates information indicating the relative position and attitude of the imaging unit 11 and the marker MK as information indicating the relative position and attitude of the imaging unit 11 and the moving object 10. The information processing device 20 controls the moving object 10 based on these two pieces of information detected from the marker MK. Such a control method is often applied, for example, when the information processing device 20 controls (controls) each moving object included in a group of multiple moving objects. In addition, the process flow for detecting the first information and the second information in such a control method may be, for example, the process flow of steps S120 to S170 shown in FIG. 7, or may be another process flow capable of detecting the first information and the second information.

[0082] As described above, the markers according to the embodiment (in the above-described example, the markers MK, MKA, MKB, and MKC) are markers in which a first marker indicating the first information (in the above-described example, the first marker MK1, the first marker MK1A, the first marker MK1B, and the first marker MK1C) and a second marker indicating the second information (in the above-described example, the second marker MK2, the second marker MK2A, the second marker MK2B, and the second marker MK2C) are superimposed, and the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding (in the above-described example, for example, an ArUco marker), and the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type (in the above-described example, for example, a QR code). This makes it possible to accurately detect more information from a captured image in which the markers are captured while suppressing an increase in the number of markers.

[0083] In addition, the marker may be configured such that at least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light, and the hue of the first marker is a different hue from the hue of the second marker.

[0084] In addition, the marker may be configured such that one of the hue of the first marker and the hue of the second marker is a hue located at a first vertex of three vertices of an equilateral triangle inscribed in a specified hue circle, and the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex different from the first vertex of the three vertices in the specified hue circle.

[0085] Also, in the marker, a configuration may be used in which the first marker is an ArUco marker.

[0086] Also, a configuration may be used in which the second marker is a QR code.

[0087] Furthermore, the marker may have a configuration in which, in addition to the first marker and the second marker, one or more markers (the third marker MK3C in the example described above) are superimposed.

[0088] Moreover, the information processing device according to the embodiment (information processing device 20 in the example described above) includes a control unit (control unit 26 in the example described above) that detects the first information and the second information based on a captured image in which the above-described markers are captured. This allows the information processing device to cause the moving object to perform a highly accurate operation while suppressing an increase in the number of markers.

[0089] In addition, in the information processing device, a configuration may be used in which the control unit detects first information indicated by a first marker based on an captured image, detects a contour of the first marker, and detects second information indicated by a second marker based on the detected contour of the first marker and the captured image.

[0090] The information processing device may further include an imaging unit (in the example described above, imaging unit 11) having a lens to which a filter (in the example described above, filter FT) that transmits electromagnetic waves in at least one of the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, and the control unit may be configured to detect the first information and the second information based on an image captured by the imaging unit.

[0091] In addition, the information processing device may further include a communication unit (in the example described above, communication unit 24) that communicates with a moving body (in the example described above, moving body 10), and the moving body may include an imaging unit having a lens to which a filter that transmits electromagnetic waves in at least one of a first wavelength band and a second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, the first information is information indicating the relative position and attitude of the imaging unit and the marker, the second information is information indicating the position and attitude of the marker in a predetermined coordinate system (in the example described above, a world coordinate system), and the control unit detects the first information and the second information based on an image captured by the imaging unit, calculates the position and attitude of the moving body in the predetermined coordinate system based on the detected first information and second information, and controls the moving body via the communication unit based on the calculated position and attitude of the moving body.

[0092] In addition, the information processing device may be configured such that the moving body is a drone or a robot.

[0093] In addition, in the information processing device, a configuration may be used in which, after detecting the first information and the second information, the control unit stores in a memory unit (memory unit 22 in the example described above) history information that associates information indicating the first marker from which the first information was detected with the detected second information.

[0094] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and may be changed, replaced, deleted, etc. without departing from the gist of the present invention.

[0095] Also, a program for realizing the function of any of the components in the above-described device (e.g., the information processing device 20, etc.) may be recorded in a computer-readable recording medium, and the program may be read and executed by a computer system. Note that the term "computer system" as used herein includes hardware such as an OS (Operating System) and peripheral devices. The term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), and a CD (Compact Disk)-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the term "computer-readable recording medium" refers to a storage device that holds a program for a certain period of time, such as a volatile memory (RAM (Random Access Memory)) inside a computer system that serves as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0096] The above program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may be for implementing some of the above functions. Furthermore, the above program may be a so-called differential file (differential program) that can implement the above functions in combination with a program already recorded in the computer system. [Explanation of symbols]

[0097] 1...mobile body control system, 10...mobile body, 11...imaging unit, 20...information processing device, 22...storage unit, 23...input reception unit, 24...communication unit, 25...display unit, 26...control unit, 261...imaging control unit, 262...image processing unit, 263...position and orientation calculation unit, 264...mobile body control unit, FT...filter, MK, MKA, MKB, MKC...marker, MK1, MK1A, MK1B, MK1C...first marker, MK2, MK2A, MK2B, MK2C...second marker, MK3C...third marker

Claims

1. a control unit that detects the first information and the second information based on a captured image of a target marker on which a first marker indicating first information and a second marker indicating second information are superimposed, the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding, the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type, the control unit detects the first information indicated by the first marker based on the captured image, and detects an outline of the first marker, and detects the second information indicated by the second marker based on the detected outline of the first marker and the captured image. Information processing device.

2. At least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light, The hue of the first marker is different from the hue of the second marker. The information processing device according to claim 1 .

3. one of the hue of the first marker and the hue of the second marker is a hue located at a first vertex among three vertices of an equilateral triangle inscribed in a predetermined hue circle, the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex, which is different from the first vertex, among the three vertices in a predetermined hue circle; The information processing device according to claim 2 .

4. The first marker is an ArUco marker; The information processing device according to claim 1 .

5. The second marker is a QR code. The information processing device according to claim 1 .

6. The target marker is superimposed with one or more markers in addition to the first marker and the second marker. The information processing device according to claim 1 .

7. an imaging unit including a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal; The control unit detects the first information and the second information based on the captured image captured by the imaging unit. The information processing device according to claim 1 .

8. Further comprising a communication unit for communicating with a mobile object, the moving body includes an imaging unit having a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal; The first information is information indicating a relative position and orientation between the imaging unit and the target marker, The second information is information indicating a position and an orientation of the target marker in a predetermined coordinate system, the control unit detects the first information and the second information based on the captured image captured by the imaging unit, calculates a position and an attitude of the moving body in the predetermined coordinate system based on the detected first information and the detected second information, and controls the moving body via the communication unit based on the calculated position and attitude of the moving body. The information processing device according to claim 1 .

9. The moving object is a drone or a robot. The information processing device according to claim 8.

10. and after detecting the first information and the second information, the control unit stores in a storage unit history information correlating information indicating the first marker from which the first information was detected with the detected second information. The information processing device according to claim 1 .

11. A computer of an information processing device according to any one of claims 1 to 6, a detection step of detecting the first information and the second information based on the captured image in which the target marker is captured; A program for executing.

12. A communication unit for communicating with a mobile object; a control unit that detects the first information and the second information based on a captured image in which a first marker indicating first information and a second marker indicating second information are superimposed; Equipped with The first marker reflects electromagnetic waves in a predetermined first wavelength band and has a predetermined first type of code. a marker which indicates the first information by encoding the first information; The second marker reflects electromagnetic waves in a second wavelength band different from the first wavelength band, and a marker indicating the second information by a second type of encoding different from the first type; the moving body includes an imaging unit having a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal; The first information is information indicating a relative position and orientation between the imaging unit and the marker, the second information is information indicating a position and an orientation of the marker in a predetermined coordinate system, the control unit detects the first information and the second information based on the captured image captured by the imaging unit, calculates a position and an attitude of the moving body in the predetermined coordinate system based on the detected first information and the detected second information, and controls the moving body via the communication unit based on the calculated position and attitude of the moving body. Information processing device.

13. A control unit detects the first information and the second information based on an image in which a first marker indicating first information and a second marker indicating second information are superimposed, the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding, the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type, and after detecting the first information and the second information, the control unit stores in a storage unit history information correlating information indicating the first marker from which the first information was detected with the detected second information. Information processing device.

Citation Information

Patent Citations

  • Bar code reader

    JP1991228193A

  • Two-dimensional code

    JP1995254037A

  • Bar code reader

    JP2004240874A

  • Two-dimensional code detecting device, program for it, and robot control information generating device and robot

    JP2007090448A

  • Composite optical mark, printed matter having composite optical mark printed, recognition method of composite optical mark, recognition system of composite optical mark, generation method of composite optical mark and generation device of composite optical mark

    JP2015230554A