Optical marker, system, optical marker detection method, and program

Abstract

[Problem] To transmit more information at a time than in a binary information transmission using a conventional optical marker. [Solution] An optical marker 101 has a structure in which a liquid crystal 301 is positioned at a distance d310 from a retroreflective film 203 used for retroreflecting laser light. By dynamically changing the liquid crystal 301 to be transmissive / non-transmissive, it is possible to temporally fluctuate the light intensity of a reflected laser. The laser light emitted from a LiDAR apparatus is reflected on a surface of the retroreflective film 203 or the liquid crystal 301 and detected by a light reception device of the LiDAR apparatus.

Description

Optical Marker, System, Optical Marker Detection Method, and Program

[0001] The present invention relates to an optical marker, a system, an optical marker detection method, and a program. More specifically, it relates to an optical marker, a system, an optical marker detection method, and a program for performing information output by utilizing changes in the light intensity of reflected light using laser image detection and ranging (LiDAR) technology.

[0002] In recent years, as part of remote sensing, Light Detection And Ranging (LiDAR) technology mainly using laser light has been used in various applications. In particular, in combination with an optical marker, it has become possible to capture a moving object or obtain information such as metadata of a specific location at a remote or separated state. For example, for the purpose of providing an information communication system and an information reading method that can miniaturize a tag and read tag information from a long distance, a light emitting unit (infrared light emitting unit 210) that emits infrared light, and among the infrared light emitted by the light emitting unit (infrared light emitting unit 210), an infrared light that passes through the liquid crystal 120 of the tag 100 and is reflected by the retroreflective material 110 of the tag 100 so as to include the information of the tag 100, a light receiving unit (infrared light receiving unit 220) that receives the infrared light, and a decoding unit that decodes the information of the tag 100 included in the infrared light received by the light receiving unit (infrared light receiving unit 220), a camera 200 has been proposed (see, for example, Patent Document 1).

[0003] Furthermore, with the aim of providing a laser beam search system that can accurately search for objects to be searched on land or in water from the air using laser light, a laser beam search system 10 has been proposed that includes a retroreflective light tag that reflects reflected laser light Lr toward a laser beam transceiver, and an optical tag 40 that assigns predetermined identification information related to a person (object to be searched) 32 to the reflected laser light Lr, and a processing unit that receives the reflected laser light Lr and extracts the identification information from the received reflected laser light Lr, wherein the reflected laser light Lr is reflected from the retroreflective light tag and has a relatively high reception gain, so that it can accurately search for a person (object to be searched) 32 on land, in snow or in water, and the reflected laser light Lr contains information that identifies the person (object to be searched) 32 and can be extracted individually (see, for example, Patent Document 2).

[0004] Japanese Patent Publication No. 2016-071663, republished; Japanese Patent Publication No. WO2020 / 044411

[0005] However, the tag disclosed in Patent Document 1, which includes a liquid crystal and a retroreflective material, can transmit tag information to the camera 200 by dynamically making the liquid crystal transparent and opaque. However, since the information must be represented by the time-varying binary properties of transparency / opacity of the liquid crystal, there is a problem that as the amount of information, for example the number of IDs to be handled, increases, it takes a lot of time to transmit the information. Similarly, the retroreflective tag 130 disclosed in Patent Document 2 can also extract identification information from the received reflected laser light Lr by modulating the reflected laser light Lr that is reflected from the retroreflective layer 144 and passes through the liquid crystal plate 146 with, for example, an on / off signal. However, similar to Patent Document 1, as the amount of identification information increases, it takes a long time to transmit the information, and there is a problem that not all of the information can be read when reading from an artificial satellite or the like where there is a time limit for reading.

[0006] In view of the problems of the prior art described above, the present invention aims to provide an optical marker, system, optical marker detection method, and program that can transmit a large amount of information in a short period of time, by opening and closing the liquid crystal of the optical marker at multiple positions and storing information in the depth direction, thereby enabling the transmission of more information at once compared to the binary information of the prior art.

[0007] The invention described in claim 1 of the present invention is an optical marker comprising a reflective layer that reflects laser light, and an optical blocking layer that dynamically blocks laser light by optical shutter means at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer, wherein the optical blocking layer has an optical reflectivity lower than that of the reflective layer when light is blocked.

[0008] The invention described in claim 2 is characterized in that, in the optical marker described in claim 1, the plurality of positions change dynamically.

[0009] The invention described in claim 3 is characterized in that, in the optical marker described in claim 1, the optical shutter means is single and dynamically blocks the laser light at multiple positions by moving it in the direction of the laser light source.

[0010] The invention described in claim 4 is characterized in that, in the optical marker described in claim 1, a plurality of optical shutter means are arranged at different positions, and the laser light is dynamically blocked at multiple positions by switching the optical shutter means that is blocking.

[0011] The invention described in claim 5 is characterized in that, in the optical marker described in claim 1, the reflective layer is formed of a retroreflective material.

[0012] The invention described in claim 6 is characterized in that, in the optical marker described in claim 1, the optical shutter means opens and closes at a predetermined period to dynamically block the laser light.

[0013] The invention described in claim 7 is characterized in that, in the light marker described in claim 6, the light-blocking layer repeats a predetermined opening and closing pattern of 3 cycles.

[0014] The invention described in claim 8 is characterized in that, in the optical marker described in any one of claims 1 to 7, the optical shutter means is a liquid crystal.

[0015] The invention described in claim 9 is characterized in that, in the optical marker described in claim 8, it further comprises a relay circuit for driving liquid crystal.

[0016] The invention described in claim 10 is characterized in that, in the optical marker described in claim 2, it further comprises a drive circuit that drives the liquid crystal and dynamically switches and opens / closes the position that blocks the optical laser.

[0017] The invention described in claim 11 is a system comprising: a laser irradiation means for irradiating laser light; a reflective layer for reflecting laser light; a light blocking layer for dynamically blocking laser light by a light shutter means at a plurality of positions between the light source and the reflective surface of the reflective layer, wherein the light blocking layer has a lower light reflectivity than the reflective layer; and a laser light receiving means for receiving reflected laser light and acquiring the reflection position and light intensity.

[0018] The invention described in claim 12 is a method for detecting an optical marker, comprising: a laser irradiation step of irradiating laser light; an optical marker comprising a reflective layer that reflects laser light and a light-blocking layer with a lower light reflectivity than the reflective layer, wherein the optical marker receives reflected laser light from the optical marker which dynamically blocks the laser light at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer to encode information; and a decoding step of obtaining the reflection position and light intensity from the received reflected laser light and decoding the encoded information.

[0019] The invention described in claim 13 is a program for performing an optical marker detection method, the method comprising: a laser irradiation step of irradiating with laser light; a laser reception step of receiving reflected laser light from an optical marker comprising a reflective layer that reflects laser light and a light-blocking layer with a lower light reflectivity than the reflective layer, the optical marker dynamically blocking the laser light at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer to encode information; and a decoding step of obtaining the reflection position and light intensity from the received reflected laser light and decoding the encoded information.

[0020] The invention described in claim 14 is a method for estimating a self-position using an optical marker, comprising: a movement amount acquisition step of scanning a laser beam, receiving the reflected laser beam, and calculating a movement amount based on point cloud information using a predetermined algorithm; a laser light reception step of receiving reflected laser beam from an optical marker, which is positioned at a predetermined coordinate in a target space and comprises a reflective layer that reflects laser beam and a light-blocking layer with a lower light reflectivity than the reflective layer, and dynamically blocks the laser beam to encode identification information; a coordinate acquisition step of obtaining the reflection position and light amount from the received reflected laser beam, decoding the encoded identification information, and obtaining the coordinates of the optical marker; and a position estimation step of estimating a self-position by performing factor graph optimization using the acquired movement amount and the coordinates of the optical marker.

[0021] The invention described in claim 15 is characterized in that, in the self-position estimation method described in claim 14, the optical marker dynamically blocks the laser light at a plurality of positions between the laser light source and the reflective surface of the reflective layer.

[0022] According to the present invention, the device comprises a reflective layer that reflects laser light, and a light-blocking layer that dynamically blocks the laser light at multiple positions between the laser light source and the reflective surface of the reflective layer by a light shutter means, wherein the light-blocking layer has a lower light reflectivity than the reflective layer when light is blocked. This makes it possible to transmit a large amount of information at once from a remote or spaced-out position.

[0023] This figure shows an example of the overall configuration of a LiDAR system according to one embodiment of the present invention. This figure schematically shows a conventional optical marker. This figure schematically shows an optical marker according to one embodiment of the present invention. This figure schematically shows the operation of an optical marker according to one embodiment of the present invention when the liquid crystal is in a transparent state. This figure schematically shows the operation of an optical marker according to one embodiment of the present invention when the liquid crystal is in an opaque state. This figure shows an example of a flowchart of a LiDAR control system according to one embodiment of the present invention. This figure schematically shows the principle of information transmission of an optical marker according to one embodiment of the present invention. This figure shows an example of a block diagram of an optical marker according to one embodiment of the present invention. This figure shows an example of a block diagram of a LiDAR system using an optical marker according to one embodiment of the present invention. This figure schematically shows the principle of information transmission of a conventional optical marker. This figure schematically shows the principle of information transmission of an optical marker that dynamically changes the position of the liquid crystal according to another embodiment of the present invention. This figure shows an example of a method for dynamically changing the position of the liquid crystal of an optical marker according to another embodiment of the present invention. This figure shows another example of a method for dynamically changing the position of the liquid crystal of an optical marker according to another embodiment of the present invention. This figure schematically shows the principle of another example of information transmission of an optical marker that dynamically changes the position of the liquid crystal according to another embodiment of the present invention. This figure illustrates an example of the concept of self-position estimation according to a third embodiment of the present invention. This figure schematically illustrates the principle of processing self-localization according to the third embodiment of the present invention. This figure shows an example of a flowchart of the LiDAR control system according to the third embodiment of the present invention. This figure illustrates an example of factor graph optimization used in self-localization according to the third embodiment of the present invention. This figure schematically illustrates an example of applying self-localization according to the third embodiment of the present invention.

[0024] Hereinafter, an embodiment of the optical marker, system, optical marker detection method, and program of the present invention will be described with reference to the drawings. Note that even in different drawings, the same reference numerals will be used to indicate the same process or configuration.

[0025] LiDAR technology, as a remote sensing method, has recently been used in a variety of applications due to its scalability and high accuracy, including accurately understanding the shape of terrain and ruins, capturing moving artificial or natural objects in combination with optical markers, and providing traffic information for autonomous driving assistance. Basically, LiDAR aims to accurately detect the location of objects and places, but especially when combined with optical markers, it can have the function of transmitting information including identification information in addition to the location information of the optical marker or the moving object to which the optical marker is attached, in order to identify them. That is, for example, the aforementioned Patent Document 2 discloses a technology in which a person to which an optical tag (optical marker) is attached is found by scanning the land or sea with LiDAR mounted on a drone, etc., and location information is provided. However, detecting the location of the optical tag is premised on being able to recognize that it is the target that the optical tag is looking for, and therefore, if identification information of the optical tag cannot be obtained from the reflected laser light received by the LiDAR, the person to whom the optical tag is attached cannot be identified, and the person cannot be searched for. Here, it is possible to transmit predetermined information by displaying, for example, a two-dimensional barcode on an optical tag and reading it. However, if the distance to the reader is long or the optical tag is small, the tag will essentially have to be treated as a point, and it will not be possible to use representations that require a certain area as information to be transmitted. For example, as shown in Figure 1, each optical marker 101 appears as a collection of points to the LiDAR 102 and is considered as a point cloud 103. Therefore, the optical marker of the present invention aims to transmit information so that the position of a target existing as a point in three-dimensional space can be identified and information such as identification information can be acquired. For this reason, optical markers that are considered as a point cloud generally have a reflective film that reflects light such as a laser and a means that dynamically blocks reflection, such as a liquid crystal, and are configured to transmit information by changing the intensity of the reflected laser over time by making the liquid crystal transparent / impermeable. However, the present invention is characterized in that it has a structure with a three-dimensional shape difference between the liquid crystal and the reflective film, making it possible to transmit more information.

[0026] (System Configuration) Figure 1 shows an example of the overall configuration of a LiDAR system according to one embodiment of the present invention. In this embodiment, for example, as shown in Figure 1, a laser is irradiated from the laser emitter of the LiDAR 102, and the reflected laser from each point group 103 is received and analyzed to acquire position information. In addition, identification information can be extracted from the reflected laser of the reflected light from an optical marker having predetermined identification information, thereby identifying it as an optical marker having predetermined identification information. Here, the LiDAR mainly scans the target area with laser light or the like to acquire the necessary position information. The specific configuration and acquisition method will not be described here, but any method known in the art can be used. Similarly, any method known in the art can be used for the specific method of capturing the target optical marker by scanning with the LiDAR and receiving the reflected light, as well as for the general method of extracting information from the received reflected light.

[0027] (Optical Marker of the First Embodiment) The optical marker of this embodiment will be described with reference to Figures 2 to 5. Figure 2 is a schematic diagram showing a conventional optical marker, and Figure 3 is a schematic diagram showing an optical marker of one embodiment of the present invention. Figure 4 is a schematic diagram showing the operation of the optical marker of one embodiment of the present invention when the liquid crystal is in a transparent state, and Figure 5 is a schematic diagram showing the operation of the optical marker of this embodiment when the liquid crystal is in an opaque state. As shown in Figure 2, the conventional optical marker 201 has a structure in which a retroreflective film 203 for retroreflectively reflecting laser light is attached behind the liquid crystal film 202, and information is transmitted by dynamically making the liquid crystal transparent / opaque, thereby changing the light intensity of the reflected laser over time. On the other hand, the optical marker 101 of the present invention differs in that, as shown in Figure 3, the liquid crystal 301 is arranged at a distance d310 from the retroreflective film 203, but it is similar to the conventional optical marker 201 in that it can change the light intensity of the reflected laser over time by dynamically making the liquid crystal transparent / opaque.

[0028] In other words, as shown in Figure 4, in the optical marker 101 of the present invention, when the liquid crystal is in a transparent state, the irradiated laser light is reflected by the surface 411 of the retroreflective film 203 and detected by the LiDAR light receiver. However, when the liquid crystal is in an opaque state, as shown in Figure 5, the irradiated laser light is reflected by the surface 503 of the liquid crystal 301. However, since the reflectivity of the laser light on the liquid crystal surface 503 is considerably lower than that of the retroreflective film 203, the reflection intensity is lower than when it is in a transparent state. Therefore, by calculating the reflection position using LiDAR, not only the difference in the light intensity of the reflected laser, but also the reflection position of the liquid crystal (i.e., when the liquid crystal is opaque) can be compared with the reflection position when it is opaque and the difference can be used as additional information. In other words, it is possible to increase the amount of information that can be transmitted by changing the position of the liquid crystal (the value of the distance d310), rather than just the binary values ​​of whether the liquid crystal is transparent or opaque. For example, assuming a two-frame repetition, if the distance between the liquid crystal 301 and the retroreflective film 203 is defined as distance d310, and the structure allows for two values, front and back, then 2 x 2 = 4 IDs can be assigned as identification information. If the structure allows for the placement of the liquid crystal at five positions from the front row to the back, then 5 x 5 = 25 IDs can be assigned. In this embodiment, the optical marker uses liquid crystal as a means of blocking laser light, but it is not limited to this; any device that can dynamically block or control light, such as an optical shutter, can be used, as is known in the art. Furthermore, although a reflective film using retroreflective material is used as the laser light reflector, any reflective layer having a higher reflectivity than the means of blocking light, such as liquid crystal, can be used, and any material or structure of reflective layer can be used, including retroreflective material, for specific structures and usage methods known in the art. Furthermore, the liquid crystal can be positioned not only in the front, as shown in the figure, but also in any position where a change in distance from the LiDAR can be detected, as long as the distance difference from the retroreflective film can be detected.

[0029] (Information Acquisition Method of the First Embodiment) The information extraction method of this embodiment will be described with reference to Figures 6 to 9. Figure 6 is a diagram showing an example of a flowchart of a LiDAR control system of one embodiment of the present invention, and Figure 7 is a diagram schematically showing the principle of information transmission of the optical marker of this embodiment. Figure 8 is a diagram showing an example of a block diagram of an optical marker of one embodiment of the present invention, and Figure 9 is a diagram showing an example of a block diagram of a LiDAR system using the optical marker of this embodiment. As described above, the liquid crystals constituting the optical marker of this embodiment can be configured to be arranged at multiple positions at different distances from the laser light receiving device, and by making the liquid crystal transparent / opaque, information such as identification information or ID can be transmitted along with its position information. That is, as shown in Figure 8, the optical marker 101 of this embodiment is controlled to transmit light irradiated by a command from the control unit 801 at a predetermined timing in synchronization with a clock (not shown) so that the laser light reaches the surface 411 of the retroreflective film 203, and also to reflect the irradiated light on the liquid crystal surface 503 by making it opaque, thereby enabling information to be transmitted on the reflected laser light in combination with LiDAR. For example, if the transparency / opacity of the liquid crystal is controlled to repeat at regular intervals based on a predetermined clock, during the transparency timing, the laser light is reflected by the surface 411 of the retroreflective film 203, resulting in high reflection intensity. When the reflection position (surface 411 of the retroreflective film 203) is calculated using LiDAR, a position further away from the liquid crystal by a distance d310 is detected. On the other hand, during the opacity timing, the laser light is reflected by the surface 503 of the liquid crystal 301, resulting in low light intensity of the reflected light, and the reflection position becomes the surface 503 of the liquid crystal 301, which can be used for information encoding. Here, the optical marker 101 has a power supply 802, which is used to operate the liquid crystal 301 and the control unit 801. The power supply 802 can use a normal battery, and since the required power is not large, it is not limited to this, and various power supply methods such as photovoltaic power generation, depending on the usage situation, can be used.

[0030] The above process will be explained with reference to Figure 7. As shown in Figure 7, if the reflection position and the light intensity of the reflected light are expressed as point cloud information (x, y, z, i (reflection intensity)), then at the opacity timing, the point cloud information will be (10, 10, 10, 1) as shown in frames 1 and 3, for example, if the coordinates of the liquid crystal surface 503 are (10, 10, 10). Similarly, if the distance d310 is 0.05, at the transmission timing, the point cloud information will be (10.05, 10, 10, 100) as shown in frame 2. Although only frames 1 to 3 are shown in Figure 7, a typical optical marker can repeat such frames at a constant clock, so the state of the optical marker can be detected by reading the frames at predetermined time intervals with a reflected light receiving device. In this example, for explanatory purposes, the x-coordinate is taken in the depth direction, so the coordinate values ​​only change along the x-axis, but generally, it can take various values, so it is necessary to calculate the distance d310 from each coordinate value. Furthermore, since the actual coordinate values ​​and reflectivity can take on various values, a detailed explanation will be omitted. However, the raw data can be processed using any method known in this art to accurately detect differences in the position of the liquid crystal. Also, the coordinate values ​​are for illustrative purposes only and can actually be processed using any method known in this art, such as normalizing them to facilitate distance comparisons.

[0031] As described above, by using LiDAR while the optical marker 101 is controlling the liquid crystal, information is encoded in the reflected and received laser light. This can be decoded to obtain not only the position information of the optical marker 101 but also information such as identification information transmitted by the optical marker. Specifically, the laser irradiation module 901 of the LiDAR control system shown in Figure 9 causes the radar light emitter 911 to scan the target area with laser light (step S601). When the irradiated laser light reaches the optical marker, it is reflected, and the reflected light with encoded information reaches the reflected light receiving device 921, where it is received by the reflected light receiving module 902 (step S602). The optical marker position calculation module 903 then detects the light intensity and reflection position of the received reflected light (step S603). In this embodiment, the reflected light received from the optical marker 101 has a pattern that repeats frame 1 and frame 2 as described above, so the reflection intensity can be compared to determine whether or not it is a reflection from the liquid crystal surface 503 (step S604). In other words, when the point cloud information shows a low reflection intensity, as in frame 1 or 3 (the "No" in step S604), it is a reflection from the surface 503 of the liquid crystal 301, so the position of the liquid crystal 301 can be determined by obtaining its coordinates (step S605). The information output module 904 transmits the position information, identification information, and ID as point cloud information of the reflected light (step S606). Although the above explanation is based on the principle of processing, in actual processing, for example, since reflected laser light with fluctuating reflection intensity is received, it is possible to acquire several fluctuating frames, for example 3 frames in the above explanation, and decode the data by analyzing the reflection intensity and reflection position to quantify the fluctuation pattern. However, any method known in the art can also be used for specific processing.

[0032] As described above, by linking the position of the liquid crystal (distance d310) with information such as identification information or ID in advance, the identification information or ID of the optical marker detected by LiDAR can be identified by the position of the liquid crystal. For example, by installing multiple optical markers or attaching them to a moving object and detecting them with LiDAR, if the identification information of each optical marker can be obtained, it is possible to perform processing such as detecting which optical marker is located where. In this embodiment, identification information is used, but it is not limited to this and can be applied to various applications that acquire the position and information of optical markers from remote or separated positions. That is, the distance between the LiDAR and the optical marker varies from long distances, such as when detecting information of a target on the ground from a satellite, to short distances, such as when detecting information of a road sign from a vehicle, and depending on the distance and the operating environment, a search device using light other than LiDAR can also be used.

[0033] In this embodiment, as shown in Figure 7, three frames are acquired and information is extracted. This is because, although it is logically possible to identify the liquid crystal position and extract information using only two frames based on the principle described above, reading at least three frames is desirable to prevent errors because the intensity and coordinate information of point clouds other than the optical marker also fluctuates. Furthermore, while the liquid crystal is controlled to repeatedly switch between transmission and opacity at regular intervals, this is not limited to this. As in the conventional technology, for example, as will be described later with reference to Figure 10, the time-series change in reflection intensity can be represented using Manchester coding or the like with many frames, and in addition to this, information on the liquid crystal position can be used to provide even more information. In this embodiment, we have described an example of how the optical marker of the present invention can be combined with LiDAR technology to enable the provision of unprecedented positional and identification information. However, the optical marker of the present invention can be used in various applications combined with remote sensing technologies other than LiDAR or systems that irradiate light and receive and process reflected light, and can also be used in applications other than remote sensing.

[0034] (Optical Marker of the Second Embodiment) The optical marker of this embodiment also shares the same basic configuration as the liquid crystal and retroreflective film, but differs in that the position of the liquid crystal changes dynamically. That is, by dynamically reflecting laser light from the surface of the liquid crystal at different positions using several methods, more information can be transmitted in a shorter time. Possible methods for dynamically changing the position of the liquid crystal include physically moving the retroreflective material and the liquid crystal with a motor or the like as shown in Figure 12 (Example 1), and installing multiple liquid crystals in layers and selecting the liquid crystal to be driven by a relay circuit as shown in Figure 13 (Example 2). However, it is not limited to these, and any method can be used as long as the laser light can be reflected at different positions dynamically.

[0035] In this embodiment, as in the first embodiment described above, frames are acquired as point cloud information and information is extracted from the temporal changes of the frames. However, it differs in that the liquid crystal can be dynamically controlled at multiple positions. For example, if reflection can be made from the liquid crystal surface at two positions, three values ​​can be obtained in one frame, including the retroreflective film, so numbers up to 9 can be represented with just two frames. By increasing the number of frames, even larger numbers can be represented, making it possible to use the optical marker as one of the information communication devices that can simultaneously provide positional information. For example, it is possible to build a system that can be attached to a moving object over a wide area and capture measurement data of the moving object and environment in real time along with positional information. Furthermore, since the optical marker of this embodiment can dynamically change the position of the liquid crystal, it can transmit dynamic information in addition to fixed information such as identification information and IDs, so it can also be used for general data communication.

[0036] (Example 1) The optical marker and its operation in this embodiment will be described with reference to Figures 10 to 12. Figure 10 is a schematic diagram showing the principle of information transmission of a conventional optical marker. Figure 11 is a schematic diagram showing the principle of information transmission of an optical marker that dynamically changes the position of the liquid crystal in another embodiment of the present invention, and Figure 12 is a diagram showing an example of a method for dynamically changing the position of the liquid crystal in the optical marker of this embodiment. As shown in Figure 12, the optical marker 101 of Embodiment 1 of this embodiment has a liquid crystal 1201 that is movable in front of the retroreflective film 203, and is structured so that the liquid crystal 1201 can change its position in the direction of arrow 1204 via an arm 1203 by controlling a motor 1202. With such a structure, the liquid crystal 1201 can be dynamically positioned at any position between the front surface 1205 of the optical marker and the retroreflective film 203. By changing the position of this liquid crystal 1201 at a predetermined timing, the point cloud information composed of the coordinates of the reflection position and the reflection intensity can be dynamically changed and information can be encoded. While it is generally possible to dynamically position these elements at any location, in practice, due to the distance measurement accuracy of LiDAR, it is usually possible to set the number of points to a finite value, such as n points.

[0037] In other words, referring to Figure 7, in the first embodiment described above, frames 1 and 3 were reflected light from the liquid crystal 301, and frame 2 was reflected light from the retroreflective film 203. However, in this embodiment, since the position of the liquid crystal 1201 can be dynamically changed, the reflection position of the laser light can take on a total of n+1 positions, for example, the number of positions n where the liquid crystal 1201 can be placed and the retroreflective film 203, and thus n+1 numbers can be represented. That is, if the movable range of the liquid crystal 1201 is L in the x direction and the coordinates of the surface of the liquid crystal 1201 are (a, b, c), then the possible point cloud information is (a + L / n, b, c, 1) ... (a + L, b, c, 100), which is n+1. Therefore, for example, if the information is encoded by repeating frames 1 and 2, the possible values ​​will be (n+1) × (n+1), and any number from (n+1) × (n+1) can be transmitted with just two frames (or three frames considering error prevention). For example, if n is 7 and one frame is encoded in 10ms, then approximately 6 bits of information (i.e., numerical values ​​from 0 to 63) can be transmitted in 20 to 30ms by arbitrarily changing the position of the liquid crystal of the same optical marker.

[0038] (Modification) Using the optical marker of this embodiment, even more information can be transmitted. This will be explained below with reference to Figures 10 and 11. Figure 10 is a schematic diagram showing the principle of information transmission using a conventional optical marker, and Figure 11 is a schematic diagram showing the principle of information transmission using an optical marker that dynamically changes the position of the liquid crystal in another embodiment of the present invention. Conventional information transmission methods using optical markers involve encoding by changing the light intensity of reflected light over time by turning the liquid crystal on and off, and decoding at the receiving end.

[0039] To explain in detail with reference to Figure 10, if the only possible values ​​for each frame as point cloud information are the difference in light intensity due to the on / off state of the liquid crystal, i.e., expressed as (x, y, z, i (reflection intensity)) then, at opaque timing, the point cloud information will be (10, 10, 10, 1) as shown in frames 1, 3, and 5, and at transparent timing, the point cloud information will be (10, 10, 10, 100) as shown in frames 2 and 4. These two values ​​form a time-varying pattern and can be encoded. For example, as shown in the example in Figure 10, frames 1, 3, and 5 are (10, 10, 10, 1), and frames 2 and 4 are (10, 10, 10, 100). If these five frames are then repeated, since each frame is binary, 2 frames will be 2 ５ This means that 32 different numbers can be represented. In contrast to conventional techniques that only utilize changes in the light intensity of reflected light, this modified example allows the liquid crystal to be turned on and off at multiple positions, thus dramatically increasing the amount of information that can be displayed, as shown in Figure 11.

[0040] Referring to Figure 11, a modified example of this embodiment can be described. If the liquid crystal is positioned in two locations and d310 is set to 0.05 and 0.10, and encoded as a repeating pattern of 5 frames, similar to the conventional method in Figure 10, then at opaque timings, frames 1 and 3 will be (10, 10, 10, 1), frame 5 will be (10.05, 10, 10, 1), and at transparent timings, frames 2 and 4 will be (10.10, 10, 10, 100). Since there are 3 possible values ​​for each frame, there are 3 for 5 frames. ５ This means that 243 different numbers can be represented. As described above, by using the encoding method of this modified form, even by simply increasing the position of the liquid crystal by one, the number of numbers that can be represented in the same number of frames, that is, within the same transmission time, can be dramatically increased from 32 to 243.

[0041] (Example 2) The optical marker of this embodiment and its operation will be described with reference to Figure 13. Figure 13 shows another example of a method for dynamically changing the position of the liquid crystals in the optical marker of this embodiment. As shown in Figure 12, the optical marker 101 of Embodiment 2 of this embodiment differs from Embodiment 1 described above in that multiple liquid crystals 1301, 1302, and 1303 are pre-arranged in front of the retroreflective film 203 at multiple positions and are controlled by wirings 1321, 1322, and 1323 from the relay circuit 1311, respectively. In this way, the liquid crystals 1301, 1302, and 1303 can be turned on and off independently, but are not limited to this and can be controlled by any method known in the art. As described above, by changing the multiple liquid crystals 1301, 1302, and 1303 at predetermined timings, the point cloud information composed of the coordinates of the reflection position and the reflection intensity can be dynamically changed and the information can be encoded.

[0042] That is, referring to FIG. 7, in the above-described first embodiment, frames 1 and 3 were the reflected light from the liquid crystal 301, and frame 2 was the reflected light from the retroreflective film 203. However, in this embodiment, when making each of the liquid crystals 1301, 1302, and 1303 opaque at their respective positions, the reflection position of the laser light, for example, as shown in FIG. 13, assuming the use of three liquid crystals 1301, 1302, and 1303, the number of positions of the liquid crystals is 3 and the retroreflective film 203, and a total of 3 + 1 = 4 numbers can be expressed. That is, if the distance between the liquid crystal 1301 and the retroreflective film 203 is L1 and the coordinates of the surface of the liquid crystal 1301 are (a, b, c), the possible point group information is (a, b, c, 1), (a + L / 3, b, c, 1), (a + 2L / 3, b, c, 1), and (a + L, b, c, 100), totaling 4. Therefore, for example, when encoding information by repeating frames 1 and 2, the possible values are 4 × 4 = 16, and with only two frames (or three frames considering error prevention), information up to 16 can be transmitted. In this example, an example using three liquid crystals has been described, but depending on the system such as the accuracy of LiDAR, many liquid crystals can be used within the possible range. For example, when adopting a configuration with n liquid crystals, a number up to (n + 1) × (n + 1) can be expressed.

[0043] (Modified Example) Similar to the above-described Example 1, a modified example can also be implemented in this example. That is, referring to FIG. 14 to explain the modified example of this embodiment, assuming that the positions of the liquid crystals are three, d310 is 0.10, 0.15, and 0.20, and encoding is performed as a repeating pattern of 7 frames, then at the timing of non-transmission, frames 1 and 3 are (10, 10, 10, 1), frame 5 is (10.05, 10, 10, 1), frame 7 is (10.10, 10, 1), and at the timing of transmission, frames 2 and 4 are (10.20, 10, 10, 100). Since the possible values for each frame are 4, with 7 frames, 4 ７ = 16,384 numbers can be expressed. In the case of a conventional optical marker, when using 7 frames, 2 ７= 128, and by using the encoding method of this modified example, even by simply increasing the position of the liquid crystal by one, the number that can be represented in the same number of frames, that is, within the same transmission time, can be dramatically increased from 128 to 16,384. As can be understood by referring to the above example, in general, in order to reflect the effect of the number of liquid crystals being n to the maximum extent in terms of the amount of information, it is necessary to use 2 × n + 1 frames, but it is not limited to this, and even without changing the number of frames due to time constraints, etc., a sufficient increase in the amount of transmitted information can be expected by increasing the number of liquid crystals.

[0044] (Self-Position Estimation Method of the Third Embodiment) In this embodiment, LiDAR technology and optical markers as used in the first and second embodiments described above are used to process self-position estimation to estimate the current location of a device incorporating LiDAR, such as a robot or an automobile. In recent years, LiDAR SLAM (Self-Position Estimation Simultaneous Mapping) has become a representative technology for such LiDAR-based self-position estimation systems. It calculates feature quantities from point cloud information obtained from the space scanned by the LiDAR, performs matching using various methods, and estimates the position of the device in space and creates a map showing the arrangement of obstacles in space. However, in environments where there is insufficient information to perform matching, as shown in Figure 15, matching may not be processed as expected, resulting in inaccurate or impossible self-position estimation. To solve the above problems, in this embodiment, optical markers are placed at predetermined positions as shown in Figure 15, enabling highly accurate self-position estimation. First, an overview of this embodiment will be described with reference to Figures 15 to 17. Figure 15 is a diagram illustrating an example of the concept of self-localization according to the third embodiment of the present invention, and Figure 16 is a diagram schematically showing the principle of processing self-localization according to this embodiment. Figure 17 is a diagram showing an example of a flowchart of the LiDAR control system according to the third embodiment of the present invention.

[0045] In this embodiment, as shown in Figure 15, it is assumed that a mobile robot 1502 equipped with LiDAR 1501 moves within a warehouse while performing self-position estimation. In such a wide-area environment, if the environmental shape is monotonous, or if the field of view of LiDAR 1501 is narrow and can only scan a limited range, there will be insufficient information to use for matching, resulting in matching failure, inability to estimate self-position, or inaccurate estimation. Therefore, in this embodiment, optical markers 1511 to 1514 are placed at appropriate positions and used as landmarks to enable accurate self-position estimation. Self-position estimation in this embodiment obtains more accurate results by adding the positional information of optical markers, which have absolute coordinate information, as landmarks, in addition to conventional LiDAR SLAM technology. Therefore, any method known in this field, such as Iterative Closest Point (ICP) or Normal Distribution Transform (NDT), can be used as scan matching in general LiDAR SLAM, so a detailed explanation will be omitted. Furthermore, when using LiDAR SLAM, various methods known in this field can be used to improve the accuracy of self-position estimation, such as an Inertial Measurement Unit (IMU) to acquire the sensor attitude of LiDAR 1501, or odometry to acquire the movement amount of the autonomous robot 1502 equipped with LiDAR 1501 by the rotation of its tires, etc. However, the method is not limited to these, and processing can be done without using such sensors, and more accurate or efficient self-position estimation can be performed using other sensor technologies.

[0046] Referring to Figures 16 and 17, the overall process is as follows: First, the LiDAR 1501 scans with laser light and receives the reflected laser light (steps S1701 and S1702). Based on the point cloud information obtained by the LiDAR 1501, the LiDAR SLAM unit 1601 calculates the amount of movement of the LiDAR 1501 (step S1703), and the optical marker detection / ID reading unit 1602 acquires information from the optical marker also obtained by the LiDAR 1501 (step S1704).

[0047] Here, the information of the optical marker is acquired during laser scanning together with other feature points, but it is not limited to this, and the reflected light of only the optical marker can also be processed. Further, any of the methods described in the above first and second embodiments can be used as a method for acquiring identification information from the reflected light from the optical marker. The identification information of the optical marker and its arranged position (coordinates) are pre-linked and stored in a database or memory not shown. When the identification information of the optical marker is acquired, a configuration can be adopted such that the arranged position can be searched, and any method for obtaining the position coordinates can be used as long as the identification information of the optical marker is read. Furthermore, as the optical marker, a conventional optical marker other than the optical markers of the above first and second embodiments, that is, an optical marker that operates an optical shutter at a fixed position instead of at a plurality of positions can also be used. However, in the case of the configuration of a conventional optical marker, as described in the above first and second embodiments, compared with the optical marker of the present invention, the amount of information that can be transmitted per unit time is small, and a considerable number of frames need to be read for reading the identification information. In a LiDAR that moves quickly, there is also a possibility of reading failure. Therefore, it is advantageous to use the optical markers of the first and second embodiments that can acquire identification information in a shorter time.

[0048] The sensor state from the IMU 1611, for example, the three rotational components (roll, pitch, yaw) and the three translational components (X, Y, Z) from the odometry 1612 are acquired (step S1705), and the self-position estimation unit 1603 estimates the self-position by factor graph optimization based on the calculated and acquired information (step S1706). However, it is not limited to this, and any method known in the art that is used for error minimization based on constraint conditions can be used to estimate the position where the error is minimized.

[0049] (Factor Graph Optimization Process) The factor graph optimization process will be explained with reference to Figure 18. Figure 18 is a diagram illustrating an example of factor graph optimization used in self-localization estimation in the third embodiment of the present invention. In this embodiment, self-localization is performed based on the calculated or acquired information, and in particular, the position of the optical marker and the sensor position at different times are used as nodes, and each constraint is used as an edge, and the position where the sum of the errors of each edge is minimized is estimated as the self-localization.

[0050] Specifically, referring to Figure 18, a given sensor state x(n) has a total of six components: three rotational components (roll, pitch, yaw) and three translational components (X, Y, Z). The sensor states at different times (t = 0, 1, 2, 3, ...) are denoted as x(0), x(1), x(2), ..., and the position of the optical marker is L(n). In Figure 18, two optical markers are used, so they are L(0) and L(1). However, as shown in Figure 15, four optical markers can also be used, and the number of optical markers observed during movement can be added to the constraint conditions to further improve accuracy. As described above, after performing steps S1701 to S1705, the following data (1) to (4) will be available when factor graph optimization is performed. (1) From the IMU, the sensor attitude, i.e., the three rotational components (roll, pitch, yaw) of sensor x(n) at a given time can be constrained. (2) Odometry can constrain the amount of movement between different sensors, i.e., the three components of translation between sensors at different times, for example, between sensors x(1) and x(2). However, since movement is usually in the xy plane, it can also be considered as two components in x and y. (3) From the results of LiDAR SLAM, it is possible to constrain the amount of movement between different sensors, i.e., the total of six components of rotation and translation between sensors at different times, for example, between sensors x(1) and x(2). (4) By observing optical markers from LiDAR, it is possible to constrain the positional relationship between the sensor and the landmark, i.e., the total of six components of rotation and translation of sensor x(n) at a certain time.

[0051] In factor graph optimization, the x(n) that best matches the data from (1) to (4) is estimated. In other words, through factor graph optimization, the three rotational components (roll, pitch, yaw) and three translational components (X, Y, Z) of the sensor at different times (t = 0, 1, 2, 3, ...) can ultimately be estimated. Through the above process, even in a space with few features, it becomes possible to estimate the self-position with higher accuracy by using optical markers whose positions are known in advance. In this embodiment, steps S1702 to S1705 have been described as being executed in a predetermined order, but this is not limited to this, and they can be performed in any order, or several processes can be performed simultaneously.

[0052] (Modification) An example of an application using the method of this embodiment described above will be explained with reference to Figure 19. Figure 19 is a schematic diagram showing an example of applying the self-position estimation of the third embodiment of the present invention. That is, as an example with few features, the self-position estimation method of this embodiment is used to estimate the self-position of a car when it passes through a tunnel. A tunnel is generally a space in which monotonous walls 1910 continue throughout, and if LiDAR SLAM is used as is, even if the self-position can be estimated, it will spread out widely, for example, as in the estimated position range 1911, making it practically unusable. Therefore, in this modification, an optical marker 1901 is placed in the tunnel, and by finally performing factor graph optimization processing using the method described above, it is possible to estimate the position accurately, as in the estimated position 1912.

[0053] Although Figure 19 shows only one optical marker, the accuracy of the final position estimation improves as the constraints on the factor graph increase. Therefore, the more times optical markers can be observed, and the more optical markers can be observed, the better the position estimation accuracy becomes. For example, observing one optical marker every 100m in a tunnel reduces the cumulative error compared to observing one optical marker only once every 1km. Also, observing three optical markers every 100m is better than observing one optical marker every 100m, resulting in higher position estimation accuracy. However, while the difference between not using any optical markers and using even one is extremely large, the degree of accuracy improvement does not increase as the frequency and number of optical markers increase. Therefore, it is desirable to set the optical markers according to the environment, system, and cost.

[0054] 101, 201, 1511, 1512, 1513, 1514, 1901 Light markers 102, 1501 LiDAR 103 Point cloud 202, 301, 1201, 1301, 1302, 1303 Liquid crystal 203 Retroreflective film 411 Surface of retroreflective film 503 Surface of liquid crystal 801 Control unit 802 Power supply 911 Laser emitter 921 Reflected light receiver 1202 Motor 1203 Drive arm 1311 Relay circuit 1321, 1322, 1323 Wiring 1502 Robot 1910 Tunnel wall 1911 Self-position range 1912 Self-position

Claims

1. An optical marker comprising a reflective layer that reflects laser light, and an optical blocking layer that dynamically blocks the laser light at multiple positions between the light source of the laser light and the reflective surface of the reflective layer by optical shutter means, wherein the optical blocking layer has an optical reflectivity lower than that of the reflective layer when light is blocked.

2. The optical marker according to claim 1, characterized in that the plurality of positions change dynamically.

3. The optical marker according to claim 1, characterized in that the optical shutter means is single and dynamically blocks the laser light at the plurality of positions by moving it in the direction of the light source of the laser light.

4. The optical marker according to claim 1, characterized in that the optical shutter means are arranged in multiple different positions, and the laser light is dynamically blocked at the multiple positions by switching the optical shutter means to block.

5. The optical marker according to claim 1, characterized in that the reflective layer is formed of a retroreflective material.

6. The optical marker according to claim 1, characterized in that the optical shutter means opens and closes at a predetermined period to dynamically block the laser light.

7. The light marker according to claim 6, characterized in that the light-blocking layer repeats a predetermined opening and closing pattern of three cycles.

8. The optical marker according to any one of claims 1 to 7, characterized in that the optical shutter means is liquid crystal.

9. The optical marker according to claim 8, further comprising a relay circuit for driving the liquid crystal.

10. The optical marker according to claim 2, further comprising a drive circuit that drives the liquid crystal and dynamically switches the position for blocking the optical laser to open and close.

11. A system comprising: a laser irradiation means for irradiating laser light; a reflective layer for reflecting the laser light; and a light-blocking layer for dynamically blocking the laser light at multiple positions between the light source of the laser light and the reflective surface of the reflective layer by a light shutter means, wherein the light-blocking layer has a lower light reflectivity than the reflective layer; and a laser light-receiving means for receiving reflected laser light and acquiring the reflection position and light intensity.

12. A method for detecting an optical marker, comprising: a laser irradiation step of irradiating with laser light; a laser reception step of receiving reflected light of laser light from an optical marker comprising a reflective layer that reflects the laser light and a light-blocking layer with a lower light reflectivity than the reflective layer, wherein the optical marker dynamically blocks the laser light at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer to encode information; and a decoding step of obtaining the reflection position and light intensity from the received reflected laser light and decoding the encoded information.

13. A program for executing a light marker detection method, the method comprising: a laser irradiation step of irradiating with laser light; a laser reception step of receiving reflected light of laser light from a light marker comprising a reflective layer that reflects the laser light and a light-blocking layer with a lower light reflectivity than the reflective layer, the light marker dynamically blocking the laser light at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer to encode information; and a decoding step of obtaining the reflection position and light intensity from the received reflected laser light and decoding the encoded information.

14. A method for estimating a self-position using an optical marker, comprising: a movement amount acquisition step of scanning a laser beam, receiving the reflected laser beam, and calculating a movement amount based on point cloud information using a predetermined algorithm; a laser light reception step of receiving reflected laser beam from an optical marker, which is positioned at a predetermined coordinate in a target space and comprises a reflective layer that reflects the laser beam and a light-blocking layer with a lower light reflectivity than the reflective layer, and dynamically blocks the laser beam to encode identification information; a coordinate acquisition step of acquiring the reflection position and light intensity from the received reflected laser beam, decoding the encoded identification information, and obtaining the coordinates of the optical marker; and a position estimation step of estimating a self-position by performing factor graph optimization using the acquired movement amount and the coordinates of the optical marker.

15. The self-position estimation method according to 14, characterized in that the optical marker dynamically blocks the laser light at a plurality of positions between the light source of the laser light and the reflective surface of the reflective layer.

Images