Positioning device, mobile device, positioning method, and program
The positioning device enhances positioning reliability by using light sources and landmarks to derive the position and orientation of moving bodies, addressing environmental interference issues in conventional methods.
Patent Information
- Application Number
- JP2022205013
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-12-22
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-12-22
Abstract
Description
Technical Field
[0001] The present invention relates to a positioning device, a moving body, a positioning method, and a program.
Background Art
[0002] Conventionally, a technique is known that performs positioning of a moving body indoors using information from a light source that emits light to position the moving body (for example, Patent Document 1).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of Invention
Problem to be Solved by the Invention
[0004] However, in the conventional technique, depending on the surrounding environment, there is a risk that light from the light source cannot be properly received and positioning cannot be performed.
[0005] The present invention aims to perform positioning more reliably.
Means for Solving the Problem
[0006] To solve the above problem, the positioning device according to the present invention is In an image captured by an imaging unit provided on a moving body that moves in a space, a light source and a target different from the light source are detected, and the positions of the light source and the target in the image, and known positions of the light source and the target in the space, and a processing unit for deriving the position of the moving body in the space based thereon, The processing unit obtains the identification result of the target shown in the image based on the last derived position of the moving body, and identifies the known position of the target in the space based on the obtained identification result. .
Effects of the Invention
[0007] According to the present invention, positioning can be performed more reliably. [Brief explanation of the drawing]
[0008] [Figure 1] This is a diagram showing the configuration of the positioning system. [Figure 2] This is a block diagram showing the functional configuration of a mobile device. [Figure 3] This figure shows an example of an image used for positioning. [Figure 4] This figure shows an example of the contents of light source position data. [Figure 5] This figure shows an example of the contents of target location data. [Figure 6] This diagram illustrates how to derive the position of a moving object. [Figure 7] This figure shows another example of the image. [Figure 8] This is a flowchart showing the control procedure for positioning and movement processing. [Figure 9] This flowchart shows another example of a control procedure for positioning and movement processing. [Modes for carrying out the invention]
[0009] Hereinafter, embodiments of the present invention will be described based on the drawings.
[0010] (Configuration of the positioning system) Figure 1 shows the configuration of the positioning system 1 in this embodiment. The positioning system 1 comprises a moving body 10 and a plurality of light sources 20a, 20b, 20c, 20d, ... (hereinafter, any one light source will be referred to as "light source 20"). In this embodiment, the case in which the positioning system 1 is applied in space 500 on the lunar surface will be described as an example. In the following, the positions of the moving body 10 and the light sources 20 in space 500 will be expressed in the XYZ Cartesian coordinate system. The Z direction is upward in the vertical direction (direction of lunar gravitational acceleration), and the X and Y directions are perpendicular to each other and perpendicular to the Z direction.
[0011] The mobile unit 10 is a lunar rover that autonomously travels (moves) and explores space 500 on the lunar surface. The mobile unit 10 is equipped with two cameras 14 (imaging units) and a positioning device 100 that derives the position and orientation of the mobile unit 10 based on images captured by the cameras 14. In this specification, "positioning" refers to deriving at least the position of the mobile unit 10 in space 500. The positioning performed by the positioning device 100 in this embodiment includes the operation of deriving both the position and orientation of the mobile unit 10. The mobile unit 10 autonomously moves toward a set destination while determining its own position and orientation based on the results of positioning performed by the positioning device 100 at a predetermined frequency.
[0012] Multiple light sources 20 are placed on the lunar surface at intervals from each other. The position (XYZ coordinates) of each light source 20 in space 500 is measured at the time of its placement (its position is known). The light sources 20 function as positional indicators representing their location on the lunar surface and are also called markers. Each light source 20 has three LEDs (Light Emitting Diodes) that emit, for example, red, green, and blue light, and emits light in a unique emission pattern. That is, each light source 20 changes its emission color over time in a predetermined pattern. By decoding the emission pattern of each light source 20, the light source ID assigned to each light source 20 can be identified. In this way, the light sources 20 are used in visible light communication to transmit information (in this case, the light source ID) via visible light.
[0013] The plurality of light sources 20 are arranged and installed in a range within the space 500 where the moving body 10 may move, such that two or more light sources 20 are imaged by the camera 14 no matter which position the moving body 10 is located at. A solar cell or the like can be used as the power source for the light sources 20. By connecting a rechargeable battery with an appropriate capacity to the light source 20, the light source 20 can be made to emit light even during the lunar night period (the period when sunlight does not directly reach due to the rotation of the moon, hereinafter referred to as "nighttime"). FIG. 1 depicts the light sources 20 emitting light at nighttime. At nighttime, the space 500 is dark, so the light emission patterns of the light sources 20 can be observed favorably. On the other hand, during the lunar daytime period (the period when sunlight directly hits due to the rotation of the moon, hereinafter referred to as "daytime"), if the light source 20 is directly exposed to sunlight, the light emission pattern of the light source 20 may be drowned out by the sunlight and become unobservable (undetectable in the image captured by the camera 14). For this reason, it is preferable to install the light source 20 in a place that is shaded as much as possible even during the daytime, for example, the depression 40 (such as a crater) shown in FIG. 1.
[0014] Further, in the present embodiment, various targets such as landscapes like the mountain peaks 30a, 30b on the lunar surface, and landmarks like the lunar base 30c are also used as position indicators. Hereinafter, when referring to any one of various targets including the mountain peaks 30a, 30b and the lunar base 30c, it is described as "target 30".
[0015] FIG. 2 is a block diagram showing the functional configuration of the moving body 10. The moving body 10 includes a CPU 11 (Central Processing Unit) (one or more processing units, processing means, one or more control units, control means, computer), a RAM 12 (Random Access Memory), a storage unit 13, a camera 14, a timing unit 15, a travel drive unit 16, an attitude angle detection unit 17, and the like. Among these, the CPU 11, the RAM 12, and the storage unit 13 constitute the positioning device 100. Each unit of the moving body 10 is connected via a communication path 18 such as a bus.
[0016] The CPU 11 is a processor that functions as a control unit for controlling the operation of the moving object 10 by reading and executing the program 131 stored in the storage unit 13, and also functions as a processing unit that executes processing related to positioning by the positioning device 100. Note that the moving object 10 (and the positioning device 100) may include a plurality of processors (a plurality of CPUs, etc.), and the plurality of processors may execute the plurality of processes executed by the CPU 11 of the present embodiment. In this case, the plurality of processors correspond to one or more control units and one or more processing units. In these cases, the plurality of processors may be involved in common processing, or the plurality of processors may independently execute different processes in parallel. In the latter case, some of the plurality of processors may correspond to one or more control units, and the other part of the plurality of processors may correspond to one or more processing units.
[0017] The RAM 12 provides a working memory space for the CPU 11 and stores temporary data.
[0018] The storage unit 13 is a non-transitory recording medium readable by the CPU 11 serving as a computer, and stores the program 131 and various types of data. The storage unit 13 includes, for example, a non-volatile memory such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The data stored in the storage unit 13 includes captured image data 132 related to an image captured by the camera 14, light source position data 133 including information related to the position of each light source 20, and information related to the position of each target 30, such as target position data 134, and positioning history data 135 including information on the positioning history obtained by the positioning device 100.
[0019] Camera 14 captures images of space 500 at a predetermined frame rate (e.g., 30fps or 60fps, etc.) and outputs the captured image data 132 to the positioning device 100. As shown in Figure 1, the moving body 10 is equipped with two cameras 14, one for capturing images in front of the moving body 10 and the other for capturing images behind it. The reason for using two cameras 14 is to ensure the detection range of the light from the light source 20 (light for visible light communication) and to prevent errors in correcting optical distortion of lenses, etc. However, if distortion correction can be performed well, a single camera 14 with a wide-angle lens may be used.
[0020] The timekeeping unit 15 calculates and stores the current time and outputs the stored time to the CPU 11. The time calculated by the timekeeping unit 15 may be, for example, UTC (Coordinated Universal Time) on Earth, or it may represent the passage of time in a day, with one lunar day and night (equivalent to approximately 27.3 days on Earth) being considered one day.
[0021] The drive unit 16 operates the wheels to propel the mobile body 10, changes the direction of movement, etc., according to the control signals transmitted from the CPU 11. Therefore, the CPU 11 can control the speed and direction of movement of the mobile body 10 by transmitting control signals to the drive unit 16.
[0022] The attitude angle detection unit 17 is a 6-axis sensor that includes, for example, a 3-axis acceleration sensor for detecting acceleration in three axes and a 3-axis angular velocity sensor (gyro sensor) for detecting angular velocity around three axes. Based on the output values of the 3-axis acceleration sensor and the 3-axis angular velocity sensor, the attitude angle detection unit 17 derives the attitude angles (yaw, pitch, roll) of the moving body 10 and outputs them to the positioning device 100. Alternatively, the attitude angle detection unit 17 may output the output values of the 3-axis acceleration sensor and the 3-axis angular velocity sensor to the positioning device 100, and the CPU 11 of the positioning device 100 may derive the attitude angles based on these output values.
[0023] (Operation of the positioning system) Next, the operation of the mobile body 10 and the positioning device 100 of the positioning system 1 will be described. As described above, the positioning device 100 of the mobile body 10 performs positioning of the mobile body 10 at a predetermined frequency. The main operator of the positioning operations described below is the CPU 11 of the positioning device 100, but for convenience, the positioning device 100 may be described as the main operator.
[0024] The positioning device 100 detects two or more position indicators whose positions in space 500 are known in the image 60 (see Figure 3) captured by the camera 14, and derives the position and orientation of the moving object 10 in space 500 based on the positions of the two or more position indicators in image 60 and the known positions of the two or more position indicators in space 500. Hereinafter, the position of the position indicator in image 60 (the coordinates of the pixels corresponding to the position indicators in image 60) will be referred to as "position in image," and the position of the position indicator in space 500 (the XYZ coordinates of the position indicators in space 500) will be referred to as "position in space."
[0025] As described above, the light source 20 is one form of a position indicator. When the positioning device 100 detects multiple light sources 20 in the image 60, it derives the position and orientation of the moving object 10 based on the image and spatial positions of at least two of the multiple light sources 20. As described above, since the space 500 is dark at night, it is usually possible to detect two or more light sources 20 from the image 60. Therefore, at night, the positioning device 100 can perform positioning using only the light sources 20 as position indicators.
[0026] On the other hand, if only one or fewer light sources 20 can be detected in image 60, the positioning device 100 uses a different object 30 on the lunar surface (within space 500) as a position indicator. That is, if only one light source 20 is detected, the positioning device 100 derives the position and orientation of the moving object 10 based on the image and spatial positions of that one light source 20 and at least one other object 30. Furthermore, if no light sources 20 can be detected in image 60, the positioning device 100 derives the position and orientation of the moving object 10 based on the image and spatial positions of at least two objects 30. Here, the object 30 can be any object whose representative point (feature point) can be identified within image 60 and space 500. For example, a part of a mountain ridge (such as a mountaintop), a part of the horizon, the edge of a distinctive terrain (such as a rock or crater), and a structure (such as a lunar base) can be objects 30. Examples of cases in image 60 where one or fewer light sources 20 can be detected include situations where sunlight during the day obscures the light from some of the light sources 20, making observation impossible, or when there is an obstacle blocking the light between the moving object 10 and the light sources 20.
[0027] Figure 3 shows an example of an image 60 used for positioning. Image 60 was obtained when camera 14 imaged a portion of space 500 during the daytime. Here, a shadow is cast by sunlight in the depression 40, so the light from light source 20a located in the depression 40 can be detected in image 60. On the other hand, the light from light sources 20b and 20c located in the sun is obscured by sunlight and cannot be detected in image 60. In this case, when only one light source 20a can be detected in image 60, the positioning device 100 detects not only light source 20a but also the mountain peak 30a and lunar base 30c as targets 30 in image 60 and uses them for positioning. The operation when performing positioning using light source 20a and mountain peak 30a in image 60 of Figure 3 will be explained below as an example.
[0028] When image 60 is acquired from camera 14, positioning device 100 first detects light source 20a in image 60. More specifically, positioning device 100 detects the light emission pattern of light source 20a from a predetermined number of recent images (moving images) including image 60, decodes the detected light emission pattern, and identifies the light source ID. The operation of identifying the light source ID corresponds to the operation of identifying light source 20. Positioning device 100 obtains the spatial position of light source 20 with the identified light source ID by referring to light source position data 133.
[0029] Figure 4 shows an example of the contents of the light source position data 133. In the light source position data 133, each light source ID is associated with and stored the installation position (in-space position) of the light source 20 assigned to that light source ID within space 500. The light source ID of light source 20a shown in Figure 3 is "1001", and its corresponding in-space position is (X11, Y11, Z11).
[0030] Furthermore, the positioning device 100 detects the target 30 (mountain peak 30a) in the image 60 using the following method. First, the positioning device 100 refers to the target position data 134 and identifies candidate targets 30 that may be visible in the image 60 based on the previous positioning result (the last determined position of the moving object 10) and the current time being measured by the timing unit 15. Furthermore, based on the identified candidate targets 30, it obtains the identification result of the target 30 (mountain peak 30a) currently visible in the image 60, and determines the spatial position of the target 30 based on the acquired identification result. The operation of obtaining the identification result of the target 30 is included in the operation of detecting the target 30 in the image 60.
[0031] Figure 5 shows an example of the contents of target position data 134. In the target location data 134, when the space 500 is divided into multiple areas R (area R1, R2, ...), candidate targets 30 observable from each area R are registered for each area R. The target location data 134 includes data related to the candidate targets 30, such as "name," "azimuth," "location," and "observable time period." "Name" is the name of the candidate for target 30. "Direction" is the direction from which a candidate target 30 is observed from a representative location (e.g., the center) in area R. "Position" refers to the pre-measured XYZ coordinates of candidate targets 30 in space 500. Therefore, the spatial position of each target 30 is known. The "observable time period" is the time period during which a candidate target 30 can be observed from a representative location in area R. For example, for a target 30 that becomes unobservable due to being hidden by a high mountain at certain times of the day, even during the daytime, the time period excluding the time when observation is impossible is recorded.
[0032] By referring to the area R to which the position of the mobile body 10 in the previous positioning result belonged in the target position data 134, candidate targets 30 observable from the current position (more precisely, the most recent position) of the mobile body 10 and their orientations can be identified. Therefore, by comparing the identified candidate targets 30 with the targets 30 shown in the image 60 using image recognition processing, etc., the identification result of the targets 30 shown in the image 60 can be obtained, and the targets 30 can be detected from this identification result. Furthermore, the spatial position of the identified targets 30 can be determined from the target position data 134. In addition, the accuracy of target identification can be further improved by referring to the orientation of the mobile body 10 in the previous positioning result to determine the orientation in which the image 60 was taken. From the image 60 shown in Figure 3, the mountaintop 30a was detected as the object 30, and its spatial position (X21, Y21, Z21) was identified.
[0033] Furthermore, the methods for identifying and detecting the target 30 are not limited to those described above. For example, instead of using area R, candidate targets 30 that can be captured in image 60 may be identified directly from the position (XYZ coordinates) of the mobile object 10 in the previous positioning result. For example, a 3D model of space 500 may be created in advance, and candidate targets 30 visible from the current position (nearest position) of the mobile object 10 may be identified by simulation based on the 3D model. By comparing the candidate targets 30 identified by the simulation with image 60, the identification result of the targets 30 can be obtained. Alternatively, instead of the above 3D model, candidate targets 30 may be identified using a map of the lunar surface that includes height information. Alternatively, the target 30 in the image 60 may be identified by inputting the image 60 into a machine learning model. That is, a machine learning model may be prepared that has been pre-trained on the position, shape, and size of the target 30 that can be observed from multiple points on the lunar surface. By inputting the image 60 and the previous positioning result (the position of the mobile body 10, and its orientation if necessary) into this machine learning model, it may be possible to obtain as output whether or not a candidate target 30 is visible in the image 60, and, if so, the identification result of the target 30. The above machine learning model may be stored in the memory unit 13 or in an external device. If the machine learning model is stored in an external device, the mobile body 10 may input input data such as the image 60 into the machine learning model in the external device via a communication unit (not shown) and receive the output result from the external device. In either case, the spatial position of the identified target 30 can be obtained from table data (which may also be target position data 134) that associates the target 30 with its spatial position.
[0034] When the positioning device 100 detects the light source 20a and the mountain peak 30a as position indicators in the image 60, it derives the position and orientation of the moving object 10 in space 500 based on the in-image positions of the light source 20a and the mountain peak 30a in the image 60 and the spatial positions of the light source 20a and the mountain peak 30a in space 500.
[0035] Figure 6 is a diagram illustrating the method for deriving the position of the moving body 10. Here, we assume that the image position of the light source 20a in image 60 is point P1(Px,Py), and the image position of the mountaintop 30a is point Q1(Qx,Qy). The positioning device 100 derives the spatial position (X0,Y0,Z0) of the moving object 10 (camera 14) using the principle of triangulation, based on the image position of the light source 20a (point P1) and the image position of the mountaintop 30a (point Q1) in image 60, and the spatial position of the light source 20a (X1,Y1,Z1) and the spatial position of the mountaintop 30a (X2,Y2,Z2) in space 500 (however, Z0 is fixed here). Furthermore, the positioning device 100 derives the orientation of the mobile body 10 (camera 14) from the spatial position (X0, Y0, Z0) of the mobile body 10 and the positions in which the light source 20a and / or the mountaintop 30a are visible in the image 60. This method allows for the deriving of the position and orientation (horizontal direction) of the mobile body 10 (camera 14), assuming that the height and attitude angle of the mobile body 10 (camera 14) are fixed in two degrees of freedom (pitch and roll). However, if the attitude angle of the mobile body 10 may fluctuate, such as when there are irregularities in the road surface, the corrected position and orientation may be derived based on the attitude angle detection result by the attitude angle detection unit 17. For example, the position may be derived after correcting the orientation of the image 60 (the positions of the light source 20a and the mountain peak 30a in the image 60) based on the attitude angle detection result by the attitude angle detection unit 17.
[0036] Incidentally, the detection accuracy of the position of the target 30 in image 60 tends to be lower than the detection accuracy of the position of the light source 20 in image 60. This is because the target 30 is not a light source and is therefore difficult to distinguish from surrounding objects in image 60, and the accuracy of other processes (such as image recognition processing) used to identify the target 30 also has an impact. For this reason, if multiple targets 30 can be detected in image 60 in addition to one light source 20, the positioning accuracy may be improved by using one light source 20 and at least two of the multiple detectable targets 30 as position indicators (i.e., using three or more position indicators) to perform positioning. For example, in the example shown in Figure 3, in addition to the light source 20a and the mountaintop 30a as a target 30, the lunar base 30c (for example, a feature point at its corner (position in image: point Q2)) may also be used for positioning.
[0037] Figure 7 shows another example of Image 60. Image 60, shown in Figure 7, was obtained when camera 14 imaged a portion of space 500 different from that shown in Figure 3 during the daytime. From this image 60, the emission patterns of two light sources 20a (image position: point P1) and light source 20d (image position: point P2), located in the shadows of the two depressions 40, can be observed, allowing detection of these light sources 20a and 20d. Furthermore, the spatial positions of light sources 20a and 20d can be obtained by referring to the light source position data 133. Therefore, the positioning device 100 derives the position and orientation of the moving object 10 in the same manner as in Figure 6, based on the image positions of light sources 20a and 20d in image 60 and the spatial positions of light sources 20a and 20d in space 500. In this case, in Figure 6, the mountaintop 30a should be replaced with light source 20d, and point Q1 in image 60 should be replaced with point P2 corresponding to light source 20d.
[0038] Furthermore, if two light sources 20 can be detected from image 60, other targets 30 may also be used as position indicators. In other words, positioning may be performed using three or more position indicators, including the light sources 20 and the targets 30. For example, in image 60 shown in Figure 7, in addition to light sources 20a and 20d, a lunar base 30c (position in the image: point Q2) and a mountain peak 30b (position in the image: point Q3) can be detected as targets 30. Therefore, in addition to light sources 20a and 20d, at least one of the lunar base 30c and the mountain peak 30b may also be used as a position indicator. This makes it possible to further improve positioning accuracy compared to using only the two light sources 20.
[0039] Furthermore, using three or more position indicators allows for positioning with more degrees of freedom. That is, while using two position indicators allows for positioning with three degrees of freedom (horizontal position (XY coordinates) and horizontal direction), using three or more position indicators enables positioning with four or more degrees of freedom, including the vertical coordinate (Z coordinate) and at least a portion of the two degrees of freedom (pitch and roll) of attitude angle. Ideally, using six position indicators enables positioning with six degrees of freedom (three-dimensional coordinates (horizontal position and height) and three-axis attitude angles (yaw (horizontal direction), pitch, and roll)). If seven or more position indicators are detected, the positioning accuracy can be further improved by prioritizing the use of light source 20 as a position indicator.
[0040] Next, we will explain the positioning and movement process that the CPU 11 executes in order to move the mobile object 10 while performing the positioning described above. Figure 8 is a flowchart showing the control procedure for positioning and movement processing. The positioning and movement process is initiated when the mobile object 10 begins to move on the lunar surface.
[0041] When the positioning and movement process begins, the CPU 11 acquires the image 60 captured by the camera 14 (step S101). The CPU 11 also determines whether it is nighttime or not (step S102). Here, the CPU 11 determines whether it is nighttime or not based on the current time measured by the timekeeping unit 15. Alternatively, the CPU 11 may determine whether it is nighttime or not based on the brightness of the image 60. For example, it may be determined to be nighttime if the average brightness of the pixels in the image 60 is below a predetermined reference brightness.
[0042] If it is determined that it is nighttime ("YES" in step S102), the CPU 11 attempts to detect the light source 20 in the image 60 using the method described above and determines whether or not multiple light sources 20 have been detected in the image 60 (step S103). If it is determined that multiple light sources 20 have been detected ("YES" in step S103), the CPU 11 performs positioning based on the image and spatial positions of two or more of the detected multiple light sources 20 (i.e., derives the position and orientation of the moving object 10) and records the positioning result in the positioning history data 135 (step S104).
[0043] On the other hand, if it is determined that multiple light sources 20 could not be detected in image 60 ("NO" in step S103), the CPU 11 sends a control signal to the drive unit 16 to stop the movement of the mobile body 10 (step S105). This is because moving while the position and orientation of the device are unknown could lead to deviating significantly from the route to the destination or to unforeseen accidents. When step S105 is completed, the CPU 11 returns to step S101.
[0044] In step S102, if it is determined that it is not nighttime (i.e., it is daytime) ("NO" in step S102), the CPU 11 attempts to detect light sources 20 in image 60 and determines whether multiple light sources 20 have been detected in image 60 (step S106). If it is determined that multiple light sources 20 have been detected ("YES" in step S106), the CPU 11 proceeds to step S104 and performs positioning using two or more light sources 20.
[0045] If it is determined that multiple light sources 20 could not be detected in image 60 ("NO" in step S106), the CPU 11 determines whether the previous positioning result is recorded in the positioning history data 135 (step S107). If it is determined that the previous positioning result is not recorded ("NO" in step S107), the CPU 11 cannot obtain the current position of its own device necessary for identifying and detecting the target 30, so it proceeds to step S105 and stops the movement of the moving object 10.
[0046] If it is determined that the previous positioning result has been recorded ("YES" in step S107), the CPU 11 determines whether or not one light source 20 has been detected in the image 60 (step S108). If it is determined that one light source 20 has been detected ("YES" in step S108), the CPU 11 identifies and acquires candidate targets 30 that can be imaged by the camera 14 based on the previous positioning result and the current time measured by the timing unit 15 (step S109).
[0047] The CPU 11 attempts to detect a target 30 that matches the candidate in the image 60 (obtain the identification result of the target 30) and determines whether or not one or more targets 30 have been detected from the image 60 (step S110). If it is determined that no targets 30 were detected ("NO" in step S110), the CPU 11 moves the process to step S105 and stops the movement of the moving object 10. If it is determined that one or more targets 30 have been detected ("YES" in step S110), the CPU 11 performs positioning based on the image position and spatial position of the one or more targets 30 detected in step S108 and in step S110, and records the positioning result in positioning history data 135 (step S111).
[0048] On the other hand, if it is determined in step S108 that no light sources 20 were detected ("NO" in step S108), the CPU 11 identifies and acquires candidate targets 30 in the same way as in step S109 (step S112).
[0049] The CPU 11 attempts to detect targets 30 that match the candidates in the image 60 and determines whether or not it has detected two or more targets 30 from the image 60 (step S113). If it is determined that it could not detect two or more targets 30 ("NO" in step S113), the CPU 11 moves the process to step S105 and stops the movement of the moving object 10. If it is determined that it has detected two or more targets 30 ("YES" in step S113), the CPU 11 performs positioning based on the in-image and spatial positions of the two or more detected targets 30 and records the positioning result in positioning history data 135 (step S114).
[0050] When step S114 is completed, the CPU 11 reduces the speed of the moving object 10 to a low speed below a predetermined reference value (step S115). When positioning is performed using only the target 30 as a position indicator, the position accuracy is lower compared to positioning using the two light sources 20 as position indicators, so the error in the moving trajectory of the moving object 10 from the planned route tends to be large. Therefore, in step S115, the speed of the moving object 10 is reduced to suppress the expansion of this error. If the speed has already been reduced below the reference value at the start of step S115, step S115 is omitted. Furthermore, even after step S111 is completed, the process may proceed to step S115 to reduce the movement speed. That is, the movement speed of the moving body 10 may be reduced when positioning is performed using one or more targets 30 as position indicators. However, even when one or more targets 30 are used as position indicators, if two or more other light sources 20 are also used as position indicators, the movement speed of the moving body 10 may not be reduced.
[0051] When any of steps S104, S111, or S115 is completed, the CPU 11 determines a route to the destination based on the positioning results and moves the mobile body 10 along that route (step S116). The route to the destination is set so that multiple light sources 20 can be observed at any point along the route. The CPU 11 determines whether or not the destination has been reached (step S117). If the CPU 11 determines that the destination has not been reached ("NO" in step S117), it returns to step S101, and if it determines that the destination has been reached ("YES" in step S117), it terminates the positioning and movement process.
[0052] Alternatively, the flowchart shown in Figure 9 may be used instead of the flowchart shown in Figure 8. Figure 9 is a flowchart that uses the target 30 as a position indicator as much as possible, regardless of the detection status of the light source 20 in image 60, as long as it is detectable in image 60. This makes it possible to improve positioning accuracy as much as possible, depending on the detection status of the light source 20 and the target 30 in image 60.
[0053] Steps S201 to S205 of the positioning and movement process shown in Figure 9 are the same as steps S101 to S105 in Figure 8, so their explanation is omitted. In step S202, if it is determined that it is not nighttime ("NO" in step S202), the CPU 11 determines whether the previous positioning result is recorded in the positioning history data 135 (step S206). If it is determined that the previous positioning result is not recorded ("NO" in step S206), the CPU 11 proceeds to step S203. If it is determined that the previous positioning result is recorded ("YES" in step S206), the CPU 11 identifies and acquires candidate targets 30 that can be imaged by the camera 14 based on the previous positioning result and the current time measured by the timing unit 15 (step S207).
[0054] The CPU 11 attempts to detect targets 30 that match the acquired target candidates in the image 60 and determines whether or not it has detected one or more targets 30 from the image 60 (step S208). If it is determined that no targets 30 were detected ("NO" in step S208), the CPU 11 moves the process to step S203. If it is determined that one or more targets 30 were detected ("YES" in step S208), the CPU 11 attempts to further detect light sources 20 in the image 60 and determines whether or not it has detected two or more light sources 20 and / or targets 30 (step S209). If it is determined that only one or fewer light sources 20 and / or targets 30 were detected ("NO" in step S209), the CPU 11 determines that it cannot perform positioning and moves the process to step S205 to stop the movement of the mobile body 10. If it is determined that two or more light sources 20 and / or targets 30 have been detected ("YES" in step S209), the CPU 11 performs positioning based on the image and spatial positions of the detected light sources 20 and / or targets 30, and records the positioning result in positioning history data 135 (step S210). If a total of seven or more light sources 20 and targets 30 as position indicators are detected, the CPU 11 selects six position indicators, prioritizing the light sources 20, before starting step S210.
[0055] When step S210 or S204 is completed, the CPU 11 moves the process to step S211. Steps S211 and S212 are identical to steps S116 and S117 in Figure 8, so their explanation is omitted.
[0056] (effect) As described above, the positioning device 100 according to this embodiment is equipped with a CPU 11. The CPU 11 detects a light source 20 and a target 30 (such as a mountain peak 30a) different from the light source 20 in an image 60 captured by a camera 14 mounted on a moving body 10 moving in space 500. Based on the positions of the light source 20 and the target 30 in the image 60 and the known positions of the light source 20 and the target 30 in space 500, the CPU 11 derives the position of the moving body 10 in space 500. This enables positioning using two position indicators even in environments where the light from the light source 20 cannot be observed. For example, positioning becomes possible even when the light source 20 cannot be observed due to the influence of strong sunlight on the lunar surface during the daytime. Therefore, positioning can be performed more reliably.
[0057] Furthermore, when the CPU 11 detects one light source 20 and a target 30 in image 60, it derives the position of the moving object 10 based on the positions of the one light source 20 and the target 30 in image 60 and the known positions of the one light source 20 and the target 30 in space 500. Also, if a light source 20 is detected in image 60 and the detected light source 20 is one of several light sources 20, the CPU 11 derives the position of the moving object 10 based on the positions of at least two of the multiple light sources 20 in image 60 and the known positions of at least two of the light sources 20 in space 500. As a result, when there is only one detectable light source 20, positioning is possible by using the target 30 as a position indicator, and when multiple light sources 20 are detectable, positioning can be performed with high accuracy by using two or more light sources 20 as position indicators.Therefore, reliable positioning can be performed regardless of the environment, and high-precision positioning can be performed in suitable environments.
[0058] Furthermore, the CPU 11 detects the light source 20 and the target 30 in the image 60, and if the detected light source 20 is one of several light sources 20, it derives the position of the moving object 10 based on the positions of at least two of the multiple light sources 20 and the target 30 in the image 60, and the known positions of at least two of the light sources 20 and the target 30 in space 500. In this way, when two or more light sources 20 can be detected, using the target 30 as a position indicator allows for more accurate positioning.
[0059] Furthermore, the CPU 11 detects the light source 20 and the target 30 in the image 60, and if the detected target 30 is one of several targets 30, it derives the position of the moving object 10 based on the positions of the light source 20 and at least two of the targets 30 in the image 60, and the known positions of the light source 20 and at least two of the targets 30 in space 500. In this way, when there is only one detectable light source 20, the positioning accuracy can be improved by using two or more targets 30 as additional position indicators.
[0060] Furthermore, the CPU 11 obtains the identification result of the target 30 shown in the image 60 based on the last derived position of the moving object 10, and identifies the known position of the target 30 in space 500 based on the obtained identification result. This makes it possible to appropriately detect the target 30 according to the current position (most recent position) of the moving object 10.
[0061] Furthermore, the CPU 11 obtains the identification result of the object 30 shown in the image 60 based on the current time. This allows objects 30 that have been predetermined to become unobservable during certain time periods to be excluded from the detection target. Thus, objects 30 can be detected more appropriately.
[0062] Furthermore, the light source 20 emits light in a unique emission pattern, and the CPU 11 identifies the light source 20 based on the emission pattern identified from the image 60, and determines the known position of the light source 20 in space 500 based on the identification result of the light source 20. This makes it possible to determine the position of the light source 20 without transmitting or receiving radio waves. In addition, since the position of the light source 20 can be determined with high precision at the pixel-level resolution of the captured image 60, the positioning accuracy can be improved.
[0063] Furthermore, since the mobile body 10 according to this embodiment is equipped with the positioning device 100 and the camera 14, positioning can be performed more reliably.
[0064] Furthermore, the mobile body 10 is equipped with a CPU 11 as a control unit. When the CPU 11, as a processing unit, derives the position of the mobile body 10 based on the position of the target 30 in space 500, the CPU 11 reduces the mobile body 10's speed to a low speed below a reference value. When the target 30 is used as a position indicator, the accuracy of the positioned position tends to be low. By reducing the mobile body 10's speed, the increase in the error of the mobile body 10's position from the planned route can be suppressed.
[0065] Furthermore, the positioning method according to this embodiment detects the light source 20 and the target 30 in image 60, and derives the position of the moving object 10 in space 500 based on the positions of the light source 20 and the target 30 in image 60 and the known positions of the light source 20 and the target 30 in space 500. This enables more reliable positioning.
[0066] Furthermore, the program 131 according to this embodiment causes the CPU 11 to function as a processing means for detecting the light source 20 and the target 30 in the image 60, and deriving the position of the moving object 10 in space 500 based on the positions of the light source 20 and the target 30 in the image 60 and the known positions of the light source 20 and the target 30 in space 500. This enables more reliable positioning.
[0067] (others) It should be noted that the present invention is not limited to the embodiments described above, and various modifications are possible. For example, the mobile vehicle 10 is not limited to an autonomous lunar rover, but may also be a human-operated rover (including in autonomous driving mode) or a robot.
[0068] Furthermore, the positioning system 1 may be operated on Earth. For example, the present invention may be applied to positioning a vehicle (such as a forklift) as a moving object in a warehouse space.
[0069] Furthermore, in the above embodiment, the light emission pattern of the light source 20 is decoded to identify the light source ID, and the spatial position of the light source 20 is obtained from the light source ID. However, if the light emission pattern of the light source 20 contains a sufficient amount of information, the spatial position may be obtained directly by decoding the light emission pattern.
[0070] Furthermore, in positioning based on the image 60, the positioning device 100 only derives the position of the moving object 10, and the orientation of the moving object 10 may be derived by other means (for example, based on the detection result by the attitude angle detection unit 17).
[0071] Furthermore, while the explanation described using a combination of three colors of light—red, green, and blue—as an example of visible light communication using light source 20, it is not limited to this. For example, the colors used may be changed, or the number of colors used may be two or fewer, or four or more. In addition, Li-Fi (Light Fidelity), which transmits information using light modulated at high speed, may be used. Alternatively, optical communication using infrared or ultraviolet light may be used instead of visible light communication.
[0072] Furthermore, while the above description discloses examples in which the HDD and SSD of the storage unit 13 are used as computer-readable media for the program according to the present invention, the invention is not limited to these examples. Other computer-readable media that can be used include information recording media such as flash memory and CD-ROM. In addition, a carrier wave can also be used as a medium for providing the data of the program according to the present invention via a communication line.
[0073] Furthermore, it goes without saying that the detailed configuration and operation of each component of the mobile body 10 and the positioning device 100 in the above embodiment can be appropriately modified without departing from the spirit of the present invention.
[0074] Although embodiments of the present invention have been described, the scope of the present invention is not limited to the embodiments described above, but includes the scope of the invention as described in the claims and its equivalents. [Explanation of Symbols]
[0075] 1. Positioning system 10 Mobile Units 11 CPU 12 RAM12 13 Storage section 131 Programs 14. Camera (imaging unit) 15 Timing section 16. Drive unit 17 Attitude angle detection unit 20, 20a~20d light source 30 Target 30a, 30b summit (target) 30c Moon base (target) 40 depression 60 images 100 Positioning device 500 space
Claims
1. The system includes a processing unit that detects a light source and a target different from the light source in an image captured by an imaging unit provided on a moving body moving in space, and derives the position of the moving body in space based on the positions of the light source and the target in the image and the known positions of the light source and the target in space, The processing unit is a positioning device that obtains an identification result of the object captured in the image based on the last derived position of the moving object, and identifies the known position of the object in the space based on the obtained identification result.
2. The aforementioned processing unit, When one light source and the target are detected in the aforementioned image, the position of the moving body is derived based on the positions of the one light source and the target in the image and the known positions of the one light source and the target in the space. If the light source is detected in the image, and the detected light source is a plurality of light sources, the position of the moving object is derived based on the positions of at least two of the plurality of light sources in the image and the known positions of the at least two light sources in the space. do, The positioning device according to claim 1.
3. The processing unit detects the light source and the target in the image, and if the detected light source is a plurality of light sources, it derives the position of the moving body based on the positions of at least two of the plurality of light sources and the target in the image and the known positions of the at least two light sources and the target in the space. The positioning device according to claim 1.
4. The processing unit detects the light source and the target in the image, and if the detected target is a plurality of targets, it derives the position of the moving body based on the positions of the light source and at least two of the plurality of targets in the image and the known positions of the light source and the at least two targets in the space. The positioning device according to claim 1.
5. The processing unit further obtains the identification result of the object shown in the image based on the current time. The positioning device according to claim 1.
6. The light source emits light in a unique emission pattern, The processing unit identifies the light source based on the light emission pattern identified from the image, and identifies the known position of the light source in the space based on the light source identification result. The positioning device according to claim 1.
7. A positioning device according to any one of claims 1 to 6, The imaging unit, A mobile device equipped with the following features.
8. A positioning device comprising: an imaging unit provided on a moving body moving in space, which detects a light source and a target different from the light source in an image captured by the imaging unit, and a processing unit which derives the position of the moving body in space based on the positions of the light source and the target in the image and the known positions of the light source and the target in space, The imaging unit, When the processing unit of the positioning device derives the position of the moving body based on the position of the target in the space, the control unit reduces the moving speed of the moving body to a low speed below a reference value, A mobile device equipped with the following features.
9. A positioning method performed by a computer, In an image captured by an imaging unit provided on a moving body moving through space, a light source and a target different from the light source are detected, and the position of the moving body in space is derived based on the positions of the light source and the target in the image and the known positions of the light source and the target in space. A positioning method comprising obtaining an identification result of the object captured in the image based on the last derived position of the moving object, and determining the known position of the object in the space based on the obtained identification result.
10. A positioning method performed by a computer, In an image captured by an imaging unit installed on a moving object that moves through space, the light source and the front A target different from the marked light source is detected, and the position of the moving object in the space is derived based on the positions of the light source and the target in the image and the known positions of the light source and the target in the space. A positioning method that, when the position of the moving body is derived based on the position of the target in the aforementioned space, reduces the speed of the moving body to a low speed below a reference value.
11. A computer is configured to function as a processing means for detecting a light source and a target different from the light source in an image captured by an imaging unit provided on a moving body moving in space, and for deriving the position of the moving body in space based on the positions of the light source and the target in the image and the known positions of the light source and the target in space, The processing means is a program that obtains an identification result of the object captured in the image based on the last derived position of the moving object, and identifies the known position of the object in the space based on the obtained identification result.
12. A computer comprising: a processing means for detecting a light source and a target different from the light source in an image captured by an imaging unit provided on a moving body moving in space; and deriving the position of the moving body in space based on the positions of the light source and the target in the image and the known positions of the light source and the target in space; When the processing means derives the position of the moving body based on the position of the target in the space, a control means reduces the moving speed of the moving body to a low speed below a reference value. A program that makes something function as such.
Citation Information
Patent Citations
Unmanned running controller
JP2000284830A
Positioning system, positioning method, signal transmitter, signal receiver, and control program
JP2021076401A
Control device for autonomous travel type automatic conveyance vehicle and conveyance system
JP2021096731A
Systems and methods for optical target based indoor vehicle navigation
US20190265722A1
Control device, mobile body, and program
WO2019167210A1
