Self-localization method and remote work system
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- HITACHI GE NUCLEAR ENERGY LTD
- Filing Date
- 2025-01-24
- Publication Date
- 2026-08-05
Smart Images
Figure 2026126848000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for estimating the self-position of a moving body and a remote operation system.
Background Art
[0002] In nuclear power plants, a remote operation system that substitutes robots for tasks such as reactor decommissioning and inspections is utilized to improve the safety and reduce the burden of workers in a high-radiation environment. In this system, in order to make the robot execute a predetermined task at a predetermined position, it is necessary to accurately and safely guide the robot to the destination.
[0003] As a typical method for guiding a robot to a destination, SLAM (Simultaneous Localization And Mapping) is known. SLAM is a method in which while measuring the surrounding environment without prior knowledge using a measuring instrument mounted on the robot, the estimation of the self-position and the creation of a three-dimensional environmental map are carried out in parallel, and there are various methods according to the type of measuring instrument.
[0004] For example, LiDAR-SLAM, which is a guiding method when LiDAR is mounted, is a method for estimating the self-position and the environmental map in a three-dimensional space using environmental information (point cloud and depth image) measured three-dimensionally by LiDAR, and can achieve excellent accuracy. However, since LiDAR is a measuring instrument using a highly integrated semiconductor, it has the characteristics of low radiation resistance and low heat resistance. Therefore, LiDAR-SLAM is a guiding method that is not suitable for a working robot planned for long-term operation in a harsh environment.
[0005] Furthermore, Visual-SLAM, a guidance method used when a 2D camera is installed, creates 3D information using triangulation by combining environmental information (camera images) measured 2D by the 2D camera, and estimates the self-position and environmental map in 3D space. However, its accuracy is inferior to LiDAR-SLAM. Nevertheless, because 2D cameras use semiconductors with a lower integration density than LiDAR, they have the characteristic of being more resistant to radiation and heat than LiDAR. Therefore, Visual-SLAM is a suitable guidance method for work robots that are scheduled to operate for long periods in harsh environments.
[0006] A robot guidance method that takes such measuring instrument characteristics into consideration is proposed in Patent Document 1. For example, the abstract of the document states: "The system comprises a measuring robot 10 and a work robot 20, and a remote control device 30 that can remotely control each robot 10 and 20 from outside the work area 1. (A) The measuring robot 10 performs three-dimensional measurement of obstacles 3 in the work area 1 and transmits the three-dimensional data to the remote control device 30. (B) The remote control device 30 creates an environmental map and a movement path 7 from the three-dimensional data and transmits them to the measuring robot 10. (C) The measuring robot 10 marks marks 4 sequentially on the road surface along the movement path 7 based on the environmental map. (D) The work robot 20 moves to the work point 8 by following the marks 4 sequentially." This discloses a technology that uses a measuring robot responsible for three-dimensional measurement and marking, and a work robot that moves by following the markings.
[0007] Furthermore, paragraph 0031 of the same document explains that "the measurement robot 10 has a three-dimensional measuring instrument 12 for three-dimensional measurement of obstacles 3 (see Figure 1) within the work area 1, a marking device 14 for marking marks 4 on the road surface 2, and a first communication device 16 capable of bidirectional wireless communication with the remote control device 30. The three-dimensional measuring instrument 12 is, for example, a laser rangefinder (LRF) or a three-dimensional laser radar. The marking device 14 is, for example, a paint sprayer, and is configured to mark single-color or multi-colored marks 4 on the road surface 2." As explained in paragraph 0034, "The work robot 20 has a work device 22 that performs predetermined work within the work area 1, a positioning camera 24 that detects marks 4 on the road surface 2, and a second communication device 26 that can communicate wirelessly in both directions with the remote control device 30. ... The positioning camera 24 is installed near the work device 22 and takes images of the marks 4." This discloses a combination in which an advanced measuring instrument such as a 3D laser radar is mounted on the measuring robot and a simple measuring instrument such as a positioning camera is mounted on the work robot. [Prior art documents] [Patent Documents]
[0008] [Patent Document 1] Japanese Patent Publication No. 2014-203146 [Overview of the project] [Problems that the invention aims to solve]
[0009] However, the technology described in Patent Document 1 involves a work robot moving by following markers marked by a measuring robot. Therefore, it was difficult to use it to guide work robots that move in the air, on the water surface, or underwater, where marking is impossible, or in environments where the visibility of markers on the road surface is easily deteriorated due to the effects of dust, etc.
[0010] Therefore, the present invention aims to provide a self-position estimation method and a remote work system that can move a work robot (second mobile unit) to a desired destination without using markers, in an environment in which a measurement robot (first mobile unit) equipped with an advanced 3D measuring instrument and a simple 2D measuring instrument is used in combination with a work robot (second mobile unit) equipped with a simple 2D measuring instrument. [Means for solving the problem]
[0011] To solve the above problems, the present invention provides a self-position estimation method for estimating the self-position of a second mobile body using the results of acquiring the working environment by a first mobile body, wherein (a) a first environment map is generated by accumulating a 3D point cloud acquired by the first mobile body from the environment in 3D space, (b) a first self-position is estimated, which is the 3D relative position of the first mobile body within the first environment map, (c) a second environment map is created by accumulating the color information of a 2D first image acquired by the first mobile body in 3D space with reference to the first self-position, and (d) a second self-position is estimated, which is the 3D relative position of the second mobile body in the second environment map, by comparing a 2D second image acquired by the second mobile body with the second environment map. [Effects of the Invention]
[0012] According to the present invention, in an environment where a measurement robot (first mobile unit) equipped with an advanced 3D measuring instrument and a simple 2D measuring instrument is used in combination with a work robot (second mobile unit) equipped with a simple 2D measuring instrument, the work robot (second mobile unit) can be moved to a desired destination without using markers. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments. [Brief explanation of the drawing]
[0013] [Figure 1] A schematic diagram of a remote work system according to one embodiment. [Figure 2] A functional block diagram of a remote work system according to one embodiment. [Figure 3]A flowchart illustrating the update process for the first self-position, first environment map, and second environment map. [Figure 4] A flowchart illustrating the process of updating the second self-position. [Figure 5A] A diagram illustrating map updating when the second environment map is a 3D point cloud. [Figure 5B] A diagram illustrating position estimation when the second environmental map is a 3D point cloud. [Figure 5C] A diagram illustrating map updates when the second environment map is a function. [Figure 5D] A diagram illustrating position estimation when the second environment map is a function. [Figure 6] An example of how to operate a remote work system according to one embodiment. [Figure 7] Another example of how to operate a remote work system according to one embodiment. [Modes for carrying out the invention]
[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that each figure is merely a schematic representation to allow for a sufficient understanding of the present invention. Therefore, the present invention is not limited to the illustrated example. Furthermore, in each figure, common or similar components are denoted by the same reference numerals, and their redundant descriptions are omitted.
[0015] <Configuration of the remote work system> First, referring to the schematic diagram of FIG. 1 and the block diagram of FIG. 2, the configuration of the remote operation system 100 according to this embodiment will be described. The remote operation system 100 shown here is a system for causing a measurement robot 1 (first moving body) and a work robot 2 (second moving body) that can be operated in a harsh working environment E such as a high-radiation environment of a nuclear power plant to execute a predetermined operation at a predetermined position in the working environment E by controlling them with a control device 3 installed at a remote location. In FIG. 1, a configuration in which the measurement robot 1, the work robot 2, and the control device 3 are connected by wire and signal transmission / reception and power supply to the robots are performed by wire is illustrated. However, instead of this configuration, a configuration may be adopted in which wireless communication is used for signal transmission / reception and necessary power is supplied from the built-in batteries of each robot. Details of each will be sequentially described below.
[0016] <Measurement robot 1> The measurement robot 1 is a moving body that acquires a low-precision two-dimensional image (first image P1) and a high-precision three-dimensional point cloud (point cloud D) while moving in the working environment E, and includes an optical measuring instrument 11, a point cloud measuring instrument 12, a communication device 13, and a moving mechanism 14. Here, the "low precision" and "high precision" mentioned here mean the relative precision when comparing the information granularity of the first image P1 and the point cloud D.
[0017] The optical measuring instrument 11 is a measuring instrument that optically acquires information (hereinafter referred to as "color information") regarding the color, brightness, etc. of an object (road surface, wall surface, obstacle, etc.) existing in the working environment E as a two-dimensional image (first image P1), and is, for example, a color camera, a monochrome camera, an infrared camera, or the like. It is desirable that this optical measuring instrument 11 has as large a measurement range as possible and a small pixel resolution (high resolution), and a plurality of units may be combined.
[0018] The point cloud measuring instrument 12 is a measuring instrument that acquires a three-dimensional point cloud D of the shape of objects (road surface, wall surface, obstacles, etc.) present in the work environment E, and is, for example, a laser rangefinder or a depth camera. It is desirable that the point cloud measuring instrument 12 has as large a field of view and point cloud density as possible, and multiple units may be combined. Although measuring instruments such as laser rangefinders and depth cameras generally tend to have low radiation resistance and heat resistance, the measuring robot 1 is a mobile device intended for short-term operation in the harsh work environment E, so even a point cloud measuring instrument 12 with low radiation resistance and heat resistance does not pose any particular problem.
[0019] The communication device 13 is a device for communicating with the control device 3 by wire or wireless connection. This communication device 13 allows the measurement robot 1 to sequentially transmit the first image P1 and point cloud D acquired by the robot to the control device 3, and to receive movement commands from the control device 3.
[0020] The movement mechanism 14 is a mechanical mechanism for the measurement robot 1 to move on land, in the air, on water, and underwater within the work environment E, and can be, for example, a crawler, multi-legged robot, tires, a screw, or a propeller. The movement mechanism 14 operates based on movement commands received from the control device 3 via the communication device 13, and sequentially controls the position of the measurement robot 1.
[0021] <Working Robot 2> The work robot 2 is a mobile unit that moves through the work environment E, acquires low-resolution two-dimensional images (second image P2), and performs predetermined tasks at predetermined locations. It has an optical measuring instrument 21, a communication device 22, and a movement mechanism 23. Here, "low-resolution" means information granularity equivalent to that of the first image P1.
[0022] Since the work robot 2 is scheduled to operate in work environment E for an extended period until its predetermined tasks are completed, if work environment E is a high-radiation environment within a nuclear power plant, it may be exposed to radiation from radioactive materials for an extended period. Therefore, it is desirable that the work robot 2 has a device configuration that is more radiation-resistant than the measurement robot 1.
[0023] The optical measuring instrument 21 is a measuring instrument that optically acquires information regarding the color and brightness of objects (road surface, wall surface, obstacles, etc.) present in the work environment E as a two-dimensional image (second image P2), such as a color camera, monochrome camera, or infrared camera. The optical measuring instrument 21 should preferably have a large measurement range and low pixel resolution (high resolution), and multiple units may be combined. To achieve high radiation resistance, it is desirable to use vacuum tubes or low-integration semiconductors.
[0024] The communication device 22 is a device for communicating with the control device 3 via wired or wireless connection. This communication device 22 allows the work robot 2 to sequentially transmit the second image P2 it has acquired to the control device 3, and to sequentially receive movement commands and work commands from the control device 3.
[0025] The mobility mechanism 23 is a mechanical mechanism for the work robot 2 to move on land, in the air, on water, and underwater within the work environment E, and can be, for example, a crawler, multiple legs, tires, a screw, or a propeller. The mobility mechanism 23 operates based on movement commands received from the control device 3 via the communication device 22, and sequentially controls the position of the work robot 2.
[0026] Furthermore, the robot 2 may be equipped with additional work mechanisms depending on the type of work to be performed in the work environment E. For example, if the work to be performed by the robot 2 is to process, cut, grip, or clean an object, then a work mechanism such as a multi-joint arm with a gripper can be added to the robot 2. On the other hand, if the work to be performed by the robot 2 is visual inspection and the visual inspection can be performed based on the second image P2 acquired by the optical measuring instrument 21, then no additional work mechanisms are necessary.
[0027] <Control device 3> The control device 3 is a device for remotely controlling the measuring robot 1 and the work robot 2, which are installed outside the harsh working environment E, and includes a display unit 31, an input unit 32, a calculation unit 33, a storage unit 34, and a communication device 35.
[0028] The display unit 31 is a device for presenting various information about the work environment E, the measurement robot 1, and the work robot 2 to the operator, and is, for example, a liquid crystal display.
[0029] The input unit 32 is a device for the operator to input commands related to the measurement robot 1 and the work robot 2, and is, for example, a keyboard, mouse, buttons, switches, levers, etc.
[0030] The calculation unit 33 is a functional unit that estimates the first self-position L1, which is the three-dimensional relative position of the measurement robot 1, and the second self-position L2, which is the three-dimensional relative position of the work robot 2, and generates and updates the first environment map M1, which is used to estimate the position of the measurement robot 1 (first self-position L1), and the second environment map M2, which is used to estimate the position of the work robot 2 (second self-position L2). Specifically, it is a computer such as a CPU. The first environment map M1 is a three-dimensional map defined by a point cloud placed in three-dimensional space, and the second environment map M2 is a three-dimensional map defined by a point cloud or function placed in three-dimensional space.
[0031] The memory unit 34 is a functional unit that stores the first self-position L1, the first environment map M1, the second self-position L2, and the second environment map M2, and specifically, it is a storage medium such as an HDD or SSD.
[0032] The communication device 35 is a device for communicating with the measurement robot 1 and the work robot 2 by wire or wireless connection. This communication device 35 can sequentially receive the first image P1 and point cloud D from the measurement robot 1, sequentially receive the second image P2 from the work robot 2, and sequentially transmit various commands to the measurement robot 1 and the work robot 2.
[0033] <Method for self-localization of measurement robot 1 and work robot 2> Here, the self-position estimation method according to this embodiment will be explained using the flowcharts in Figures 3 and 4. Note that the processes in Figures 3 and 4 are performed in parallel, and it is assumed that the first self-position L1, second self-position L2, environment map M1, and second environment map M2, which were estimated or generated in the past (i.e., at the positions where the measurement robot 1 and the work robot 2 were previously located), are registered in the storage unit 34.
[0034] <<Update process for the second environmental map M2>> Figure 3 corresponds to the process of sequentially updating the first self-position L1, the first environment map M1, and the second environment map M2 within the memory unit 34.
[0035] In step S1, the communication device 35 receives the latest first image P1 and point cloud D acquired by the measurement robot 1 at its current position from the communication device 13.
[0036] In step S2, the arithmetic unit 33 reads the previous first self-position L1 and the first environment map M1 from the storage unit 34.
[0037] In step S3, the calculation unit 33 compares the latest point cloud D acquired in step S1 with the previous first environment map M1 acquired in step S2, and estimates the latest first self-position L1 in the previous first environment map M1 by searching for the position where the two match most. For example, in this step, the current position may be determined by calculating the movement speed of the measurement robot 1 from the time series change of the first self-position L1, calculating the product of this speed and the elapsed time since the time when the previous first self-position L1 was estimated, and adding this to the most recent first self-position L1. Alternatively, the current position may be determined by finding the position where the error between the two point clouds is minimized using the ICP (Iterative Closest Point) algorithm. These two position search methods may also be combined.
[0038] In step S4, the calculation unit 33 updates the first environment map M1 based on the latest first self-position L1 estimated in step S3 and the latest point cloud D acquired in step S1. For example, for areas not included in the previous first environment map M1, the first environment map M1 is expanded by adding the information from the latest point cloud D. For areas already included in the previous first environment map M1, the first environment map M1 is updated with the information from the latest point cloud D.
[0039] In step S5, the calculation unit 33 registers the first environment map M1 updated in step S4 and the first self-position L1 estimated in step S3 in the storage unit 34.
[0040] In step S6, the arithmetic unit 33 reads the previous second environment map M2 from the storage unit 34.
[0041] In step S7, the calculation unit 33 updates the second environment map M2 by arranging the color information of the first image P1 received in step S1 in three-dimensional space, based on the first self-position L1 estimated in step S3. Details of this step will be described later.
[0042] In step S8, the arithmetic unit 33 registers the second environment map M2, which was updated in step S7, in the storage unit 34.
[0043] Based on the processing shown in Figure 3, the second environmental map M2 for self-position estimation of the work robot 2 can be updated based on the first image P1 and point cloud D received from the measurement robot 1.
[0044] <<Update process for the second self-position L2>> Figure 4 corresponds to the process of sequentially updating the second self-position L2 within the memory unit 34.
[0045] In step S11, the communication device 35 receives the latest second image P2 acquired by the work robot 2 at its current position from the communication device 22.
[0046] In step S12, the arithmetic unit 33 reads the previous second self-position L2 and the second environment map M2 from the storage unit 34.
[0047] In step S13, the calculation unit 33 compares the latest second image P2 acquired in step S11 with the previous second environment map M2 acquired in step S12 in the three-dimensional coordinate system of the second environment map M2, and estimates the latest second self-position L2 in the previous second environment map M2 by searching for the position where the two match most. Since the second image P2 is two-dimensional information and the second environment map M2 is three-dimensional information, the position can be searched by projecting the second image P2 onto a three-dimensional space and comparing it with the second environment map M2, or conversely, by projecting the second environment map M2 onto a two-dimensional surface and comparing it with the second image P2. Details of this step will be described later.
[0048] In step S14, the arithmetic unit 33 adds the second self-position L2, which was searched in step S13, to the storage unit.
[0049] Following the above flow, the second self-position L2 of the work robot can be estimated from the second image P2 using the second environment map M2, which is based on information estimated with high accuracy from the point cloud D of the measurement robot 1. This eliminates the need for marking the environment and provides a self-position estimation method that is superior to simultaneous execution of map generation and self-position estimation using only the camera image of the work robot (Visual-SLAM).
[0050] <Details of the update process for the second environmental map M2 and the estimation process for the second self-position L2> Next, using Figures 5A and 5C, the details of the update process of the second environment map M2 in step S7 will be explained, and using Figures 5B and 5D, the details of the estimation process of the second self-position L2 in step S13 will be explained. Hereafter, the point cloud D and first image P received by the control device 3 at time (t) will be described. 1、 The second image P2 is denoted by the subscript (t). Also, the point cloud D at reception time (t) and the first image P 1、 The first self-position L1, second self-position L2, first environment map M1, and second environment map M2, which were updated in the second image P2, are denoted by the subscript (t).
[0051] <<When the second environmental map M2 is composed of a 3D point cloud>> Figure 5A illustrates the update process for the second environment map M2 when it is composed of a three-dimensional point cloud.
[0052] Step S7 describes the process of updating the second environment map M2(t-1), which was read from the storage unit in step S6, to M2(t) based on the first self-position L1(t), the first image P1(t), and the first self-position L1(tn) and first image P1(tn) at a past time (tn).
[0053] In step S7, feature points on the image plane (pixels with values unique to the surrounding pixels in a 2D image, specifically pixels corresponding to object boundaries, corners, or specific patterns) are extracted from both first images P1. Methods for extracting characteristic pixels from the first image P1 based on their pixel values include, for example, SIFT using an explicit function or methods using a pre-trained neural network.
[0054] After extracting feature points from both first images P1, the point cloud of the second environment map M2 is updated so that the same points as those extracted from the first image P1(t) at time (t) are reproduced on the plane of the first self-position L1(t) at time (t). More specifically, for identical feature points included in the image plane of past time (tn) and the image plane of time (t), the 3D coordinates are calculated from the 2D coordinates on the image plane and the first self-position L1 at the time each image was captured, and these points are added to and updated in the second environment map M2. If multiple feature points can be extracted from the first image P1(t) at time (t), the 3D coordinates may be obtained using feature points from the image planes of different past times (t-n1) and (t-n2), as shown in Figure 5A.
[0055] Figure 5B illustrates the estimation process for the second self-position L2 when the second environment map M2 is composed of a 3D point cloud.
[0056] Step S13 describes the process of estimating the second self-position L2(t'), which is the current position of the work robot 2, based on the second image P2(t') at time (t') and the most recent second environment map M2(t'-n3) at time (t'). As mentioned above, the flow in Figure 3 and the flow in Figure 4 may be executed asynchronously, so time (t) and time (t') may be the same or different.
[0057] In step S13, feature points are extracted from the second image P2(t') using the same method as used in step S7. Then, the feature points extracted in step S13 are compared with the point cloud contained in the second environment map M2(t'-n3), and the position that is most consistent is determined as the second self-position L2(t'). For example, the epipolar lines are projected onto the second environment map M2(t'-n3), and the position that intersects most frequently with the point cloud of the second environment map M2(t'-n3) is determined as the second self-position L2(t').
[0058] Thus, in step S13, the second self-position L2, which is the self-position of the work robot 2, is estimated by comparing the feature points in the second image P2 acquired by the optical measuring instrument 21 of the work robot 2 with the feature points in the second environment map M2 based on the first image P1 acquired by the optical measuring instrument 11 of the measurement robot 1, which are homogeneous (same granularity) point clouds. Therefore, the second self-position L2 can be estimated more accurately than when comparing the feature points in the second image P2 with the heterogeneous (different granularity) point cloud in the first environment map M1 acquired by the point cloud measuring instrument 12 of the measurement robot 1.
[0059] <<If the second environmental map M2 consists of a function capable of generating a 2D image>> Figure 5C illustrates the update process for the second environment map M2 when the second environment map M2 is composed of a function capable of generating a two-dimensional image viewed from an arbitrary position in an arbitrary direction.
[0060] In step S7, a function capable of generating a third image P3, which corresponds to a 2D image viewed from any position and in any direction within the working environment E, is learned by using time (t) and numerous pairs of first images P1 and first self-position L1 from past time points as training data, and this is designated as the second environment map M2(t). Known learning methods for such functions include, for example, photogrammetry, NeRF, and Gaussian splatting. By using any of these learning methods, a function that minimizes the error between the first image P1 and the third image P3 under equivalent shooting conditions can be learned through optimization calculation.
[0061] Figure 5D illustrates the estimation process of the second self-position L2 when the second environment map M2 is composed of a function capable of generating a two-dimensional image viewed from an arbitrary position in an arbitrary direction.
[0062] In step S13, the current second image P2(t') is compared with the third image P3(t') generated from the second environment map M2(t'-n3), which is the most recent function at time (t'). The position where the third image P3 best matches the second image P2 is searched for, and this position is estimated to be the second self-position L2(t'), which is the current position of the work robot 2. As mentioned above, the flow in Figure 3 and the flow in Figure 4 may be executed asynchronously, so time (t) and time (t') may be the same or different. Also, the comparison between the second image P2 and the third image P3 may be done by directly comparing each pixel value, or by comparing feature points extracted from each image.
[0063] Through the above process, it becomes possible to update the second environment map M2 with high accuracy based on the first self-position L1, and to estimate the second self-position L2 with high accuracy based on the second environment map M2.
[0064] Generally, the methods in Figures 5A and 5B have a lower computational load than the methods in Figures 5C and 5D, and have the advantage of being able to update the second environment map M2 and estimate the second self-position L2 at a high frequency. On the other hand, unlike the methods in Figures 5A and 5B, which update the second environment map M2 with extracted feature point information, the methods in Figures 5C and 5D update the second environment map M2 based on all the information of the first image P1, so they can more appropriately correct for differences in how the same object looks from different viewpoints. For example, it is known that even for the same object, the positional relationship with the light source changes depending on the viewpoint, and images with different color information are obtained. However, in the methods in Figures 5A and 5B, the viewpoint of the first image P1 when updating the second environment map M2 and the viewpoint of the second image P2 when comparing them do not necessarily match, and the color information may differ. Because feature points are extracted based on color information, the extracted points of the first image P1 and the extracted points of the second image P2 registered in the second environment map M2 may not match, which can lead to a decrease in accuracy. On the other hand, the methods in Figures 5C and 5D generate a third image P3 from the same viewpoint as the second image P2 and then compare it with the second image P2, allowing for highly accurate estimation of the second self-position L2. Therefore, depending on the estimation accuracy and period of the second self-position L2 required for performing remote work, the methods in Figures 5A and 5B or Figures 5C and 5D should be selected. Alternatively, the high-period, low-accuracy methods in Figures 5A and 5B may be combined with the low-period, high-accuracy methods in Figures 5C and 5D.
[0065] <Operation method for measurement robot 1 and work robot 2> The specific operation method of the remote work system 100 of this embodiment will be described below using Figures 6 and 7. The operation method shown in both figures is designed to enable accurate self-position estimation of the work robot 2, which works for a long period of time near radioactive materials with high dose rates, while suppressing the cumulative radiation exposure of the measurement robot 1, which is equipped with a point cloud measuring instrument 12 with low radiation resistance.
[0066] <<First Operation Method>> Figure 6 is a schematic diagram of the operational arrangement of two robots: a work robot 2 stationed at a high-dose-rate work site to perform tasks such as collecting radioactive materials, and a measurement robot 1 stationed at a low-dose-rate location (for example, a location relatively far from the radioactive material and behind a wall from the perspective of the radioactive material) to perform environmental measurements.
[0067] Generally, the amount of radiation exposure from a radioactive material decreases inversely proportional to the square of the distance from that radioactive material. Therefore, if a radioactive material is localized within the work environment E, or if radioactive nuclide dust is generated near the work site due to the processing of radioactive materials, the increase in the cumulative dose of the measurement robot 1 can be slowed down by ensuring a sufficient distance between the measurement robot 1 and the work site.
[0068] The self-position estimation method described in Figures 3 and 4 is a method for estimating the second self-position L2, which is the current position of the work robot 2, by comparing the second environment map M2, created based on the first image P1 and point cloud D received from the measurement robot 1, with the second image P2, received from the work robot 2. Therefore, even if the work robot 2 cannot be directly seen from the measurement robot 1, if the first image P1 acquired by the measurement robot 1 and the second image P2 acquired by the work robot 2 contain a common object (the same area of the wall in the example of Figure 6), the second self-position L2 can be accurately estimated based on the latest information. Thus, by hiding behind structures in the work environment E and shielding from radiation directly reaching from the work point to reduce the radiation exposure of the measurement robot 1, while including walls and ceilings to the sides and above in the first image P1 and second image P2, it is possible to estimate the second self-position L2 of the work robot 2 based on the latest information.
[0069] Furthermore, increasing the distance between the work site and the measurement robot 1 tends to decrease the number of pixels occupied by objects common to both the first image P1 and the second image P2, thus reducing the estimation accuracy of the second self-position L2. In other words, with the distance between the work site and the measurement robot 1 as a variable, the radiation exposure of the measurement robot 1 and the estimation accuracy of the second self-position L2 of the work robot 2 are inversely proportional.
[0070] Therefore, when searching for the second self-position L2 in step S13, the magnitude of the discrepancy between the remaining second image P2 and the second environment map M2 can be used as a quantitative indicator of the estimation accuracy of the second self-position L2, thereby adjusting the conflicting radiation exposure of the measurement robot 1 and the estimation accuracy of the second self-position L2 of the work robot 2.
[0071] <<Second Operation Method>> Figure 7 is a schematic diagram illustrating a configuration in which a work robot 2 is kept at a high-dose-rate work site for an extended period to continue tasks such as collecting radioactive materials, while a measurement robot 1 is engaged in environmental measurement of the work environment E for short periods only, and is evacuated from the work environment E for the rest of the time, thereby allowing the measurement robot 1 to intermittently measure the work environment E.
[0072] Generally, the radiation resistance of each piece of equipment can be evaluated by its cumulative dose, which is the total amount of radiation exposure before it malfunctions due to radiation. Since the cumulative dose is determined by the product of the dose rate per unit time and time, the increase in the cumulative dose can be suppressed by limiting the time spent in a radiation environment.
[0073] In the method for updating the second environmental map M2, as explained in Figure 3, objects that were included in the past first image P1 are reflected in the current second environmental map M2. Furthermore, since the current second self-position L2 is estimated by comparing the current second environmental map M2 with the current second image P2, it is not necessary for the current first image P1 and the current second image P2 to contain common objects. In other words, it is not necessary to operate the measurement robot 1 and the work robot 2 simultaneously within the work environment E at all times. After the measurement robot 1 acquires the first image P1 and point cloud D across the entire work area, it is possible to move the measurement robot 1 outside the work environment until the work environment E changes significantly due to the effects of work by the work robot 2, etc. Thus, by shortening the period of stay in the radiation environment, the increase in the cumulative dose of the measurement robot 1 can be suppressed.
[0074] Furthermore, in the method for estimating the second self-position L2 explained in Figure 4, in step S13, the second image P2 and the second environment map M2 are compared to search for the second self-position L2 that minimizes the discrepancy between the two pieces of information. Therefore, even if there is some discrepancy between the information in the second image P2 and the second environment map M2, it is possible to accurately estimate the second self-position L2. In other words, for a certain area shown in the current second image P2, the second environment map M2 is created based on the past first image P1, and even if there has been a change in that area from that point to the present, it is acceptable as long as the proportion of the changed area that occupies within the second image P2 is relatively small. That is, after acquiring the first image P1 and point cloud D for the entire work environment, the measurement robot 1 is basically moved out of the work environment, and thereafter, in response to the occurrence of a large change in the work environment, it is sufficient to repeatedly put it back into the work environment E for remeasurement and then move it back after the remeasurement is complete.
[0075] In the actual work environment E, in addition to the work object whose shape and surface pattern change as the work progresses, there are structures such as the floor, walls, and ceiling that do not change as the work progresses, and structures often make up the majority of the environment. Furthermore, even in cases where the work object is scattered on the surface of a structure and the work object makes up the majority of the environment, the proportion of the area that changes due to the work per unit time that covers the entire work environment is often small. Therefore, the frequency of having to reintroduce the measurement robot 1 into the work environment E due to the effects of environmental changes is low, and a certain degree of reduction in the time the measurement robot 1 stays in the radiation environment by temporarily moving the measurement robot 1 outside the work environment can be expected.
[0076] Furthermore, increasing the remeasurement interval by the measurement robot 1 increases the discrepancy between the update time of the second environmental map M2 and the time of the second image P2, which tends to decrease the estimation accuracy of the second self-position L2. In other words, with the remeasurement interval by the measurement robot 1 as a variable, the radiation exposure of the measurement robot 1 and the estimation accuracy of the second self-position L2 of the work robot 2 are inversely proportional.
[0077] Therefore, when searching for the second self-position L2 in step S13, the magnitude of the discrepancy between the remaining second image P2 and the second environment map M2 can be used as a quantitative indicator of the estimation accuracy of the second self-position L2, thereby adjusting the conflicting radiation exposure of the measurement robot 1 and the estimation accuracy of the second self-position L2 of the work robot 2.
[0078] As described above, a series of remote operations are performed by either the first operation method in Figure 6 or the second operation method in Figure 7, or by switching the operation method depending on the work content of the work robot and changes in the work environment E. This makes it possible to suppress the cumulative radiation exposure of the measurement robot 1, which is equipped with a point cloud measuring instrument 12 that has relatively low radiation resistance, in a radiation environment. In other words, the measurement robot 1 can be operated for a long period of time in a radiation environment, and the estimation of the second self-position L2 of the work robot 2 can be continued. Thus, an operation method for the remote work system 100 is provided for work robot 2 to perform work near radioactive materials with high dose rates and for long periods of time.
[0079] Furthermore, the present invention is not limited to radiation environments, but is equally effective for other harsh working environments E that contain or generate toxic gases, dust, water, fire, high-temperature heat sources, etc. This is because, like radiation, the impact of these risk factors can be reduced by maintaining distance from the risk source, and the time to failure can be evaluated by exposure time, just like with radiation.
[0080] The present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above are described in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those having all the configurations described.
[0081] Furthermore, for example, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them, for example, using integrated circuits. Alternatively, each of the above-mentioned configurations, functions, etc., may be implemented in software by a processor interpreting and executing programs that realize each function. Information such as programs and files that realize each function can be stored in recording media such as semiconductor memory, HDDs (Hard Disk Drives) and SSDs (Solid State Drives), semiconductor memory cards, and optical discs.
[0082] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and do not necessarily represent all control lines and information lines in the actual product. In practice, it can be assumed that almost all components are interconnected. [Explanation of Symbols]
[0083] 100 Remote Work Systems 1. Measurement robot 11 Optical measuring instruments 12-point cloud measuring instrument 13. Communication equipment 14 Moving mechanism 2. Work robots 21 Optical Measuring Instruments 22 Communication equipment 23 Moving mechanism 3. Control device 31 Display section 32 Input section 33 Arithmetic section 34 Storage section 35 Communication equipment
Claims
1. A self-position estimation method for estimating the self-position of a second mobile object using the results of acquiring the working environment by a first mobile object, (a) The first mobile body generates a first environmental map by accumulating the three-dimensional point cloud acquired from the environment in three-dimensional space, (b) Estimate the first self-position, which is the three-dimensional relative position of the first moving object within the first environmental map. (c) Create a second environmental map by accumulating the color information of the two-dimensional first image acquired by the first moving object in three-dimensional space with reference to the first self-position, (d) A self-position estimation method characterized by estimating a second self-position, which is the three-dimensional relative position of the second mobile body in the second environmental map, by comparing a two-dimensional second image acquired by the second mobile body with the second environmental map.
2. In the self-localization method according to claim 1, In (c) above, (c1) Extract feature points on the image plane based on the pixel values in the first image, (c2) For the same feature point on the image plane of multiple first images taken at different times, the three-dimensional coordinates of the feature point are calculated based on the first self-position at the time of each first image and the two-dimensional coordinates of the feature point on the image plane. (c3) The second environmental map is created by accumulating the three-dimensional coordinates of multiple feature points. In (d) above, (d1) Based on the pixel values in the second image, feature points are extracted from the image plane. (d2) A self-position estimation method characterized by estimating the second self-position by matching feature points in the second image with feature points in the second environment map.
3. In the self-localization method according to claim 1, In (c) above, a function capable of generating a two-dimensional third image viewed from an arbitrary position in an arbitrary direction is generated as a second environment map by learning the pair of the first image and the first self-position as training data. The self-position estimation method in (d) above is characterized by comparing each of the multiple third images generated from the function with the second image, and estimating the position where the third image that best matches the second image is obtained as the second self-position.
4. In the self-localization method described in claim 3, A self-localization method characterized in that the function is learned by one of the following learning methods: photogrammetry, NeRF, or Gaussian splatting.
5. A remote work system comprising a first mobile body and a second mobile body positioned within the work environment, and a control device positioned outside the work environment, The first mobile body is A first optical measuring instrument that acquires the color information of the aforementioned work environment as a two-dimensional first image, A point cloud measuring instrument that acquires the positional information of the aforementioned work environment as a three-dimensional point cloud, A first communication device that communicates with the control device, It has a first moving mechanism for moving the aforementioned work environment, The second mobile body is A second optical measuring instrument that acquires the color information of the aforementioned work environment as a two-dimensional second image, A second communication device that communicates bidirectionally with the control device, It has a second moving mechanism for moving the aforementioned work environment, The control device is A storage unit that holds a first self-position which is the three-dimensional relative position of the first moving object, a first environment map used to estimate the first self-position, a second self-position which is the three-dimensional relative position of the second moving object, and a second environment map used to estimate the second self-position, A third communication device that communicates with the first mobile body and the second mobile body, A remote work system comprising: a calculation unit that estimates the second self-position using the self-position estimation method described in any one of claims 1 to 4.
6. In the remote work system described in claim 5, The control device further includes a display unit that displays images, It has an input unit into which movement instructions for the first mobile body and movement instructions and work instructions for the second mobile body are input, The aforementioned arithmetic unit, Based on the first image and the three-dimensional point cloud sequentially obtained from the first moving object while it is in motion, and the second image sequentially obtained from the second moving object while it is in motion, the first and second environmental maps held by the storage unit are updated, and the first and second self-positions are estimated. A remote work system characterized by displaying on the display unit images showing the status of the work environment, the first mobile object, and the second mobile object, which are created based on the first environmental map, the second environmental map, the first self-position, and the second self-position.