Information processing device, information processing method, and program

Abstract

This information processing device comprises: a weight processing unit that determines a weight for an image on the basis of information that indicates the state of a work machine for performing work by a plurality of cyclic operations and that is determined on the basis of information related to one or both of the work machine and a camera that captures a work region to acquire a plurality of images; and an image selection unit that selects, on the basis of the weight, an image to be used for three-dimensional data generation from among the plurality of images.

Description

Information Processing Apparatus, Information Processing Method, and Program 【0001】 This technology relates to an information processing apparatus, an information processing method, and a program. 【0002】 As a conventional technique, a system has been proposed that generates three-dimensional data of the terrain around a shovel based on an image captured by a camera mounted on a revolving body during the operation of the shovel (Patent Document 1). As a result, three-dimensional data of the area where the work by the shovel in the work site has progressed can be obtained. 【0003】 Japanese Patent Application Laid-Open No. 2023-151682 【0004】 In such a system, when work performed by a plurality of consecutive periodic operations, such as work by a shovel at a civil engineering or construction work site, is carried out for a long time, the number of images increases. In the generation of three-dimensional data based on a plurality of images as in the technology described in Patent Document 1, problems such as an increase in the processing load of three-dimensional data generation and a need for a large-capacity storage for storing a large number of images occur. 【0005】 This technology has been made in view of such problems, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that can suppress the processing load and select images for efficiently generating three-dimensional data. 【0006】 To solve the above-described problems, a first technology is an information processing apparatus including a weight processing unit that determines a weight for an image based on information indicating a state of a working machine determined based on information regarding either or both of a working machine that performs work by a plurality of periodic operations and a camera that captures a work area to obtain a plurality of images, and an image selection unit that selects an image to be used for generating three-dimensional data from the plurality of images based on the weight. 【0007】The second technology is an information processing method that determines weights for images based on information indicating the state of the work machine, which is determined based on information about either or both of the work machine that performs work through multiple periodic movements and the camera that photographs the work area and acquires multiple images, and then selects an image to be used for generating three-dimensional data from the multiple images based on the weights. 【0008】 Furthermore, the third technology is a program that causes a computer to execute an information processing method that determines weights for images based on information indicating the state of the work machine, which is determined based on information about either or both of the work machine that performs work through multiple periodic movements and the camera that photographs the work area and acquires multiple images, and then selects an image to be used for generating three-dimensional data from the multiple images based on the weights. 【0009】This is a side view showing the configuration of the work machine 10 and camera CA. This is a plan view showing the operation of the work machine 10. This is an explanatory diagram of the surrounding area, work area, and non-work area. This is a diagram showing the configuration of the processing block of the information processing device 100 in the first embodiment. This is a diagram showing the hardware configuration of the information processing device 100. This is a flowchart showing the processing of the information processing device 100 in the first embodiment. This is a graph showing an example of input position information. Figure 8A is a graph showing position information for the latest 80 steps, and Figure 8B is a graph showing correlograms for 80 steps. Figure 9A is a graph showing position information and weights for one period, and Figure 9B is a graph showing position / weight conversion information (relationship between phase and weight). This is a diagram showing pseudocode used for phase determination. This is a graph showing position information and phase. This is a diagram explaining the synthesis of correlograms when the input is multidimensional. This is a diagram explaining image selection based on weight. This is a diagram showing the configuration of the processing block of the information processing device 100 in the second embodiment. This is an explanatory diagram of the period ID. This is a flowchart showing the processing of the information processing device 100 in the second embodiment. This is a diagram showing the configuration of the processing block of the information processing device 100 in the third embodiment. This is a diagram explaining the shooting position of camera CA. This is a diagram illustrating the accumulation of unnecessary images. This is a flowchart showing the processing of the information processing device 100 in the third embodiment. This is a diagram illustrating image selection based on weights. Figure 22A is a side view showing the configuration of the work machine 10 and camera CA in the fourth embodiment, and Figure 22B is a plan view showing the configuration of the work machine 10 and camera CA. This is a diagram showing the configuration of the processing block of the information processing device 100 in the fourth embodiment. This is a flowchart showing the processing of the information processing device 100 in the fourth embodiment. This is a diagram illustrating image selection based on weights. This is a diagram showing the configuration of the processing block of the information processing device 100 in the fifth embodiment. This is a diagram showing a modified example of the periodic operation of the work machine. 【0010】The embodiments of this technology will be described below with reference to the drawings. The description will be in the following order. <First Embodiment> [Configuration of work machine 10 and camera CA] [Configuration of information processing device 100] [Processing in information processing device 100] <Second Embodiment> [Configuration of information processing device 100] [Processing in information processing device 100] <Third Embodiment> [Configuration of information processing device 100] [Processing in information processing device 100] <Fourth Embodiment> [Configuration of work machine 10 and camera CA] [Configuration of information processing device 100] [Processing in information processing device 100] <Fifth Embodiment> [Configuration of information processing device 100] [Processing in information processing device 100] <Modification> 【0011】 <First Embodiment> [Configuration of Work Machine 10 and Camera CA] The configuration of the work machine 10 will be described with reference to Figure 1. In this embodiment, the work machine 10 is assumed to be an excavator (hydraulic shovel) capable of excavating and transporting soil and sand. 【0012】 As shown in Figure 1, the work machine 10 is equipped with a traveling body 11 that travels on the ground surface. A slewing body 13 is mounted on the upper surface of the traveling body 11 via a slewing mechanism 12 so as to be able to rotate 360 ​​degrees. A boom 14 is attached to the slewing body 13, and the boom 14 is hydraulically driven by a cylinder 15. An arm 16 is attached to the tip of the boom 14, and the arm 16 is hydraulically driven by an arm cylinder 17. A bucket 18 is attached to the tip of the arm 16, and the bucket 18 is hydraulically driven by a bucket cylinder 19. The slewing body 13 is equipped with an operator's cab and a power source such as an engine (not shown). 【0013】 As shown in Figure 2, the work machine 10 performs a periodic operation by repeating four operations: a first operation of excavating soil with the bucket 18, a second operation of moving the position of the bucket 18 by rotating the slewing body 13, a third operation of releasing soil from the bucket 18, and a fourth operation of moving the position of the bucket 18 for excavation by rotating the slewing body 13. The period is defined as the time from the start of the first operation to the end of the fourth operation, and the period is represented by N. The first periodic operation is N=1 and period 1, the next periodic operation is N=2 and period 2, and thereafter the period is represented by increasing N by 1 each time. 【0014】 Furthermore, the start of the periodic operation does not necessarily have to be the first operation; it may be the second, third, or fourth operation. Any of the repeating operations can be designated as the start of the periodic operation. 【0015】 As shown in Figure 3, the area covering the entire vicinity of the work machine 10 during periodic operation is defined as the surrounding area. If the periodic operation includes positional movement by the traveling body 11, the surrounding area is expanded by the positional movement of the work machine 11. Furthermore, as described above, the area in the surrounding area where the state changes due to the work machine 10 excavating soil is defined as the working area. In addition, the area in the surrounding area where the work machine 10 releases soil can also be defined as the working area because its state changes due to the release of that soil. Furthermore, as described above, the area in the surrounding area where the work machine 10 does not perform work, such as moving the position of the bucket 18 by the rotation of the rotating body 13, and where the state does not change or changes little is defined as the non-working area. The working area and the non-working area are areas that exist within the surrounding area. A change in state refers to a change in the state of the land or soil due to operations such as excavation and release by the work machine 10. 【0016】 The user can determine which part of the surrounding area will be designated as the work area by generating the necessary three-dimensional data. The work area may be limited to the area where the work machine 10 performs excavation, limited to the area where the work machine 10 discharges soil, or both the area where the work machine 10 performs excavation and the area where it discharges soil. In the first embodiment, both the area where the work machine 10 performs excavation and the area where it discharges soil are designated as the work area. Note that the surrounding area, work area, and non-work area are not necessarily limited to rectangular shapes, but may also be other shapes, such as circles or freeform shapes. 【0017】A camera CA is attached to boom 14. The camera CA is equipped with an image sensor, signal processing circuitry, etc., and can obtain RGB (Red, Green, Blue) or monochrome image data through imaging. CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor) are used as image sensors. 【0018】 The camera CA is mounted so that its lens faces the ground surface in order to acquire images of the work area and non-work area. Alternatively, the camera CA may be attached to the boom 14 via a gimbal, and the gimbal's drive may be used to adjust the camera CA's lens so that it faces the ground surface. The camera CA may also be attached to a part of the work machine 10 other than the boom 14. 【0019】 The camera CA continuously captures images at a predetermined frame rate or higher, for example, 30 fps (frames per second) or higher, while the work machine 10 is in operation, and continuously transmits the images to the information processing device 100. The reason the camera CA captures images at 30 fps is that capturing images at a frame rate of 30 fps or higher is required to ensure the accuracy of the camera CA's position and orientation estimation by SLAM. Therefore, the frame rate may be 30 fps or higher. If the frame rate required to ensure the accuracy of position and orientation estimation by SLAM is a different value, the capture frame rate should be set according to that value. 【0020】The camera CA is connected to the information processing device 100 by wired or wireless connection and transmits images acquired through shooting to the information processing device 100. Wired connection methods include, for example, HDMI (High-Definition Multimedia Interface) and USB (Universal Serial Bus), while wireless connection methods include, for example, Wi-Fi, wireless LAN (Local Area Network), 4G (fourth-generation mobile communication system), 5G (fifth-generation mobile communication system), Bluetooth (registered trademark), NFC (Near Field Communication), and Ethernet (registered trademark). Images may also be supplied from the camera CA to the information processing device 100 via recording media such as USB flash memory or SD memory cards. 【0021】 [Configuration of Information Processing Device 100] Next, the configuration of the information processing device 100 will be described with reference to Figure 4. The information processing device 100 selects an image to be used for generating three-dimensional data from multiple images captured by the camera CA installed on the work machine 10, and uses the selected image to generate three-dimensional data of the area around the work machine 10. 【0022】 Three-dimensional data can be, for example, a three-dimensional point cloud. A three-dimensional point cloud is a collection of data consisting of multiple points that have positional and color information. It can represent terrain, objects, and other things as a large collection of points, and is geographic information that can be used in various fields such as civil engineering, architecture, and manufacturing. By generating three-dimensional data for the work area, the progress of the work can be accurately grasped. By generating three-dimensional data for the non-work area, the surrounding conditions of the work machine 10 can be grasped. 【0023】 The receiving unit 101 receives the image transmitted from the camera CA and outputs it to the position and attitude estimation unit 102. 【0024】The position and orientation estimation unit 102 estimates the image capture position and orientation, i.e., the position (X, Y, Z) and orientation (Φ, θ, Ψ) of the camera CA, from multiple images input from the receiving unit 101 using Visual SLAM, and acquires position and orientation information. Note that other methods for estimating the position and orientation of the camera CA from images include Structure From Motion (SfM). 【0025】 Furthermore, the position and orientation of the camera CA may be estimated using an attitude detection unit such as an IMU (Inertial Measurement Unit) built into or attached to the camera CA, a two-axis or three-axis acceleration sensor, an angular velocity sensor, or a gyroscope sensor. The position and orientation of the camera CA may also be estimated using GNSS (Global Navigation Satellite System) or GPS (Global Positioning System) technology. These methods of position and orientation detection may be combined with position and orientation estimation using Visual SLAM or SfM. The position and orientation estimation of the camera CA may be performed outside the information processing device 100. The position and orientation estimation unit 102 may use any method as long as it can estimate the position and orientation of the camera CA. The position and orientation estimation unit 102 outputs the image and the position and orientation information for that image to the first storage unit 103. The position and orientation estimation unit 102 also outputs the position and orientation information to the position and orientation information storage unit 105. 【0026】 The first storage unit 103 is composed of a large-capacity storage medium such as a hard disk or flash memory, and stores multiple images and their corresponding positional information. 【0027】 The weight processing unit 104 determines the weight for each image. A weight is information that serves as a criterion for selecting images to be used to generate three-dimensional data from multiple images captured by the camera CA. The weight processing unit 104 consists of a position and orientation information storage unit 105, an autocorrelation calculation unit 106, a period detection unit 107, an information generation unit 108, a phase determination unit 109, a phase interpolation unit 110, and a weight determination unit 111. 【0028】The information addition unit 112 adds weight information corresponding to the image read from the first storage unit 103 and outputs it to the second storage unit 113. 【0029】 The second storage unit 113 is composed of a large-capacity storage medium such as a hard disk or flash memory, and stores images to which weight information has been added. The second storage unit 113 may also store camera CA position and orientation information and other information (such as image timestamps). 【0030】 The image selection unit 114 selects multiple images to be used for generating three-dimensional data based on weight information and outputs them to the three-dimensional data generation unit 115. 【0031】 The three-dimensional data generation unit 115 creates three-dimensional data based on a plurality of images selected by the image selection unit 114. The three-dimensional data generation unit 115 creates the three-dimensional data, for example, using SfM. 【0032】 Next, the hardware configuration of the information processing device 100 will be described with reference to Figure 5. 【0033】 The CPU (Central Processing Unit) 151 functions as an arithmetic processing unit that performs various processing tasks and controls the entire information processing device 100 and its individual parts. The CPU 151 executes various processes according to programs stored in the ROM (Read Only Memory) 152 or programs loaded from the storage unit 159 into the RAM (Random Access Memory) 153. The RAM 153 appropriately stores data necessary for the CPU 151 to execute various processes. Each processing block constituting the information processing device 100 can be realized by the processor, which is composed of the CPU 151, ROM 152, and RAM 153, executing a program. 【0034】 The CPU 151, ROM 152, and RAM 153 are interconnected via a bus 154, which is connected to a bridge 155. 【0035】 Interface 157 is connected to bridge 155 via bus 156. 【0036】 Interface 157 is connected to an input unit 158, an output unit 130, a storage unit 159, a drive 160, a connection port 161, and a communication unit 162. 【0037】 The input unit 158 ​​is, for example, various operators and operating devices such as a keyboard, mouse, keys, dial, touch panel, touchpad, or remote controller. User operations are detected by the input unit 158, and signals corresponding to the input operations are interpreted by the CPU 151. 【0038】 The output unit 130 includes liquid crystal displays or organic EL displays for displaying video, images, GUIs, messages, etc., and speakers for outputting sound. 【0039】 The storage unit 159 is a large-capacity storage medium such as a hard disk or flash memory. Various applications, data, and information are stored in the storage unit 159. In the information processing device 100, the first storage unit 103, the position information storage unit 105, and the second storage unit 113, which store information, can all be composed of the storage unit 159. The position and orientation information storage unit 105 can also be composed of a buffer memory or the like for temporarily storing information. 【0040】 The information processing device 100 can be connected to a removable storage medium 163 via a drive 160. The removable storage medium 163 includes USB flash memory, SD memory cards, magnetic disks, optical disks, magneto-optical disks, semiconductor memory, and the like. 【0041】 The drive 160 can read data files such as programs used for each process from the removable storage medium 163. The read data files are stored in the storage unit 159. In addition, programs and other data read from the removable storage medium 163 are installed in the storage unit 159 as needed. Furthermore, the information processing device 100 may transfer information and data to an external device via the removable storage medium 163. 【0042】 External devices 164 can be connected to the information processing device 100 via the connection port 161. 【0043】 The communication unit 162 includes various communication terminals and communication modules for performing communication processes via a network NW such as the Internet, wired / wireless communication with various devices, and bus communication. An external device can be connected to the information processing apparatus 100 via the communication unit 162. The communication method can be either wired or wireless. Examples of communication methods include cellular communication, 4G, 5G, Wi-Fi, Bluetooth (registered trademark), NFC, Ethernet (registered trademark), HDMI (registered trademark), USB (registered trademark), etc. 【0044】 Other external devices can be connected via the connection port 161 and the communication unit 162. 【0045】 Note that the information processing apparatus 100 does not necessarily have to include all the configurations shown in FIG. 5. For example, when the information processing apparatus 100 only performs processing and outputs the processing results externally, a display or a speaker as the output unit 130 is not necessary. 【0046】 In the information processing apparatus 100, for example, a program for the processing of the present technology can be installed via network communication by the communication unit 162 or via a removable storage medium 163. Also, the program may be stored in the ROM 152, the storage unit 159, etc. in advance. 【0047】 The information processing apparatus 100 can be configured as a single device and installed as a fixed station within or in the vicinity of the surrounding area of the work machine 10, or can be configured by an electronic device having information processing and communication functions such as a personal computer, a smartphone, or a tablet terminal, and installed at a position away from the surrounding area. The information processing apparatus 100 may be mounted on the work machine 10. Also, the information processing apparatus 100 and the information processing method may be realized when an electronic device having a function as a computer executes a program. The program may be installed in the electronic device in advance, or may be distributed via download, a storage medium, etc. so that a user or the like installs it. 【0048】The functions realized by the components described in this specification may be implemented in circuitry or processing circuitry including a general-purpose processor, a specific-purpose processor, an integrated circuit, ASICs (Application Specific Integrated Circuits), a CPU, a conventional circuit, and / or a combination thereof programmed to realize the described functions. The processor includes transistors and other circuits and is regarded as circuitry or processing circuitry. The processor may be a programmed processor that executes a program stored in a memory. In this specification, circuitry, unit, and means are hardware programmed to realize the described functions or hardware that executes. The hardware may be any hardware disclosed in this specification or any hardware known as being programmed or executing to realize the described functions. When the hardware is a processor regarded as a type of circuitry, the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or the processor. 【0049】 The information processing apparatus 100 may be configured in a cloud server. Also, the information processing apparatus 100 may transmit the generated three-dimensional data to an external device, a cloud server, or the like. Further, the information processing apparatus 100 may transfer the generated three-dimensional data to an external device via an external storage medium such as a USB flash memory or an SD memory card. 【0050】A cloud server is not limited to being composed of a single computer device; it may also be composed of a system of multiple computer devices. These multiple computer devices may be systematized, for example, by a LAN (Local Area Network). Alternatively, multiple computer devices located in remote locations may be systematized by a VPN (Virtual Private Network) using the internet, etc. These multiple computer devices may include computer devices that constitute a group of servers (cloud) available through cloud computing services. 【0051】 [Processing in the Information Processing Device 100] Next, the processing in the information processing device 100 will be described with reference to Figure 6. 【0052】 As described above with reference to Figure 2, the work machine 10 performs a periodic operation by repeating four operations: a first operation of excavating soil with the bucket 18, a second operation of moving the position of the bucket 18 by rotating the slewing body 13, a third operation of releasing soil from the bucket 18, and a fourth operation of moving the position of the bucket 18 by rotating the slewing body 13. The work machine 10 performs the work by repeating this periodic operation. 【0053】 During the periodic operation of the work machine 10, the camera CA continuously takes pictures at a predetermined frame rate (e.g., 30 fps) or higher. While taking pictures, the camera CA continuously outputs the images acquired through the taking to the information processing device 100. 【0054】 In step S101, the receiving unit 101 receives the image transmitted from the camera CA and outputs the image to the position and orientation estimation unit 102. 【0055】 Next, in step S102, the position and orientation estimation unit 102 estimates the position and orientation of the camera CA using Visual SLAM for the input image and acquires position and orientation information. The position and orientation estimation unit 102 associates the image with the position and orientation information for that image and outputs it to the first storage unit 103 for storage. The position and orientation estimation unit 102 also outputs the position and orientation information to the position and orientation information storage unit 105 for storage. 【0056】Alternatively, the position and orientation estimation unit 102 may estimate only the position of camera CA and store only the position information in the position and orientation information storage unit 105. Alternatively, the position and orientation estimation unit 102 may estimate the position and orientation of camera CA and store the position and orientation information in the position and orientation information storage unit 105, and only the position information may be input from the position and orientation information storage unit 105 to each part of the weight processing unit 104. In the following description, it will be assumed that position information is input from the position and orientation information storage unit 105 to each part of the weight processing unit 104. The one-dimensional position information to be input is shown, for example, in Figure 7. In Figure 7, the vertical axis represents the (one-dimensional) position, which is normalized to ±1, so the actual position corresponds to ±10 meters. In Figure 7, the horizontal axis represents time, and the vertical axis represents position information obtained by discretely sampling the time shown on the horizontal axis at step intervals of 1 second. In this embodiment, for the sake of explanation, the information used is limited to position information and further limited to one dimension, but this technology is not limited to this content, and information other than position information may be used, or it may be multi-dimensional (three-dimensional, etc.). 【0057】 Next, in step S103, the autocorrelation calculation unit 106 creates a correlogram showing the autocorrelation based on the position information. First, the autocorrelation calculation unit 106 acquires the latest 80 steps of position information from the position and orientation information storage unit 105. The 80 steps of position information are shown, for example, in Figure 8A. Here, assuming a step interval of 1 second, the 80 steps of position information represent the time from the current time to 79 seconds past at 1-second intervals. In this embodiment, N=30 data points are used to calculate the correlation coefficient, and a time shift of 50 steps is performed. Therefore, the autocorrelation calculation unit 106 uses a total of 80 steps of position information to calculate a correlogram with N=30 and a shift of 50 steps. 【0058】 Since the system requires position information for the past 80 steps, including the current position, the autocorrelation calculation unit 106 starts calculating the correlogram 80 steps after the position and orientation information storage unit 105 has started accumulating position information. 【0059】The correlogram for 80 steps (N=30) will look like the one shown in Figure 8B. As indicated by the dashed circle in Figure 8B, a characteristic of the correlogram is that the autocorrelation value is 1 when the lag is 0. The autocorrelation calculation unit 106 outputs the correlogram to the period detection unit 107. 【0060】 Next, in step S104, the period detection unit 107 detects the period of the periodic operation of the work machine 10 by detecting the peaks in the correlogram. The period is the time required for one periodic operation of the work machine 10, and specifically it is the elapsed time between peaks in the correlogram. The period can also be defined as the elapsed time in the period from the start of the first operation to the end of the fourth operation. 【0061】 In the correlogram shown in Figure 8B, peaks exist at t=0, t=-20, and t=-40. Here, we detect the latest period and determine that the period is 20 seconds by multiplying the difference between t=0 and t=-20 (20) by the step interval of 1 second. The period detection unit 107 outputs this period information to the information generation unit 108 and the phase determination unit 109. 【0062】 Next, in step S105, the information generation unit 108 takes position information and time information for the past 80 steps from the current time (t=80) as input and generates period start time information and phase / weight conversion information. 【0063】 In this embodiment, it is assumed that within the surrounding area, a work area where the state changes (where the state changes significantly due to the work of the work machine 10) and a non-work area where the state does not change or changes little (for example, where the work machine 10 does not perform any work and the state changes little) are given in advance by coordinate values. Here, as an example, the area where "-0.2 < coordinate value ≤ 0.2" is the work area, and the other areas are the non-work area. Note that if the input is multidimensional data, parameters other than coordinate values ​​may be used. 【0064】Furthermore, in this embodiment, the current time (t=80) is assumed to be the step in which the periodic operation of the work machine 10 is completed. Since the current time (t=80) is the step in which the period is completed, the time that is earlier than the current time (t=80) by the amount of the period is the time when the period of interest began. Since the period indicated by the period information is 20 seconds, the time when the period that is completed at the current time (t=80) began, i.e., the time t=61, is taken as the period start time. The information generation unit 108 outputs the period start time information "t=61" to the phase determination unit 109 and the weight determination unit 111. 【0065】 Furthermore, the information generation unit 108 extracts the position information for the latest period (a period that starts at t=61 and ends at t=80) from the position information of the past 80 steps, based on the period information (20 seconds). The extracted position information for the latest period is shown by the solid line in Figure 9A. 【0066】 Based on the latest position information for one cycle, as shown in Figure 9A, the weight at time t corresponding to the coordinate value range "-0.2 < coordinate value ≤ 0.2" indicating the work area where the state changes is set to 1.0, and the weight at all other times t is set to 0.2. This determines the weight for the latest one cycle, as shown by the dashed line in Figure 9A. 【0067】 Next, the information generation unit 108 converts the most recent period into a phase. The phase is information representing the state of the work machine 10 that performs periodic operation, determined based on one-dimensional or combined multi-dimensional input information. In this embodiment, the phase is determined based on the position information of the one-dimensional camera CA. The phase at t=61 is 0.0, and increases by the reciprocal of the numerical value 20 representing the period in steps (1 / 20 = 0.05) for each step. Therefore, after increasing by 20 steps, the phase becomes 1.00 at t=80. By converting one period into a phase in this way, the time t, which is the horizontal axis of the weight graph shown in Figure 9A, is converted into a phase, as shown in Figure 9B. In this way, the weights corresponding to the phase of the period are determined. The information generation unit 108 outputs this information as phase-weight conversion information to the weight determination unit 111. 【0068】Next, in step S106, the phase determination unit 109 takes as input the time information for the past 80 steps output from the position and attitude information storage unit 105, the period information output from the period detection unit 107, and the period start time information output from the information generation unit 108, and determines the phase relative to the position information for the past 80 steps. The period start time information is "t = 61", and the period is "20 seconds". 【0069】 Although t=1 and t=21 could also be considered period start times, here we will consider t=41 and t=61 as period start times, and not t=1 and t=21. This is based on the criterion that times when the time elapsed to the current time exceeds twice the period are not treated as period start times. This is because time periods with a large time difference from the current time are likely to be unsuitable for the period detected at this time. Therefore, t=1 and t=21 are not considered period start times because the time elapsed to the current time (t=80) exceeds twice the period. However, this criterion is not always necessary to apply. 【0070】 The phase determination unit 109 assigns phase information to the position information for the past 80 steps. Specifically, it uses pseudocode as shown in Figures 10A and 10B. Note that the time period 0 ≤ t < 41 is treated as having no period. Specifically, None is substituted as the phase value. 【0071】 Substitute 0.0 for t=41 and t=61 as the phase start times, and increase by the reciprocal of the period (1 / 20 = 0.05) for each increase of t during the period. However, stop the phase determination at the current time. 【0072】 The phases of the 80 steps, determined by the above process and occurring every second, are shown as dashed lines in Figure 11. Note that while position information is shown as a solid line in Figure 11, this only indicates the relationship between the determined phase and the corresponding position information; the position information is not used in the processing of the phase determination unit 109. The phase determination unit 109 outputs the phase information to the phase interpolation unit 110. 【0073】Next, in step S107, the phase interpolation unit 110 interpolates the phases of 80 steps at one-second intervals determined by the phase determination unit 109. This determines the phase corresponding to the time of each image in the 80-step time period stored in the first storage unit 103. 【0074】 There are various methods, such as linear interpolation, for calculating the phase corresponding to the time of each image. Here, the phase determination unit 109 extracts the phase of the step immediately following the time of the image to be interpolated based on the information of the 80 steps for which the phase has been calculated, and uses that as the phase of the target image. If the phase of the step immediately following the target image is None, the phase of the target image is set to None. 【0075】 Next, in step S108, the weight determination unit 111 uses the phase-weight conversion information shown in Figure 9B, which is input from the information generation unit 108, to convert the phase of all images corresponding to the past 80 steps, which are input from the phase interpolation unit 110, into weights. As a result, the weight processing unit 104 determines the weight for each image. 【0076】 The processes from step S101 to step S108 are performed to calculate the weight for each image. After a certain period of time has elapsed, the process is repeated to update the weight for each image. Here, the certain period of time is at least the time of one step and at most the time of two cycles (because the phase was set for two cycles of images as explained above). 【0077】 Note that the phase and weights may be represented not only by free curves but also by other linear shapes such as steps. 【0078】 In the above explanation, for the sake of explanation, the position information of the camera CA is used as one-dimensional information to determine the weights. However, it is also possible to determine the weights using one-dimensional or multi-dimensional information related to either or both the work machine 10 and the camera CA as input, such as the position information of the camera CA, the orientation information of the camera CA, the position and orientation detection results of the work machine 10 and the camera CA by sensors such as the IMU (Inertial Measurement Unit), time information, the amount of operation and operation information of levers and handles for operating the work machine 10, and the state of each part of the work machine 10 (for example, the angle of the bucket 18). 【0079】In this embodiment, for the sake of explanation, the information used is limited to location information and further limited to one dimension. However, this technology is not limited to this content, and other information besides location information may be used, or it may be multidimensional (e.g., three dimensions). However, when multidimensional information is used as input, it is time-consuming to generate a transformation table to determine weights from the multidimensional information. Therefore, the efficiency of weight determination can be improved by detecting a period from the input information and assigning weights that change in accordance with that period. 【0080】 When multidimensional information is used as input, the autocorrelation calculation unit 106 receives the multidimensional information. As shown in Figure 12, the autocorrelation calculation unit 106 generates multiple correlograms by calculating the autocorrelation for each of the multiple inputs, and then combines them with pre-set weights (for example, 1:1) to generate a one-dimensional correlogram. By combining multiple correlograms to generate a one-dimensional correlogram, multidimensional information can be treated as one-dimensional information. The period detection unit 107 then detects the period of operation of the work machine 10 from the one-dimensional correlogram that is the result of the combination. 【0081】 In the above explanation, it was assumed that the range of coordinate values ​​indicating the work area where the state changes is predetermined. However, it is also possible to estimate the range of coordinate values ​​indicating the work area where the state changes from the position and orientation of the camera CA and its changes, various sensor values ​​and their changes, and operation information such as the levers of the work machine 10. Furthermore, it is possible to set the range of coordinate values ​​indicating the work area where the state changes using AI (Artificial Intelligence). 【0082】 When using the changes in position (X, Y, Z) and orientation (Φ, θ, Ψ) of the camera CA attached to the boom 14, the boom 14 often only moves up and down (in the Z direction) during excavation, so the changes in the X and Y directions ΔX and ΔY are likely to be smaller compared to other movements. Changes in orientation may show a similar distribution, and these characteristics can be used to predict the region where the state changes. It is also possible to process and estimate this information using AI. 【0083】In the above explanation, it was assumed that the current time corresponds to the step in the periodic operation of the work machine 10 that completes, and that the time obtained by going back one period from the current time is the start time of the period. However, the completion of the period is not limited to the current time; it may occur at other times as well. 【0084】 In the above explanation, the autocorrelation calculation unit 106 uses N = 30 data points to calculate the correlation coefficient. Since one step is 1 second, this amounts to 30 seconds of data, and therefore the maximum period length is limited to 30 seconds. However, N = 30 is just an example value set for explanatory purposes, and N can be any other value, greater than or less than 30. If the period to be detected is longer, the value of N should be increased. 【0085】 Period detection and weight assignment to periods can also be performed using AI, machine learning methods such as DNN (Deep Neural Network), CNN (Convolutional Neural Network), and RF (Random Forest). Examples of machine learning training methods include neural networks and deep learning. 【0086】 In determining weights, various methods and mechanisms can be employed to enhance the robustness of the processing, such as signal processing techniques, AI, and machine learning. 【0087】 Returning to the flowchart explanation, in step S109, the information addition unit 112 adds weight information determined based on the same positional information to each image read from the first storage unit 103 by referring to the positional information or positional orientation information associated with the image. The addition of weight information to an image can also be done by referring to information that can uniquely identify the image, such as the image name or image ID, or the time the image was taken. The information addition unit 112 outputs the images with added weight information to the second storage unit 113 for storage. 【0088】 Next, in step S110, the image selection unit 114 selects an image to be used for generating three-dimensional data from a plurality of images stored in the second storage unit 113 based on the weight information attached to the image, and outputs it to the three-dimensional data generation unit 115. 【0089】Here, we will explain using an example where the weights are determined as shown in Figure 13. Figure 13 shows the periodic operation of the work machine 10 and the corresponding phase and weights. The image selection unit 114 sets multiple thresholds for the weights to divide them into multiple stages. In the example in Figure 13, two thresholds are set to divide the weights into three stages: high, medium, and low. 【0090】 Images with a "high" weight are selected to be used for generating three-dimensional data and output to the three-dimensional data generation unit 115. Images with a "medium" weight are selected to be used for generating three-dimensional data but are thinned out and output to the three-dimensional data generation unit 115. Images with a "low" weight are not selected to be used for generating three-dimensional data and are not output to the three-dimensional data generation unit 115. Images with a "high" weight are always used, images with a "medium" weight are thinned out and used, and images with a "low" weight are never used. A "high" weight is the first weight in the claim, a "medium" weight is the third weight in the claim, and a "low" weight is the second weight in the claim. 【0091】 For example, if we consider the weight W as the probability of selecting an image, and use a method to select images according to probability as shown in Equation 1 below, then if W = 1.0, it means "always use", if 0.0 < W < 1.0, it means "use after thinning", and if W = 0.0, it means "never use". 【0092】 (Formula 1) 【0093】 The image with weight W1 added at phase P1 shown in Figure 13 is output to the three-dimensional data generation unit 115 because the weight is "high". Phase P1 is during the first operation, in which the work machine 10 is excavating in the work area, so the state of the work area is changing. Images of the work area whose state is changing due to excavation are important for the generation of three-dimensional data, so many of the images taken during the first operation are output to the three-dimensional data generation unit 115. 【0094】The image with weight W2 added at phase P2 shown in Figure 13 has a "medium" weight, so it is thinned out according to a predetermined rule and output to the three-dimensional data generation unit 115. As a predetermined rule, for example, thinning may be done by selecting every other image from a series of consecutive images with a "medium" weight, or by selecting every three images. The rule for thinning images may be set in advance in the information processing device 100, or it may be made possible for the user of the information processing device 100 to set it. 【0095】 Phase P2 is during the second operation, in which the work machine 10 rotates the slewing body 13 to move the position of the bucket 18, so camera CA is capturing images of a non-working area where the change in state is very small. Therefore, the images captured during the second operation are thinned out and output to the three-dimensional data generation unit 115. Thinning out the images captured during the second operation has little effect on the accuracy of the three-dimensional data. Thinning out the images reduces the processing load in the generation of three-dimensional data, and enables efficient generation of three-dimensional data. 【0096】 The image with weight W3 added at phase P3, as shown in Figure 13, is output to the three-dimensional data generation unit 115 because the weight is "high". Phase P3 is during the third operation, in which the work machine 10 is releasing soil and sand in the work area, so the state of the work area is changing. Images of the work area whose state is changing due to the release are important for the generation of three-dimensional data, so many of the images taken during the third operation are output to the three-dimensional data generation unit 115. 【0097】The image with weight W4 added at phase P4 shown in Figure 13 is not output to the three-dimensional data generation unit 115 because the weight is "low". Phase P4 is in the middle of the fourth operation, and in the fourth operation the work machine 10 rotates the slewing body 13 to move the position of the bucket 18, so the camera CA is capturing images of a non-working area where the change in state is very small. Furthermore, the operation in which the work machine 10 rotates the slewing body 13 to move the position of the bucket 18 is an operation that captures the same non-working area as the second operation. Therefore, not using the images captured during the fourth operation does not affect the accuracy of the three-dimensional data, or the effect is minimal. By not selecting unnecessary images, the processing load in the generation of three-dimensional data can be reduced, and three-dimensional data can be generated efficiently. 【0098】 Note that phases P1 to P4 in Figure 13 are specific phases used to explain the weight-based image selection, and do not mean that images are only captured by camera CA during those phases. As mentioned above, image capture by camera CA is performed continuously at a predetermined frame rate from the start of operation of the work machine 10. 【0099】 Next, in step S111, the three-dimensional data generation unit 115 generates three-dimensional data using the image input from the image selection unit 114. 【0100】 The processing of the first embodiment is carried out as described above. In order to ensure the accuracy of position and orientation estimation of the camera CA by SLAM, multiple images taken at a predetermined frame rate (e.g., 30 fps) or higher are required. However, if all images taken at such a frame rate are used for three-dimensional data generation, the processing load becomes large. Therefore, in the first embodiment, images to be used for three-dimensional data generation are selected from among multiple images taken by the camera CA during the operation of the work machine 10 based on their weights. This reduces the processing load in three-dimensional data generation and enables efficient generation of three-dimensional data. 【0101】By assigning a higher weight to images of the work area, where the state changes significantly due to the operation of the work machine 10, images of the work area are preferentially used for 3D data generation, thereby generating highly accurate 3D data. Conversely, by assigning a lower weight to images of the non-work area, where the state does not change due to the operation of the work machine 10, images of the non-work area are either thinned out and not used for 3D data generation, or not used at all. This reduces the processing load in 3D data generation and enables efficient 3D data generation. 【0102】 In the process described above, the information addition unit 112 adds weight information to each image and stores it in the second storage unit 113. However, instead of adding weight information to each image, a table may be prepared that links the weight information to the corresponding image, and the image selection unit 114 may refer to this table to select the images to be used for generating three-dimensional data. 【0103】 In the process described above, the image selection unit 114 selects an image to be used for generating three-dimensional data from among multiple images stored in the second storage unit 113. However, it is also possible to store only the image selected by the image selection unit 114 from among multiple images to which weight information has been added by the information addition unit 112 in the second storage unit 113, and not store the images not selected by the image selection unit 114 in the second storage unit 113. In this case, the three-dimensional data generation unit 115 reads multiple images from the second storage unit 113 and generates three-dimensional data. This reduces the amount of images stored in the second storage unit 113. 【0104】 Although it was explained that the camera CA outputs multiple images at predetermined intervals while continuing to capture images at a predetermined frame rate, the camera CA may output images one by one. Alternatively, the receiving unit 101, which receives multiple images output from the camera CA, may output each image one by one, and the information processing device 100 may process them one by one. 【0105】<Second Embodiment> [Configuration of Information Processing Device 100] The configuration of the information processing device 100 in the second embodiment will be described with reference to Figure 14. The configuration and operation of the work machine 10 and the configuration and operation of the camera CA are the same as in the first embodiment. In the second embodiment, the area in which the work machine 10 performs excavation is defined as the work area. However, the user can determine which part of the surrounding area is defined as the work area by what kind of three-dimensional data is generated. The work area may be defined as only the area in which the work machine 10 performs excavation, or only the area in which the work machine 10 discharges soil and sand, or both the area in which the work machine 10 performs excavation and the area in which it discharges soil and sand. 【0106】 The information processing device 100 in the second embodiment includes a period ID determination unit 201. The other configurations are the same as in the first embodiment, so their description is omitted. 【0107】 The period ID determination unit 201 determines the period ID of each image based on the phase information output from the phase interpolation unit 110. 【0108】 The period is a parameter that indicates the start and end of one cycle of operation of the work machine 10. The period ID is information that identifies the period and indicates which cycle it is. The period ID is used to order and distinguish each of multiple consecutive cycles. 【0109】 As shown in Figure 15, the work machine 10 performs a first action of excavating soil with the bucket 18, a second action of moving the position of the bucket 18 by rotating the slewing body 13, a third action of releasing soil from the bucket 18, and a fourth action of moving the position of the bucket 18 by rotating the slewing body 13. After that, the first to fourth actions are repeated to perform a periodic operation. In this case, the period ID of the first periodic operation is "1", the period ID of the second periodic operation is "2", and the period ID of the third periodic operation is "3". 【0110】The state of the work area differs between the first periodic operation (period ID = 1) and the second periodic operation (period ID = 2) due to excavation by the work implement 10. Similarly, the state of the work area also differs between the second periodic operation (period ID = 2) and the third periodic operation (period ID = 3) due to excavation by the work implement 10. Because the state of the work area changes significantly due to excavation by the work implement 10, using images before and after the change simultaneously for 3D data generation will reduce the accuracy of the 3D data. In order to generate 3D data of the work area with high accuracy, it is necessary to input multiple images of the work area taken within the same period, that is, multiple images of the work area in the same state, into the 3D data generation unit 115. 【0111】 Therefore, in the second embodiment, images taken during one periodic operation are grouped by adding the same period ID. Then, multiple images with the same period ID are output together to the three-dimensional data generation unit 115. Images with different period IDs are not output to the three-dimensional data generation unit 115 simultaneously. 【0112】 [Processing in the Information Processing Device 100] The processing in the information processing device 100 of the second embodiment will be described with reference to Figure 16. 【0113】 Steps S101 to S108 are the same as in the first embodiment, so their explanation will be omitted. 【0114】 In step S121, the period ID determination unit 201 determines the period ID. The period ID is determined based on the period start time detected by the period detection unit 107. If, as in the first embodiment, t=41 and t=61 are determined as the period start times, when determining the period ID, t=41 is used as the starting point, and the period ID from t=41 to t=60 is determined as period ID=0, and thereafter the value of the period ID is incremented by +1 for each period start time. Therefore, from t=61 to t=80 onwards, the period ID is 1. The period ID determination unit 201 outputs the period ID information to the information addition unit 112. 【0115】Next, in step S122, the information adding unit 112 adds weight information and a period ID to each image read from the first storage unit 103 by referring to the position information or position / orientation information associated with the image. The addition of weight information to an image can also be done by referring to information that can uniquely identify the image, such as the image name or image ID, or the time the image was taken. The information adding unit 112 outputs the image with the added weight information and period ID to the second storage unit 113 for storage. 【0116】 Next, in step S123, the image selection unit 114 selects an image to be used for generating three-dimensional data from a plurality of images stored in the second storage unit 113 based on the weight information and outputs it to the three-dimensional data generation unit 115. This process is the same as in the first embodiment. 【0117】 Furthermore, in the second embodiment, the image selection unit 114 limits the multiple images to be simultaneously output to the three-dimensional data generation unit 115 to images that have the same period ID assigned to them. Therefore, the image selection unit 114 outputs multiple images that have been selected based on weights and that have the same period ID assigned to them together to the three-dimensional data generation unit 115. 【0118】 In other words, multiple images with the same period ID captured during a single periodic operation are treated as a single group, and multiple images with the same period ID are output together to the three-dimensional data generation unit 115. As described above, multiple images captured during a single periodic operation, from the first operation to the fourth operation, are assigned the same period ID, so multiple images captured during that single periodic operation are treated as a single group and output together to the three-dimensional data generation unit 115. The image selection unit 114 does not output images with different period IDs to the three-dimensional data generation unit 115 simultaneously. 【0119】 Next, in step S111, the three-dimensional data generation unit 115 generates three-dimensional data using the image selected by the image selection unit 114. 【0120】The processing in the second embodiment is carried out as described above. According to the second embodiment, in addition to the effects of the first embodiment, even when the state of the work area changes significantly due to the work of the work machine 10, high-precision three-dimensional data can be generated by using multiple images of the work area in the same state, taken within the same period. Therefore, it becomes possible to generate three-dimensional data that is adapted to drastic changes in terrain. 【0121】 In the above description, images with a period ID attached are stored in the second storage unit 113. However, instead of attaching a period ID to the images, the image selection unit 114 may treat multiple images corresponding to the same period ID as a single group by referring to a pre-prepared table that associates each image with a period ID, and output them to the three-dimensional data generation unit 115. 【0122】 Furthermore, in determining the period ID, various types of information indicating the periodic operation of the work machine 10 may be used in addition to phase information. 【0123】 <Third Embodiment> [Configuration of Information Processing Device 100] The configuration of the information processing device 100 in the third embodiment will be described with reference to Figure 17. The configuration and operation of the work machine 10 and the configuration and operation of the camera CA are the same as in the first embodiment. In the third embodiment, the work area is defined as only the area in which the work machine 10 performs excavation. 【0124】 The information processing device 100 in the third embodiment includes a period ID determination unit 301 and a period weight determination unit 302. The other configurations are the same as in the first embodiment, so their description is omitted. 【0125】 The period ID determination unit 301 determines the period ID for each image based on the phase information output from the phase interpolation unit 110, similar to the second embodiment. The period and period ID are the same as in the second embodiment. 【0126】 The period-specific weight determination unit 302 determines weights for the image based on the period ID and phase. 【0127】 [Processing in the Information Processing Device 100] First, the problems to be solved in the third embodiment will be explained with reference to Figures 18 and 19. 【0128】 As described in the first embodiment, this technology generates three-dimensional data of the surroundings of the work machine 10 using multiple images captured by the camera CA. To this end, multiple images are stored in the first storage unit 103 and the second storage unit 113. 【0129】 As shown in Figure 18, consider multiple shooting positions by camera CA. Figures 18A, 18B, and 18C each represent one cycle of operation of the work machine 10, with Figure 18A representing the cycle of the first period, Figure 18B representing the cycle of the second period, and Figure 18C representing the cycle of the third period. Note that the cycle of operation of the work machine 10 may continue beyond the third period, but here only the cycle of the first to third periods is shown for explanatory purposes. 【0130】 In Figure 18A, positions 1 to 10 represent the shooting positions of camera CA during the periodic operation of the first period. In Figure 18B, positions 11 to 20 represent the shooting positions of camera CA during the periodic operation of the first period. In Figure 18C, positions 21 to 30 represent the shooting positions of camera CA during the periodic operation of the first period. 【0131】 The work machine 10 operates in a cyclical manner, following a first, second, and third operation sequence. During the first period of cyclical operation, camera CA takes images in the order of position 1 to position 10. During the second period of cyclical operation, camera CA takes images in the order of position 11 to position 20. During the third period of cyclical operation, camera CA takes images in the order of position 21 to position 30. Since camera CA continuously takes images at a predetermined frame rate (e.g., 30 fps), multiple images are taken at a single location. 【0132】 Note that this shooting position does not indicate that camera CA only takes pictures at that specific location, but rather it is a location identified for explanatory purposes. As described above, camera CA continuously takes pictures at a predetermined frame rate or higher, for example, 30 fps or higher, while the work machine 10 is in operation. Therefore, multiple images are taken at each location, and the multiple images taken at each location are referred to as an image group. 【0133】Since the non-working area does not change or changes only slightly due to the work of the work machine 10, the same unchanging information can be obtained from images of the non-working area even as time progresses. The work machine 10 repeatedly performs periodic operations, and during that time, the camera CA continues to take pictures. Therefore, if all images of such non-working areas are saved, the amount of unnecessary images accumulated in the first storage unit 103 and the second storage unit 113 will increase as time progresses. 【0134】 During the first period, positions 2-5 and 6-9 are the same location and within a non-working area where there is no change in state. Therefore, the images taken at positions 2-5 and 6-9 only provide the same information without any changes. Thus, as shown in Figure 19, the images taken at positions 6-9 are unnecessary and will be stored as unnecessary images if saved. 【0135】 During the first period, position 10 is the same as position 1, but it is the position where the work area is photographed, and the timing of the photography is different. Therefore, new information about the work area can be obtained from the image set taken at position 10. Thus, the image sets taken at position 1 and position 10 are necessary images, and saving them will not result in the accumulation of unnecessary images. 【0136】 In the second period, positions 12-15 are the same as positions 2-5 in the first period, and are within a non-working area where there is no change in state. Therefore, the images taken at positions 12-15 only provide the same information as the images taken at positions 2-5. Thus, as shown in Figure 19, the images taken at positions 12-15 are unnecessary and will be stored as unnecessary images if saved. The same applies to positions 16-19. 【0137】 In the second period, position 11 is the same as position 1, but it is the position where the work area is photographed, and the timing of the photography is different. Therefore, new information about the work area can be obtained from the image set taken at position 11. Thus, the image set taken at position 11 is a necessary image, and saving it will not result in the accumulation of unnecessary images. The same applies to position 20. 【0138】In the third period, positions 22-25 are the same as positions 2-5 in the first period, and are within a non-working area where there is no change in state. Therefore, the images taken at positions 22-25 only provide the same information as the images taken at positions 2-5. Thus, as shown in Figure 19, the images taken at positions 22-25 are unnecessary and will be stored as unnecessary images if saved. The same applies to positions 26-29. 【0139】 In the third period, position 21 is the same as position 1, but it is the position where the work area is photographed, and the timing of the photography is different. Therefore, new information about the work area can be obtained from the image set taken at position 21. Thus, the image set taken at position 21 is a necessary image, and saving it will not result in the accumulation of unnecessary images. The same applies to position 30. 【0140】 Next, referring to Figure 20, the processing in the information processing device 100 of the third embodiment will be described. Steps S101 to S107 and S111 are the same as in the first embodiment, so their explanation will be omitted. 【0141】 In step S131, the period ID determination unit 301 determines the period ID based on the phase information output from the phase determination unit 109. The method for determining the period ID is the same as in the second embodiment. The period ID determination unit 301 outputs the period ID information to the information addition unit 112. 【0142】 Next, in step S132, the period-specific weight determination unit 302 determines the weights for the image based on the phase and period ID. The method for determining the weights based on the phase is the same as in the first embodiment. 【0143】Furthermore, in order to solve the above-mentioned problems, the period-specific weight determination unit 302 determines the weight for images captured in the non-working area based on the period ID so that it decreases as the period progresses. This can also be described as switching the weights for each period. Alternatively, the period-specific weight determination unit 302 can be described as determining the weights so that they decrease as multiple consecutive periodic operations progress, according to the order of periodic operations specified by the period ID. However, since images captured in the working area are necessary for three-dimensional data generation, their weight is determined to be the highest. In this embodiment, as an example, the weights are determined as shown in Figure 21. 【0144】 Next, in step S133, the information addition unit 112 adds weight information to each image read from the first storage unit 103. The information addition unit 112 outputs the images with added weight information to the second storage unit 113 for storage. 【0145】 Next, in step S134, the image selection unit 114 selects an image to be used for generating three-dimensional data from a plurality of images stored in the second storage unit 113 based on the weight information attached to the image, and outputs it to the three-dimensional data generation unit 115. 【0146】 In this embodiment, as shown in Figure 21, the image selection unit 114 sets five thresholds for the weights as an example, dividing the weights into six levels from 5 to 0. A weight of 5 is the highest, and a weight of 0 is the lowest. 【0147】 Images with a weight of "5" are selected as images to be used for generating three-dimensional data and output to the three-dimensional data generation unit 115. Images with a weight of "4" are selected as images to be used for generating three-dimensional data but are thinned out and output to the three-dimensional data generation unit 115. In Figure 21, the amount of thinning is represented as (small). Images with a weight of "3" are selected as images to be used for generating three-dimensional data but are thinned out and output to the three-dimensional data generation unit 115. The amount of thinning is greater than when the weight is "4", and in Figure 21, the amount of thinning is represented as (small). 【0148】Images with a weight of "2" are selected as images to be used for generating three-dimensional data, but are thinned out and output to the three-dimensional data generation unit 115. The amount of thinning is greater than when the weight is "3", and in Figure 21, this amount of thinning is represented as (medium). Images with a weight of "1" are selected as images to be used for generating three-dimensional data, but are thinned out and output to the three-dimensional data generation unit 115. The amount of thinning is greater than when the weight is "2", and in Figure 21, this amount of thinning is represented as (high). Furthermore, images with a weight of "0" are not used for generating three-dimensional data and are not output to the three-dimensional data generation unit 115. Therefore, the image selection unit 114 thins out multiple images so that the amount of thinning increases as the period progresses and the weight decreases, and selects them as images to be used for generating three-dimensional data. 【0149】 As shown in Figure 21, in the first period, the image groups taken at positions 1, 2-5, and 10 have a weight of "5". Also, the image groups taken at positions 6-9 have a weight of "4". In the second period, the image groups taken at positions 11 and 20 have a weight of "5". The image groups taken at positions 12-15 have a weight of "3". The image groups taken at positions 16-19 have a weight of "2". In the third period, the image groups taken at position 21 and 30 have a weight of "5". The image groups taken at positions 22-25 have a weight of "1". The image groups taken at positions 26-29 have a weight of "0". 【0150】 Therefore, in the first period, the image selection unit 114 outputs the image group taken at positions 1, 2-5, and 10, which have a weight of "5", to the three-dimensional data generation unit 115. The image selection unit 114 also omits the image group taken at positions 6-9, which have a weight of "4", and outputs them to the three-dimensional data generation unit 115. The reason why the image group taken at positions 2-5, which are outside the working area, has a weight of "5" in the first period is that even though the images taken at positions 2-5 are outside the working area, they are the first images taken, and therefore the image group taken at positions 2-5 is necessary for generating three-dimensional data for the non-working area. 【0151】In the second period, the image selection unit 114 outputs the image group taken at positions 11 and 20, which have a weight of "5", to the three-dimensional data generation unit 115. The image selection unit 114 also thins out the image group taken at positions 12 to 15, which have a weight of "3", and outputs it to the three-dimensional data generation unit 115. The amount of thinning is greater for this group than for the image group taken at positions 6 to 9. The image selection unit 114 also thins out the image group taken at positions 16 to 19, which have a weight of "2", and outputs it to the three-dimensional data generation unit 115. The amount of thinning is greater for this group than for the image group taken at positions 12 to 15. 【0152】 In the third period, the image selection unit 114 outputs the image group taken at positions 21 and 30, which have a weight of "5", to the three-dimensional data generation unit 115. The image selection unit 114 also thins out the image group taken at positions 22 to 25, which have a weight of "1", and outputs this to the three-dimensional data generation unit 115. The amount of thinning is greater for the image group taken at positions 16 to 19 than for the image group taken at positions 16 to 19. The image selection unit 114 does not output the image group taken at positions 26 to 29, which have a weight of "0", to the three-dimensional data generation unit 115. 【0153】 Next, in step S111, the three-dimensional data generation unit 115 generates three-dimensional data using the image input from the image selection unit 114. 【0154】 The processing in the third embodiment is carried out as described above. According to the third embodiment, in addition to the effects of the first embodiment, it is possible to prevent unnecessary images from being stored in the first storage unit 103 and the second storage unit 113. Furthermore, by not using unnecessary images for the generation of three-dimensional data, the amount of computation required for three-dimensional data generation can be reduced, thereby improving the efficiency of three-dimensional data generation and shortening the time required for generation. 【0155】 <Fourth Embodiment> [Configuration of the work machine 10 and camera CA] The fourth embodiment will now be described. The configuration and operation of the work machine 10 are the same as in the first embodiment. In the fourth embodiment, the work area is defined as only the area in which the work machine 10 performs excavation. 【0156】In the fourth embodiment, the work machine 10 is equipped with multiple cameras, namely a first camera CA1 and a second camera CA2. The first camera CA1 and the second camera CA2 are mounted so that their lenses face towards the ground surface in order to acquire images of the area around the work machine 10, mainly the work area where the bucket 18 excavates soil and sand. 【0157】 As shown in Figure 22, the first camera CA1 is attached to the boom 14 of the work machine 10. The second camera CA2 is attached to the side of the slewing body 13 so that it can photograph the work area when the bucket 18 of the work machine 10 is facing sideways to the work area. This allows the second camera CA2 to photograph the work area at a different timing than the first camera CA1. However, the mounting positions of the first camera CA1 and the second camera CA2 are not limited to the boom 14 and the slewing body 13, but can be set up anywhere on the work machine 10 that allows them to photograph the work area at different times. 【0158】 It is assumed that the first camera CA1 is mounted in a position that allows it to capture a wider working area than the second camera CA2. The first camera CA1 and the second camera CA2 continuously capture images at a predetermined frame rate or higher, for example, 30 fps or higher, while the work machine 10 is in operation, and continuously transmit the images to the information processing device 100. The frame rate may be 30 fps or higher. Images captured by the first camera CA1 are referred to as first camera images, and images captured by the second camera CA2 are referred to as second camera images. 【0159】 [Configuration of Information Processing Device 100] The configuration of the information processing device 100 in the fourth embodiment will be described with reference to Figure 23. 【0160】 The information processing device 100 in the fourth embodiment includes a first camera image storage unit 401, a first weight determination unit 402, a first information addition unit 403, a second camera image storage unit 404, a second weight determination unit 405, and a second information addition unit 406. The other configurations are the same as in the first embodiment, so their description is omitted. 【0161】The first camera image storage unit 401 is composed of a large-capacity storage medium such as a hard disk or flash memory, and stores multiple first camera images and positional orientation information for each image in association with each other. 【0162】 The first weight determination unit 402 determines the weights for the first camera image. The method for determining the weights is the same as that of the weight determination unit 111 in the first embodiment. 【0163】 The first information addition unit 403 adds the weight information determined by the first weight determination unit 402 to the first camera image read from the first camera image storage unit 401. 【0164】 The second camera image storage unit 404 is composed of a large-capacity storage medium such as a hard disk or flash memory, and stores multiple second camera images and positional orientation information for each image in association. The first camera image storage unit 401 and the second camera image storage unit 404 may be composed of a single storage medium. 【0165】 The second weight determination unit 405 determines the weights for the second camera image. The method for determining the weights is the same as that of the weight determination unit 111 in the first embodiment. 【0166】 The second information addition unit 406 adds the weight information determined by the second weight determination unit 405 to the second camera image read from the second camera image storage unit 404. 【0167】 [Processing in the Information Processing Device 100] The processing in the information processing device 100 of the fourth embodiment will be described with reference to Figure 24. Steps S103 to S107, S110, and S111 are the same as in the first embodiment, so their explanation will be omitted. 【0168】 In step S141, the receiving unit 101 receives the first camera image transmitted from the first camera CA1 and the second camera image transmitted from the second camera CA2. The receiving unit 101 outputs the first camera image and the second camera image to the position and attitude estimation unit 102. 【0169】Next, in step S142, the position and orientation estimation unit 102 estimates the position and orientation of the first camera CA1 based on the first camera image. The position and orientation estimation unit 102 associates the position and orientation information of the first camera CA1 with the first camera image and outputs it to the first camera image storage unit 401 for storage. The position and orientation estimation unit 102 also outputs the position and orientation information of the first camera CA1 to the position and orientation information storage unit 105 for storage. 【0170】 Furthermore, the position and orientation estimation unit 102 estimates the position and orientation of the second camera CA2 based on the second camera image. The position and orientation estimation unit 102 associates the position and orientation information of the second camera image with that of the second camera CA2 and outputs it to the second camera image storage unit 404 for storage. The position and orientation estimation unit 102 also outputs the position and orientation information of the second camera CA2 to the position and orientation information storage unit 105 for storage. 【0171】 Next, in step S143, the information generation unit 108 generates first phase-weight conversion information using time information and the (one-dimensional) position information of the first camera CA1, and generates second phase-weight conversion information using time information and the (one-dimensional) position information of the second camera CA2. The information generation unit 108 outputs the first phase-weight conversion information to the first weight determination unit 402, and outputs the second phase-weight conversion information to the second weight determination unit 405. At this time, for the first camera CA1, the first phase-weight conversion information is determined when "-0.2 < coordinate value ≤ 0.2" is determined to be a work area, as in the first embodiment, and for the second camera CA2, the second phase-weight conversion information is determined when "-0.4 < coordinate value ≤ 0.4" is determined to be a work area. Of course, the same judgment criteria may be applied. 【0172】 In step S107, the phase interpolation unit 110 determines the phase of each image in the same manner as in the first embodiment. The phase interpolation unit 110 determines the phase corresponding to the time for the first camera image stored in the first camera image storage unit 401 by interpolation and outputs it to the first weight determination unit 402. The phase interpolation unit 110 also determines the phase corresponding to the time for the second camera image stored in the second camera image storage unit 404 by interpolation and outputs it to the second weight determination unit 405. 【0173】Next, in step S144, the first weight determination unit 402 determines the weight for the first camera image using the first phase-weight conversion information in the same manner as in the first embodiment. 【0174】 Next, in step S145, the first information adding unit 403 adds weight information determined based on the same position and orientation information to the first camera image read from the first camera image storage unit 401 by referring to the position and orientation information associated with the first camera image. The first information adding unit 403 outputs the first camera image with the added weight information to the second storage unit 113 for storage. 【0175】 Furthermore, in step S146, the second weight determination unit 405 determines the weight for the second camera image using the second phase-weight conversion information in the same manner as in the first embodiment. 【0176】 Next, in step S147, the second information adding unit 406 adds weight information determined based on the same position and orientation information to the second camera image read from the second camera image storage unit 404 by referring to the position and orientation information associated with the second camera image. The second information adding unit 406 outputs the second camera image with the added weight information to the second storage unit 113 for storage. 【0177】 The second storage unit 113 stores the first camera image with weight information added and the second camera image with weight information added. 【0178】 In this fourth embodiment, the weights for the first camera image captured by the first camera CA1 and the weights for the second camera image captured by the second camera CA2 are determined individually based on the phase relationship between each of the images. 【0179】 Steps S144 and S145, and steps S146 and S147 may be performed in the reverse order, or simultaneously or almost simultaneously. 【0180】 Next, in step S110, the image selection unit 114 selects images to be used for generating three-dimensional data from the first camera images and second camera images stored in the second storage unit 113 based on the weight information and outputs them to the three-dimensional data generation unit 115. 【0181】 Here, we will explain assuming that the weights attached to the first camera image (shown by solid lines) and the weights attached to the second camera image (shown by dashed lines) are as shown in Figure 25. The image selection unit 114 sets multiple thresholds for the weights to divide them into multiple levels. In the example in Figure 25, two thresholds are set to divide the weights into three levels: high, medium, and low. 【0182】 Images with a "high" weight are selected to be used for generating three-dimensional data and output to the three-dimensional data generation unit 115. Images with a "medium" weight are selected to be used for generating three-dimensional data but are thinned out and output to the three-dimensional data generation unit 115. Images with a "low" weight are not selected to be used for generating three-dimensional data and are not output to the three-dimensional data generation unit 115. 【0183】 As shown in Figure 25, the first group of camera images with weight W1-1 added in phase P1 have a "medium" weight, so they are thinned out and output to the three-dimensional data generation unit 115. Also, the second group of camera images with weight W1-2 added in phase P1 have a "low" weight, so they are not output to the three-dimensional data generation unit 115. In phase P1, the first camera CA1 captures the work area, so the first camera image is necessary for the generation of three-dimensional data, but the second camera CA2 does not capture the work area, so the second camera image is not necessary for the generation of three-dimensional data. 【0184】 As shown in Figure 25, the first group of camera images with weight W2-1 added in phase P2 have a "low" weight and are therefore not output to the three-dimensional data generation unit 115. Also, the second group of camera images with weight W2-2 added in phase P2 have a "medium" weight and are therefore decimated before being output to the three-dimensional data generation unit 115. In phase P2, the first camera CA1 does not capture the work area, so the first camera image is not necessary for the generation of three-dimensional data, but the second camera CA2 captures the work area, so the second camera image is necessary for the generation of three-dimensional data. 【0185】In the first operation, the work machine 10 performs excavation, while in the second operation, the work machine 10 rotates the slewing body 13 and does not perform excavation. Therefore, the state of the work area is less affected and more stable during the second operation than during the first operation. As a result, it is thought that the work area can be captured more clearly during the second operation than during the first operation. Therefore, the weight of the second camera image of the work area captured during the second operation is made greater than the weight of the first camera image of the work area captured during the first operation. 【0186】 The first group of camera images with weight W3-1 added at phase P3, as shown in Figure 25, have a "low" weight and are therefore not output to the three-dimensional data generation unit 115. Similarly, the second group of camera images with weight W3-2 added at phase P3 also have a "low" weight and are therefore not output to the three-dimensional data generation unit 115. During the third operation, neither the first camera CA1 nor the second camera CA2 photographs the work area, so neither the first group of camera images nor the second group of camera images are necessary for generating three-dimensional data. 【0187】 As shown in Figure 25, the first group of camera images with weight W4-1 added in phase P4 have a "medium" weight, so they are thinned out and output to the three-dimensional data generation unit 115. Similarly, the second group of images with weight W4-2 added in phase P4 have a "high" weight, so they are also output to the three-dimensional data generation unit 115. In phase P4, the second camera CA2 captures the work area, so the second camera images are necessary for the generation of three-dimensional data, while the first camera CA1 does not capture the work area, so its importance in the generation of three-dimensional data is low. 【0188】 As shown in Figure 25, the first camera image group with weight W5-1 added at phase P5 has a "high" weight and is therefore output to the three-dimensional data generation unit 115. On the other hand, the second camera image group with weight 5-2 added at phase P5 has a "low" weight and is therefore not output to the three-dimensional data generation unit 115. At phase P5, the first camera CA1 captures the work area, so the first camera image is necessary for the generation of three-dimensional data, but the second camera CA2 does not capture the work area and is therefore not necessary for the generation of three-dimensional data. 【0189】Since the viewpoint of the first camera CA1 can capture a wider work area than that of the second camera CA2, the weight of the first camera image during the fourth operation in the first period is increased compared to the second camera image during the first operation in the second period. 【0190】 The work machine 10 performs excavation in the first operation, and then transitions to the fourth operation after going through the second and third operations. During the fourth operation, the state of the work area is less affected and more stable than during the first, second, and third operations. As a result, it is thought that the work area can be clearly photographed during the fourth operation. Therefore, the weight of the second camera image of the work area taken during the fourth operation is made greater than the weight of the first camera image taken during the first operation and the weight of the second camera image taken during the second operation. 【0191】 Note that phases P1 to P5 in Figure 25 are specific phases used to explain the weight-based image selection, and do not mean that images are only captured by camera CA during those phases. As mentioned above, image capture by camera CA is performed continuously at a predetermined frame rate from the moment the work machine 10 starts operating. 【0192】 Next, in step S111, the three-dimensional data generation unit 115 generates three-dimensional data using the image input from the image selection unit 114. 【0193】 The processing of the fourth embodiment is carried out as described above. According to the fourth embodiment, in addition to the effects of the first embodiment, it is possible to select images that are more suitable for three-dimensional data generation from images captured by multiple cameras CA and generate highly accurate three-dimensional data. 【0194】 <Fifth Embodiment> [Configuration of Information Processing Device 100] The configuration of the information processing device 100 in the fifth embodiment will be described with reference to Figure 26. The configuration and operation of the work machine 10 and the configuration and operation of the camera CA are the same as in the first embodiment. 【0195】 The information processing device 100 in the fifth embodiment includes a periodic operation determination unit 501. The other configurations are the same as in the first embodiment, so their description is omitted. 【0196】The periodic motion determination unit 501 determines whether the operation of the work machine 10 is periodic based on the position and orientation information of the camera CA, and outputs the determination result to the weight determination unit 111. 【0197】 [Processing in the Information Processing Device 100] The processing in the information processing device 100 of the fifth embodiment will be described with reference to Figure 27. Steps S101 to S107 and steps S108 to S111 are the same as in the first embodiment, so their description will be omitted. 【0198】 In step S151, the periodic motion determination unit 501 determines whether the operation of the work machine 10 is periodic based on the position and orientation information input from the position and orientation estimation unit 102. For example, the periodic motion determination unit 501 can determine that the operation of the work machine 10 is periodic if, based on the position and orientation information, the camera CA attached to the boom 14 of the work machine 10 is in the same position a predetermined number of times at predetermined timings or predetermined time intervals. 【0199】 If the operation of the work machine 10 is cyclical, the process proceeds to step S108 (Yes in step S151), and the weight determination unit 111 determines the weight in the same manner as in the first embodiment. On the other hand, if the operation of the work machine 10 is not cyclical, the process proceeds to step S152 (No in step S151), and the weight is determined to a predetermined value. 【0200】 The processing of the fifth embodiment is carried out as described above. According to the fourth embodiment, in addition to the effects of the first embodiment, even when the operation of the work machine 10 is not a periodic operation, weights can be added to the images, and based on those weights, images can be selected to generate three-dimensional data. 【0201】 <Modifications> Although embodiments of this technology have been described in detail above, this technology is not limited to the embodiments described above, and various modifications are possible based on the technical concept of this technology. 【0202】 This technology can be implemented by combining any two or more of the first to fifth embodiments. 【0203】The periodic operation of the work machine 10 is not limited to that shown in Figure 2, but may be other operations as shown in Figure 28, or any operation that is repeated. In Figure 28, the work machine 10 performs a periodic operation by repeating four operations: a first operation in which the bucket 18 collects the soil loaded on the truck; a second operation in which the position of the bucket 18 is moved by the rotation of the slewing body 13 while the vehicle moves by the movement of the vehicle 11; a third operation in which the soil is released from the bucket 18 to fill in the area; and a fourth operation in which the position of the bucket 18 is moved by the rotation of the slewing body 13 while the vehicle moves by the movement of the vehicle 11. 【0204】 In this embodiment, the area where soil is excavated is defined as the work area, but the area where soil is discharged may be defined as the work area in addition to or instead of the area where excavation is performed. Furthermore, the work area may be defined as any area where the work machine 10 performs work, not limited to excavation and discharge. 【0205】 In the first and fourth embodiments, the weights are divided into three stages, and in the third embodiment, the weights are divided into six stages, but the way the weights are divided is not limited to these. They may be divided into fewer or more stages. Furthermore, the manufacturer or user of the information processing device 100 may be allowed to set how the weights are divided. 【0206】 The information processing device 100 may generate other data using multiple images in addition to, or instead of, three-dimensional data. Examples of other data include orthomosaic images. Orthomosaic images can be created using existing methods such as SfM-MVS (Structure-from-Motion / Multi-View-Stereo). The data generated by the information processing device 100 can be any data generated using multiple images. 【0207】 The generation of three-dimensional data may be performed by an external device, such as a cloud server, rather than by the information processing device 100. In that case, the information processing device 100 does not need to have a three-dimensional data generation unit 115, and the image selection unit 114 sends the selected image to the external device or cloud server via the network. 【0208】In this embodiment, the work machine 10 is described as an excavator (hydraulic shovel) and the work as excavating soil and sand. However, the work machine 10 and the work may be other. For example, the work machine 10 may be a bulldozer or a wheel loader, and the work may be the transportation of soil and sand. Alternatively, the work machine 10 may be a crane, and the work may be the movement or stacking of cargo, materials, or supplies. Furthermore, the work machine 10 may be construction machinery, and the work may be the construction or demolition of buildings. 【0209】 Furthermore, in forestry, the implement 10 may be a chainsaw and the work may be cutting trees; the implement 10 may be a feller buncher and the work may be collecting felled trees; or the implement 10 may be a harvester and the work may be a series of operations from felling standing trees, pruning branches, and stacking to timber production. 【0210】 Furthermore, the work machine 10 may be a machine or robot in a factory, and the work may be the manufacturing of electronic equipment, robots, mobile vehicles such as automobiles, ships, and airplanes, general merchandise, clothing, food, etc., performed by that machine or robot. 【0211】 In other words, the work can be anything as long as it involves a periodic, repeating action that changes the state of the object being worked on. 【0212】 Sensors such as LiDAR (Light Detection and Ranging), depth sensors, and ToF (Time of Flight) sensors may be attached to the work machine 10, and the position and orientation of the camera CA for image capture may be estimated using the sensing data from these sensors. In that case, image capture and sensing by the sensors must be synchronized, and the sensing data detected at the same position must be linked to the captured image. 【0213】The technology can also take the following configurations: (1) An information processing device comprising: a weight processing unit that determines weights for images based on information indicating the state of a work machine performing periodic movements, which is determined based on information relating to either or both of a work machine that performs work by multiple periodic movements and a camera that photographs the work area and acquires multiple images; and an image selection unit that selects an image to be used for generating three-dimensional data from a plurality of images based on the weights. (2) The information processing device according to (1), wherein the image selection unit selects the image corresponding to a first weight from the plurality of images to be used for generating the three-dimensional data. (3) The information processing device according to (1) or (2), wherein the image selection unit does not select an image corresponding to a second weight lower than the first weight to be used for generating the three-dimensional data. (4) The information processing device according to any one of (1) to (3), wherein the image selection unit thins out a plurality of images corresponding to a third weight lower than the first weight and higher than the second weight to select an image to be used for generating the three-dimensional data. (5) An information processing device according to any one of (1) to (4), comprising an ID determination unit that determines an ID for ordering and distinguishing each of a series of periodic operations, wherein the same ID is assigned to a series of images taken during one periodic operation. (6) An information processing device according to (5), wherein the image selection unit selects a series of images assigned the same ID as images to be used for generating the three-dimensional data. (7) An information processing device according to (5), wherein the weight processing unit determines the weight according to the order of the periodic operations identified by the ID. (8) An information processing device according to (7), wherein the weight processing unit determines the weight based on the ID such that it decreases as the series of periodic operations progress. (9) An information processing device according to (8), wherein the image selection unit thins out a series of images such that the amount of thinning increases as the weight decreases and selects images to be used for generating the three-dimensional data.(10) An information processing device according to any one of (1) to (9), wherein the image of the work area is captured from different positions by a first camera and a second camera, the weight processing unit has a first weight determination unit and a second weight determination unit, the first weight determination unit determines the weight for the image captured by the first camera based on information indicating the state of the work machine, and further determines the weight for the image captured by the second camera based on information indicating the state of the work machine. (11) An information processing device according to (10), wherein the image selection unit selects an image to be used for generating three-dimensional data from a plurality of images captured by the first camera and a plurality of images captured by the second camera based on the weight. (12) An information processing device according to any one of (1) to (11), comprising a periodic operation determination unit that determines whether the periodic operation is being performed, wherein if the periodic operation determination unit determines that the periodic operation is not being performed, the weight processing unit determines a predetermined value as the weight instead of the weight based on the periodic operation information. (13) An information processing device according to any one of (1) to (12), further comprising a three-dimensional data generation unit that generates three-dimensional data based on the image selected by the image selection unit. (14) An information processing device according to any one of (1) to (13), wherein the three-dimensional data is a three-dimensional point cloud. (15) An information processing device according to any one of (1) to (14), wherein the operation is at least one of excavation by the work machine and discharge of soil. (16) An information processing device according to any one of (1) to (15), wherein the camera that takes the image is attached to the work machine that performs the periodic operation. (17) An information processing method that determines a weight for the image based on information indicating the state of the work machine that performs the periodic operation, which is determined based on information relating to either or both of the work machine that performs the operation by multiple periodic operations and the camera that takes pictures of the work area and acquires multiple images, and selects an image to be used for generating three-dimensional data from a plurality of images based on the weight.(18) A program that causes a computer to execute an information processing method which determines a weight for an image based on information indicating the state of a work machine that performs a work by performing a work by multiple periodic movements and a camera that photographs the work area and acquires multiple images, and which determines a weight for the image based on the weight and selects an image to be used for generating three-dimensional data from the multiple images. 【0214】 10...Work machine 100...Information processing device 102...Position and orientation estimation unit 104...Weight processing unit 114...Image selection unit 115...Three-dimensional data generation unit 201...ID determination unit 402...First weight determination unit 405...Second weight determination unit 501...Periodic motion determination unit CA...Camera CA1...First camera CA2...Second camera

Claims

1. An information processing device comprising: a work machine that performs work through multiple periodic movements; a weight processing unit that determines weights for the images based on information indicating the state of the work machine, which is determined based on information related to either or both of the work machine and a camera that photographs the work area and acquires multiple images; and an image selection unit that selects an image to be used for generating three-dimensional data from the multiple images based on the weights.

2. The information processing apparatus according to claim 1, wherein the image selection unit selects from the plurality of images the image corresponding to the first weight as the image to be used for generating the three-dimensional data.

3. The information processing apparatus according to claim 1, wherein the image selection unit does not select images corresponding to a second weight lower than the first weight as images to be used for generating the three-dimensional data.

4. The information processing apparatus according to claim 1, wherein the image selection unit thins out a plurality of images corresponding to a third weight which is lower than the first weight and higher than the second weight, and selects them as images to be used for generating the three-dimensional data.

5. The information processing apparatus according to claim 1, comprising an ID determination unit that determines an ID for ordering and distinguishing each of a series of periodic operations, wherein the same ID is assigned to a plurality of images captured during one periodic operation.

6. The information processing apparatus according to claim 5, wherein the image selection unit selects a plurality of images that have the same ID as images to be used for generating the three-dimensional data.

7. The information processing apparatus according to claim 5, wherein the weight processing unit determines the weight according to the order of the periodic operations identified by the ID.

8. The information processing apparatus according to claim 7, wherein the weight processing unit determines the weight based on the ID such that it decreases as a series of cycle operations progress.

9. The information processing apparatus according to claim 8, wherein the image selection unit thins out multiple images such that the amount of thinning increases as the weight decreases, and selects the images to be used for generating the three-dimensional data.

10. The information processing apparatus according to claim 1, wherein the image of the work area is captured from different positions by a first camera and a second camera, the weight processing unit comprises a first weight determination unit and a second weight determination unit, the first weight determination unit determines a weight for the image captured by the first camera based on information indicating the state of the work machine, and further determines a weight for the image captured by the second camera based on information indicating the state of the work machine.

11. The information processing apparatus according to claim 10, wherein the image selection unit selects an image to be used for generating three-dimensional data from a plurality of images captured by the first camera and a plurality of images captured by the second camera, based on the weight.

12. The information processing device according to claim 1, further comprising a periodic operation determination unit for determining whether the periodic operation is being performed, wherein if the periodic operation determination unit determines that the periodic operation is not being performed, the weight processing unit determines a predetermined value as the weight instead of the weight based on the periodic operation information.

13. The information processing apparatus according to claim 1, further comprising a three-dimensional data generation unit that generates three-dimensional data based on the image selected by the image selection unit.

14. The information processing device according to claim 1, wherein the three-dimensional data is a three-dimensional point cloud.

15. The information processing apparatus according to claim 1, wherein the work is at least one of excavation using the work machine and discharge of soil and sand.

16. The information processing apparatus according to claim 1, wherein the camera that takes the image is attached to the work machine that performs the periodic operation.

17. An information processing method that determines weights for images based on information indicating the state of a work machine, which is determined based on information about either or both of a work machine that performs work through multiple periodic movements and a camera that photographs a work area and acquires multiple images, and selects an image to be used for generating three-dimensional data from the multiple images based on the weights.

18. A program that causes a computer to execute an information processing method which determines weights for images based on information indicating the state of a work machine, which is determined based on information about either or both of a work machine that performs work through multiple periodic movements and a camera that photographs a work area and acquires multiple images, and selects an image to be used for generating three-dimensional data from the multiple images based on the weights.

Images