Information processing device, information processing method, and program

Abstract

To realize new technologies that provide information about resources. [Solution] The disclosed information processing device comprises an acquisition unit, an image analysis unit, and an estimation unit. The acquisition unit acquires images of resource sites where resource-related work is performed. The image analysis unit detects work machines performing resource-related work within the images and detects actions performed by the work machines. The estimation unit estimates the amount of resource utilization based on the actions of the work machines.

Description

Technical Field

[0006] , ,

[0001] This disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] A technique related to this disclosure is disclosed in Patent Document 1. Patent Document 1 discloses a technique for estimating the type of work being performed at a work site based on an image obtained by photographing the work site from above.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As an index related to resources, the mining and industrial production index is known. The mining and industrial production index indicates the status (quantity, weight, amount, etc.) of monthly mining and industrial production activities in each country. For example, the mining and industrial production index is used to grasp or predict the trend of the economy. This mining and industrial production index indicates the status of mining and industrial production activities on a monthly basis and has problems such as lack of immediacy.

[0005] An example of the object of this disclosure is to realize a new technique for providing information related to resources.

Means for Solving the Problems

[0006] According to one aspect of this disclosure, an acquisition means for acquiring an image of a resource site where work related to resources is performed, an image analysis means for detecting a work machine that performs work related to resources in the image and detecting an action performed by the work machine, An estimation means for estimating the amount of resources used based on the actions of the aforementioned work machine, An information processing device having the following is provided.

[0007] Furthermore, according to one aspect of this disclosure, One or more computers, We obtained images taken at resource sites where resource-related work was being carried out. The system detects a work machine that performs resource-related work within the image, and also detects the actions performed by the work machine. An information processing method is provided for estimating the amount of resources used based on the actions of the aforementioned work machine.

[0008] Furthermore, according to one aspect of this disclosure, Computers, A means for acquiring images of resource sites where resource-related work is being carried out. Image analysis means for detecting a work machine that performs resource-related work in the image and for detecting the actions performed by the work machine. Estimation means for estimating the amount of resource utilization based on the actions of the aforementioned work machine, A program is provided to enable it to function as such. [Effects of the Invention]

[0009] According to one example of this disclosure, new technologies will be realized that provide information about resources. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows an example of a functional block diagram of an information processing device. [Figure 2] Figure 2 is a flowchart showing an example of the processing flow of an information processing device. [Figure 3] Figure 3 shows an example of the hardware configuration of an information processing device. [Figure 4] Figure 4 is a diagram illustrating an example of a resource site. [Figure 5]FIG. 5 is a diagram showing an example of information output by the information processing apparatus. [Figure 6] FIG. 6 is a diagram showing another example of information output by the information processing apparatus. [Figure 7] FIG. 7 is a flowchart showing another example of the processing flow of the information processing apparatus. [Figure 8] FIG. 8 is a flowchart showing another example of the processing flow of the information processing apparatus. [Figure 9] FIG. 9 is a flowchart showing another example of the processing flow of the information processing apparatus. [Figure 10] FIG. 10 is a flowchart showing another example of the processing flow of the information processing apparatus. [Figure 11] FIG. 11 is a diagram showing another example of the functional block diagram of the information processing apparatus. [Figure 12] FIG. 12 is a flowchart showing another example of the processing flow of the information processing apparatus. [Figure 13] FIG.  13 is a diagram showing another example of the functional block diagram of the information processing apparatus.

MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of this disclosure will be described with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. Also, in all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[0012] <<First Embodiment>> FIG. 1 is a functional block diagram showing an overview of the information processing apparatus 10. FIG. 2 is a flowchart showing an example of the processing flow executed by the information processing apparatus 10.

[0013] As shown in FIG. 1, the information processing apparatus 10 includes an acquisition unit 11, an image analysis unit 12, and an estimation unit 13. By these functional units, the processing of the flowchart in FIG. 2 is executed.

[0014] In S10, the acquisition unit 11 acquires images of the resource site where resource-related work is being carried out. In S11, the image analysis unit 12 detects a work machine that performs resource-related work within the image, and also detects the actions performed by the work machine. In S12, the estimation unit 13 estimates the amount of resources used based on the actions of the work machine.

[0015] In this way, the information processing device 10 analyzes images taken at resource sites where resource-related work is performed to estimate the amount of resources used. With such an information processing device 10, by processing images in real time, the amount of resources used can be estimated in real time. As a result, it becomes possible to provide information (resource utilization amount) with excellent immediacy.

[0016] Furthermore, the information processing device 10 detects work machines that perform resource-related tasks from images taken at the resource site and estimates the amount of resources utilized based on the actions performed by those work machines. By estimating the amount of resources utilized based on the actions of work machines used for resource extraction and transportation, the information processing device 10 can estimate the amount of resources utilized with high accuracy.

[0017] Thus, the information processing device 10 enables a new technology that provides information about resources (the amount of resource utilization and information generated based on it).

[0018] <<Second Embodiment>> <Overview> The information processing device 10 in the second embodiment detects on-site transport machines that transport resources within the resource site and also detects the number of predetermined movements made by the on-site transport machines within the resource site. The information processing device 10 then estimates the amount of resources utilized based on the number of movements. This will be explained in detail below.

[0019] <Hardware Configuration> First, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the implementation method and the device. The software includes programs that are pre-installed at the time of shipment of the device, as well as programs downloaded from recording media such as CDs (Compact Discs) or from servers on the Internet.

[0020] Figure 3 is a block diagram illustrating the hardware configuration of the information processing device 10. As shown in Figure 3, the information processing device 10 includes a processor 1A, memory 2A, input / output interface 3A, peripheral circuitry 4A, and bus 5A. Peripheral circuitry 4A includes various modules. The information processing device 10 does not necessarily have peripheral circuitry 4A. The information processing device 10 may also be composed of multiple physically and / or logically separated devices. In this case, each of the multiple devices may have the above hardware configuration.

[0021] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuits 4A, and input / output interface 3A to send and receive data to and from each other. Processor 1A is a processing unit such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit). Memory 2A is a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. Input / output interface 3A also includes interfaces for connecting to communication networks such as the Internet. Input devices include, for example, keyboards, mice, microphones, physical buttons, touch panels, etc. Output devices include, for example, displays, projectors, speakers, printers, mailers, etc. Processor 1A can issue commands to each module and perform calculations based on their calculation results.

[0022] <Functional Configuration> Next, the functional configuration of the information processing device 10 will be described in detail. Figure 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an acquisition unit 11, an image analysis unit 12, and an estimation unit 13.

[0023] The acquisition unit 11 acquires images of resource sites where resource-related work is being carried out.

[0024] "Resources" are materials used in industry. The state (activity level) of an industry can be understood based on the amount of resources used. Examples of resources include, but are not limited to, mineral resources, water resources, forest resources, marine resources, and food resources.

[0025] A "resource site" is a site where various tasks related to resources are performed using work equipment. Examples of resource sites include, but are not limited to, the following. • Sites for acquiring new resources from nature (mining sites, logging sites, etc.) • Sites where resources are stored (mining sites, logging sites, warehouses, etc.) • Sites (factories, etc.) where resources are processed (refined, etc.) • Workplaces (factories, etc.) that utilize resources to create new things.

[0026] "Resource-related work" will vary depending on the type of resource and the location of the resource. Examples of resource-related work include, but are not limited to, work to acquire new resources from nature (mining, logging, etc.), transporting resources, loading resources onto transport vehicles, and inputting resources into machinery.

[0027] The "image" may be a still image or a moving image. For example, the acquisition unit 11 may acquire satellite images taken by an artificial satellite. Alternatively, the acquisition unit 11 may acquire images taken by an aircraft (such as a drone) moving in the sky. Alternatively, the acquisition unit 11 may acquire images taken by a surveillance camera installed at the resource site. It is preferable that the acquisition unit 11 acquires the image with minimal time lag from the image generation timing. By configuring it in this way, it becomes possible to generate and provide information (resource utilization amount) with excellent immediacy.

[0028] "Acquisition" includes at least one of the following: the device retrieving data or information stored in another device or storage medium (active acquisition), and the device inputting data or information output from another device into its own device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a reply, and accessing and reading data from another device or storage medium. Examples of passive acquisition include receiving information that is delivered (or transmitted, push notification, etc.). Furthermore, acquisition may also involve selecting and acquiring data or information from among the received data or information, or selecting and receiving data or information that has been delivered.

[0029] Here, we will illustrate an example of a resource site using Figure 4. Note that the resource site illustrated is merely an example and is not limited to this example.

[0030] Resource site A in Figure 4 is a mining site where resources obtained from nature are extracted. At resource site A, multiple types of work machines are in operation, including on-site transport machine 1, loading machine 2, mining machine 3, and off-site transport machine 5.

[0031] Mining machine 3 is used for extracting resources. The type of mining machine 3 used depends on the type and location of the resource to be extracted. For example, mining machine 3 may be a large hydraulic excavator, a large wheel loader, a large bulldozer, etc., but is not limited to these. The number of mining machines 3 used at resource site A is not limited to the one shown in the illustration; multiple mining machines 3 may be used at resource site A.

[0032] Loading machine 2 is used to load the mined resources onto the on-site transport machine 1. The type of loading machine 2 used depends on the type and location of the resources being mined. For example, loading machine 2 may be a hydraulic excavator, wheel loader, bulldozer, etc., but is not limited to these. The number of loading machines 2 used at resource site A is not limited to the one shown in the diagram; multiple loading machines 2 may be used at resource site A.

[0033] On-site transport vehicle 1 is used for transportation within resource site A. For example, on-site transport vehicle 1 is used to transport mined resources to storage area 4. On-site transport vehicle 1 is used depending on the type and location of the resources being mined. For example, on-site transport vehicle 1 may be a truck, ship, aircraft, etc., but is not limited to these. The number of on-site transport vehicles 1 used at resource site A is not limited to the one shown in the figure; multiple on-site transport vehicles 1 may be used at resource site A.

[0034] Storage Area 4 is a place for temporarily storing mined resources. Storage Area 4 is located in Area C, for example, where the headquarters for mining operations is set up. The number of Storage Areas 4 installed at Resource Site A is not limited to the one shown in the diagram; multiple Storage Areas 4 may be installed at Resource Site A.

[0035] The off-site transport vehicle 5 is used to transport the extracted resources from resource site A to an external location. The type of off-site transport vehicle 5 used depends on the type and location of the resources being extracted. For example, the off-site transport vehicle 5 may be a truck, ship, aircraft, etc., but is not limited to these. The number of off-site transport vehicles 5 used at resource site A is not limited to the one shown in the diagram; multiple off-site transport vehicles 5 may be used at resource site A.

[0036] In the example shown in Figure 4, resource site A is a vast area. Therefore, mining operations are carried out in sections a within resource site A. The time required to complete mining operations in section a may range from a few hours to several days, weeks, or even months. Alternatively, mining operations may be carried out on the entire area of ​​resource site A at once, rather than in sections a.

[0037] As shown in the diagram, the loading machine 2 and the mining machine 3 are located in a section a where mining operations are being carried out. Mining operations are then performed by operating the mining machine 3 at that location. The mined resources are loaded onto the on-site transport machine 1 by the loading machine 2. The on-site transport machine 1 transports the loaded resources to the storage area 4. The resources are then stored in the storage area 4. The resources stored in the storage area 4 can be loaded onto the off-site transport machine 5 at any time and transported to an external location.

[0038] Returning to Figure 1, the image analysis unit 12 analyzes the image acquired by the acquisition unit 11 using widely known techniques. For example, the image analysis unit 12 can utilize techniques such as object detection, semantic segmentation, image classification, and super-resolution.

[0039] First, the image analysis unit 12 detects the work equipment that performs resource-related work within the image acquired by the acquisition unit 11. The image analysis unit 12 can identify and detect multiple types of work equipment (on-site transport equipment 1, loading equipment 2, mining equipment 3, off-site transport equipment 5, etc.). Alternatively, the image analysis unit 12 may identify and detect each type of work equipment (for example, by manufacturer or model).

[0040] The image analysis unit 12 can detect a work machine within an image, for example, using a classifier generated by machine learning. The image analysis unit 12 may also utilize other techniques such as object detection, semantic segmentation, and super-resolution in this detection.

[0041] Furthermore, the image analysis unit 12 detects the actions performed by the detected work machine based on the images acquired by the acquisition unit 11. Specifically, the image analysis unit 12 detects that the detected work machine has performed a predetermined action. The image analysis unit 12 can then count the number of times the detected work machine has performed the predetermined action.

[0042] "Specified actions" are actions related to resource utilization and are defined for each piece of work equipment. For example, specified actions include, but are not limited to, transporting resources to a specified location, extracting resources, loading resources, and feeding resources into the machine.

[0043] In the second embodiment, the image analysis unit 12 detects a predetermined action performed by the on-site transport machine 1 and counts the number of times it is performed.

[0044] The "predetermined action by the on-site transport machine 1" refers to movement related to the transportation of resources by the on-site transport machine 1. One example of such a "predetermined action by the on-site transport machine 1" is movement by the on-site transport machine 1 to a predetermined location. In the example explained using Figure 4, the on-site transport machine 1 transports the mined resources within resource site A to storage area 4. In this example, the image analysis unit 12 can detect movement by the on-site transport machine 1 to storage area 4 (predetermined action). Note that the detection of movement to storage area 4 is merely an example, and the image analysis unit 12 may also detect movement by the on-site transport machine 1 to other locations.

[0045] The image analysis unit 12 detects when the on-site transport vehicle 1 has moved to a predetermined location (for example, storage area 4) by tracking the position of the on-site transport vehicle 1 detected within the image. The image analysis unit 12 can track the on-site transport vehicle 1 within the image using widely known tracking techniques. Then, in response to the detection that the on-site transport vehicle 1 has moved back to the predetermined location after leaving it, the image analysis unit 12 can increment the number of predetermined actions (number of movements) by the on-site transport vehicle 1 by 1.

[0046] The criteria for determining whether an object has "left its designated location (e.g., storage area 4)" or "moved to its designated location" can vary. For example, the image analysis unit 12 may determine that the on-site transport device 1 has left its designated location when the distance between the designated location and the on-site transport device 1 exceeds a threshold. The image analysis unit 12 may also determine that the on-site transport device 1 has moved to its designated location when the distance between the designated location and the on-site transport device 1 falls below the threshold.

[0047] In addition, there may be cases where multiple on-site transport devices 1 are used within resource site A. In this case, the image analysis unit 12 may not identify the multiple on-site transport devices 1 from each other, but may count the number of predetermined actions (number of movements) performed by the multiple on-site transport devices 1 together. In this case, the image analysis unit 12 counts how many times the multiple on-site transport devices 1 have moved to a predetermined location (for example, storage area 4).

[0048] In addition, the image analysis unit 12 may identify multiple on-site transport devices 1 by type and count the number of predetermined actions (number of movements) for each type of on-site transport device 1. The image analysis unit 12 can identify and detect multiple on-site transport devices 1 by type from within the image using a classifier generated by machine learning.

[0049] The estimation unit 13 estimates the amount of resource utilization based on the actions of the work machine. "Utilization amount" indicates the amount of resource being utilized. Utilization amount includes, but is not limited to, the amount of resource extracted, the amount of resource transported, and the amount of resource input into the machine.

[0050] In the second embodiment, the estimation unit 13 estimates the amount of resources used based on the number of predetermined actions (number of movements) of the on-site transport machine 1.

[0051] The estimation unit 13 can calculate the amount of resources utilized within a predetermined time period by multiplying the number of predetermined actions (number of movements) performed by the on-site transport machine 1 within a predetermined time period by the amount of resources transported by the on-site transport machine 1 in one movement. Various values ​​can be used for the predetermined time period. For example, the predetermined time period can be 1 to a few minutes, 1 to a few hours, 1 to a few days, 1 to a few weeks, 1 to a few months, 1 to a few years, etc. The amount of resources transported by the on-site transport machine 1 in one movement is predetermined and stored in the information processing device 10.

[0052] Furthermore, the amount of resources transported in a single trip may differ depending on the type of on-site transport machine 1. Also, multiple types of on-site transport machines 1 may be used at resource site A. In this case, the image analysis unit 12 and the estimation unit 13 can perform the following processing.

[0053] The image analysis unit 12 identifies multiple on-site transport devices 1 by type and counts the number of predetermined actions (number of movements) for each type of on-site transport device 1.

[0054] The estimation unit 13 calculates the amount of resources utilized within a predetermined time period by multiplying the number of predetermined actions (number of movements) within a predetermined time period by the amount of resources transported in one movement for each type of on-site transport machine 1. The estimation unit 13 then calculates the total amount of resources utilized within a predetermined time period by summing the amounts of resources utilized for each type of on-site transport machine 1. The amount of resources transported by each type of on-site transport machine 1 in one movement is predetermined and stored in the information processing device 10.

[0055] The estimation unit 13 can output the estimation result (amount of resource utilization within a predetermined time). "Output" includes outputting the estimation result via an output device such as a display, projection device, or printer. Output also includes transmitting the estimation result to an external device or storing it in a predetermined storage device. The predetermined storage device may be provided within the information processing device 10, or it may be provided in an external device configured to communicate with the information processing device 10.

[0056] Furthermore, the estimation unit 13 may output information showing the trend (time change) in the amount of resources used at predetermined intervals. For example, the estimation unit 13 may generate and output a table or graph showing the trend.

[0057] Furthermore, the information processing device 10 may acquire images of multiple resource sites A, or images of resource sites A of multiple types of resources. The information processing device 10 may then analyze each image to estimate the amount of resource utilization. In this case, the estimation unit 13 can statistically process these diverse estimation results (amount of resource utilization within a predetermined time) and output the results.

[0058] For example, the estimation unit 13 can calculate the amount of resources utilized within a predetermined time for each country. Furthermore, the estimation unit 13 can calculate the amount of resources utilized within a predetermined time for each region (Asia, Europe, etc.). Additionally, the estimation unit 13 can calculate the amount of resources utilized within a predetermined time for each sub-region (a portion of a country). Based on map information and information indicating the shooting location of each image, the estimation unit 13 can identify the country, region, and sub-region where the resource site A shown in each image exists.

[0059] Furthermore, the estimation unit 13 can calculate the amount of resource used within a predetermined time for each type of resource. For example, the information processing device 10 may store in advance which resource-related work is performed at the shooting location of each image. The estimation unit 13 may then identify which resource the calculated amount of use based on each image relates to. Alternatively, the estimation unit 13 may identify which resource-related work is being performed at the shooting location of each image based on the type of work machine identified by the image analysis unit 12. In this case, the type of work machine used for each type of resource may be defined in advance and stored in the information processing device 10.

[0060] Figure 5 shows an example of the information output by the estimation unit 13. In the example in Figure 5, the trend of the calculated daily amount of oil used in region L of country M is shown in a table. Furthermore, in the example in Figure 5, the day-over-day, month-over-month, and year-over-year changes in the calculated amount of oil used are also shown.

[0061] In the example shown in Figure 5, a UI (User Interface) component is displayed to change the information being shown. By manipulating this UI component, the resources, country, sub-region, and display unit can be changed.

[0062] Figure 6 shows another example of the information output by the estimation unit 13. In the example in Figure 6, the calculated amount of oil utilization in each region on a specified day is mapped and displayed on the map. x1 to x6 displayed in each region on the map represent the calculated amount of oil utilization in each region on a specified day.

[0063] In the example in Figure 6, a UI component for changing the displayed information is shown. By manipulating this UI component, the resource, display unit, and date can be changed.

[0064] Next, an example of the processing flow of the information processing device 10 will be explained using the flowchart in Figure 7. The purpose here is simply to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0065] First, the information processing device 10 acquires an image of resource site A where resource-related work is performed (S20). Next, the information processing device 10 detects the on-site transport machine 1 in the image and detects the number of times the on-site transport machine 1 moves to a predetermined position within resource site A (S21).

[0066] Then, the information processing device 10 estimates the amount of resources used based on the number of movements of the on-site transport device 1 (S22). The information processing device 10 can output the estimation result.

[0067] For example, the information processing device 10 can calculate the amount of resources utilized within a predetermined time period by multiplying the number of movements made by the on-site transport device 1 within a predetermined time period by the amount of resources transported by the on-site transport device 1 in one movement.

[0068] In addition, the information processing device 10 can calculate the amount of resources utilized within a predetermined time period by multiplying the number of movements within a predetermined time period by the amount of resources transported in one movement for each type of on-site transport device 1. The information processing device 10 can then calculate the total amount of resources utilized within a predetermined time period by summing the amounts of resources utilized for each type of on-site transport device 1.

[0069] In addition, the information processing device 10 can analyze images of multiple resource sites A, or images of resource sites A with multiple types of resources, to estimate the amount of resources utilized. The information processing device 10 can then statistically process these diverse estimation results (amount of resources utilized within a predetermined time) and output the results. For example, the information processing device 10 can output information as shown in Figures 5 and 6.

[0070] <Effects and Effects> According to the information processing device 10 of the second embodiment, the same effects and advantages as those of the information processing device 10 of the first embodiment can be achieved.

[0071] Furthermore, the information processing device 10 of the second embodiment can detect the number of movements involved in the transportation of resources by the on-site transport machine 1 that transports resources within the resource site A, and estimate the amount of resources utilized based on the number of movements. When the economy and production activities are good and the demand for resources is high, the amount of resources extracted, etc. will be large, and the number of movements by the on-site transport machine 1 will be large. On the other hand, when the economy and production activities are not very good and the demand for resources is low, the amount of resources extracted, etc. will be small, and the number of movements by the on-site transport machine 1 will be small.

[0072] According to the information processing device 10, which estimates the amount of resource utilization based on the number of movements involved in the transportation of resources by the on-site transport machine 1 that transports resources within resource site A, useful information regarding the economy and production activities can be generated and provided. In other words, the information processing device 10 realizes a new technology that provides information about resources (the amount of resource utilization and information generated based on it).

[0073] Furthermore, in the case of the information processing device 10 that calculates resource utilization through image analysis, resource utilization can be calculated for various time periods such as 1 to a few minutes, 1 to several hours, 1 to several days, 1 to several weeks, 1 to several months, 1 to several years, etc. As a result, it becomes possible to provide information at various levels of granularity that were not possible before.

[0074] Furthermore, the information processing device 10 can analyze images of multiple resource sites A, or images of resource sites A with multiple types of resources, to estimate the amount of resources utilized, and then statistically process and output a wide variety of estimation results. For example, the information processing device 10 can provide information like that shown in Figures 5 and 6. In this way, the information processing device 10 enables the provision of useful information that had not been possible before.

[0075] <<Third Embodiment>> <Overview> In the second embodiment, the amount of resources to be transported by the on-site transport machine 1 in one movement was predetermined. The information processing device 10 then used this predetermined value to calculate the amount of resources utilized.

[0076] However, the amount of resources loaded onto the on-site transport vehicle 1 may vary significantly with each move. For example, this can occur if the system operates under a rule that, regardless of the amount loaded at a predetermined time (e.g., a predetermined time has elapsed since the start of loading), the on-site transport vehicle 1 will begin moving to storage area 4 (transporting the resources). It should be noted that this is just one example, and other factors may also cause the amount of resources loaded onto the on-site transport vehicle 1 to vary significantly with each move.

[0077] Thus, even though the amount of resources loaded onto the on-site transport vehicle 1 varies greatly with each movement, calculating resource utilization using a predefined fixed value for "the amount of resources transported by the on-site transport vehicle 1 in one movement" results in poor accuracy.

[0078] Therefore, the information processing device 10 of the second embodiment detects the loading machine 2 that loads resources onto the on-site transport machine 1, and also detects the number of predetermined first actions performed by the loading machine 2 on the on-site transport machine 1. Then, the information processing device 10 estimates the amount of resources loaded onto the on-site transport machine 1 based on the number of first actions, and estimates the amount of resources utilized based on this estimation result. This will be explained in detail below.

[0079] <Hardware Configuration> This is the same as the first and second embodiments.

[0080] <Functional Configuration> Figure 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an acquisition unit 11, an image analysis unit 12, and an estimation unit 13.

[0081] The configuration of the acquisition unit 11 is the same as in the first and second embodiments.

[0082] The image analysis unit 12 detects the loading machine 2 in the image acquired by the acquisition unit 11. The image analysis unit 12 then detects a predetermined first action performed by the loading machine 2 on the on-site transport machine 1 and counts the number of times this action is performed.

[0083] "The predetermined first action performed by the loading machine 2 on the on-site transport machine 1" is an action performed when loading resources onto the on-site transport machine 1, and is one of the following: • Movement to on-site transport machine 1 • A predetermined action of a predetermined part of the loading machine 2

[0084] First, let's explain "movement to the on-site transport machine 1." For example, if the loading machine 2 moves between the on-site transport machine 1 and the location where the mined resources are located, loading the resources onto the on-site transport machine 1, the amount of resources loaded onto the on-site transport machine 1 can be estimated based on the number of times the loading machine 2 makes the above movements.

[0085] The image analysis unit 12 detects when the loading machine 2 has moved to the on-site transport machine 1 by tracking the position of the loading machine 2 detected within the image. The image analysis unit 12 can track the loading machine 2 within the image using widely known tracking techniques. Then, in response to the detection that the loading machine 2 has moved back to the on-site transport machine 1 after leaving it, the image analysis unit 12 can increment the number of predetermined first actions (number of movements) by the loading machine 2 by 1.

[0086] There are various criteria for determining whether "the loading machine 1 has moved away from the on-site transport machine 1" or "the loading machine 2 has moved to the on-site transport machine 1." For example, the image analysis unit 12 may determine that the loading machine 2 has moved away from the on-site transport machine 1 when the distance between the on-site transport machine 1 and the loading machine 2 exceeds a threshold. Conversely, the image analysis unit 12 may determine that the loading machine 2 has moved to the on-site transport machine 1 when the distance between the on-site transport machine 1 and the loading machine 2 falls below a threshold.

[0087] Next, we will explain the "predetermined actions of a predetermined part of the loading machine 2." The "predetermined actions of a predetermined part of the loading machine 2" are the actions of a predetermined part of the loading machine 2 when the loading machine 2 performs the operation of loading resources onto the on-site transport machine 1.

[0088] Here, we will explain an example of the actions taken when a hydraulic excavator, which is an example of a loading machine 2, loads resources onto the loading platform of the on-site transport machine 1. First, the loading machine 2 rotates its arm to position the bucket at the location of the excavated resources, and then performs the action of collecting the resources into the bucket. At this time, the bucket is positioned above the resources and not above the loading platform of the on-site transport machine 1. After collecting the resources into the bucket, the loading machine 2 rotates its arm to position the bucket above the loading platform of the on-site transport machine 1. Next, in this position, the loading machine 2 directs the opening of the bucket vertically downward and performs the action of dropping the resources in the bucket onto the loading platform of the on-site transport machine 1. At this time, the bucket is positioned above the loading platform of the on-site transport machine 1.

[0089] Thus, in one example, when the loading machine 2 performs the action of loading resources onto the platform of the on-site transport machine 1, it performs the action of orienting the opening of the bucket vertically downwards and releasing (dropping) the contents (resources) from the bucket. Also in one example, when the loading machine 2 performs the action of loading resources onto the platform of the on-site transport machine 1, it performs the action of changing from a state in which "the bucket is not positioned above the platform of the on-site transport machine 1" to a state in which "the bucket is positioned above the platform of the on-site transport machine 1". Therefore, based on the predetermined actions of predetermined parts of the loading machine 2 as described above, the amount of resources loaded onto the on-site transport machine 1 can be estimated.

[0090] For example, a predetermined action of a predetermined part of the loading machine 2 may be an action that directs the opening of the hydraulic excavator's bucket vertically downward and removes (drops) the material (resource) inside the bucket. The image analysis unit 12 can detect the bucket and arm in the image and, based on the state (posture, etc.) of the bucket and arm, can detect that the action has been performed.

[0091] In addition, a predetermined action of a predetermined part of the loading machine 2 may be an action that rotates the arm of the hydraulic excavator to change the state from "the bucket is not positioned above the loading platform of the on-site transport machine 1" to "the bucket is positioned above the loading platform of the on-site transport machine 1".

[0092] The specific actions of the designated parts of the loading machine 2 exemplified here are merely examples; other actions performed when loading resources onto the loading platform of the on-site transport machine 1 can also be adopted.

[0093] In addition, multiple on-site transport machines 1 may be used within resource site A. In this case, the image analysis unit 12 can count the number of predetermined first actions performed by the loading machine 2 for each on-site transport machine 1.

[0094] Furthermore, there may be cases where multiple loading machines 2 are used within resource site A. In this case, the image analysis unit 12 may not identify the multiple loading machines 2 from each other, but may count the number of predetermined first actions performed by the multiple loading machines 2 together. In this case, the image analysis unit 12 counts how many times the predetermined first action was performed on each site transport machine 1 by the multiple loading machines 2.

[0095] In addition, the image analysis unit 12 may identify multiple loading machines 2 by type and count the number of predetermined first actions performed on each in-site transport machine 1 for each type of loading machine 2. The image analysis unit 12 can identify and detect multiple loading machines 2 by type from within the image using a classifier generated by machine learning.

[0096] The other configurations of the image analysis unit 12 can be the same as those in the first and second embodiments.

[0097] The estimation unit 13 estimates the amount of resources loaded into each on-site transport machine 1 based on the number of first actions performed by the loading machine 2.

[0098] For example, the estimation unit 13 estimates the amount of resources loaded onto each on-site transport machine 1 as the product of the number of predetermined first actions performed by the loading machine 2 on each on-site transport machine 1 and the amount of resources loaded by the loading machine 2 in one first action.

[0099] "The number of predetermined first actions performed by the loading machine 2 on each on-site transport machine 1" is the number of predetermined first actions performed by the loading machine 2 on the on-site transport machine 1 from the storage area 4 until it moves back towards the storage area 4.

[0100] The amount of resources that the loading machine 2 loads in one first action is predetermined and stored in the information processing device 10.

[0101] Furthermore, the amount of resources loaded in a single first action may differ depending on the type of loading machine 2. Also, multiple types of loading machines 2 may be used at resource site A. In this case, the image analysis unit 12 and the estimation unit 13 can perform the following processing.

[0102] As described above, the image analysis unit 12 identifies multiple loading machines 2 by type and counts the number of predetermined first actions performed on each in-site transport machine 1 for each type of loading machine 2.

[0103] The estimation unit 13 calculates the amount of resources loaded onto each on-site transport machine 1 by the various loading machines 2 by multiplying the number of predetermined first actions performed on each on-site transport machine 1 by the number of first actions performed on each on-site transport machine 1. The estimation unit 13 then calculates the total amount of resources loaded onto each on-site transport machine 1 by the various loading machines 2, and uses this calculated total as the amount of resources loaded onto each on-site transport machine 1.

[0104] The estimation unit 13 calculates the amount of resources to be utilized based on the amount of resources loaded into each on-site transport machine 1 calculated as described above.

[0105] For example, the estimation unit 13 may calculate the amount of resources used within a predetermined period by accumulating the amount of resources loaded onto each on-site transport machine 1 that has moved to the storage area 4 within that predetermined period. The predetermined period can be, for example, 1 to a few minutes, 1 to a few hours, 1 to a few days, 1 to a few weeks, 1 to a few months, 1 to a few years, etc.

[0106] The other configurations of the estimation unit 13 can be the same as those in the first and second embodiments.

[0107] Next, an example of the processing flow of the information processing device 10 will be explained using the flowchart in Figure 8. Note that the purpose here is to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0108] First, the information processing device 10 acquires an image of the resource site A where resource-related work is performed (S30). Next, the information processing device 10 detects the loading machine 2 in the image and detects the number of predetermined first actions performed by the loading machine 2 on the on-site transport machine 1 (S31). The information processing device 10 detects the number of predetermined first actions performed by the loading machine 2 on the on-site transport machine 1 from the storage area 4 until the on-site transport machine 1 moves back towards the storage area 4.

[0109] Furthermore, the information processing device 10 detects the on-site transport machine 1 in the image and detects the movement of the on-site transport machine 1 to a predetermined location (storage area 4) within resource site A (S32).

[0110] Then, the information processing device 10 estimates the amount of resources loaded onto the on-site transport machine 1, which has moved to a predetermined location (storage area 4), based on the number of first actions detected in S31, and estimates the amount of resources to be utilized based on the estimation result (S33).

[0111] For example, the information processing device 10 estimates the amount of resources loaded onto the on-site transport device 1 as the product of the number of predetermined first actions performed by the loading device 2 on the on-site transport device 1 and the amount of resources loaded by the loading device 2 in one first action.

[0112] Alternatively, the information processing device 10 may identify multiple loading machines 2 by type and count the number of predetermined first actions performed on the on-site transport machine 1 for each type of loading machine 2. In this example, the information processing device 10 calculates the amount of resources loaded onto the on-site transport machine 1 by each type of loading machine 2 by multiplying the number of predetermined first actions performed on the on-site transport machine 1 by multiplying the amount of resources loaded in one first action. The estimation unit 13 then calculates the total amount of resources loaded by each type of loading machine 2 as the amount of resources loaded onto the on-site transport machine 1.

[0113] The information processing device 10 can then calculate the amount of resources utilized within a predetermined period by accumulating the amount of resources loaded onto each of the on-site transport machines 1 that have moved to the storage area 4 within a predetermined period.

[0114] Other configurations of the information processing device 10 in the third embodiment can be the same as those in the first and second embodiments.

[0115] <Effects and Effects> According to the information processing device 10 of the third embodiment, the same effects and advantages as those of the information processing device 10 of the first and second embodiments can be achieved.

[0116] Furthermore, the information processing device 10 of the third embodiment can estimate the amount of resources loaded onto the on-site transport vehicle 1 based on a predetermined first action of the loading machine 2 that loads resources onto the on-site transport vehicle 1, and estimate the amount of resources utilized based on the estimation result. With such an information processing device 10, the amount of resources utilized can be estimated with high accuracy even in situations where the amount of resources loaded onto the on-site transport vehicle 1 may vary greatly.

[0117] <<Fourth Embodiment>> <Overview> The information processing device 10 in the fourth embodiment detects the mining machines 3 used in resource extraction and estimates the amount of resources utilized based on the number of mining machines 3 and their operating hours. This will be explained in detail below.

[0118] <Hardware Configuration> This is the same as the first to third embodiments.

[0119] <Functional Configuration> Figure 1 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an acquisition unit 11, an image analysis unit 12, and an estimation unit 13.

[0120] The configuration of the acquisition unit 11 is the same as in the first to third embodiments.

[0121] The image analysis unit 12 detects the mining machine 3 used for resource extraction within the image acquired by the acquisition unit 11. Note that multiple mining machines 3 may be used within resource site A. In this case, the image analysis unit 12 may identify and detect the multiple mining machines 3 by type. The image analysis unit 12 can detect the mining machine 3 within the image using a classifier generated by machine learning.

[0122] The other configurations of the image analysis unit 12 can be the same as those in the first and second embodiments.

[0123] The estimation unit 13 estimates the amount of resources utilized based on the number of mining machines 3 and their operating time detected within the image. An estimation model is pre-generated and stored in the information processing device 10, with the number of mining machines 3 and their operating time as inputs and the amount of resources utilized as an output. The estimation unit 13 calculates the amount of resources utilized by inputting the number of mining machines 3 and their operating time detected within the image into this estimation model. The estimation model may consist of calculation formulas, a table showing the relationship between the number of mining machines 3 and their operating time and the amount of resources utilized, or other configurations. The estimation model is configured to calculate a larger amount of resources utilized as the number of mining machines 3 increases. Furthermore, the estimation model is configured to calculate a larger amount of resources utilized as the operating time increases.

[0124] The estimation unit 13 can calculate the amount of resources utilized within a predetermined time period based on the number of mining machines 3 detected within that time period and the operating time within that time period. The predetermined time period can be, for example, 1 to a few minutes, 1 to a few hours, 1 to a few days, 1 to a few weeks, 1 to a few months, 1 to a few years, etc.

[0125] The "operating time within a predetermined period" can be determined by analyzing the images acquired by the acquisition unit 11. For example, the image analysis unit 12 tracks the mining machine 3 detected in the image and monitors whether or not there is movement. The image analysis unit 12 then determines that a mining machine 3 that is not moving, or a mining machine 3 that has not moved for a predetermined period of time or longer, is not operating. The image analysis unit 12 can also determine that a mining machine 3 that is moving, or a mining machine 3 that has moved within the most recent predetermined period, is operating.

[0126] The image analysis unit 12 then determines that the group of mining machines 3 is operational when at least one of the mining machines 3 detected in the image is in operation, and calculates the operating time within a predetermined period. In this example, the operating time within a predetermined period is the time when at least one of the mining machines 3 detected in the image is in operation.

[0127] The other configurations of the estimation unit 13 can be the same as those in the first and second embodiments.

[0128] Next, an example of the processing flow of the information processing device 10 will be explained using the flowchart in Figure 9. The purpose here is simply to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0129] First, the information processing device 10 acquires images of the resource site where resource-related work is being carried out (S40). Next, the information processing device 10 detects the mining machine 3 in the images (S41). Then, the information processing device 10 estimates the amount of resources utilized based on the number of mining machines detected and their operating time (S42).

[0130] Next, a modified version of this embodiment will be described. The information processing device 10 of this embodiment can adopt one or more of the following first to fourth modified versions.

[0131] "The first variation" In the first modified example, multiple mining machines 3 are used within resource site A, and the above estimation models are prepared corresponding to each type of mining machine 3. The estimation unit 13 inputs the number of each type of mining machine 3 detected from the image and the operating time identified by the method of this embodiment described above into the estimation model corresponding to each type of mining machine 3 to estimate the amount of resources utilized. The estimation unit 13 then calculates the total amount of resources utilized, which is the sum of the estimated amounts of resources utilized for each type of mining machine 3, as the total amount of resources utilized.

[0132] "The second variation" In the second modified example, the image analysis unit 12 detects the operating mining machine 3 within the image. The estimation unit 13 then inputs the "number of operating mining machines 3" detected from the image and the operating time identified using the method of this embodiment described above into the estimation model to estimate the amount of resources utilized.

[0133] "The third variation" In the third modified example, the image analysis unit 12 detects an operational mining machine 3 within the image. The image analysis unit 12 then calculates the operating time for each detected mining machine 3. The image analysis unit 12 can use the method of this embodiment described above to determine whether each mining machine 3 is operational and calculate the operating time for each mining machine 3 based on the determination result.

[0134] The estimation unit 13 inputs the "number of mining machines 3 in operation" and the operating time of each mining machine 3 into the estimation model to estimate the amount of resources utilized. The estimation unit 13 then calculates the total amount of resources utilized by each of the mining machines 3 in operation as the total amount of resources utilized. Note that the "number of mining machines 3" input into the estimation model when calculating the amount of resources utilized by each mining machine 3 is 1.

[0135] "The fourth variation" In the fourth modification, the image analysis unit 12 identifies areas within resource site A where mining operations are being carried out and areas where mining operations are not being carried out. The estimation unit 13 then inputs the number of mining machines 3 detected in the areas where mining operations are being carried out, and the operating time identified using the method of this embodiment described above, into the estimation model to estimate the amount of resources being utilized.

[0136] Here, we will explain a method for distinguishing between areas where mining operations are taking place and areas where they are not.

[0137] The image analysis unit 12 can identify areas where mining is being performed and areas where mining is not being performed based on the operating status of the mining machine 3 detected in the image. An example of the process is described below, but it is not limited to this example.

[0138] First, the image analysis unit 12 divides resource site A into multiple sections. The division method can vary. The shape and size of each section can also vary and are not restricted. The image analysis unit 12 determines whether each section meets the conditions for determining that mining work is being carried out. The image analysis unit 12 determines that a set of sections that meet the conditions is an area where mining work is being carried out. The image analysis unit 12 then determines that a set of sections that do not meet the conditions is an area where mining work is not being carried out.

[0139] In addition, the image analysis unit 12 may determine whether each section meets the conditions for determining that no mining work is being performed. In this case, the image analysis unit 12 determines that a set of sections that meet the conditions is an area where no mining work is being performed. The image analysis unit 12 then determines that a set of sections that do not meet the conditions is an area where mining work is being performed.

[0140] The "conditions for determining that mining operations are being conducted" may include any of the following: • At least one mining machine 3 is in operation. • More than the prescribed number of mining machines 3 are in operation. • Surrounded by an area where mining operations are being conducted (e.g., surrounded on all four or eight sides)

[0141] The "conditions for determining that no mining operations are being conducted" may include any of the following: • There are 3 inactive mining machines. • There are more than the specified number of non-operational mining machines. • None of the mining machines (3) located within the area are operational. • There is a mining machine 3 that has not been operating for more than the specified time. • There are more than the specified number of mining machines 3 that have not been operating for the specified time. • None of the mining machines 3 located within the area have been operating for the specified time. • Surrounded by areas where mining operations are not being conducted (e.g., surrounded on all four or eight sides)

[0142] The image analysis unit 12 can determine whether or not each mining machine 3 is operational by employing the method of this embodiment described above.

[0143] Here, using the flowchart in Figure 10, we will explain an example of the processing flow of the information processing device 10 in the fourth modified example. The purpose here is to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0144] First, the information processing device 10 acquires an image of the resource site where resource-related work is being carried out (S50). Next, the information processing device 10 detects the mining machine 3 in the image (S51).

[0145] Next, the information processing device 10 identifies areas within resource site A where mining operations are being carried out and areas where mining operations are not being carried out, based on the operating status of the detected mining machines 3 (S52). Then, the information processing device 10 estimates the amount of resources to be utilized based on the number of mining machines and their operating hours detected in the areas where mining operations are being carried out (S53).

[0146] Other configurations of the information processing device 10 of the fourth embodiment can be the same as those of the first to third embodiments.

[0147] <Effects and Effects> According to the information processing device 10 of the fourth embodiment, the same effects and advantages as those of the information processing device 10 of the first to third embodiments can be achieved.

[0148] Furthermore, the information processing device 10 of the fourth embodiment can calculate the amount of resources utilized based on the number of mining machines 3, such as large hydraulic excavators, large wheel loaders, and large bulldozers, that are used in resource extraction, and their operating hours. With such an information processing device 10, the amount of resources utilized can be estimated with high accuracy.

[0149] <<Fifth Embodiment>> <Overview> The information processing device 10 of the fifth embodiment has a function to calculate the accuracy of the resource utilization amount estimated by the estimation unit 13. This will be described in detail below.

[0150] <Hardware Configuration> This is the same as the first to fourth embodiments.

[0151] <Functional Configuration> Figure 11 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 includes an acquisition unit 11, an image analysis unit 12, an estimation unit 13, and an accuracy calculation unit 14.

[0152] The configuration of the acquisition unit 11 is the same as in the first to fourth embodiments. The configurations of the image analysis unit 12 and the estimation unit 13 can be the same as in the first to fourth embodiments.

[0153] The accuracy calculation unit 14 calculates the accuracy of the resource utilization amount estimated by the estimation unit 13 by comparing the resource utilization amount estimated by the estimation unit 13 with a reference value.

[0154] The "reference value" is the correct amount of resource utilization. While it is difficult to determine this correct amount in real time, it is possible to determine it retrospectively. Therefore, the accuracy calculation unit 14 calculates the accuracy of the resource utilization amount estimated by the estimation unit 13 by comparing the resource utilization amount estimated by the estimation unit 13 with the correct amount of resource utilization amount (reference value) determined retrospectively.

[0155] Various techniques and criteria can be used to calculate accuracy. For example, the accuracy calculation unit 14 may calculate the difference between the amount of resource utilization estimated by the estimation unit 13 and a reference value as a value indicating accuracy. In this case, a smaller difference means higher accuracy.

[0156] In addition, the accuracy calculation unit 14 may calculate the rate of change when the amount of resource utilization and the reference value estimated by the estimation unit 13 change from one to the other, as a value indicating accuracy. In this case, a smaller rate of change means higher accuracy.

[0157] Next, we will explain the method for determining the reference value. In one example, the user determines the correct amount of resource utilization using any method and inputs it into the information processing device 10 as a reference value. The accuracy calculation unit 14 can use the reference value input by the user to calculate the accuracy described above.

[0158] In other examples, the accuracy calculation unit 14 determines a reference value based on the results of image analysis performed by the image analysis unit 12. The processing in this example will be described in detail below.

[0159] The image analysis unit 12 detects the off-site transporter 5 within the image acquired by the acquisition unit 11. The image analysis unit 12 then detects the number of times the off-site transporter 5 moves from inside resource site A to the outside. As described in the second embodiment, the off-site transporter 5 is used to transport resources from resource site A to the outside.

[0160] In addition, there are cases where multiple off-site transport vehicles 5 are used at resource site A. In this case, the image analysis unit 12 may identify and detect the multiple off-site transport vehicles 5 by type. The image analysis unit 12 can detect the off-site transport vehicles 5 from the image using a classifier generated by machine learning.

[0161] The accuracy calculation unit 14 determines a reference value based on the number of movements detected by the off-site transporter 5. For example, the accuracy calculation unit 14 can calculate a reference value as the product of the number of movements detected by the off-site transporter 5 within a predetermined time and the amount of resources transported by the off-site transporter 5 in one movement. Various values ​​can be used for the predetermined time. For example, the predetermined time can be 1 to a few minutes, 1 to a few hours, 1 to a few days, 1 to a few weeks, 1 to a few months, 1 to a few years, etc. The amount of resources transported by the off-site transporter 5 in one movement is predetermined and stored in the information processing device 10.

[0162] As mentioned above, the image analysis unit 12 may identify and detect multiple off-site transport vehicles 5 by type. In this case, the accuracy calculation unit 14 can calculate the product of the number of movements detected by each off-site transport vehicle 5 within a predetermined time and the amount of resources transported by each off-site transport vehicle 5 in one movement, and use the sum of these as a reference value. The amount of resources transported by each off-site transport vehicle 5 in one movement is predetermined and stored in the information processing device 10.

[0163] The accuracy calculation unit 14 calculates a reference value for a predetermined observation period (e.g., from June 3, 2024 to June 7, 2024) based on the number of movements detected by the off-site transport machine 5 during that period. Then, the accuracy calculation unit 14 can calculate the accuracy described above based on the amount of resource utilization during that observation period estimated by the estimation unit 13 and the reference value calculated above.

[0164] Here, we will explain why the value determined based on the actions of the off-site transport device 5 is suitable as a reference value.

[0165] (Reason 1) Not all resources mined and moved to storage area 4 during a given period are necessarily transported from resource site A to the outside within that period. However, to avoid excessive increases or decreases in inventory in storage area 4, the amount of resources mined is usually adjusted to match the amount transported from resource site A to the outside. In other words, in order to keep the inventory level in storage area 4 constant, the amount of resources mined during a given period may be adjusted to be equal to the amount of resources transported from resource site A to the outside within that period. In such cases, the amount of resources moved to storage area 4 during a given period (amount of resources mined) becomes equal to the amount of resources transported from resource site A to the outside within that period. Therefore, the amount of resources transported from resource site A to the outside, calculated based on the actions of off-site transport machine 5, is suitable for comparison with the amount of resources utilized, calculated based on the actions of on-site transport machine 1, loader 2, mining machine 3, etc.

[0166] (Reason 2) The "amount of resources to be transported from resource site A to the outside, calculated based on the actions of the off-site transport machine 5" and the "amount of resources utilized, calculated based on the actions of the on-site transport machine 1, loading machine 2, mining machine 3, etc." are both identified by analyzing the images acquired by the acquisition unit 11. However, for the following reasons, the "amount of resources to be transported from resource site A to the outside, calculated based on the actions of the off-site transport machine 5" is more accurate (higher reliability) and therefore more suitable as a reference value.

[0167] As described in the first to fourth embodiments, the estimation unit 13 estimates the amount of resources to be utilized based on detection results in images of the on-site transport machine 1, loading machine 2, mining machine 3, etc., present within resource site A. However, because there are numerous work machines and various other equipment and tools within resource site A, and the location where work machines operate may change, the detection of work machine actions as described above may be missed. As a result, the accuracy of the amount of resources utilized estimated by the estimation unit 13 may be reduced.

[0168] In contrast, detection of the off-site transporter 5 can be achieved by monitoring predetermined locations, such as the entrance and exit of resource site A. Furthermore, the entrance and exit of resource site A are usually less chaotic than the actual mining site, making it less likely that the off-site transporter 5's actions (movements) will be missed. As a result, a highly reliable reference value can be calculated based on the actions (movements) of the off-site transporter 5.

[0169] The accuracy calculation unit 14 can perform error processing if the calculated accuracy is below a threshold.

[0170] For example, the accuracy calculation unit 14 can notify the user of this fact. The accuracy calculation unit 14 may also output information indicating that the accuracy is below a threshold via an output device such as a display or speaker. In addition, the accuracy calculation unit 14 may turn on or flash a warning lamp, output a warning sound via a speaker, or display a warning image on a display.

[0171] In addition, the accuracy calculation unit 14 may, as an error handling measure, have the image analysis unit 12 and the estimation unit 13 re-perform the image analysis and the estimation of resource utilization. Although the results obtained from the re-performance may lack immediacy, there is a possibility that more reliable results can be obtained through the re-performance.

[0172] Next, an example of the processing flow of the information processing device 10 will be explained using the flowchart in Figure 12. Note that the purpose here is to explain the processing flow. Details of each process have been described above, so explanations will be omitted here as appropriate.

[0173] First, the information processing device 10 acquires an image of the resource site where resource-related work is being performed (S60). Next, the information processing device 10 detects the work machine in the image and detects the action performed by the work machine (S61). Then, the information processing device 10 estimates the amount of resource utilized based on the action of the work machine (S62). The processing in S60 to S62 can be the same as in the first to fourth embodiments.

[0174] Next, the information processing device 10 compares the amount of resource utilization estimated in S62 with a reference value and calculates the accuracy of the amount of resource utilization estimated in S62 (S63). If the calculated accuracy is below a threshold (Yes in S64), the information processing device 10 performs error processing (S65).

[0175] Other configurations of the information processing device 10 of the fifth embodiment can be the same as those of the first to fourth embodiments.

[0176] <Effects and Effects> According to the information processing device 10 of the fifth embodiment, the same effects and advantages as those of the information processing device 10 of the first to fourth embodiments can be achieved.

[0177] Furthermore, the information processing device 10 of the fifth embodiment can calculate the accuracy of the estimated resource utilization amount based on a comparison result between the estimated resource utilization amount and the correct answer (reference value) obtained afterward. The information processing device 10 can then perform error processing according to the accuracy. With such an information processing device 10, it is possible to estimate the resource utilization amount with high reliability.

[0178] <<Sixth Embodiment>> <Overview> The information processing device 10 of the sixth embodiment has a function to calculate the relationship between the estimated amount of resources buried at resource site A, estimated by a predetermined technique, and actual values ​​(actual amount extracted, recoverable reserves) based on the amount of resources utilized, estimated by the estimation unit 13. This will be explained in detail below.

[0179] <Hardware Configuration> This is the same as the first to fifth embodiments.

[0180] <Functional Configuration> Figure 13 is an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 includes an acquisition unit 11, an image analysis unit 12, an estimation unit 13, and a calculation formula determination unit 15. The information processing device 10 may also include an accuracy calculation unit 14.

[0181] The configuration of the acquisition unit 11 is the same as in the first to fifth embodiments. The configurations of the image analysis unit 12, the estimation unit 13, and the accuracy calculation unit 14 can be the same as in the first to fifth embodiments.

[0182] The calculation formula determination unit 15 determines a calculation formula for calculating the "actual value of resources extracted at resource site A," which is calculated based on the amount of resources utilized estimated by the acquisition unit 11, using the "estimated amount of resource reserves at resource site A."

[0183] The "estimated reserves" is an estimated value of the resource reserves at resource site A, estimated using a predetermined technique. The means for estimating the estimated resource reserves at resource site A are not particularly limited, and widely known techniques can be employed. For example, the estimated resource reserves at resource site A may be estimated based on the results of analyzing resource site A using satellite remote sensing technology or the results of geological surveys at resource site A. For example, the estimated reserves may be estimated using methods such as the volumetric method, the decay curve method, the mass balance method, or oil reservoir simulation.

[0184] The user recognizes the estimated amount of resources buried at resource site A, which has been estimated using a predetermined technology, and inputs the recognized estimated amount of resources into the information processing device 10.

[0185] The "actual value" is a value calculated based on the amount of resource utilization estimated by the estimation unit 13. The actual value represents the amount actually extracted (recoverable reserves) at resource site A. The estimation unit 13 calculates the amount of resource utilized as the actual value during the period from the start of mining to the end of mining at resource site A.

[0186] The calculation formula determination unit 15 may determine a calculation formula to calculate the "actual value of resources actually extracted (recoverable reserves)" from the "estimated amount of resources estimated by a predetermined technology" based on the estimated amount of resources and actual values ​​at one resource site A. In this case, for example, the calculation formula determination unit 15 may determine a correction coefficient to obtain the actual value at one resource site A from the estimated amount of resources at that one resource site A, and then determine a calculation formula to obtain the actual value (recoverable reserves) as the product of the determined correction coefficient and the estimated amount of resources.

[0187] In addition, the calculation formula determination unit 15 may determine a calculation formula for calculating the "actual value of resources actually extracted (recoverable reserves)" from the "estimated resource reserves estimated by a predetermined technology" using methods such as regression analysis, based on the estimated reserves and actual values ​​at multiple resource sites A.

[0188] Other configurations of the information processing device 10 of the sixth embodiment can be the same as those of the first to fifth embodiments.

[0189] <Effects and Effects> According to the information processing device 10 of the sixth embodiment, the same effects and advantages as those of the information processing device 10 of the first to fifth embodiments can be achieved.

[0190] Furthermore, the information processing device 10 of the sixth embodiment can determine a calculation formula for calculating the amount actually extracted at resource site A (recoverable reserves) from the estimated amount of resources buried at resource site A. Based on this calculation formula and the estimated amount of resources buried, it becomes possible to estimate recoverable reserves with high accuracy.

[0191] <<Variation>> The following describes modifications applicable to the information processing device 10 of the first to sixth embodiments. The information processing device 10 of the first to sixth embodiments can adopt one or more of the following modifications from fifth to seventh. The same effects and advantages as those of the first to sixth embodiments are achieved in these modifications as well.

[0192] "The fifth variation" The information processing device 10 can determine a calculation formula for calculating the correct amount of resource utilization from the estimated amount of resource utilization by the estimation unit 13, based on the amount of resource utilization in each of the multiple periods estimated by the estimation unit 13 and the correct amount of resource utilization in each period.

[0193] The correct amount of resource utilization may be determined by the user through any means and input into the information processing device 10. Alternatively, the information processing device 10 may calculate the correct amount of resource utilization using the method (reference value determination method) described in the fifth embodiment. In this case, the determined reference value is treated as the correct answer.

[0194] "The sixth variation" As explained with reference to Figure 4 in the second embodiment, mining operations may be carried out on a portion of resource site A. In this case, mining operations are usually carried out sequentially starting from the portion a closest to area C where the mining headquarters is located. Therefore, the distance traveled by the site transport vehicle 1 increases each time the portion a on which mining operations are carried out changes. Based on this, the information processing device 10 identifies areas within resource site A where mining operations have been completed and areas where they have not yet been completed.

[0195] Specifically, the information processing device 10 analyzes the images acquired by the acquisition unit 11 and calculates the travel distance D of the on-site transport device 1. The information processing device 10 then determines that areas where the distance from area C is less than the travel distance D are areas where mining work has been completed. Conversely, the information processing device 10 determines that areas where the distance from area C is greater than or equal to the travel distance D are areas where mining work has not been completed.

[0196] The information processing device 10 may calculate and output, for example, the percentage of the area where mining work has been completed at resource site A, or the percentage of the area where mining work has not been completed at resource site A.

[0197] "The seventh variation" The information processing device 10 may use satellite remote sensing technology to investigate the amount of resources at resource site A and output the information. The information processing device 10 may also output information such as the country, jurisdiction, and type of resources at resource site A.

[0198] <<Usage scenarios>> Next, we will describe the usage scenarios of the information processing device 10 according to the first to sixth embodiments.

[0199] For example, the information processing device 10 is used to provide users with information to help them understand and predict economic trends. Specifically, the information processing device 10 can present users with estimated resource utilization figures. For instance, the information processing device 10 can present information as shown in Figures 5 and 6. The information processing device 10 may present this information through a predetermined application or web page, or it may present the information by other means.

[0200] In another example, the information processing device 10 can propose an investment portfolio based on the estimated amount of resource utilization. For example, the information processing device 10 stores information that links each type of resource to sectors that are highly dependent on that resource (e.g., petroleum sector, textile / chemical sector).

[0201] The information processing device 10 can provide the user with advice to increase the proportion of sectors (proportion in the investment portfolio) associated with resource types whose resource utilization satisfies a predetermined first condition. When the information processing device 10 determines the investment portfolio, it can decide to increase the proportion of sectors (proportion in the investment portfolio) associated with resource types whose resource utilization satisfies a predetermined first condition.

[0202] The first condition is, but is not limited to, "an upward trend in resource utilization" and "a monotonically increasing amount of resource utilization over the past few months."

[0203] In addition, the information processing device 10 can provide the user with advice to reduce the proportion of sectors (proportion in the investment portfolio) associated with resource types whose resource utilization satisfies a predetermined second condition. When the information processing device 10 determines the investment portfolio, it can decide to reduce the proportion of sectors (proportion in the investment portfolio) associated with resource types whose resource utilization satisfies a predetermined second condition.

[0204] The second condition includes, but is not limited to, "a declining trend in resource utilization" and "a monotonically decreasing amount of resource utilization over the past few months."

[0205] In another example, the information processing device 10 can issue a price increase warning for a product based on the estimated amount of resource utilization. For example, information linking each type of resource to the products produced using that resource is stored in the information processing device 10 beforehand.

[0206] The information processing device 10 can notify the user that products linked to resource types whose resource utilization meets a predetermined third condition are products that may increase in price.

[0207] The third condition includes, but is not limited to, "a declining trend in resource utilization" and "a monotonically decreasing amount of resource utilization over the past few months."

[0208] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0209] Furthermore, the flowcharts used in the above description show multiple steps (processes) in sequence. However, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content.

[0210] Some or all of the above embodiments may also be described as follows, but are not limited to the following. 1. An acquisition means for acquiring images of resource sites where resource-related work is carried out, Image analysis means for detecting work machines that perform resource-related work within the image and for detecting actions performed by the work machines, An estimation means for estimating the amount of resources used based on the actions of the aforementioned work machine, An information processing device having 2. The image analysis means is: The system detects on-site transport machines that transport resources within the resource site, and also detects the movement of the on-site transport machines to a predetermined location within the resource site. The estimation means is, An information processing device according to claim 1, which estimates the amount of resource utilization based on the number of movements of the on-site transport device. 3. The image analysis means is: The system detects a loading machine that loads resources onto an on-site transport machine that transports resources within the resource site, and also detects a predetermined first action performed by the loading machine on the on-site transport machine. The estimation means is, An information processing device according to 1 or 2, which estimates the amount of resources loaded onto the on-site transport machine based on the number of times the first action has been performed, and estimates the amount of resources to be utilized based on the estimation result. 4. The image analysis means is: It detects mining machines used in resource extraction, The estimation means is, An information processing device according to any one of 1 to 3, which estimates the amount of resources utilized based on the number of mining machines and their operating hours. 5. The image analysis means is: Identify areas where mining operations are being conducted and areas where mining operations are not being conducted. The estimation means is, The information processing device according to 4, which estimates the amount of resources utilized based on the number of mining machines and their operating hours detected within the area where mining operations are being carried out. 6. An information processing device according to any one of 1 to 5, having an accuracy calculation means for calculating the accuracy of the estimated resource utilization by comparing the estimated resource utilization with a reference value. 7. The image analysis means is: The system detects off-site transport vehicles that transport resources from the resource site to the outside, and also detects the number of times the off-site transport vehicles move from inside the resource site to the outside. The accuracy calculation means is The information processing device according to 6, which determines the reference value based on the number of movements of the off-site transport machine. 8. The aforementioned resource site is a resource extraction site, An information processing device according to any one of 1 to 7, having a calculation formula determination means for determining a calculation formula for calculating the actual value of resources extracted at the resource site, which is calculated based on the estimated amount of resource reserves at the resource site and the estimated amount of resource utilization. 9. One or more computers, We obtained images taken at resource sites where resource-related work was being carried out. The system detects a work machine that performs resource-related work within the image, and also detects the actions performed by the work machine. An information processing method for estimating the amount of resources used based on the actions of the aforementioned work machine. 10. Computers, A means for acquiring images of resource sites where resource-related work is being carried out. Image analysis means for detecting a work machine that performs resource-related work in the image and for detecting the actions performed by the work machine. Estimation means for estimating the amount of resource utilization based on the actions of the aforementioned work machine, A program that makes it function as such.

[0211] Some or all of the appendices 2 through 8, which are dependent on the information processing device described in appendice 1 above, may also be dependent on the information processing method in appendice 9 and the program in appendice 10 in the same dependent relationship as between appendice 1 and appendices 2 through 8. Furthermore, within the scope that does not depart from each of the embodiments described above, some or all of the configurations described as appendices can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of symbols]

[0212] 1 On-site transport machine 2 Loading machine 3 mining machine 4. Storage Area 5 Off-site transport equipment 10 Information Processing Devices 11 Acquisition Department 12 Image Analysis Unit 13 Estimation part 14. Accuracy Calculation Unit 15. Formula Determination Section 1A Processor 2A Memory 3A input / output I / F 4A Peripheral Circuits 5A bus

Claims

1. A means for acquiring images of resource sites where resource-related work is carried out, Image analysis means for detecting work machines that perform resource-related work within the image and for detecting actions performed by the work machines, An estimation means for estimating the amount of resources used based on the actions of the aforementioned work machine, An information processing device having

2. The aforementioned image analysis means is The system detects on-site transport machines that transport resources within the resource site, and also detects the movement of the on-site transport machines to a predetermined location within the resource site. The estimation means is, The information processing device according to claim 1, which estimates the amount of resource utilization based on the number of movements of the on-site transport device.

3. The aforementioned image analysis means is The system detects a loading machine that loads resources onto an on-site transport machine that transports resources within the resource site, and also detects a predetermined first action performed by the loading machine on the on-site transport machine. The estimation means is, The information processing device according to claim 1, which estimates the amount of resources loaded onto the on-site transport machine based on the number of times the first action has been performed, and estimates the amount of resources to be utilized based on the estimation result.

4. The aforementioned image analysis means is It detects mining machines used in resource extraction, The estimation means is, The information processing device according to claim 1, which estimates the amount of resources utilized based on the number of mining machines and their operating hours.

5. The aforementioned image analysis means is Identify areas where mining operations are being conducted and areas where mining operations are not being conducted. The estimation means is, The information processing device according to claim 4, which estimates the amount of resource utilization based on the number of mining machines and their operating hours detected within the area where mining operations are being carried out.

6. The information processing apparatus according to claim 1, comprising an accuracy calculation means for calculating the accuracy of the estimated resource utilization by comparing the estimated resource utilization with a reference value.

7. The aforementioned image analysis means is The system detects off-site transport vehicles that transport resources from the resource site to the outside, and also detects the number of times the off-site transport vehicles move from inside the resource site to the outside. The accuracy calculation means is The information processing device according to claim 6, which determines the reference value based on the number of movements of the off-site transport machine.

8. The aforementioned resource site is a resource extraction site, The information processing apparatus according to claim 1, comprising a calculation formula determination means for determining a calculation formula for calculating the actual value of resources extracted at the resource site, which is calculated based on the estimated amount of resource reserves at the resource site and the estimated amount of resource utilization.

9. One or more computers, We obtained images taken at resource sites where resource-related work was being carried out. The system detects a work machine that performs resource-related work within the image, and also detects the actions performed by the work machine. An information processing method for estimating the amount of resources used based on the actions of the aforementioned work machine.

10. Computers, A means for acquiring images of resource sites where resource-related work is being carried out. Image analysis means for detecting a work machine that performs resource-related work in the image and for detecting the actions performed by the work machine. Estimation means for estimating the amount of resource utilization based on the actions of the aforementioned work machine, A program that makes it function as such.

Images