Weight estimation method, weight estimation device, weight estimation system and program

Abstract

Provided is a technology for suitably evaluating the weight of an object placed in a container. A weight estimation method includes: a first acquisition step (S101) for acquiring first imaging data which is learning-use imaging data from a first type of imaging data which is imaging data of an object scooped into a container by an excavator; a learning step (S102) for referencing the first imaging data and the measured weight of the object to train an estimation model that outputs the weight of said object on the basis of the first type of imaging data; a second acquisition step (S103) for acquiring second imaging data which is estimation-use imaging data from the first type of imaging data; and an estimation step (S104) for estimating the weight of the object on the basis of the estimation model and the second imaging data.

Description

[Technical field]

[0001] The present invention relates to a weight estimation method and a weight estimation device. , heavy Quantity Estimation System and Program Regarding. [Background technology]

[0002] 2. Description of the Related Art Conventionally, in construction work, civil engineering work and the like, a method has been developed for evaluating the weight of an object scooped up by a hydraulic excavator or the like while the object is mounted on the hydraulic excavator or the like.

[0003] For example, Patent Document 1 discloses a load measuring device that calculates a plurality of load values ​​and determines an optimal load value based on the output values ​​of an angle sensor that detects the front posture of a hydraulic excavator and a pressure sensor that detects the pressure of a hydraulic cylinder. Patent Document 2 describes estimating the volume of an excavated object in a bucket from an image captured by a camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2012-103029 [Patent Document 2] WO2019 / 117166 publication Summary of the Invention [Problem to be solved by the invention]

[0005] However, the device disclosed in Patent Document 1 requires the installation of a new hydraulic sensor inside the hydraulic system, making it difficult to incorporate into existing hydraulic excavators. In addition, the method disclosed in Patent Document 2 cannot evaluate the weight of the excavated material.

[0006] One aspect of the present invention is to provide a technique that can suitably evaluate the weight of an object placed in a container. [Means for solving the problem]

[0007] A weight estimation method according to one aspect of the present invention includes a first acquisition step of acquiring first imaging data, which is imaging data for learning, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning step of learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object; a second acquisition step of acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; and an estimation step of estimating the weight of the object based on the estimation model and the second imaging data.

[0008] A weight estimation device according to one aspect of the present invention includes a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs a weight of the object based on the first type of imaging data, by referring to the first imaging data and a measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data.

[0009] A weight estimation system according to one aspect of the present invention includes a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs a weight of the object based on the first type of imaging data, by referring to the first imaging data and a measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data. Furthermore, a program according to one aspect of the present invention is a program for causing a computer to function as a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data. Effect of the Invention

[0010] According to one aspect of the present invention, it is possible to provide a technique that can suitably evaluate the weight of an object placed in a container. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a learning device according to a first exemplary embodiment of the present invention. [Diagram 2] 1 is a flowchart showing a flow of a learning method according to the first exemplary embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating a first type of imaging data and a second type of imaging data according to the first exemplary embodiment of the present invention. [Figure 4] FIG. 1 is a block diagram showing a configuration of an estimation device according to a first exemplary embodiment of the present invention. [Diagram 5] 1 is a flowchart showing the flow of an estimation method according to a first exemplary embodiment of the present invention. [Figure 6] FIG. 11 is a block diagram showing a configuration of a weight estimation device according to an exemplary embodiment 2 of the present invention. [Figure 7] 10 is a flowchart showing the flow of a weight estimation method according to a second exemplary embodiment of the present invention. [Figure 8] FIG. 11 is a block diagram of a learning system for learning a model to estimate the weight of soil contained in a bucket according to an exemplary embodiment 3 of the present invention. [Figure 9] 11 is a heat map showing an example of imaging data of soil contained in a backhoe bucket according to an exemplary embodiment 3 of the present invention. [Figure 10] FIG. 11 is a diagram illustrating a theoretical weight estimation formula according to an exemplary embodiment 4 of the present invention. [Figure 11] FIG. 13 illustrates the results of kernel ridge regression of measurements according to an exemplary embodiment 4 of the present invention. [Figure 12] FIG. 11 is a configuration diagram of an estimation system for estimating weight according to an exemplary embodiment 5 of the present invention. [Figure 13] FIG. 13 is a configuration diagram of an information processing system including a learning unit according to a sixth exemplary embodiment of the present invention. [Figure 14] FIG. 13 is a configuration diagram of another example of an information processing system including a switching unit according to the seventh exemplary embodiment of the present invention. [Figure 15] FIG. 13 is a diagram showing two patterns of distribution of maximum values ​​of a kernel function. [Figure 16] FIG. 13 is a configuration diagram of a distributed information processing system according to an eighth exemplary embodiment of the present invention. [Figure 17] This is a diagram showing how each part is configured using software. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Example Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later. FIG. 1 is a block diagram showing the configuration of a learning device 1 according to the first exemplary embodiment. The learning device 1 is a learning device of an estimation model that estimates the weight of an object placed in a container.

[0013] In this exemplary embodiment, the container may be in a state where an object is placed therein and in a state where an object is not placed therein. In this exemplary embodiment, the container may be a bucket of an excavator. In this exemplary embodiment, the object is a granular or amorphous solid or liquid that is placed or contained in the container. As an example, the object is soil, sand, sediment, snow, grain, cement, etc., and objects that can be placed in a storage facility or the like or that can be contained in a container or the like are included in the object. In addition, in this exemplary embodiment, the excavator is a device that scoops up objects such as soil, sand, sediment, snow, grain, cement, etc. The objects to be scooped up are not limited to soil and rocks.

[0014] (Configuration of learning device 1) As shown in FIG. 1, the learning device 1 includes an acquisition unit 10 and a learning unit 11. The acquisition unit 10 acquires a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container. The first type of imaging data is imaging data in a state in which the object is placed in the container. The learning unit 11 learns an estimation model for estimating the weight of the object by referring to the first type of imaging data and the measured weight of the object. The measured weight of the object is acquired by, for example, a weight measuring device. The acquisition unit 10 is one form of the acquisition means described in the claims, and the learning unit 11 is one form of the learning means described in the claims.

[0015] (Type 1 imaging data and type 2 imaging data) Fig. 3 is a diagram for explaining the first type of imaging data and the second type of imaging data. As shown in 3001 in Fig. 3, the first type of imaging data is data acquired by the imaging device C in a state where a target object TO is placed in a container TA (target area). As shown in 3002 in Fig. 3, data acquired by the imaging device C in a state where the target object TO is not placed in the container TA is called the second type of imaging data.

[0016] The imaging device C includes, for example, a camera such as a 3D camera, or a 3D scanner. The camera includes a depth camera, etc. The 3D scanner includes a 3D Lidar (Light Detection And Ranging) and the like. The imaging data refers to data acquired by the imaging device C such as a camera or a 3D scanner. The imaging data may be, for example, image data represented by a color according to depth, or contour lines, etc.

[0017] The learning unit 11 learns an estimation model using teacher data (learning data) including a plurality of pairs of a first type of imaging data captured by the imaging device C and a measured weight of an object. In this exemplary embodiment, the estimation model is a model that outputs the weight of an object based on the first type of imaging data. As an example, the estimation model may receive the first type of imaging data as input and output the weight of an object. As an example, the estimation model may be a model using an algorithm such as a theoretical function, a regression function, or a CNN (Convolutional Neural Network).

[0018] In this exemplary embodiment, "learning a model" can also be expressed as "learning a model (as a learning target)" or "training a model."

[0019] In the exemplary embodiment shown in FIG. 1, the acquisition unit 10 and the learning unit 11 are described as being incorporated in one learning device 1, but they do not necessarily have to be incorporated in one learning device. For example, the acquisition unit 10 and the learning unit 11 may be disposed separately. They may be connected by wired or wireless communication. In addition, both or one of the acquisition unit 10 and the learning unit 11 may be on the cloud. This also applies to the device configurations shown below.

[0020] (Effect of learning device 1) As described above, the learning device 1 according to this exemplary embodiment is configured to include an acquisition unit that acquires a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container, and a learning unit that learns an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0021] Therefore, according to the learning device 1 of this exemplary embodiment, it is possible to obtain the effect of providing a technology that can appropriately evaluate the weight of an object scooped up by an excavator and placed in a container.

[0022] In addition, since the weight of the object scooped up by the excavator can be evaluated while the object is contained in a container, work efficiency can be improved compared to a method in which the object is evaluated using a separate weight measuring device.

[0023] (Flow of learning method S1) Next, the flow of the learning method S1 according to the present exemplary embodiment 1 will be described with reference to FIG. 2. The learning method S1 is a learning method of an estimation model that estimates the weight of an object scooped up by an excavator and placed in a container. FIG. 2 is a flowchart showing the flow of the learning method S1 executed by the learning device 1. As shown in FIG. 2, the learning method S1 includes the following steps. First, in step S10 (acquisition step), the acquisition unit 10 acquires a first type of imaging data that is imaging data of an object scooped up by an excavator and placed in a container. As an example, the acquisition unit 10 acquires the first type of imaging data captured by the imaging device C. Note that the method by which the acquisition unit 10 acquires the imaging data from the imaging device C is not limited. As an example, the acquisition unit 10 can acquire the first type of imaging data from the imaging device C using wired communication, wireless communication, or a combination thereof.

[0024] Next, in step S11 (learning step), the learning unit 11 learns an estimation model for estimating the weight of the object by referring to the first type of imaging data and the measured weight of the object. As an example, the learning unit 11 learns the estimation model by referring to the first type of imaging data acquired in step S10 and the measured weight of the object measured by, for example, a weight measuring device. A specific example of the estimation model will be described later.

[0025] (Effect of learning method S1) As described above, the learning method S1 according to this exemplary embodiment is configured to include acquiring a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container, and learning an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0026] Therefore, according to the learning method S1 of this exemplary embodiment, it is possible to obtain the effect of providing a technology that can appropriately evaluate the weight of an object scooped up by an excavator and placed in a container.

[0027] (Configuration of the estimation device) Next, the configuration of the estimation device 2 according to the exemplary embodiment 1 will be described with reference to the drawings. Fig. 4 is a block diagram showing the configuration of the estimation device 2. The estimation device 2 is a device that estimates the weight of an object placed in a container by using an estimation model.

[0028] As shown in Fig. 4, the estimation device 2 includes an acquisition unit 20 and an estimation unit 21. The acquisition unit 20 acquires a first type of imaging data that is imaging data of an object scooped up by an excavator and placed in a container. The estimation unit 21 estimates the weight of the object by referring to the first type of imaging data acquired by the acquisition unit 20. As an example, the estimation unit 21 estimates the weight of the object by using an estimation model learned by the learning unit 11 of the learning device 1 described above. A specific example of the estimation model used by the estimation unit 21 will be described later.

[0029] As described above, the estimation device 2 according to the present exemplary embodiment 1 is configured to include an acquisition unit 20 that acquires first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container, and an estimation unit 21 that estimates the weight of the object by referring to the first type of imaging data acquired by the acquisition unit 20. Therefore, the estimation device 2 according to the present exemplary embodiment 1 has the effect of providing a technique for evaluating the weight of objects placed in various regions by using information acquired by imaging the object.

[0030] Furthermore, the acquisition unit 20 and the estimation unit 21 of the estimation device 2, and the acquisition unit 10 and the learning unit 11 of the learning device 1 may be mounted on the same device.

[0031] (Flow of estimation method S2) Next, the flow of the estimation method S2 according to the present exemplary embodiment 1 will be described with reference to FIG. 5. The estimation method S2 is a method for estimating the weight of an object scooped up by an excavator and placed in a container. FIG. 5 is a flowchart showing the flow of the estimation method S2 executed by the estimation device 2. As shown in FIG. 5, the estimation method S2 includes the following steps. First, in step S20 (acquisition step), the acquisition unit 20 acquires a first type of imaging data that is imaging data of an object scooped up by an excavator and placed in a container. This step S20 is similar to the acquisition step S10 of the learning method S1 described above. Next, in step S21, the estimation unit 21 estimates the weight of the object based on the first type of imaging data using an estimation model that outputs the weight of the object. As an example, the estimation model may be the estimation model learned in step S11 of the learning method S1 described above.

[0032] As described above, the estimation method S2 according to the present exemplary embodiment 1 employs a configuration that includes acquiring a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container, and estimating the weight of the object using an estimation model that outputs the weight of the object based on the first type of imaging data.

[0033] Therefore, according to the estimation method S2 according to the present exemplary embodiment 1, it is possible to obtain an effect of providing a technique capable of suitably evaluating the weight of an object placed in a container.

[0034] Exemplary embodiment 2 (Configuration of weight estimation device 100) Next, a weight estimation device 100 according to the present exemplary embodiment 2 will be described with reference to the drawings. Descriptions of components having the same functions as those described in the exemplary embodiment 1 will be omitted as appropriate.

[0035] 6 is a block diagram showing the configuration of a weight estimation device 100 according to this exemplary embodiment. As shown in FIG. 6, the weight estimation device 100 includes a first acquisition unit 101, a learning unit 102, a second acquisition unit 103, and an estimation unit 104.

[0036] The first acquisition unit 101 acquires first imaging data, which is imaging data for learning, from among the first type of imaging data, which is imaging data of an object in a container scooped up by an excavator. The first acquisition unit 101 has a configuration corresponding to the acquisition unit 10 in the exemplary embodiment 1, as an example. The first type of imaging data has been described in the exemplary embodiment 1, and therefore further description thereof will be omitted here.

[0037] The learning unit 102 learns an estimation model that outputs the weight of the object based on the first type of imaging data by referring to the first imaging data and the measured weight of the object. As an example, the learning unit 102 has a configuration corresponding to the learning unit 11 in the exemplary embodiment 1. The estimation model has been described in the exemplary embodiment 1, so further description will be omitted here.

[0038] The second acquisition unit 103 acquires second imaging data, which is imaging data for estimation, from the first type of imaging data. The second acquisition unit 103 corresponds to the acquisition unit 20 in the first exemplary embodiment, for example.

[0039] The estimation unit 104 estimates the weight of the object based on the estimation model and the second imaging data. The estimation unit 104 has a configuration corresponding to the estimation unit 21 in the first exemplary embodiment, for example.

[0040] In the exemplary embodiment shown in FIG. 6, the first acquisition unit 101, the learning unit 102, the second acquisition unit 103, and the estimation unit 104 are described as being incorporated into one weight estimation device 100, but they do not necessarily need to be incorporated into one weight estimation device. For example, the first acquisition unit 101, the learning unit 102, the second acquisition unit 103, and the estimation unit 104 may be arranged separately. These may be connected by wired communication or wireless communication. In addition, both or one of the learning unit 102 and the estimation unit 104 may be on the cloud. In addition, the functions of the first acquisition unit 101 and the second acquisition unit 103 may be realized by one acquisition unit.

[0041] (Effects of the weight estimation device 100) As described above, the weight estimation device 100 according to this exemplary embodiment has the advantage of being able to provide a technology that can appropriately evaluate the weight of an object that has been scooped up by an excavator and placed in a container.

[0042] (Additional Notes Regarding Weight Estimation Device 100) The first acquisition unit 101 may be configured to further acquire third imaging data, which is imaging data for learning, from the second type of imaging data, which is imaging data of the inside of the container before the object is scooped. Here, the expression "imaging data of the inside of the container before the object is scooped" is not intended to specify the timing of acquiring imaging data, but is intended to specify the situation inside the container. More specifically, in the "container before the object is scooped", the object does not exist or only a small amount of remaining object exists. Therefore, the "imaging data of the inside of the container before the object is scooped" can also be rephrased as "imaging data when the object does not exist in the container or a small amount of remaining object exists in the container".

[0043] The estimation model may output a weight of the object based on the first type of imaging data and the second type of imaging data.

[0044] The learning unit 102 may be configured to learn the estimation model by further referring to the third imaging data.

[0045] The second acquisition unit 103 may be configured to further acquire fourth imaging data, which is imaging data for estimation, from among the second type of imaging data.

[0046] The estimation unit 104 may be configured to estimate the weight of the object based on the estimation model, the second imaging data, and the fourth imaging data.

[0047] According to the above configuration, the weight of the object is estimated using an estimation model that outputs the weight of the object based on the first type of imaging data and the second type of imaging data, so that the weight of the object can be more appropriately evaluated.

[0048] (Flow of weight estimation method S100) Next, a description will be given of the flow of the weight estimation method S100 according to the present exemplary embodiment 2. Fig. 7 is a flowchart showing the flow of the weight estimation method S100 executed by the weight estimation device 100. As shown in Fig. 7, the weight estimation method S100 includes the following steps.

[0049] First, in step S101 (first acquisition step), the first acquisition unit 101 acquires first imaging data, which is imaging data for learning, from among first type imaging data, which is imaging data of an object in a container scooped up by an excavator.

[0050] Next, in step S102 (learning step), the learning unit 102 learns an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object.

[0051] Subsequently, in step S103 (second acquisition step), the second acquisition unit 103 acquires second imaging data, which is imaging data for estimation, from among the first type of imaging data.

[0052] Subsequently, in step S104 (estimation step), the estimation unit 104 estimates the weight of the object based on the estimation model and the second imaging data.

[0053] (Effect of weight estimation method S100) As described above, the weight estimation method S100 according to this exemplary embodiment has the advantage of providing a technology that can appropriately evaluate the weight of an object scooped up by an excavator and placed in a container.

[0054] (Additional notes regarding weight estimation method S100) In addition, in the first obtaining step S101, a configuration may be adopted in which third imaging data, which is imaging data for learning, is further obtained from the second type of imaging data, which is imaging data of the inside of the container before the object is scooped.

[0055] The weight of the object may be output based on the estimation model, the first type of imaging data, and the second type of imaging data.

[0056] In the learning step S102, the estimation model may be learned by further referring to the third imaging data.

[0057] In the second acquiring step S103, a fourth imaging data, which is imaging data for estimation, may be further acquired from the second type of imaging data.

[0058] In addition, in the estimation step S104, a weight of the object may be estimated based on the estimation model, and the second imaging data and the fourth imaging data.

[0059] According to the above configuration, the weight of the object is estimated using an estimation model that outputs the weight of the object based on the first type of imaging data and the second type of imaging data, so that the weight of the object can be more appropriately evaluated.

[0060] Exemplary embodiment 3 (Configuration of learning system 300) Next, a learning system 300 according to the present exemplary embodiment 3 will be described with reference to the drawings. Descriptions of components having the same functions as those described in the exemplary embodiments 1 and 2 will be omitted as appropriate.

[0061] Fig. 8 is a block diagram showing the configuration of a learning system 300 that learns an estimation model for estimating the weight of soil and sand contained in a bucket 353 of a backhoe 35. As shown in Fig. 8, the learning system 300 includes a learning device 3. The learning system 300 may further include a weight measuring device 34 and a backhoe 35.

[0062] The learning device 3 includes a calculation unit 30, a memory 31, a communication unit 32, and a storage unit 33. The calculation unit 30 includes an acquisition unit 301 and a learning unit 302. As an example, the acquisition unit 301 and the learning unit 302 are equivalent to the acquisition unit 10 and the learning unit 11 described in exemplary embodiment 1, respectively. As another example, the acquisition unit 301 and the learning unit 302 are equivalent to the acquisition unit (first acquisition unit 101) and the learning unit 102 described in exemplary embodiment 2, respectively, and therefore a description thereof will be omitted here.

[0063] The memory 31 temporarily or non-temporarily records the programs executed by the calculation unit 30 and various data or parameters referenced by the calculation unit 30. The communication unit 32 performs data communication with the controller 351 of the backhoe 35 or a communication unit (not shown) via a wireless communication network N1.

[0064] The storage unit 33 records various data for the learning unit 302 to learn the estimation model. Specifically, the storage unit 33 records the first type of imaging data 331, the second type of imaging data 332, the weight data 333, the parameters 334 of the estimation model, and the like. The first type of imaging data 331 and the second type of imaging data 332 are as described in the exemplary embodiment 1. The weight data 333 is data obtained by measuring the soil and sand contained in the bucket 353 of the backhoe 35 by the weight measuring device 34, as will be described later. The parameters 334 of the estimation model are various parameters included in the function formula of the estimation model.

[0065] The weight measuring device 34 measures the weight of the soil contained in the bucket 353. The weight measuring device 34 is capable of data communication with the learning device 3 via the wireless communication network N1. Alternatively, the weight measuring device 34 may be configured to be capable of wireless or wired communication with the controller 351 of the backhoe 35, and may be capable of data communication with the learning device 3 via the controller 351. The weight measuring device 34 may be a device that detects distortion of parts caused by loading soil and measures the weight of the soil, or may be a device that measures the weight of the entire backhoe 35. In the latter case, the weight of the backhoe 35 before scooping the soil with the bucket 353 and the weight of the backhoe 35 after scooping the soil with the bucket 353 are measured and transmitted to the learning device 3. The learning device 3 can calculate the weight of the soil from the weight of the backhoe 35 before and after scooping the soil.

[0066] The backhoe 35 is an excavator that excavates soil using a bucket 353 and moves the excavated soil to a predetermined position. The backhoe 35 includes a learning device 3 and a controller 351 that can perform data communication via a wireless communication network N1. The backhoe 35 also includes a depth camera 352 at a position where the contents of the bucket 353 can be imaged. In this exemplary embodiment, the image data is data imaged by the depth camera. The data imaged by the depth camera is, for example, data indicating the distance between the depth camera (i.e., the image capturing means) and an object. The depth camera 352 can generate an image including information on the depth (distance from the depth camera 352). Although the method of acquiring the depth information is not limited, examples include a parallax angle method, a TOF (Time of Flight) method, and a pattern method that detects a pattern of reflected waves of dot-shaped or striped light rays. The depth camera 352 is attached, for example, to the middle of the arm 354 toward the bucket 353.

[0067] In this exemplary embodiment 3, the second type of imaging data 332 recorded in the storage unit 33 is imaging data of the bucket 353 before scooping up soil and sand. The second type of imaging data is acquired by capturing an image with the depth camera 352 in a state in which the bucket 353 is rotated so that the inner surface faces the depth camera 352. The rotation angle of the bucket 353 with respect to the arm 354 when capturing the second type of imaging data is recorded in the storage unit 33.

[0068] On the other hand, the first type of imaging data 331 is imaging data of the bucket 353 after scooping up earth and sand. The rotation angle of the bucket 353 with respect to the arm 354 when capturing the first type of imaging data is the same as the rotation angle of the bucket 353 when capturing the second type of imaging data.

[0069] Fig. 9 is a heat map showing an example of imaging data of bucket 353 acquired by depth camera 352. Reference numeral 7001 in Fig. 9 is imaging data (second type imaging data) of bucket 353 in a state where no soil or sand is contained. Reference numeral 7002 in Fig. 9 is imaging data (first type imaging data) of bucket 353 in a state where soil or sand is contained. In Fig. 9, the darker the black color, the deeper the depth of bucket 353, that is, the less soil or sand is contained. Because bucket 353 has a slope on the front side, as shown in 7001, there is a difference in depth even in a state where no soil or sand is contained.

[0070] The first type of imaging data after the soil is contained, the second type of imaging data before the soil is contained, and weight data of the contained soil are associated with each other as a set of data sets and stored in the storage unit 33. The acquisition unit 301 repeatedly acquires data sets of the first type of imaging data 331, the second type of imaging data 332, and weight data 333. The storage unit 33 records these multiple data sets as teacher data.

[0071] The learning unit 302 learns the estimation model using teacher data (learning data). Specifically, the learning unit 302 learns the estimation model by further referring to the second type of imaging data 332, which is imaging data of the inside of the container in a situation where no target object is placed, in addition to the first type of imaging data 331, as learning data. As an example, learning means updating parameters based on the estimation model and the first and second imaging data so that the output weight value is as close as possible to the actually measured weight data. Note that since the second type of imaging data 332 is imaging data of the bucket 353 in an empty state, the first acquired second type of imaging data may be used in common thereafter. More specifically, in the process of repeatedly learning the estimation model, the first acquired second type of imaging data may be used again in the subsequent learning process. In that case, the estimation model can be learned by referring to the newly acquired first type of imaging data and the measured weight data. A specific example of the estimation model learned by the learning unit 302 and its parameters will be described later.

[0072] An example of the flow of the learning method performed by the learning system 300 is as follows. First, the acquisition unit 301 acquires the first type of imaging data and the second type of imaging data. Either the first type of imaging data or the second type of imaging data may be acquired first. Next, the learning unit 302 learns an estimation model by referring to the first type of imaging data and the second type of imaging data.

[0073] (Effects of Learning System 300) As described above, the learning system 300 according to this exemplary embodiment 3 employs a configuration including an acquisition unit that acquires a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container, and a learning unit that learns an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0074] Therefore, the learning system 300 according to this exemplary embodiment has the effect of providing a technique for suitably evaluating the weight of an object scooped up by an excavator and placed in a container. By further referencing the second type of imaging data to learn the estimation model, more accurate learning can be achieved. Also, even if the container is not horizontal (has bumps or dents), for example, learning can be achieved with high accuracy.

[0075] Exemplary embodiment 4 (Various estimation models) Next, an estimation model according to the present exemplary embodiment 4 will be described with reference to the drawings. The estimation model described in this exemplary embodiment is a specific example of the estimation model learned by the learning unit 11 of the learning device 1 described above, the estimation model used by the estimation unit 21 of the estimation device 2 for weight estimation, and the estimation model learned by the learning unit 302 of the learning system 300. It is also a specific example of the estimation model learned by the learning system described below or used by the estimation system. However, the estimation model is not limited to the one described below. Note that the description of components having the same functions as the components described in the exemplary embodiments 1 to 3 will be omitted as appropriate.

[0076] (1) Theoretical model First, a theoretical formula model based on a theoretical formula will be described with reference to the drawings. The theoretical formula in this exemplary embodiment is a theoretical formula for deriving the weight of an object from values ​​obtained from the first type of imaging data and the second type of imaging data. This theoretical formula is derived from a formula for calculating the volume of a cone formed between an imaged area and an imaging device. That is, the volume of the placed object is calculated by subtracting the volume of the cone obtained from the imaging data (first type of imaging data) after scooping the object from the volume of the cone obtained from the imaging data (second type of imaging data) before scooping the object. The weight of the object is calculated by multiplying the volume of the object by the specific gravity of the object.

[0077] The volume of the pyramid can be calculated by dividing the imaged area into minute areas, approximating the volume of the minute pyramids formed between the minute areas and the image capture device, and integrating the volumes of all the minute pyramids. An example of this is described below.

[0078] FIG. 10 is a diagram showing the concept of an estimation formula for estimating the weight of soil and sand based on imaging data captured by a depth camera, for example. As shown in 8001 of FIG. 10, assume that the area captured by the depth camera is a square, and the angle of view of one side is, for example, 72°. Each side is divided into, for example, 128 parts, and the entire area is divided into 128×128=16384 infinitesimal areas dS. At this time, as shown in 8002 of FIG. 10, consider an infinitesimal square pyramid with the infinitesimal area dS as the base and the position of the depth camera as the apex, and let d(i) be the distance (height) from the apex to the base dS. In this case, since the apex angle of the side surface is infinitesimal at Δθ=72 / 128(°), the length of one side of the base is approximated as d(i)Δθ, and the base area is (d(i)Δθ). 2 Therefore, the volume ΔV(i) of this small square pyramid is approximated as follows using the formula base area × height × (1 / 3).

[0079]

number

[0080] If the distance from the depth camera 352 to the bottom of the bucket 353 measured when there is soil is e(i), and the distance from the depth camera 352 to the surface of the soil when there is no soil is d(i), then the pile of soil V is expressed by the following formula (1), and the weight W of the soil is expressed by the following formula (2). In other words, the theoretical formula for estimating the weight is expressed by formula (2) which includes formula (1). Here, γ is the specific gravity of the soil. Note that the specific gravity of typical dry soil is about 1.3. The specific gravity can be set arbitrarily depending on the type of object.

[0081]

number

[0082] When the weight obtained by inputting the distance obtained by imaging an empty bucket and the soil scooped up by the bucket using a depth camera into the above equation (2) was compared with the actually measured weight of the soil using 212 sets of data, the RMSE (Root Mean Square Error) was, for example, 0.396.

[0083] (2) Cubic simple regression estimation model This estimation model is a model that performs simple regression on cubic terms. Specifically, the following formula (3) is used as an estimation formula for weight using the simple regression model. That is, a0 and a1 (constants) in the following formula (3) are obtained by simple regression using the least squares method. By performing simple regression, systematic errors such as measurement errors from a depth camera and errors such as specific gravity errors can be reduced compared to the method of estimation from the above theoretical formula.

[0084]

number

[0085] Here, the explanatory variable v n The following equation (4) is used, and the objective variable is w n Then, a0 and a1 can be calculated by the following formula (5). Here, Var is the variance, Cov is the covariance, n is the data number in the acquired learning dataset, and w is the objective variable. n is the measured weight. Also, e(i) and d(i) are values ​​related to the soil to be estimated, and as described above, e(i) is the distance from the depth camera 352 to the bottom surface of the bucket 353 measured in the presence of soil, and d(i) is the distance from the depth camera 352 to the surface of the soil measured in the absence of soil.

[0086] As shown in equation (4), the explanatory variable v n is a value obtained from the first type of imaging data in the acquired learning data set and the second type of imaging data in the acquired learning data set. Specifically, e is a value obtained from the first type of imaging data belonging to the data set n.n The cube of (i) and the value obtained from the second type of imaging data, d n This is the difference from the cube of (i).

[0087]

number

number

[0088] The data is the same as the data mentioned above, a dataset of 212 pairs. However, the dataset of 212 pairs was divided into 10 groups, of which 9 groups were used as learning data and the remaining group was used as validation data. This operation was repeated 10 times by changing the validation data group, and the error between the regression equation and the actual measured values ​​was evaluated. As a result, the RMSE was 0.364 as an example.

[0089] (3) Linear simple regression estimation model This estimation model uses a linear regression model instead of a cubic regression model. That is, in the following equation (6), which is the estimation equation for estimating weight, a0 and a1 (constants) are calculated by simple regression using the least squares method. n is expressed by the following formula (7), and the objective variable is w n The notational methods that have already been explained will not be explained again.

[0090] In this example, a0 and a1 are calculated as shown in formula (8) below. In the regression model using the cubic equation described above, the error is amplified by the cubic term and becomes large, but the error can be made smaller by using a linear equation model. The data is the same as the data described above, a data set of 212 pairs, and the method of dividing the data into 10 groups and using them as training data and validation data is the same as the method described above. As a result, the RMSE was 0.353, for example.

[0091]

number

number

number

[0092] (4) Multiple regression estimation model using 1st, 2nd, and 3rd degree equations This estimation model is a model obtained by multiple regression of an equation including linear, quadratic, and cubic terms. In other words, in the following equation (9), which is an estimation equation for estimating weight, a0 and a r (each a constant) is calculated by multiple regression. The explanatory variable is the following formula (10), and the objective variable is w n The notational methods that have already been explained will not be explained again.

[0093] In this example, a0, a1, a2, and a3 can be calculated by the following formulas (11) and (12). According to the model of this example, the error can be further reduced by multiple regression of an equation including a linear term, a quadratic term, and a cubic term. The data and the processing method are the same as described above. As a result, the RMSE was 0.273, for example.

[0094]

number

number

number

number

[0095] (5) Logarithmic multiple regression estimation model This estimation model expresses nonlinearity by logarithmic transformation of each term. The explanatory variables are the following formula (13), and the objective variable is w n The weight estimation formula using the multiple regression model is expressed by the following formula (14). The method for calculating each coefficient is the same as that for the multiple regression estimation model using the first, second, and third degree formulas described above. However, v (r) (r=1,2,3) is related to the soil and sand to be estimated, and is defined by the above equation (10) with the subscript n removed.

[0096] The regression model described above is a linear regression model, but by making it a nonlinear model as in this example, the accuracy can be improved. The data and the processing method are the same as described above. As a result, the RMSE was 0.250 as an example.

[0097]

number

number

[0098] (6) Kernel ridge regression estimation model This model is a ridge regression model using a kernel function (kernel ridge regression model). The kernel ridge regression estimation model can achieve even higher accuracy as a nonlinear regression model. Various functions can be used as the kernel function, but here we use the Gaussian kernel function. The Gaussian kernel function has the advantage of being able to construct an estimation model with high accuracy. The Gaussian kernel function is expressed by the following formula (15).

[0099]

number

[0100] Explanatory variable x in the kernel ridge regression estimation model n is expressed by the following formula (16). Moreover, the objective variable w is expressed by the following formula (17). The notation has already been explained, so the explanation will not be repeated here.

[0101] In addition, in the kernel ridge estimation model, the Gram matrix K N is expressed by the following equation (18), and the estimation equation for estimating the weight is expressed by the following equation (19). Here, the regression coefficient (weighting coefficient) a appearing in the following equation (19) is obtained by the following equation (20). Also, x in equation (19) relates to the soil and sand to be estimated, and is defined by the equation obtained by removing the subscript n from equation (16).

[0102] Here, n is the data number, n = 1, 2, ... N, where N = 212. i is the data coordinate, i = 1, 2, ..., L, where L = 128 × 128 = 16384. λ is a real hyperparameter that is the coefficient of the regularization term and is set appropriately by the user. When regressing with the kernel ridge regression estimation model described above, the error is very small, as shown in Figure 11, and the RMSE was 0.099 as an example. The data and the processing method are the same as described above.

[0103]

number

number

number

number

number

[0104] Exemplary embodiment 5 (Configuration of Estimation System 400) Next, an estimation system 400 according to the present exemplary embodiment 5 will be described with reference to the drawings. The estimation system 400 is a system that estimates the weight of soil stored in a bucket 443 of a backhoe 44. Note that the description of components having the same functions as those described in the exemplary embodiments 1 to 4 will be omitted as appropriate.

[0105] Fig. 12 is a block diagram showing a configuration of the estimation system 400. As shown in Fig. 12, the estimation system 400 includes an estimation device 4. The estimation system 400 may further include a backhoe 44.

[0106] The estimation device 4 includes a calculation unit 40, a memory 41, a communication unit 42, and a storage unit 43. The calculation unit 40 includes an acquisition unit 401 and an estimation unit 402. The acquisition unit 401 and the estimation unit 402 are equivalent to the acquisition unit 20 and the estimation unit 21 described in the exemplary embodiment 1, respectively, as an example. In addition, the acquisition unit 401 and the estimation unit 402 are equivalent to the acquisition units (the first acquisition unit 101 and the second acquisition unit 103) and the estimation unit 104 described in the exemplary embodiment 2, respectively, as another example. The estimation unit 402 may use, as an example, the estimation model described in the exemplary embodiment 4, which is learned by the learning unit 302 of the learning device 3 described in the exemplary embodiment 3. The estimation models include a regression model (the regression estimation model described in exemplary embodiment 4) in which values ​​obtained from the first type of imaging data and the second type of imaging data are used as explanatory variables and the weight of the object is used as the objective variable, and a theoretical model based on a theoretical formula (theoretical formula model described in exemplary embodiment 4).

[0107] The estimation model used by the estimation unit 402 may include both a theoretical model and a regression model. For example, the estimation unit 402 may estimate the weight by taking a weighted average of the result of a regression model including ridge regression and the result of a theoretical formula. The estimation unit 402 may also estimate the weight by taking a weighted average of a ridge regression model and another regression model. When multiple models are used as the estimation model used by the estimation unit 402, a value obtained by averaging or weighting the output of each model may be used.

[0108] The memory 41 temporarily or non-temporarily records the programs executed by the calculation unit 40 and various data or parameters referenced by the calculation unit 40. The communication unit 42 performs data communication with the controller 441 of the backhoe 44 or a communication unit (not shown) via a wireless communication network N1.

[0109] The storage unit 43 records various parameters 431 of the estimation model included in the estimation unit 402. The various parameters 431 may be, for example, parameters of the learned estimation model described in the fourth exemplary embodiment and recorded in the storage unit 33 of the learning device 3 described in the third exemplary embodiment.

[0110] The backhoe 44 is equipped with a controller 441 capable of data communication with the estimation device 4 via the wireless communication network N1. The backhoe 44 is also equipped with a depth camera 442 at a position where it can capture an image of the contents of the bucket 443. The depth camera 442 is attached, for example, to the middle of the arm 444, facing the direction of the bucket 443.

[0111] The backhoe 44 places the empty upper body bucket 443 at a predetermined angle with respect to the arm 444, and takes an image with the depth camera 442. That is, the image data includes data taken by the depth camera 442. As a result, second image data is acquired. This angle is an angle at which the inside of the bucket 443 is positioned facing the depth camera 442. Also, the backhoe 44 places the bucket 443 that has scooped up the soil at a predetermined angle with respect to the arm 444, and takes an image with the depth camera 442. As a result, first image data is acquired. These operations may be performed by the controller 441, or may be performed by the operator.

[0112] The acquired first and second imaging data are transmitted from the controller 441 to the estimation device 4 via the wireless communication network N1, and are acquired by the acquisition unit 401. As an example, the estimation unit 402 inputs the first and second imaging data acquired by the acquisition unit 401 into an estimation model used by the estimation unit 402. The estimation unit 402 may output a weight value output from the estimation model to the outside. For example, this weight value may be transmitted to a monitoring center (not shown) or may be displayed on a display device. The user (or operator) can use this information (weight value) to evaluate the amount of work.

[0113] (Operation of Estimation System 400) Next, the operation (estimation method) of the estimation system 400 will be described. First, the acquisition unit 401 acquires a first type of imaging data, which is imaging data of an object scooped up by an excavator and placed in a container. The acquisition unit 401 also acquires a second type of imaging data, which is imaging data of the inside of the container in a situation where the object is not placed. Next, the estimation unit 402 estimates the weight of the object using, as an estimation model, a model that outputs the weight of the object based on the first type of imaging data and the second type of imaging data. In this exemplary embodiment, the container is a bucket 443. The object is soil scooped up by the bucket 443. The first type of imaging data is imaging data of the bucket 443 in a state where soil is scooped up. The second type of imaging data is imaging data of an empty bucket 443.

[0114] (Effect of Estimation System 400) As described above, the acquisition unit 401 of the estimation system 400 acquires the first type of imaging data and the second type of imaging data, which are imaging data of an object, and the estimation unit 402 is configured to estimate the weight of the object using an estimation model that outputs the weight of the object based on the first type of imaging data and the second type of imaging data.

[0115] Therefore, the estimation system 400 according to the present exemplary embodiment 5 can provide a technology that can suitably evaluate the weight of an object scooped up by an excavator and placed in a container.

[0116] Exemplary Embodiment 6 (Configuration of information processing system 500) Next, an information processing system 500 according to the sixth exemplary embodiment will be described with reference to the drawings. The information processing system 500 is a system that estimates the weight of soil stored in a bucket 553 of a backhoe 55 and learns an estimation model. Note that the description of components having the same functions as those described in the first to fifth exemplary embodiments will be omitted as appropriate.

[0117] Fig. 13 is a block diagram showing a configuration of an information processing system 500. As shown in Fig. 13, the information processing system 500 includes an information processing device 5. The information processing system 500 may further include a weight measuring device 54 and a backhoe 55.

[0118] The information processing device 5 includes a calculation unit 50, a memory 51, a communication unit 52, and a storage unit 53. The calculation unit 50 includes an acquisition unit 501, a learning unit 502, and an estimation unit 503. The acquisition unit 501 and the estimation unit 503 are equivalent to the acquisition unit 401 and the estimation unit 402 described in the exemplary embodiment 5, respectively, and therefore their explanations will be omitted. In addition, the memory 51, the communication unit 52, the backhoe 55 and the controller 551 and the depth camera 552 provided therewith are equivalent to the corresponding elements described in the exemplary embodiment 5, respectively, and therefore their explanations will be omitted.

[0119] The information processing system 500 differs from the estimation device 4 according to the exemplary embodiment 5 in that it includes a learning unit 502. Also, the storage unit 53 records a first type of imaging data 531, a second type of imaging data 532, and weight data 533 in addition to parameters 534 of the estimation model. The learning unit 502 is equivalent to the learning unit 302 of the learning system 300 according to the exemplary embodiment 3, and the first type of imaging data 531, the second type of imaging data 532, and the weight data 533 recorded by the storage unit are equivalent to the first type of imaging data 331, the second type of imaging data 332, and the weight data 333 recorded by the storage unit 33 of the learning system 300 according to the exemplary embodiment 3.

[0120] (Operation of Information Processing System 500) The learning unit 502 of the information processing system 500 according to the sixth exemplary embodiment learns the estimation unit 503 by at least referring to the first type of imaging data and the measured weight of the object. That is, the learning unit 502 uses at least the acquired imaging data of the bucket 553 after scooping up the soil and the actual measured value of the weight of the soil as learning data to learn an estimation model used by the estimation unit 503. The estimation unit 503 estimates the weight of the soil scooped up by the bucket 553 using the estimation model learned by the learning unit 502.

[0121] Even after the learning unit 502 has trained the estimation model, the learning unit 502 can further train the estimation model by using at least the image data of the bucket 553 after scooping up the acquired soil and the actual measured weight of the soil as learning data.

[0122] (Effects of Information Processing System 500) As described above, the information processing system 500 is configured such that the learning unit 502 uses the actual measured value of the weight of the soil and the acquired image data of the bucket 553 before and after scooping the soil as learning data to learn an estimation model used by the estimation unit 503.

[0123] Therefore, the information processing system 500 according to the sixth exemplary embodiment can provide a technique for appropriately evaluating the weight of an object scooped up by an excavator and placed in a container. In addition, the estimation model can be further trained.

[0124] Exemplary Embodiment 7 (Configuration of information processing system 600) Next, an information processing system 600 according to the seventh exemplary embodiment will be described with reference to the drawings. The information processing system 600 is a system that estimates the weight of soil stored in a bucket 653 of a backhoe 65 and learns an estimation model. Note that the description of components having the same functions as those described in the first to sixth exemplary embodiments will be omitted as appropriate.

[0125] Fig. 14 is a block diagram showing a configuration of an information processing system 600. As shown in Fig. 14, the information processing system 600 includes an information processing device 6. The information processing system 600 may further include a weight measuring device 64 and a backhoe 65.

[0126] The information processing device 6 includes a calculation unit 60, a memory 61, a communication unit 62, and a storage unit 63. The calculation unit 60 includes an acquisition unit 601, a learning unit 602, and an estimation unit 603. The acquisition unit 601 and the learning unit 602 are equivalent to the acquisition unit 501 and the learning unit 502 described in the exemplary embodiment 6, respectively, and therefore their explanations will be omitted. In addition, the configurations of the memory 61, the communication unit 62, the storage unit 63, the backhoe 65, and the weight measuring device 64 are equivalent to the configurations of the memory 51, the communication unit 52, the storage unit 53, the backhoe 55, and the weight measuring device 54 described in the exemplary embodiment 6, respectively, and therefore their explanations will be omitted.

[0127] The information processing system 600 differs from the information processing system 500 according to the sixth exemplary embodiment in that the estimation unit 603 includes a plurality of estimation models including a kernel ridge regression model as an estimation model, and further includes a switching unit 6031. The memory unit 63 records parameters 634 for each of the plurality of learned estimation models.

[0128] The switching unit 6031 uses a kernel ridge regression model according to the learning accuracy of the estimation model. As an example, the switching unit 6031 determines whether to use the kernel ridge regression model or an estimation model other than the kernel ridge regression model as the estimation model according to the value of a kernel function including, as arguments, values ​​obtained from the first imaging data 631 and the second imaging data 632 acquired by the acquisition unit 601, and switches the estimation model to be used.

[0129] As an example, the switching unit 6031 determines the following as a kernel function including a value obtained from the first imaging data 631 and the second imaging data 632 acquired by the acquisition unit 601 as an argument x: k(x,xi) The value of is referred to, and depending on the value, the estimation model to be used is switched between the kernel ridge regression model and an estimation model other than the kernel ridge regression model.

[0130] As mentioned above, the kernel ridge regression estimation model can obtain a smaller RMSE. In addition, the maximum error is also smaller than other models. However, in a regression model based on a norm in a high-dimensional space, such as the kernel ridge regression model, the data used to train the regression model (hereinafter simply referred to as training data) may become sparse, resulting in a decrease in accuracy. In other words, if there is little training data and no training data exists near the data to be estimated, the estimation accuracy may decrease.

[0131] The learning data x is placed near the data x related to the sediment to be estimated. i When there is no learning data, the maximum value of the kernel function k(x,xi) will be a small numerical value. FIG. 15 is a graph showing the relationship between the maximum value of the kernel function and the estimation error. 1301 in FIG. 15 is a graph when there is a small amount of training data, and 1302 is a graph when there is a large amount of training data. As shown in 1301, when the maximum value of the kernel function falls below 0.965, the error of the estimation value tends to become large. Conversely, when there is a large amount of training data, the maximum values ​​of the kernel function are all 0.965 or more.

[0132] In such a case, for example, the maximum value of the kernel function can be used as a criterion for determining whether to use the kernel ridge regression model or an estimation model other than the kernel ridge regression model. Specifically, first, the estimation unit 603 calculates the above-mentioned kernel function k(x,xi) for each of i=1 to N, and obtains the maximum value of the kernel function. Next, the switching unit 6031 determines whether the maximum value is 0.965 or more. If it is determined that the maximum value is 0.965 or more, the switching unit 6031 selects the kernel ridge regression model as the estimation model. If it is determined that the maximum value is less than 0.965, the switching unit 6031 selects a model other than the kernel ridge regression model as the estimation model. For example, a theoretical formula model can be used as the model other than the kernel ridge regression model. Note that the judgment value does not necessarily have to be the maximum value of the kernel function. Also, the judgment value of 0.965 described above is merely an example, and can be appropriately determined according to various conditions.

[0133] (Effects of Information Processing System 600) As described above, the switching unit 6031 is configured to be able to switch between the kernel ridge regression model and an estimation model other than the kernel ridge regression model as the estimation model depending on the value of the kernel function that includes values ​​obtained from the first imaging data and the second imaging data as arguments.

[0134] Therefore, the information processing system 600 according to the seventh exemplary embodiment can provide a technique for suitably evaluating the weight of an object scooped up by an excavator and placed in a container. Furthermore, since a more accurate estimation model is selected and switched to, it is possible to obtain an effect of improving the weight estimation accuracy.

[0135] Exemplary embodiment 8 (Configuration of information processing system 700) Next, an information processing system 700 according to the eighth exemplary embodiment will be described with reference to the drawings. The information processing system 700 is a system that estimates the weight of soil stored in a bucket 763 of a backhoe 76 and learns an estimation model. Note that the description of components having the same functions as those described in the first to seventh exemplary embodiments will be omitted as appropriate.

[0136] FIG. 16 is a block diagram showing a configuration of an information processing system 700. As shown in FIG. 16, the information processing system 700 includes an acquisition unit 71, a learning unit 72, an estimation unit 73, and a storage unit 74. The estimation unit 73 includes a switching unit 731. The storage unit 74 records the first type of imaging data 741, the second type of imaging data 742, the weight data 743, and the parameters 744 of the estimation model. The acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 have the same configuration as the acquisition unit 601, the learning unit 602, the estimation unit 603, and the storage unit 63 described in the seventh exemplary embodiment. In addition, the operations of the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 are the same as the operations of the acquisition unit 601, the learning unit 602, the estimation unit 603, and the storage unit 63 described in the seventh exemplary embodiment.

[0137] The information processing system 700 may further include a weight measuring device 75 and a backhoe 76. The weight measuring device 75 and the backhoe 76 have the same configuration as the weight measuring device 64 and the backhoe 65 described in the seventh exemplary embodiment. The operation of the weight measuring device 75 and the backhoe 76 is similar to the operation of the weight measuring device 64 and the backhoe 65 described in the seventh exemplary embodiment. The acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 are capable of data communication with the weight measuring device 75 and the backhoe 76 via the wireless communication network N1.

[0138] The information processing system 700 according to the present exemplary embodiment 8 is different from the information processing system 600 according to the exemplary embodiment 7 in that the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 of the information processing system 700 are each arranged in a distributed manner, and the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 can communicate data with each other via the wireless communication network N1. In addition, the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 may each independently communicate data with the weight measuring device 75 and the backhoe 76. A part or all of the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 may be arranged on the cloud. Although not shown, the acquisition unit 71, the learning unit 72, the estimation unit 73, and the storage unit 74 are each provided with a communication unit. In addition, the acquisition unit 71, the learning unit 72, and the estimation unit 73 each have a memory.

[0139] (Effects of the information processing system 700) As described above, the acquisition unit 71, learning unit 72, estimation unit 73, and storage unit 74 of the information processing system 700 are each disposed in a distributed manner and configured to be capable of data communication with one another. In addition, the acquisition unit 71, learning unit 72, estimation unit 73, and storage unit 74 may be capable of data communication with the weight measuring device 75 and the backhoe 76.

[0140] Therefore, according to the information processing system 700 according to the eighth exemplary embodiment, it is possible to provide a technique for suitably evaluating the weight of an object scooped up by an excavator and placed in a container. In addition, it is possible to further learn an estimation model. In addition, the components of the information processing system 700 can be distributed and placed in any position as appropriate, which provides the effect of improving the degree of freedom of the system configuration.

[0141] [Software implementation example] Some or all of the functions of the learning devices 1, 3, the estimation devices 2, 4, and the information processing devices 5, 6, the weight estimation device 100, the information processing systems 500, 600, 700, or the acquisition units (first acquisition unit, second acquisition unit), learning units, estimation units, calculation units, etc. included in these devices and systems (hereinafter referred to as "learning device 1, etc.") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0142] In the latter case, the learning device 1 etc. is realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 17. The computer C has at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the learning device 1 etc. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the learning device 1 etc.

[0143] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0144] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0145] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0146] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0147] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the aspects described below.

[0148] (Appendix 1) A learning method including: acquiring a first type of imaging data, which is imaging data of an object inside a container scooped up by an excavator; and learning an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0149] According to the above configuration, it is possible to provide a learning method that can suitably evaluate the weight of an object placed in a container.

[0150] (Appendix 2) The learning method according to claim 1, wherein in the learning step, the estimation model is trained by referring to a second type of imaging data, which is imaging data of the inside of the container in a situation in which the target object is not placed.

[0151] According to the above configuration, by referring to the second type of imaging data, it is possible to learn an estimation model with higher accuracy.

[0152] (Appendix 3) 3. The learning method according to claim 1, wherein the imaging data is data indicating a distance between an imaging means and an object.

[0153] According to the above configuration, a model for estimating the weight of an object can be trained using distance data acquired by the imaging means.

[0154] (Appendix 4) The learning method according to claim 3, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and the weight of the object is a response variable.

[0155] According to the above configuration, by using a regression model as the estimation model, it is possible to learn an estimation model with higher accuracy.

[0156] (Appendix 5) 5. The method of claim 4, wherein the regression model includes a kernel ridge regression model.

[0157] According to the above configuration, by using a kernel ridge regression model as the estimation model, it is possible to learn an estimation model with higher accuracy.

[0158] (Appendix 6) A learning device including: an acquisition means for acquiring a first type of imaging data, which is imaging data of an object inside a container scooped up by an excavator; and a learning means for learning an estimation model for estimating the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0159] According to the above configuration, it is possible to provide a learning device that can suitably evaluate the weight of an object placed in a container.

[0160] (Appendix 7) The learning device according to claim 6, wherein the learning means learns the estimation model by referring to a second type of imaging data, which is imaging data of the inside of the container in a situation in which the target object is not placed.

[0161] According to the above configuration, by referring to the second type of imaging data, it is possible to provide a learning device for a more accurate estimation model.

[0162] (Appendix 8) The learning device according to claim 6 or 7, wherein the imaging data is data indicating a distance between an imaging means and an object.

[0163] According to the above configuration, it is possible to provide a model learning device that estimates the weight of an object using distance data acquired by an imaging means.

[0164] (Appendix 9) The learning device described in Appendix 8, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and the weight of the object is a response variable.

[0165] According to the above configuration, by using a regression model as the estimation model, it is possible to provide a learning device for a more accurate estimation model.

[0166] (Appendix 10) 10. The learning device of claim 9, wherein the regression model includes a kernel ridge regression model.

[0167] According to the above configuration, by using a kernel ridge regression model as an estimation model, it is possible to provide a learning device for a more accurate estimation model.

[0168] (Appendix 11) A learning system including: an acquisition means for acquiring a first type of imaging data, which is imaging data of the object inside a container scooped up by an excavator; and a learning means for learning an estimation model for estimating the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0169] According to the above configuration, it is possible to provide a learning system that can suitably evaluate the weight of an object placed in a container.

[0170] (Appendix 12) The learning system of claim 11, wherein the learning means learns the estimation model by further referring to a second type of imaging data, which is imaging data of the inside of the container in a situation where the target object is not placed.

[0171] According to the above configuration, by referring to the second type of imaging data, it is possible to provide a learning system for a more accurate estimation model.

[0172] (Appendix 13) The learning system according to claim 11, wherein the imaging data is data indicating a distance between an imaging means and an object.

[0173] According to the above configuration, it is possible to provide a model learning system that estimates the weight of an object using distance data acquired by an imaging means.

[0174] (Appendix 14) The learning system of claim 13, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and the weight of the object is a response variable.

[0175] According to the above configuration, by using a regression model as the estimation model, it is possible to provide a learning system for a more accurate estimation model.

[0176] (Appendix 15) 15. The learning system of claim 14, wherein the regression model includes a kernel ridge regression model.

[0177] According to the above configuration, by using a kernel ridge regression model as an estimation model, it is possible to provide a learning system for a more accurate estimation model.

[0178] (Appendix 16) An estimation method for estimating a weight of an object in a container scooped up by an excavator, the estimation method including: acquiring a first type of imaging data which is imaging data of the object placed in the container; and estimating the weight of the object using an estimation model which outputs the weight of the object based on the first type of imaging data.

[0179] According to the above configuration, it is possible to provide an estimation method capable of suitably evaluating the weight of an object placed in a container.

[0180] (Appendix 17) 17. The estimation method according to claim 16, wherein the imaging data is data indicating a distance between an imaging means and an object.

[0181] According to the above configuration, it is possible to provide a method for estimating the weight of an object using distance data acquired by an imaging means.

[0182] (Appendix 18) The estimation method according to claim 16 or 17, wherein in the acquiring step, a second type of imaging data is further acquired which is imaging data of the inside of the container in a situation where the object is not placed therein, and in the estimating step, a weight of the object is estimated using, as the estimation model, a model which outputs a weight of the object based on the first type of imaging data and the second type of imaging data.

[0183] According to the above configuration, by referring to the second type of imaging data, it is possible to provide a more accurate estimation method.

[0184] (Appendix 19) 19. The estimation method according to claim 18, wherein the estimation model includes a theoretical formula for deriving a weight of the object from values ​​obtained from the first type of imaging data and the second type of imaging data.

[0185] According to the above configuration, it is possible to provide an estimation method using a theoretical model as an estimation model. The theoretical model can be suitably used when the error of the regression model is large.

[0186] (Appendix 20) 20. The estimation method according to claim 18, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and a weight of the object is a response variable.

[0187] According to the above configuration, by using a regression model as the estimation model, it is possible to provide a more accurate estimation method.

[0188] (Appendix 21) 21. The method of claim 20, wherein the estimation model comprises a kernel ridge regression model.

[0189] According to the above configuration, by using a kernel ridge regression model as the estimation model, it is possible to provide a more accurate estimation method.

[0190] (Appendix 22) The estimation method according to claim 21, wherein in the estimating step, either the kernel ridge regression model or an estimation model other than the kernel ridge regression model is used as the estimation model depending on a value of a kernel function that includes, as arguments, values ​​obtained from the first type of imaging data and the second type of imaging data acquired in the acquiring step.

[0191] According to the above configuration, it is possible to provide an estimation method that, when an error in estimation using the kernel ridge regression model is large, can reduce the error by switching to another estimation model.

[0192] (Appendix 23) The estimation method of any one of Appendices 16 to 22, comprising training the estimation model by at least referring to a first type of imaging data, which is imaging data of the object in a container scooped up by an excavator, and a measured weight of the object.

[0193] According to the above configuration, it is possible to perform estimation using a trained estimation model. It is also possible to provide an estimation method that can further train the estimation model to obtain a more accurate estimation model.

[0194] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.

[0195] A learning device for an estimation model that estimates the weight of an object in a container scooped up by an excavator, the learning device having at least one processor that executes an acquisition process that acquires a first type of imaging data, which is imaging data of the object placed in the container, and a learning process that learns an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0196] In addition, the learning device may further include a memory, and the memory may store a program for causing the processor to execute the acquisition process and the learning process.

[0197] A program for causing a computer to function as a means for learning an estimation model that estimates the weight of an object in a container scooped up by an excavator, the means including an acquisition means for acquiring first type of imaging data, which is imaging data of the object placed in the container, and a learning means for learning an estimation model that estimates the weight of the object by referring to the first type of imaging data and the measured weight of the object.

[0198] A program for causing a computer to function as a means for estimating the weight of an object in a container scooped up by an excavator, the means including an acquisition means for acquiring first type of imaging data which is imaging data of the object placed in the container, and an estimation means for estimating the weight of the object using an estimation model which outputs the weight of the object based on the first type of imaging data.

[0199] Furthermore, the above program may be recorded on a computer-readable, non-transitory, tangible recording medium.

[0200] [Additional Note 4] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the aspects described below.

[0201] (Appendix 101) A weight estimation method including: a first acquisition step of acquiring first imaging data, which is imaging data for learning, from first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning step of learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object; a second acquisition step of acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; and an estimation step of estimating the weight of the object based on the estimation model and the second imaging data.

[0202] (Appendix 102) The weight estimation method according to claim 101, wherein the first acquisition step further acquires third imaging data, which is imaging data for learning, among second type of imaging data, which is imaging data of the inside of the container before the object is scooped, and the estimation model outputs the weight of the object based on the first type of imaging data and the second type of imaging data, and the learning step further refers to the third imaging data to learn the estimation model, and the second acquisition step further acquires fourth imaging data, which is imaging data for estimation, among the second type of imaging data, and the estimation step estimates the weight of the object based on the estimation model, the second imaging data, and the fourth imaging data.

[0203] (Appendix 103) 13. The weight estimation method according to claim 102, wherein the first type of imaging data and the second type of imaging data are data indicating a distance between an imaging means and an object.

[0204] (Appendix 104) 14. The weight estimation method according to claim 102, wherein the estimation model includes a theoretical formula for deriving a weight of the object from values ​​obtained from the first type of imaging data and the second type of imaging data.

[0205] (Appendix 105) The weight estimation method according to any one of Appendices 102 to 104, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are used as explanatory variables and the weight of the object is used as a response variable.

[0206] (Appendix 106) In the estimation step, 106. The weight estimation method according to claim 105, wherein a kernel ridge regression model is used depending on the learning accuracy of the estimation model.

[0207] (Appendix 107) The weight estimation method described in Appendix 106, wherein in the estimation step, either a kernel ridge regression model or an estimation model other than the kernel ridge regression model is used as the estimation model depending on a value of a kernel function that includes as arguments values ​​obtained from the first type of imaging data and the second type of imaging data acquired in the second acquisition step.

[0208] (Appendix 108) A weight estimation device including: a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and a measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data.

[0209] (Appendix 109) The weight estimation device described in Appendix 108, wherein the first acquisition means further acquires third imaging data which is imaging data for learning among second type of imaging data which is imaging data of the inside of the container before the object is scooped, the estimation model outputs a weight of the object based on the first type of imaging data and the second type of imaging data, the learning means further refers to the third imaging data to learn the estimation model, the second acquisition means further acquires fourth imaging data which is imaging data for estimation among the second type of imaging data, and the estimation means estimates a weight of the object based on the estimation model, the second imaging data, and the fourth imaging data.

[0210] (Appendix 110) 110. The weight estimation device according to claim 109, wherein the first type of imaging data and the second type of imaging data are data indicating a distance between an imaging means and an object.

[0211] (Appendix 111) 111. The weight estimation device according to claim 109 or 110, wherein the estimation model includes a theoretical formula for deriving a weight of the object from values ​​obtained from the first type of imaging data and the second type of imaging data.

[0212] (Appendix 112) 122. The weight estimation device of claim 109, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and the weight of the object is a response variable.

[0213] (Appendix 113) 13. The weight estimation device according to claim 112, wherein the estimation means uses a kernel ridge regression model depending on the learning accuracy of the estimation model.

[0214] (Appendix 114) The weight estimation device described in Appendix 113, wherein the estimation means uses, as the estimation model, either a kernel ridge regression model or an estimation model other than the kernel ridge regression model, depending on a value of a kernel function that includes, as arguments, values ​​obtained from the first type of imaging data and the second type of imaging data acquired by the second acquisition means.

[0215] (Appendix 115) A weight estimation system including: a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and a measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data.

[0216] (Appendix 116) The weight estimation system described in Appendix 115, wherein the first acquisition means further acquires third imaging data which is imaging data for learning among second type of imaging data which is imaging data of the inside of the container before the object is scooped, the estimation model outputs the weight of the object based on the first type of imaging data and the second type of imaging data, the learning means further refers to the third imaging data to learn the estimation model, the second acquisition means further acquires fourth imaging data which is imaging data for estimation among the second type of imaging data, and the estimation means estimates the weight of the object based on the estimation model, the second imaging data, and the fourth imaging data.

[0217] (Appendix 117) 17. The weight estimation system according to claim 116, wherein the first type of imaging data and the second type of imaging data are data indicating a distance between an imaging means and an object.

[0218] (Appendix 118) 18. The weight estimation system according to claim 116 or 117, wherein the estimation model includes a theoretical formula for deriving a weight of the object from values ​​obtained from the first type of imaging data and the second type of imaging data.

[0219] (Appendix 119) 19. The weight estimation system of any one of Appendices 116 to 118, wherein the estimation model includes a regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are explanatory variables and the weight of the object is a response variable.

[0220] (Appendix 120) 120. The weight estimation system according to claim 119, wherein the estimation means uses a kernel ridge regression model depending on the learning accuracy of the estimation model.

[0221] (Appendix 121) The weight estimation system described in Appendix 120, wherein the estimation means uses, as the estimation model, either a kernel ridge regression model or an estimation model other than the kernel ridge regression model, depending on a value of a kernel function that includes, as arguments, values ​​obtained from the first type of imaging data and the second type of imaging data acquired by the second acquisition means.

[0222] [Additional Note 5] A part or all of the above-described embodiments can be further expressed as follows.

[0223] A weight estimation device having at least one processor, which executes a first acquisition process to acquire first imaging data, which is imaging data for learning purposes, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning process to learn an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object; a second acquisition process to acquire second imaging data, which is imaging data for estimation purposes, from the first type of imaging data; and an estimation process to estimate the weight of the object based on the estimation model and the second imaging data.

[0224] In addition, the weight estimation device may further include a memory, and the memory may store a program for causing the processor to execute the first acquisition process, the learning process, the second acquisition process, and the estimation process.

[0225] A program for causing a computer to function as a first acquisition means for acquiring first imaging data, which is imaging data for learning purposes, from among first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs the weight of the object based on the first type of imaging data, by referring to the first imaging data and the measured weight of the object; a second acquisition means for acquiring second imaging data, which is imaging data for estimation purposes, from among the first type of imaging data; and an estimation means for estimating the weight of the object based on the estimation model and the second imaging data. [Explanation of symbols]

[0226] 1,3 Learning device 10,20,301,401,501,601 Acquisition part 11,102,302,502,602,72 Learning Department 2,4 Estimation device 21,104,402,503,603,73 Estimation part 31,41,51,61 Memory 32, 42, 52, 62 Communications Department 33,43,53,63,74 Storage section 40,50,60 Arithmetic unit 5,6 Information processing equipment 100 Weight estimation device 101 First Acquisition Section 103 Second Acquisition Section 300 Learning System 400 Estimation System 331,531,631,741 First type of imaging data 332,532,632,742 Second type of imaging data 333,533,633,743 Weight Data 334,431,534,634,744 Estimated model parameters 35,44,55,65,76 Backhoe 34,54,64,75 Weight measuring device 351,441,551,651,761 Controller 352,442,552,652,762 Depth Camera 353,443,553,653,763 buckets 354,444,554,654,764 Arm 500,600,700 Information Processing System

Claims

1. a first acquisition step of acquiring first imaging data, which is imaging data for learning, from first type imaging data, which is imaging data of an object in the container scooped up by the excavator; a learning process of learning an estimation model that outputs a weight of the object based on the first type of imaging data by referring to the first imaging data and a measured weight of the object; a second acquisition step of acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; a switching step of determining whether to use a kernel ridge regression model or an estimation model other than the kernel ridge regression model as the estimation model in accordance with a value of a kernel function that includes values ​​obtained from the first imaging data and the second imaging data as arguments; an estimation step of estimating a weight of the object based on the estimation model and the second imaging data; A weight estimation method comprising:

2. In the first acquisition step, third imaging data is further acquired as learning imaging data among second type imaging data which is imaging data of the inside of the container before the object is scooped, the estimation model outputs a weight of the object based on the first type of imaging data and the second type of imaging data, the learning step further refers to the third imaging data to learn the estimation model; In the second acquisition step, fourth imaging data, which is imaging data for estimation, is further acquired from the second type of imaging data, In the estimation step, a weight of the object is estimated based on the estimation model, the second imaging data, and the fourth imaging data. The method of claim 1 .

3. The first type of imaging data and the second type of imaging data are data indicating a distance between an imaging means and an object. The weight estimation method according to claim 2.

4. The estimation model includes a theoretical formula for deriving a weight of the object from values ​​obtained from the first type of imaging data and the second type of imaging data. The weight estimation method according to claim 2 or 3.

5. The estimation model includes the kernel ridge regression model in which values ​​obtained from the first type of imaging data and the second type of imaging data are used as explanatory variables and the weight of the object is used as a response variable. The weight estimation method according to any one of claims 2 to 4.

6. a first acquisition means for acquiring first imaging data, which is imaging data for learning, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs a weight of the object based on the first type of imaging data by referring to the first imaging data and a measured weight of the object; A second acquisition means for acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; a switching means for determining whether to use a kernel ridge regression model or an estimation model other than the kernel ridge regression model as the estimation model in accordance with a value of a kernel function that includes values ​​obtained from the first imaging data and the second imaging data as arguments; an estimation means for estimating a weight of the object based on the estimation model and the second imaging data; A weight estimation device comprising:

7. a first acquisition means for acquiring first imaging data, which is imaging data for learning, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs a weight of the object based on the first type of imaging data by referring to the first imaging data and a measured weight of the object; A second acquisition means for acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; a switching means for determining whether to use a kernel ridge regression model or an estimation model other than the kernel ridge regression model as the estimation model in accordance with a value of a kernel function that includes values ​​obtained from the first imaging data and the second imaging data as arguments; an estimation means for estimating a weight of the object based on the estimation model and the second imaging data; A weight estimation system including:

8. Computer, a first acquisition means for acquiring first imaging data, which is imaging data for learning, from a first type of imaging data, which is imaging data of an object in a container scooped up by an excavator; a learning means for learning an estimation model that outputs a weight of the object based on the first type of imaging data by referring to the first imaging data and a measured weight of the object; A second acquisition means for acquiring second imaging data, which is imaging data for estimation, from the first type of imaging data; a switching means for determining whether to use a kernel ridge regression model or an estimation model other than the kernel ridge regression model as the estimation model in accordance with a value of a kernel function that includes, as arguments, values ​​obtained from the first imaging data and the second imaging data; an estimation means for estimating a weight of the object based on the estimation model and the second imaging data; A program to make it function as such.

Images