MEASUREMENT SYSTEM, SUPPORT CONTROL SYSTEM, MEASUREMENT METHOD, AND MEASUREMENT PROGRAM

Abstract

This weighing system comprises: an amount estimation unit that calculates, for each of at least one candidate separation position, an estimation amount of a subset of material pieces obtained by separating, at the candidate separation position, a set of material pieces controlled by mass, the estimation amount being calculated on the basis of a set image captured of the set of material pieces; and a separation position determination unit that determines a position at which a predetermined amount of material pieces is separated from the set as a separation position on the basis of the at least one estimation amount.

Description

[Technical field]

[0001] One aspect of the present disclosure relates to a weighing system, a support control system, a weighing method, and a weighing program. [Background technology]

[0002] Patent Document 1 describes a sorting device that sorts products based on the product dimensions, the ordered number, and the dimensions of the box. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-52626 A Summary of the Invention [Problem to be solved by the invention]

[0004] What is desired is a mechanism for identifying a predetermined amount of material. [Means for solving the problem]

[0005] A weighing system according to one aspect of the present disclosure includes a quantity estimation unit that calculates, for each of at least one candidate separation position based on a collection image depicting a collection of material pieces managed by mass, an estimated quantity of a subset of material pieces obtained by separating the collection at the candidate separation position, and a separation position determination unit that determines, based on the at least one estimated quantity, a position at which a predetermined quantity of material pieces will be separated from the collection as a separation position.

[0006] A weighing method according to one aspect of the present disclosure is a weighing method performed by a weighing system having at least one processor, and includes a step of calculating, for each of at least one candidate separation position based on a collection image depicting a collection of material pieces managed by mass, an estimate of a subset of material pieces obtained by separating the collection at the candidate separation position, and a step of determining, as a separation position, a position at which a predetermined amount of material pieces is separated from the collection based on the at least one estimate.

[0007] A weighing program according to one aspect of the present disclosure causes a computer to perform the steps of calculating, for each of at least one candidate separation position based on a collection image depicting a collection of material pieces managed by mass, an estimate of a subset of material pieces obtained by separating the collection at the candidate separation position, and determining, based on the at least one estimate, a separation position as a position at which a predetermined amount of material pieces will be separated from the collection. Effect of the Invention

[0008] According to one aspect of the present disclosure, a predetermined amount of a piece of material may be identified. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of a weighing system. [Diagram 2] FIG. 13 is a diagram showing an example of a separation position. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer used in the weighing system. [Figure 4] 1 is a flowchart illustrating an example of a process in a weighing system. [Diagram 5] FIG. 2 is a diagram for explaining a voxel. [Figure 6] FIG. 13 is a diagram showing an example of setting separation candidate positions. [Figure 7] 1 is a graph showing an example of a relationship between an estimated amount and an actual amount. [Figure 8] 11 is a graph for explaining the determination of a separation position. [Figure 9] FIG. 2 is a diagram illustrating an example of a functional configuration and application of a weighing system. [Figure 10] 13 is a flowchart showing an example of determining an insertion position. [Figure 11] FIG. 13 is a diagram showing an example of setting an insertion candidate position. [Figure 12] FIG. 13 is a diagram illustrating an example of a virtual route. [Figure 13] 13 is a flowchart showing an example of determining a separation position. [Figure 14] 1 is a flow chart showing an example of a series of steps for removing and separating material pieces. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] [First example] (System Configuration) The weighing system 1 according to an example is a computer system that determines a position to separate a predetermined amount of material pieces from a collection of material pieces managed by mass. The material pieces refer to individual materials formed relatively small. For example, the material pieces are used to manufacture a given product. The "material pieces managed by mass" refer to material pieces that are managed by mass because it is very difficult or practically impossible to manage them by number. Examples of material pieces managed by mass include, but are not limited to, granular or powdered materials, finely cut materials, and materials provided as kneaded products. In one example, the material pieces are food that are quantitatively managed by mass. In this disclosure, "food" includes food additives. Examples of the management of material pieces include, but are not limited to, storage, measurement, separation, and use. In this disclosure, the "material pieces managed by mass" are also simply referred to as "material pieces". The weighing system 1 determines a position to separate a predetermined amount of material pieces from such a collection of material pieces. In this disclosure, the position is also referred to as a "separation position". Due to the nature of the material pieces described above, it is very difficult or practically impossible to obtain a predetermined amount of material pieces depending on the number of material pieces. In order to separate such material pieces, the weighing system 1 analyzes an aggregate image, which is an image showing an aggregate of material pieces, to determine a separation position.

[0012] FIG. 1 is a diagram showing an example of a functional configuration of a weighing system 1. In one example, the weighing system 1 includes an image processing unit 11, a density estimation unit 12, an amount estimation unit 13, a correction unit 14, a separation position determination unit 15, and a learning unit 16 as functional modules. The image processing unit 11 is a functional module that preprocesses a set image of a set of material pieces in order to determine a separation position. The density estimation unit 12 is a functional module that estimates the density of the set of material pieces based on the set image. In one example, the density estimation unit 12 performs the estimation using a machine learning model 20 that receives an input of the set image and outputs the density. The amount estimation unit 13 is a functional module that calculates an estimated amount of a subset of material pieces obtained by separating the set of material pieces based on the density. The amount estimation unit 13 sets at least one separation candidate position (e.g., a plurality of separation candidate positions) that are candidates for the separation position, and calculates the estimated amount for each separation candidate position. The correction unit 14 is a functional module that corrects the estimated amount as necessary. The separation position determination unit 15 is a functional module that determines the separation position based on the estimated density. The separation position determination unit 15 determines a separation position from at least one separation candidate position based on at least one estimated quantity calculated using the density, for example. The learning unit 16 is a functional module that generates a machine learning model 20 used in the density estimation unit 12 by machine learning. Therefore, the machine learning model 20 is a trained model. In one example, the learning unit 16 generates two types of machine learning models 20 described later, and the two types are hereinafter distinguished as machine learning models 21 and 22 as necessary.

[0013] 2 shows an example of a separation position. In this example, a collection of food items 210, which are quantitatively controlled by mass, is placed on a support 200, and the weighing system 1 determines a separation position 220 for adding a predetermined amount of food items 210 to a predetermined food item. The food item is, for example, a product provided to a customer, such as a meal, a lunch box, etc. The food item 210 is an ingredient of the food item, for example, a cut food item.

[0014] The weighing system 1 can be realized by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes a specific process. The weighing system 1 may be realized by one computer, or may be realized by a distributed system having multiple computers.

[0015] 3 is a diagram showing an example of a hardware configuration of the computer 100 used in the weighing system 1. In this example, the computer 100 includes a main body 110, a monitor 120, and an input device .

[0016] The main body 110 is a device that executes the main functions of a computer. The main body 110 has a circuit 160, and the circuit 160 has at least one processor 161, a memory 162, a storage 163, an input / output port 164, and a communication port 165. The storage 163 records a program for configuring each functional module of the main body 110. The storage 163 is a computer-readable recording medium such as a hard disk, a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 162 temporarily stores a program loaded from the storage 163, a calculation result of the processor 161, and the like. The processor 161 configures each functional module by executing a program in cooperation with the memory 162. The input / output port 164 inputs / outputs an electric signal between the monitor 120 or the input device 130 in response to a command from the processor 161. The input / output port 164 may input / output an electric signal between another device. The communication port 165 performs data communication with another device via a communication network N in response to a command from the processor 161.

[0017] Monitor 120 is a device for displaying information output from main body 110. Monitor 120 may be any device capable of displaying graphics, and a specific example of such a device is a liquid crystal panel.

[0018] The input device 130 is a device for inputting information to the main body 110. The input device 130 may be any device capable of inputting desired information, and specific examples thereof include an operation interface such as a keypad, a mouse, or an operation controller.

[0019] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main body 110, the monitor 120, and the input device 130 may be integrated as a tablet computer.

[0020] Each functional module of the weighing system 1 is realized by loading a weighing program onto the processor 161 or memory 162 and having the processor 161 execute the program. The weighing program includes code for realizing each functional module of the weighing system 1. The processor 161 operates the input / output port 164 or the communication port 165 in accordance with the weighing program, and executes reading and writing of data in the memory 162 or the storage 163.

[0021] The weighing program may be provided by being fixedly recorded on a non-transitory recording medium such as a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the weighing program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0022] (Measuring method) As an example of a weighing method according to the present disclosure, an example of a processing procedure executed by the weighing system 1 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of processing in the weighing system 1 as a processing flow S1. That is, the weighing system 1 executes the processing flow S1.

[0023] In step S11, the image processing unit 11 acquires a group image showing a group of material pieces. This group image may be a still image or a frame image constituting a video. In one example, the image processing unit 11 acquires a depth image showing a group of material pieces. Alternatively, the image processing unit 11 may acquire a set of an RGB image and a depth image showing a group of material pieces. The image processing unit 11 may receive a group image sent from a camera or another computer. Alternatively, the image processing unit 11 may accept a group image input by a user, or may read a group image from a given storage device based on a user input.

[0024] In step S12, the image processor 11 sets a plurality of voxels for the aggregate image. The image processor 11 sets a plurality of voxels for the aggregate of material pieces captured in the aggregate image. In this disclosure, a voxel is a virtual three-dimensional shape that is processed as the smallest unit.

[0025] FIG. 5 is a diagram for explaining a voxel. In one example, each voxel 300 is a rectangular parallelepiped that is set based on the X-axis and Y-axis defined along the surface direction of the collective image and the Z-axis defined along the depth direction of the collective image. In one example, the image processing unit 11 sets a voxel 300 for each pixel of the collective image. Each voxel 300 is defined by a side with a length Δx along the X-axis direction, a side with a length Δy along the Y-axis direction, and a side with a length z(x, y) along the Z-axis direction. z(x, y) is the length in the Z-axis direction of a voxel placed at a position (x, y) on the XY plane. The lengths Δx and Δy are common to all voxels 300. Meanwhile, the length z(x, y) is the distance from the lower surface to the upper surface of the collection of material pieces along the Z-axis direction, and may differ for each voxel 300 depending on at least one of the shape of the collection of material pieces and the shape of the support surface on which the collection is placed.

[0026] In one example, the image processor 11 analyzes the aggregate image to identify an area where a piece of material may exist, and sets multiple voxels in that area. When a depth image is used, the image processor 11 analyzes the depth image to calculate the distance in the depth direction for each voxel, and sets the length z(x, y) of each voxel based on the distance.

[0027] Returning to FIG. 4, in step S13, the density estimation unit 12 calculates a voxel density and a class value for each of the multiple voxels. The voxel density is the density of material pieces in the voxel. The class value is a value indicating whether or not a material piece exists in the voxel. In one example, the class value is expressed by a binary value of "0" indicating that no material piece exists in the voxel and "1" indicating that a material piece exists in the voxel.

[0028] In one example, the density estimation unit 12 calculates the voxel density for each of the multiple voxels using a machine learning model 21 that receives an input of a set image and outputs a voxel density. The machine learning model 21 processes the set image to calculate a voxel density d x,y (I;θ) is calculated. x,y The notation "(I;θ)" refers to the voxel density of a voxel located at position (x, y) obtained by processing a set of images I by a machine learning model 21 having a set of parameters θ.

[0029] In one example, the density estimation unit 12 calculates a class value for each of the multiple voxels using a machine learning model 22 that receives an input of a set image and outputs a class value. The machine learning model 22 processes the set image to calculate a class value c x,y (I;θ) is calculated. x,y The notation "(I;θ)" refers to the class value of a voxel located at position (x, y) obtained by processing a set of images I with a machine learning model 22 having a set of parameters θ.

[0030] In step S14, the quantity estimation unit 13 calculates an estimated quantity of a subset of material pieces for each of a plurality of separation candidate positions based on the individual voxel quantities. The voxel quantity refers to the quantity of material pieces in a voxel. The quantity estimation unit 13 sets a plurality of separation candidate positions in an area in which a plurality of voxels are set. Then, for each separation candidate position, the quantity estimation unit 13 calculates an estimated quantity of a subset of material pieces obtained by separation at the separation candidate position. For example, the quantity estimation unit 13 sets a plurality of separation candidate positions at a given interval from an end of the set of material pieces toward the center of the set, and calculates an estimated quantity for each separation candidate position.

[0031] In one example, the quantity estimator 13 calculates an estimated quantity m(I;θ) at each separation candidate position by using equation (1).

number

[0032] Equation (1) means that the sum of multiple voxel quantities corresponding to multiple voxels existing in the section between the reference position set in the area where a group of material pieces may exist and the separation candidate position is calculated as an estimate of the subset of material pieces. The estimated individual voxel quantities are calculated based on the voxel density d x,y The class value c is the product of (I;θ) and the voxel volume z(x,y)ΔxΔy, and thus indicates the mass. In one example, the mass estimator 13 calculates the voxel volume based on the depth distance obtained from the depth image. x,y When (I;θ) is one of the two values, if the class value of a certain voxel is 1, the voxel amount of that voxel is used as is, and if the class value is 0, the voxel amount is regarded as 0.

[0033] As shown in formula (1), the quantity estimation unit 13 calculates a voxel quantity for each of a plurality of voxels based on the voxel density. For example, the quantity estimation unit 13 calculates a voxel quantity for each voxel based on the class value and the voxel density. The quantity estimation unit 13 calculates an estimated quantity m(I;θ) based on the voxel quantity of each voxel. Formula (1) shows an example of calculating an estimated mass of a subset based on the density of the set of material pieces and the volume of the subset as an estimated quantity.

[0034] In one example, the quantity estimation unit 13 sets a reference position at one end of an area where a collection of material pieces may exist, and sets a plurality of separation candidate positions at given intervals from the reference position. The quantity estimation unit 13 may set the intervals based on at least one of Δx and Δy, which are the lengths of the sides of a voxel, and may set at least one of a multiple of Δx and a multiple of Δy as the intervals. FIG. 6 is a diagram showing an example of setting of separation candidate positions, and shows the support 200 shown in FIG. 2 as an area where a collection of material pieces may exist. For example, the quantity estimation unit 13 sets one end of the support 200 as a reference position 230, and sets a plurality of separation candidate positions 240 at intervals (n×Δy) along the Y axis (n is a positive integer). That is, the quantity estimation unit 13 may set a plurality of separation candidate positions along one coordinate axis of a coordinate system indicating a real space in which a collection of material pieces is located.

[0035] In one example, in step S14, the correction unit 14 corrects the estimated amount for each of the multiple separation candidate positions. The correction unit 14 corrects the estimated amount based on a relationship between a sample group of estimated amounts calculated in the past and a sample group of actual amounts of material pieces actually separated in the past at the separation positions corresponding to the estimated amounts. Data of these sample groups is stored in a given database in advance. The correction unit 14 refers to the database to determine the relationship between the estimated amount and the actual amount.

[0036] FIG. 7 is a graph showing an example of this relationship. The vertical and horizontal axes of this graph indicate the estimated amount and the actual amount, respectively. Each sample point 400 on the graph indicates the correspondence between the estimated amount and the actual amount. The correction unit 14 performs a regression analysis based on these sample points 400 to obtain the estimated amount m pred and the actual amount m act m is a linear equation that shows the relationship between pred = a m act +b. Estimate m pred and the actual amount m act The linear equation that shows the ideal relationship with m pred =m act FIG. 7 shows a linear equation 410 obtained by regression analysis and an ideal linear equation 420 in which the actual quantity is regarded as a true value. The correction unit 14 obtains a correction coefficient for converting the relationship between the estimated quantity and the actual quantity from linear equation 410 to linear equation 420, and corrects the estimated quantity for each separation candidate position using this correction coefficient. In the example of FIG. 7, the correction coefficient is the inverse 1 / a of the slope of linear equation 410 and -b for canceling the intercept of linear equation 410. For example, the estimated quantity m calculated by the quantity estimation unit 13 is pred ´ is finally estimated by m pred * When correcting to m pred * =m pred Calculate ´ / ab.

[0037] Returning to Fig. 4, in step S15, the separation position determination unit 15 determines a separation position from a plurality of separation candidate positions. In one example, the separation position determination unit 15 determines a separation candidate position that minimizes an error between an estimated amount and a predetermined amount as the separation position. In a determination based on mass, the separation position determination unit 15 determines a separation candidate position that minimizes an error between an estimated mass and a predetermined mass as the separation position. Since the estimated amount is calculated based on the amount of each voxel, it can be said that the separation position is determined based on the amount of each of the plurality of voxels.

[0038] The determination of the separation position based on the error will be described with reference to Fig. 6 again. In the example of Fig. 6, the range along the Y-axis direction is [0,y max y=0 corresponds to the reference position 230, and y=ymax corresponds to the separation candidate position 240 that is the furthest from the reference position 230. In this case, the separation position determination unit 15 determines the separation candidate position y * is determined as the separation position.

number

[0039] 8 is a graph for explaining the determination of the separation position using equation (2). The vertical axis of graph 501 indicates the estimated amount, and the horizontal axis indicates the separation candidate position along the Y direction. The vertical axis of graph 502 indicates the error between the estimated amount and the predetermined amount, and the horizontal axis indicates the separation candidate position along the Y direction. 0 on the horizontal axis represents the reference position 230, and * indicates the target position. When the estimates and errors are calculated in order from the reference position 230, the estimates become larger as the separation candidate positions change, and the errors become smaller. After a certain separation candidate position, the error becomes larger. The minimum error can be a value greater than 0.

[0040] Returning to Fig. 4, in step S16, the separation position determination unit 15 outputs the determined separation position. For example, the separation position determination unit 15 may output the separation position to a device that separates a predetermined amount of material pieces from a collection of material pieces. Alternatively, the separation position determination unit 15 may store the separation position in a storage unit such as the storage 163. Alternatively, the separation position determination unit 15 may display the separation position on the monitor 120 in the form of text, a computer graphic (CG) video or still image, or the like.

[0041] [Machine Learning] As described above, the density estimation unit 12 may calculate the voxel density or class value using the machine learning model 20. In one example, in order to execute the process flow S1, the learning unit 16 generates the machine learning model 20 in advance, and the density estimation unit 12 calculates the voxel density or class value using the machine learning model 20. The generation of the machine learning model 20 by the learning unit 16 corresponds to a learning phase. The use of the machine learning model 20 by the density estimation unit 12 corresponds to an operation phase.

[0042] As described above, the learning unit 16 generates the machine learning model 20 by machine learning. Machine learning refers to a method of autonomously finding laws or rules by repeatedly learning based on given information. The machine learning model 20 is constructed using algorithms and data structures. In one example, the machine learning model 20 is a computational model constructed by a neural network such as a convolutional neural network (CNN).

[0043] In one example, the learning unit 16 performs machine learning based on the error between the estimated class value of each pixel (voxel) set in the image and the correct answer corresponding to the estimated result, to generate a machine learning model 22. For this machine learning, an original image I org For each pixel in gt_x,y (I) is the associated mask image I mask are prepared as training data. The learning unit 16, for example, mask As training data, a given database is accessed to obtain each mask image I mask That is, the learning unit 16 performs the following process on the mask image I mask is input to a given machine learning model, and an estimated result of the class value of each pixel is output from the machine learning model. The learning unit 16 executes backpropagation based on the error between the estimated result and the correct answer for each class value, and updates the parameter group in the machine learning model. The learning unit 16 repeats this learning until a given termination condition is met, and obtains the machine learning model 22.

[0044] In one example, the learning unit 16 performs machine learning based on the error between the estimator of the subset and the correct answer corresponding to the estimator to generate a machine learning model 21. For this machine learning, a mask image I mask and separation position y obs and the correct answer m obs and the correct distance in the depth direction for each pixel z obs A record including (x, y) is prepared as training data. The learning unit 16, for example, accesses a given database that stores a plurality of such records as training data, and executes the following process for each record. That is, the learning unit 16 inputs the mask image to a given machine learning model, and obtains an estimation result of the voxel density of each pixel output from the machine learning model. Based on the estimation result, the learning unit 16 calculates the separation position y obs Then, the learning unit 16 calculates an estimate of the subset corresponding to the correct answer m obs The learning unit 16 executes backpropagation based on the error between the training data and the training data, and updates the parameter set in the machine learning model. The learning unit 16 repeats this learning until a given termination condition is satisfied, thereby obtaining the machine learning model 21.

[0045] Please note that the generated machine learning model 20 is a computational model that is estimated to be optimal, and is not necessarily a "computational model that is actually optimal."

[0046] The machine learning model 20 is portable between computer systems, so the density estimator 12 may use a machine learning model 20 that was created in another computer system.

[0047] [Second example] (System Configuration) The weighing system 2 according to an example is a computer system that determines an insertion position of a support for removing a collection of material pieces from a group of material pieces accumulated in a given area, and determines a separation position for separating a predetermined amount of material pieces from the collection. The insertion position refers to a position where the support is inserted into the group of material pieces. In one example, the insertion position is a position where the support is inserted into the group of material pieces to remove a predetermined amount or more of the group of material pieces from the group of material pieces. The support refers to a structure for supporting the group of material pieces. For example, the support may be a tool having a shape such as a scoop, spoon, or shovel.

[0048] FIG. 9 is a diagram showing an example of the functional configuration and application of the weighing system 2. In one example, the weighing system 2 is connected to the first robot 3, the second robot 4, the camera 5, and the weighing machine 6 via a communication network. At least a part of the communication network may be a wired network or a wireless network. At least a part of the communication network may be configured to include at least one of the Internet and an intranet. Alternatively, at least a part of the communication network may be simply realized by one communication cable. A first container 81 that contains a group of material pieces 90 is placed on the weighing machine 6. That is, the group of material pieces 90 is accumulated in a predetermined area called the first container 81. A second container 82 for receiving the weighed material pieces, i.e., a predetermined amount of material pieces, is placed near the first container 81.

[0049] The first robot 3 is a machine that takes out a collection of material pieces from a group of material pieces 90 in a first container 81, and transfers a predetermined amount of the material pieces obtained from the collection to a second container 82. The first robot 3 has a shovel-shaped support 3a as an end effector for performing this task.

[0050] The second robot 4 is a machine that separates a predetermined amount of material pieces from the collection of material pieces on the support 3a. In one example, the second robot 4 smoothes the collection of material pieces before the separation. The second robot 4 has a separation plate 4a as an end effector for performing these operations.

[0051] In one example, the outer edge of the partition plate 4a is formed to correspond to the shape of the inner surface of the support 3a. For example, in response to the support 3a having a flat bottom surface, the lower end of the partition plate 4a extends linearly along the horizontal direction. In this case, the lower end can also be used to level the collection of material pieces.

[0052] The camera 5 is an imaging device that captures an area image that is an image of the material piece group 90 and an aggregate image that is an image of an aggregate of the material pieces. The camera 5 may be an RGB-D camera that outputs both a color image (RGB) and a depth (D).

[0053] The weighing device 6 is a device that measures the mass of the first container 81 that contains the material piece group 90. The weighing device 6 has a precision that allows it to measure the amount of reduction in mass of the material piece group 90 when the collection of material pieces is scooped up from the material piece group 90 by the support 3a.

[0054] The weighing system 2 comprises, as functional components, a take-out function 30, a separation function 40, and a robot control unit 50. The take-out function 30 is a group of functional modules for taking out a collection of material pieces from a material piece group 90. The separation function 40 is a group of functional modules for separating a predetermined amount of material pieces from the collection. The robot control unit 50 is a functional module for controlling the first robot 3 and the second robot 4.

[0055] The take-out function 30 includes an insertion position determination unit 31, a shape specification unit 32, a path setting unit 33, and an amount calculation unit 34. The insertion position determination unit 31 is a functional module that determines an insertion position of the support 3a for taking out a collection of material pieces from the material piece group 90 based on an area image. The shape specification unit 32 is a functional module that specifies the shape of the upper surface of the material piece group 90 based on an area image. The path setting unit 33 is a functional module that sets a virtual path of the support 3a passing through the material piece group 90. The amount calculation unit 34 is a functional module that calculates the amount of the collection of material pieces taken out by the support 3a passing through the virtual path as the take-out amount. The insertion position determination unit 31 determines the insertion position in cooperation with at least one of the shape specification unit 32, the path setting unit 33, and the amount calculation unit 34. For example, the insertion position determination unit 31 may set at least one insertion candidate position (e.g., a plurality of insertion candidate positions) that are candidates for the insertion position, and determine the insertion position based on the take-out amount of each insertion candidate position. Alternatively, the insertion position determination unit 31 may determine the insertion position based on the shape of the upper surface of the material piece group 90. In the example shown in Fig. 9, the insertion position determination unit 31 has a shape specification unit 32, a path setting unit 33, and an amount calculation unit 34 as sub-modules. The hierarchical relationship between these four functional modules is not limited, and for example, the insertion position determination unit 31, the shape specification unit 32, the path setting unit 33, and the amount calculation unit 34 may be independent of each other while cooperating with each other. In any case, the removal function 30 is an example of a support control system.

[0056] The separation function 40 includes a density estimation unit 41, an amount estimation unit 42, a correction unit 43, and a separation position determination unit 44. The density estimation unit 41 is a functional module that estimates the density of a set of material pieces. The amount estimation unit 42 is a functional module that calculates an estimated amount of a subset of material pieces obtained by separating the set of material pieces based on the density. The amount estimation unit 42 sets at least one separation candidate position (e.g., a plurality of separation candidate positions) that are candidates for the separation position, and calculates an estimated amount for each separation candidate position. The correction unit 43 is a functional module that corrects the estimated amount as necessary. The separation position determination unit 44 is a functional module that determines a separation position based on the estimated density. For example, the separation position determination unit 44 determines a separation position from at least one separation candidate position based on at least one estimated amount calculated using the density.

[0057] The robot control unit 50 controls the first robot 3 or the second robot 4 based on the output from the removal function 30 or the separation function 40. The robot control unit 50 controls the first robot 3 based on the insertion position determined by the insertion position determination unit 31, and the first robot 3 scoops up the collection of material pieces from the group of material pieces 90 with the support 3a. The robot control unit 50 may control the second robot 4 to cause the second robot 4 to perform a process of smoothing the upper surface of the collection of material pieces. The robot control unit 50 controls the second robot 4 based on the separation position determined by the separation position determination unit 44, and the second robot 4 separates a predetermined amount of material pieces from the collection of material pieces with the partition plate 4a. The robot control unit 50 controls the first robot 3 to cause the first robot 3 to perform a process of transferring a predetermined amount of material pieces into the second container 82.

[0058] Each functional module of the weighing system 2 is realized by loading a weighing program onto the processor 161 or memory 162 and having the processor 161 execute the program. The weighing program includes code for realizing each functional module of the weighing system 2. The processor 161 operates the input / output port 164 or the communication port 165 in accordance with the weighing program, and executes reading and writing of data in the memory 162 or the storage 163.

[0059] The weighing program may be provided in a form separated into a first program (support control program) for realizing the removal function 30 and a second program for realizing the separation function 40. Both the first program and the second program may be provided by a recording medium or via a communication network.

[0060] (Support Control Method) As an example of a support control method according to the present disclosure, a method for determining an insertion position of a support into a group of material pieces 90 will be described with reference to Fig. 10. Fig. 10 is a flow chart showing an example of the process by the weighing system 2 as process flow S2. That is, the weighing system 2 executes process flow S2.

[0061] In step S21, the insertion position determination unit 31 acquires an area image capturing the material piece group 90 from the camera 5. This area image captures at least the top surface (upper exposed surface) of the material piece group 90. The area image may be a still image or a frame image constituting a video. In one example, the insertion position determination unit 31 acquires a depth image capturing the material piece group 90. Alternatively, the insertion position determination unit 31 may acquire a set of an RGB image capturing the material piece group 90 and a depth image. When the camera 5 captures the area image, the robot control unit 50 controls the posture of each of the first robot 3 and the second robot 4 so that the material piece group 90 is not blocked by the first robot 3 and the second robot 4.

[0062] In step S22, the insertion position determination unit 31 calculates the take-out amount for each of the multiple insertion candidate positions based on the area image. In one example, the path setting unit 33 sets multiple insertion candidate positions for the material piece group 90 shown in the area image, and sets a virtual path by simulation for each insertion candidate position. The start point of the virtual path is the insertion candidate position, and the end point of the virtual path is the position of the support 3a at the time when the support 3a exits the material piece group 90. For each of the multiple virtual paths, the amount calculation unit 34 calculates the take-out amount obtained by the support 3a moving along the virtual path.

[0063] FIG. 11 is a diagram showing an example of setting of insertion candidate positions. For example, the path setting unit 33 sets a plurality of insertion candidate positions 600 along a coordinate axis (Y axis in FIG. 11) corresponding to the moving direction of the support 3a in the horizontal direction. Alternatively, the path setting unit 33 may set a plurality of insertion candidate positions 600 along two coordinate axes (X axis and Y axis in FIG. 11) in the horizontal direction. In one example, the path setting unit 33 sets a plurality of insertion candidate positions 600 within a range in which the support 3a can be inserted and the set of material pieces can be taken out of the area in which the material piece group 90 is prepared. For example, the path setting unit 33 sets a plurality of insertion candidate positions 600 within a range in which the support 3a can be inserted into the material piece group 90 without interfering with the first container 81 and the set of material pieces can be taken out. The path setting unit 33 may set the insertion candidate positions 600 at a predetermined interval. Alternatively, the shape specifying unit 32 may specify the shape of the upper surface 91 of the material piece group 90 based on the area image, and the path setting unit 33 may set a plurality of insertion candidate positions 600 based on the shape. For example, the path setting unit 33 may set the insertion candidate position 600 near the base of a raised portion of the upper surface 91, or may set the insertion candidate position 600 in a recessed portion of the upper surface 91. In other words, the path setting unit 33 may set a plurality of insertion candidate positions 600 based on the unevenness of the upper surface 91.

[0064] FIG. 12 is a diagram showing an example of a virtual path. This diagram shows a virtual path of the support 3a that is inserted into the material piece group 90 from the insertion candidate position 600 and then removed from the material piece group 90. In state S101, the tip of the bottom surface of the tilted support 3a is in contact with the material piece group 90 at the insertion candidate position 600. In state S102, the support 3a is inserted into the material piece group 90 along the vertical direction. Next, from state S103 to state S104, the inserted support 3a is rotated until the bottom surface becomes horizontal while maintaining the position of the tip of the bottom surface in the vertical direction. In state S105, the horizontal support 3a is moved forward a predetermined distance. In state S106, the moved support 3a is pulled up to above the material piece group 90. A collection of material pieces 95 is placed on the pulled up support 3a. In one example, the path setting unit 33 calculates the path of the support 3a moving through states S101 to S106 by geometric calculation. The path setting unit 33 may perform the geometric calculation taking into consideration at least one of the dimensions of the support 3a, the tilt angle of the support 3a at the time of insertion, the dimensions of the first container 81, and the distance between the inner circumferential surface of the first container 81 and the outer circumferential surface of the support 3a. The dimensions of the support 3a include at least one of the length, width, and depth. The dimensions of the first container 81 include at least one of the length, width, and depth (height).

[0065] The amount calculation unit 34 calculates the amount of material taken out by the support 3a moving along each of the virtual routes for each of the multiple virtual routes. This amount of material taken out corresponds to the insertion candidate position that is the starting point of the virtual route. In one example, the amount calculation unit 34 calculates the average of the distance from the bottom surface of the support 3a to the top surface of the material piece group 90 on the virtual route (i.e., the average height of the top surface of the material piece group 90 based on the bottom surface of the support 3a) as the amount of material taken out. Alternatively, the amount calculation unit 34 may calculate the total volume of the material pieces located above the bottom surface of the support 3a on the virtual route as the amount of material taken out. Alternatively, the amount calculation unit 34 may calculate the mass obtained from the total volume and a preset density of the material piece group as the amount of material taken out. Even when the separation position is determined based on the mass, the amount of material taken out may be expressed by the volume or average height. For example, a range of the volume or average height in which a predetermined amount of material pieces can be expected to be obtained in the end is set in advance in relation to the volume of the support 3a. The amount calculation unit 34 can use the total volume or average height as the amount of material taken out based on the set range.

[0066] Returning to FIG. 10, in step S23, the insertion position determination unit 31 determines an insertion position from among a plurality of insertion candidate positions based on the calculated plurality of removal amounts. In one example, the insertion position determination unit 31 determines, as the insertion position, one insertion candidate position corresponding to a removal amount that can finally separate a predetermined amount of material pieces. That is, the insertion position determination unit 31 may determine, as the insertion position, one insertion candidate position corresponding to a removal amount equal to or greater than the predetermined amount. For example, the insertion position determination unit 31 may determine, as the insertion position, an insertion candidate position corresponding to the maximum value of the calculated plurality of removal amounts. Alternatively, the insertion position determination unit 31 may determine an insertion position based on the difference between the calculated plurality of removal amounts and the predetermined amount, and may determine, for example, an insertion candidate position with the smallest difference as the insertion position.

[0067] In step S24, the insertion position determination unit 31 outputs the determined insertion position. In one example, the insertion position determination unit 31 outputs the determined insertion position to the robot control unit 50.

[0068] (Measuring method) As an example of a weighing method according to the present disclosure, the determination of a separation position for obtaining a predetermined amount of material pieces will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the process as process flow S3. The weighing system 2 executes process flow S3 after the first robot 3 removes the collection of material pieces from the first container 81 using the support 3a based on the insertion position determined by process flow S2.

[0069] In step S31, the density estimation unit 41 obtains the amount of reduction in the mass of the material piece group 90. The density estimation unit 41 obtains the mass of the first container 81 before the collection of material pieces is taken out and the mass of the first container 81 after the collection is taken out from the weighing device 6. Then, the density estimation unit 41 obtains the difference between the two masses as the amount of reduction.

[0070] In step S32, the density estimation unit 41 estimates the volume of the collection of material pieces that have been taken out. In one example, the density estimation unit 41 acquires a collection image captured by the camera 5, and similar to step S12 above, sets a plurality of voxels for the collection of material pieces captured in the collection image. Then, the density estimation unit 41 calculates the volume (i.e., the voxel volume) for each of the plurality of voxels. As described in the first example, the voxel volume is expressed as z(x, y)ΔxΔy. The density estimation unit 41 calculates the sum of the plurality of voxel volumes as the volume of the collection of material pieces.

[0071] In step S33, the density estimation unit 41 estimates the density of the collection of material pieces based on the amount of reduction and the volume.

[0072] In step S34, the quantity estimation unit 42 calculates an estimated quantity of a subset of material pieces for each of the plurality of separation candidate positions. In one example, the quantity estimation unit 42 calculates an estimated quantity (e.g., an estimated mass) for each separation candidate position, similar to step S14 above. That is, the quantity estimation unit 42 sets a plurality of separation candidate positions, and calculates an estimated quantity for each separation candidate position based on a plurality of voxel quantities corresponding to a plurality of voxels. The quantity estimation unit 42 may set a reference position for setting the separation candidate positions to the leading end or trailing end of the support 3a. The quantity estimation unit 42 uses the density estimated in step S33 as the voxel density, and therefore the voxel density is the same among the plurality of voxels. The quantity estimation unit 42 may calculate the product of the voxel density and the voxel volume as the voxel quantity without using the class value. In step S34, the correction unit 43 may correct the estimated quantity for each of the plurality of separation candidate positions. This correction is also similar to step S14.

[0073] In step S35, the separation position determination unit 44 determines a separation position from a plurality of separation candidate positions. This process is the same as step S15 described above. The separation position determination unit 44 may determine the separation candidate position that minimizes the error between the estimated amount and the predetermined amount as the separation position. In the determination based on mass, the separation position determination unit 44 may determine the separation candidate position that minimizes the error between the estimated mass and the predetermined mass as the separation position.

[0074] In step S36, the separation position determination unit 44 outputs the determined separation position. In one example, the insertion position determination unit 31 outputs the separation position to the robot control unit 50.

[0075] (Metering method including support control method) A series of processes for extracting a set of material pieces from the material piece group 90 and obtaining a predetermined amount of material pieces from the set will be described with reference to Fig. 14. Fig. 14 is a flow chart showing an example of the process as process flow S4. That is, the weighing system 2 executes process flow S4.

[0076] In step S41, the unloading function 30 of the weighing system 2 determines the insertion position of the support 3a into the material piece group 90. This process corresponds to the process flow S2 described above.

[0077] In step S42, the robot control unit 50 controls the first robot 3 based on the insertion position to pick up the collection of material pieces from the group of material pieces 90. In one example, the robot control unit 50 generates a path indicating a trajectory of the first robot 3 for inserting the support 3a from the insertion position into the group of material pieces 90 to pick up the collection of material pieces. The robot control unit 50 may adopt a virtual route corresponding to the determined insertion position as the path. The robot control unit 50 generates an instruction signal based on the path and transmits the instruction signal to the first robot 3. In accordance with the instruction signal, the first robot 3 inserts the support 3a into the insertion position on the group of material pieces 90, advances the support 3a within the group of material pieces 90 (for example, as shown in FIG. 12), and scoops up the collection of material pieces from the group of material pieces 90. In one example, after picking up the collection of material pieces, the robot control unit 50 controls the second robot 4 to smooth the top surface of the collection. For example, the robot control unit 50 generates a path indicating the trajectory of the second robot 4 for smoothing the top surface of the collection of material pieces with the partition plate 4a, based on the collection image obtained from the camera 5. The robot control unit 50 generates an instruction signal based on the path, and transmits the instruction signal to the second robot 4. The second robot 4 smooths the top surface of the collection of material pieces with the partition plate 4a in accordance with the instruction signal.

[0078] In step S43, the separation function 40 of the weighing system 2 determines a separation position for obtaining a predetermined amount of material pieces. This process corresponds to the process flow S3 described above.

[0079] In step S44, the robot control unit 50 controls the second robot 4 based on the separation position to separate a predetermined amount of material pieces from the collection of material pieces. In one example, the robot control unit 50 generates a path indicating a trajectory of the second robot 4 for inserting the partition plate 4a into the collection of material pieces at the separation position to separate the collection into a portion of a predetermined amount (hereinafter, this will be referred to as a "designated portion") and a remaining portion. The robot control unit 50 generates an instruction signal based on the path and transmits the instruction signal to the second robot 4. In accordance with the instruction signal, the second robot 4 inserts the partition plate 4a into the separation position on the collection of material pieces and lowers the partition plate 4a to the bottom surface of the support 3a. The robot control unit 50 further generates a first path indicating a trajectory of the first robot 3 and a second path indicating a trajectory of the second robot 4 to transfer the predetermined amount of material pieces to the second container 82. For example, the robot control unit 50 may generate the first pass and the second pass to realize a series of operations in which the remaining portion is returned to the first container 81 by the partition plate 4a, a designated portion is left on the support 3a, and the designated portion is transferred to the second container 82. Alternatively, for example, the robot control unit 50 may generate the first pass and the second pass to realize a series of operations in which the designated portion is transferred to the second container 82 by the partition plate 4a, and then the remaining portion on the support 3a is returned to the first container 81. The robot control unit 50 generates a first instruction signal based on the first pass and transmits the first instruction signal to the first robot 3. In addition, the robot control unit 50 generates a second instruction signal based on the second pass and transmits the second instruction signal to the second robot 4. The first robot 3 and the second robot 4 work together according to these instruction signals to transfer a predetermined amount of material pieces to the second container 82.

[0080] The process flow S4 may be repeatedly executed on the group of material pieces 90. In this case, the material pieces are transferred from the first container 81 to the second container 82 in predetermined amounts.

[0081] [Variations] The present disclosure has been described in detail above based on various examples. However, the present disclosure is not limited to the above examples. The present disclosure can be modified in various ways without departing from the spirit and scope of the present disclosure.

[0082] The metrology system may estimate the density of a collection of material pieces and determine separation locations based on the density without setting voxels for the collection image.

[0083] The metrology system may directly determine the voxel volume for each of a plurality of voxels without calculating the voxel density.

[0084] The weighing system may determine the separation location based on the density of the collection of material pieces without setting multiple candidate separation locations, or the weighing system may set one candidate separation location, calculate an estimate of the collection of material pieces corresponding to the candidate separation location, and determine the separation location based on the estimate.

[0085] As described above, the density of the material pieces processed by the weighing system or the support control system may be uniform or may vary depending on the location within the collection of material pieces. In general, examples of material pieces with uniform density include granular or powdered food products, and examples of material pieces with different densities include cut food products. Regardless of the type of material pieces, the weighing system may estimate the density of the material pieces by either the method in the first example or the method in the second example described above. If it is assumed that the density of the material pieces is uniform, the weighing system may determine the separation position based on a preset density without estimating the density.

[0086] The insertion position determination unit may set one candidate insertion position, calculate the removal amount corresponding to the candidate insertion position, and determine the insertion position based on the removal amount. Alternatively, the insertion position determination unit may identify the shape of the upper surface of the material piece group based on the area image and determine the insertion position based on the shape, without setting a candidate insertion position or estimating the removal amount. For example, the insertion position determination unit may determine a position near the base of a raised portion of the upper surface of the material piece group as the insertion position, or may set a recessed portion of the upper surface as the insertion position. In other words, the insertion position determination unit may determine the insertion position based on the unevenness of the upper surface.

[0087] The ejection function 30 and the separation function 40 in the second example may be provided independently. For example, the ejection function 30 may be provided as a support control system. Alternatively, the separation function 40 may be provided as a weighing system without the ejection function 30.

[0088] The weighing system may not include a separation location determiner, in which case the weighing system estimates the quantity of a subset of the pieces of material as in the first or second example above and outputs the estimate.

[0089] The hardware configuration of the weighing system is not limited to the implementation of each functional module by executing a program. For example, at least a part of the functional modules may be configured with a logic circuit specialized for the function, or may be configured with an ASIC (Application Specific Integrated Circuit) that integrates the logic circuit.

[0090] The processing steps of the method executed by at least one processor are not limited to the above examples. For example, some of the steps or processes described above may be omitted, or each step may be executed in a different order. In addition, two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to each of the steps described above.

[0091] When comparing the magnitude of two numbers within a computer system or computer, either of the two criteria "greater than or equal to" and "greater than" can be used, or either of the two criteria "less than or equal to" and "less than" can be used.

[0092] [First Note] As can be seen from the various examples above, the present disclosure includes the following aspects. (Appendix 1) a quantity estimator that calculates, for each of at least one separation candidate position based on a set image of a set of material pieces managed by mass, an estimated quantity of a subset of the material pieces obtained by separating the set at the separation candidate position; a separation position determination unit that determines a position at which a predetermined amount of the material pieces are separated from the set as a separation position based on at least one of the estimated quantities; A weighing system comprising: (Appendix 2) an insertion position determination unit that determines an insertion position of a support for removing the predetermined amount or more of the material pieces from the material pieces based on an area image of the material pieces accumulated in a given area; the amount estimation unit calculates the estimated amount for each of the at least one separation candidate position based on the set image capturing the set taken out by the support inserted into the material piece group from the insertion position; 4. A weighing system as described in Appendix 1. (Appendix 3) The insertion position determination unit is establishing a virtual path for the support through the stack of material; Calculating the amount of the set taken out by the support passing through the virtual path as a take-out amount; determining the insertion position based on the amount of removal; A weighing system as described in Appendix 2. (Appendix 4) The insertion position determination unit is Identifying a shape of an upper surface of the group of material pieces based on the area image; determining the insertion position based on the shape of the upper surface; 2. A weighing system as described in appendix 2 or 3. (Appendix 5) An image processing unit that sets a plurality of voxels for the set captured in the set image, The quantity estimator, For each of the plurality of voxels, estimating a volume of the piece of material corresponding to the voxel as a voxel volume; calculating the estimated quantity based on a plurality of the voxel quantities; A weighing system according to any one of appendices 1 to 4. (Appendix 6) the at least one separation candidate location includes a plurality of separation candidate locations; the separation position determination unit determines, from the plurality of separation candidate positions, the separation candidate position that minimizes an error between the estimated amount and the predetermined amount as the separation position. 6. A weighing system according to any one of claims 1 to 5. (Appendix 7) the quantity estimator sets the plurality of separation candidate positions along one coordinate axis of a coordinate system indicating a real space in which the set is located; The separation position determination unit determines the separation position from the plurality of separation candidate positions. A weighing system as described in Appendix 6. (Appendix 8) A density estimation unit that estimates a density of the collection based on the collection image, the quantity estimator calculates the estimated quantity based on the density and the volume of the subset; A weighing system according to any one of appendices 1 to 7. (Appendix 9) the aggregate image is a depth image, The quantity estimation unit calculates the volume based on a distance in a depth direction obtained from the depth image. A weighing system as described in Appendix 8. (Appendix 10) the density estimation unit estimates the density based on a mass reduction amount of the material pieces when the collection is taken out from the material pieces accumulated in a given area and a volume of the taken out collection. 6. A weighing system as described in appendix 8 or 9. (Appendix 11) An image processing unit that sets a plurality of voxels for the set captured in the set image, the density estimation unit calculates, for each of the plurality of voxels, a density of the material piece in the voxel as a voxel density; The quantity estimator, calculating, for each of the plurality of voxels, an amount of the material piece in the voxel as a voxel amount based on the voxel density; calculating the estimator based on a plurality of the voxel quantities; A weighing system as described in Appendix 8. (Appendix 12) the density estimation unit calculates the voxel density for each of the plurality of voxels using a machine learning model that receives an input of the set image and outputs the voxel density; 13. A weighing system as described in Appendix 11. (Appendix 13) 13. The weighing system of claim 12, further comprising a learning unit that performs machine learning based on an error between an estimate of the subset and a correct answer corresponding to the estimate to generate the machine learning model. (Appendix 14) the density estimator estimates, for each of the plurality of voxels, a class value indicating whether the material piece is present in the voxel; the quantity estimator calculates the voxel quantity for each of the plurality of voxels based on the class value and the voxel density; 13. A weighing system as described in Appendix 11. (Appendix 15) the density estimation unit calculates the class value for each of the plurality of voxels using a machine learning model that receives an input of the set image and outputs the class value; 14. A weighing system as described in Appendix 14. (Appendix 16) a correction unit that corrects the estimated amount based on a relationship between a sample group of the estimated amount previously calculated by the amount estimation unit and a sample group of an actual amount of the material pieces actually separated in the past at the separation position corresponding to the estimated amount, the separation position determination unit determines the separation position based on the corrected plurality of estimated quantities. Attachment 1 to 15. A weighing system according to any one of claims 1 to 15. (Appendix 17) a path setting unit that sets a virtual path for a support passing through a group of material pieces that are managed by mass and that are accumulated in a given area based on an area image of the group of material pieces; an insertion position determination unit that determines an insertion position of the support for removing a set of material pieces managed by mass from the group of material pieces based on the virtual path; A support control system comprising: (Appendix 18) an image processing unit that sets a plurality of voxels for a set of material pieces managed by mass based on a set image of the set; a density estimation unit that calculates, for each of the plurality of voxels, a density of the material piece in the voxel as a voxel density; a voxel amount calculation unit that calculates, for each of the plurality of voxels, an amount of the material piece in the voxel as a voxel amount based on the voxel density; a quantity estimator that calculates an estimated quantity of a subset of the material pieces obtained by separating the set based on a plurality of the voxel quantities; A weighing system comprising: (Appendix 19) 1. A weighing method performed by a weighing system comprising at least one processor, comprising: calculating, for each of at least one candidate separation location based on a collection image of the collection of material pieces managed by mass, an estimate of a subset of the material pieces obtained by separating the collection at the candidate separation location; determining a separation location for separating a predetermined amount of the piece of material from the collection based on at least one of the estimated quantities; A weighing method including: (Appendix 20) calculating, for each of at least one candidate separation location based on a collection image of the collection of material pieces managed by mass, an estimate of a subset of the material pieces obtained by separating the collection at the candidate separation location; determining a separation location for separating a predetermined amount of the piece of material from the collection based on at least one of the estimated quantities; A weighing program that causes a computer to execute the following.

[0093] According to Supplementary Notes 1, 19, and 20, an estimated amount at each separation candidate position is calculated based on the collective image, and a separation position is determined based on the estimated amount. Thus, a predetermined amount of material pieces can be identified.

[0094] According to Supplementary Note 2, the insertion position of the support for removing a set of material pieces having a predetermined amount or more from the material piece group is determined based on an area image of the material piece group. This process makes it possible to reliably remove a set from the material piece group that will ultimately give the predetermined amount of material pieces.

[0095] According to Appendix 3, the insertion position is determined taking into account the path of the support within the material piece group and the amount of material to be removed corresponding to that path, making it possible to more reliably remove a collection of material pieces of a predetermined amount or more from the material piece group.

[0096] According to Supplementary Note 4, since the insertion position is determined based on the shape of the upper surface of the material piece group, the insertion position can be appropriately determined according to the condition of the material piece group.

[0097] According to Supplementary Note 5, multiple voxels are set for the set image, and the amount in each voxel is estimated. This process allows a predetermined amount of material pieces to be identified even if the distribution of material pieces in the set is non-uniform.

[0098] According to Supplementary Note 6, for each of the plurality of separation candidate positions, the estimated amount of the material pieces to be sorted is compared with the predetermined amount, and the separation candidate position with the smallest error between the two is determined as the separation position. Therefore, the material pieces of the predetermined amount can be more reliably identified.

[0099] According to Supplementary Note 7, since a plurality of separation candidate positions are set along one coordinate axis, the setting of separation candidate positions becomes simple, and accordingly the separation positions can be determined easily.

[0100] According to Supplementary Note 8, the density of the set of material pieces is estimated based on the set image, and the estimated amount is calculated based on the density and the volume of the subset. Therefore, the estimated amount of the subset can be calculated more accurately.

[0101] According to Supplementary Note 9, the volume can be calculated with high accuracy by using a depth image, and the accuracy of the calculation of the estimated amount can be improved accordingly.

[0102] According to Appendix 10, by using the amount of mass reduction of the group of material pieces when the group of material pieces is removed as the mass of the group, it is not necessary to actually measure the mass of the group of material pieces, and therefore it is possible to easily obtain an estimated mass of the group of material pieces.

[0103] According to Appendix 11, by setting multiple voxels for the set image and determining the density and amount of material pieces in each voxel, the separation position can be determined with high accuracy even if the density of material pieces in the set is non-uniform.

[0104] According to Appendix 12, by using machine learning, it is possible to accurately determine the voxel density in a manner that is adaptable to the diverse distribution of material pieces.

[0105] According to Addendum 13, the learning load can be reduced by training a machine learning model that outputs voxel densities based on estimates of subsets rather than values ​​for individual voxels.

[0106] According to Supplementary Note 14, by considering the presence or absence of material fragments in each voxel, voxels that should not be considered can be excluded, and the estimated amount of material fragments to be separated can be calculated with high accuracy. Therefore, it is possible to further improve the accuracy of the separation position.

[0107] According to Appendix 15, by using machine learning, class values ​​can be determined with high accuracy for diverse distributions of material pieces.

[0108] According to Appendix 16, the calculated estimated amount is corrected based on the relationship between the estimated amount and the actual amount obtained in the past, so that the accuracy of the separation position can be further improved.

[0109] According to Addendum 17, since the insertion position is determined taking into consideration the path of the support within the group of material pieces, the insertion position of the support for removing the group of material pieces can be appropriately determined.

[0110] According to Supplementary Note 18, multiple voxels are set for the set image, and the density and amount of material particles in each voxel are calculated. This process allows accurate calculation of the estimated amount of the material particle set even when the density within the set is non-uniform.

[0111] [Second Note] As can be seen from the various examples above, the present disclosure also includes the following aspects. (Appendix 1) a density estimation unit that estimates the density of a collection of material pieces managed by mass based on an image of the collection; a determination unit that determines a position for separating a predetermined amount of the material pieces from the collection based on the density as a separation position; A weighing system comprising: (Appendix 2) a quantity estimator that calculates, for each of a plurality of candidate positions that are candidates for the separation position based on the density, an estimated quantity of a subset of the material pieces obtained by separating the set at the candidate position; the determination unit determines, from the plurality of candidate positions, the candidate position that minimizes an error between the estimated amount and the predetermined amount as the separation position. 2. A weighing system as described in Appendix 1. (Appendix 3) the quantity estimator sets the plurality of candidate positions along one coordinate axis of a coordinate system indicating a real space in which the set is located; The determination unit determines the separation position from the plurality of candidate positions. A weighing system as described in Appendix 2. (Appendix 4) the quantity estimator calculates an estimated mass of the subset based on the density and a volume of the subset as the estimated quantity; The determination unit determines, as the separation position, the candidate position that minimizes the error between the estimated mass and a predetermined mass. 2. A weighing system as described in appendix 2 or 3. (Appendix 5) the image is a depth image, The quantity estimation unit calculates the volume based on a distance in a depth direction obtained from the depth image. A weighing system as described in Appendix 4. (Appendix 6) An image processing unit is further provided for setting a plurality of voxels for the set captured in the image, The quantity estimator, calculating, for each of the plurality of voxels, an amount of the material piece in the voxel as a voxel amount; calculating the estimate for the subset based on the voxel volume for each of the plurality of voxels; A weighing system according to any one of appendixes 2 to 5. (Appendix 7) the density estimation unit calculates, for each of the plurality of voxels, a density of the material piece in the voxel as a voxel density; the quantity estimator calculates the voxel quantity for each of the plurality of voxels based on the voxel density; A weighing system as described in Appendix 6. (Appendix 8) the density estimation unit calculates the voxel density for each of the plurality of voxels using a machine learning model that receives an input of the image and outputs the voxel density; A weighing system as described in Appendix 7. (Appendix 9) 9. The weighing system of claim 8, further comprising a learning unit that performs machine learning based on an error between an estimate of the subset and a correct answer corresponding to the estimate to generate the machine learning model. (Appendix 10) the density estimator estimates, for each of the plurality of voxels, a class value indicating whether the material piece is present in the voxel; the quantity estimator calculates the voxel quantity for each of the plurality of voxels based on the class value and the voxel density; A weighing system according to any one of appendices 7 to 9. (Appendix 11) the density estimation unit calculates the class value for each of the plurality of voxels using a machine learning model that receives an input of the image and outputs the class value; 10. A weighing system as described in claim 10. (Appendix 12) a correction unit that corrects the estimated amount based on a relationship between a sample group of the estimated amount previously calculated by the amount estimation unit and a sample group of an actual amount of the material pieces actually separated in the past at the separation position corresponding to the estimated amount, the determination unit determines, as the separation position, the candidate position that minimizes an error between the corrected estimated amount and the predetermined amount. A weighing system according to any one of appendixes 2 to 11. (Appendix 13) an image processing unit that sets a plurality of voxels for a collection of material pieces managed by mass based on an image of the collection; an amount estimator for estimating, for each of the plurality of voxels, an amount of the material piece corresponding to the voxel; a determination unit that determines a separation position at which a predetermined amount of the material piece is separated from the set based on the estimated amount of each of the plurality of voxels; A weighing system comprising: (Appendix 14) The material pieces are quantitatively controlled food products; the determination unit determines the separation location for adding the predetermined amount of the food product to a predetermined food product. A weighing system according to any one of appendixes 1 to 13. (Appendix 15) 1. A weighing method performed by a weighing system comprising at least one processor, comprising: estimating a density of a collection of mass-controlled pieces of material based on an image of the collection; determining a separation location for separating a predetermined amount of the material pieces from the collection based on the density; A weighing method including: (Appendix 16) estimating a density of a collection of mass-controlled pieces of material based on an image of the collection; determining a separation location for separating a predetermined amount of the material pieces from the collection based on the density; A weighing program that causes a computer to execute the following. (Appendix 17) 1. A weighing method performed by a weighing system comprising at least one processor, comprising: determining a plurality of voxels for a collection of mass-managed pieces of material based on an image of the collection; for each of the plurality of voxels, estimating a quantity of the piece of material corresponding to the voxel; determining a separation location for separating a predetermined amount of the piece of material from the set based on the estimated amount for each of the plurality of voxels; A weighing method including: (Appendix 18) determining a plurality of voxels for a collection of mass-managed pieces of material based on an image of the collection; for each of the plurality of voxels, estimating a quantity of the piece of material corresponding to the voxel; determining a separation location for separating a predetermined amount of the piece of material from the set based on the estimated amount for each of the plurality of voxels; A weighing program that causes a computer to execute the following.

[0112] According to Supplementary Notes 1, 15, and 16, the density of the collection of material pieces is estimated based on the image, and the separation position is determined based on the density. Thus, a predetermined amount of material pieces can be identified.

[0113] According to Supplementary Note 2, for each of the multiple candidate positions, the estimated amount of the material piece to be separated is compared with the predetermined amount, and the candidate position with the smallest error between the two is determined as the separation position, so that the predetermined amount of the material piece can be more reliably identified.

[0114] According to Supplementary Note 3, since a plurality of candidate positions are set along one coordinate axis, the candidate positions can be set easily, and accordingly the separation position can be determined easily.

[0115] According to Appendix 4, since the separation position is set based on the mass, the separation position can be determined with high accuracy in accordance with the properties of the material pieces.

[0116] According to Appendix 5, the volume can be calculated with high accuracy by using the depth image, and the accuracy of the mass calculation can be improved accordingly.

[0117] According to Supplementary Note 6, by setting multiple voxels in an image and calculating the voxel amount in each voxel, the separation position can be determined with high accuracy even when the distribution of material pieces in the set is non-uniform.

[0118] According to Supplementary Note 7, by taking the voxel density into consideration, the voxel amount can be calculated with high accuracy, and the separation position can be determined with high accuracy accordingly.

[0119] According to Appendix 8, by using machine learning, it is possible to accurately determine the voxel density in a manner that is adaptable to the diverse distribution of material pieces.

[0120] According to Appendix 9, the learning load can be reduced by training a machine learning model that outputs voxel densities based on estimates of subsets rather than values ​​for individual voxels.

[0121] According to Supplementary Note 10, by considering the presence or absence of material fragments in each voxel, voxels that should not be considered can be excluded, and the estimated amount of material fragments to be separated can be calculated with high accuracy. Therefore, it is possible to further improve the accuracy of the separation position.

[0122] According to Appendix 11, by using machine learning, class values ​​can be accurately determined for diverse distributions of material pieces.

[0123] According to Appendix 12, the calculated estimated amount is corrected based on the relationship between the estimated amount and the actual amount obtained in the past, so that the accuracy of the separation position can be further improved.

[0124] According to Supplementary Notes 13, 17, and 18, a plurality of voxels are set for an image, the amount in each voxel is estimated, and the separation position is determined based on these amounts. This process allows the identification of a predetermined amount of material pieces even when the distribution of the material pieces in the set is non-uniform.

[0125] According to Appendix 14, a predetermined amount of food can be specified, which in turn allows for easier production of the food product. [Explanation of symbols]

[0126] Reference Signs List 1, 2...weighing system, 3...first robot, 3a...support, 4...second robot, 4a...partition plate, 5...camera, 6...weighing device, 11...image processing unit, 12...density estimation unit, 13...amount estimation unit, 14...correction unit, 15...separation position determination unit, 16...learning unit, 20-22...machine learning model, 30...removal function, 31...insertion position determination unit, 32...shape identification unit, 33...path setting unit, 34...amount calculation unit, 40...separation function, 41...density estimation unit, 42...amount estimation unit, 43...correction unit, 44...separation position determination unit, 50...robot control unit, 81...first container, 82...second container, 90...group of material pieces, 210...food, 200...support, 220...separation position, 230...reference position, 240...separation candidate position, 300...voxel, 600...insertion candidate position.

Claims

1. a quantity estimator that calculates, for each of at least one separation candidate position based on a set image of a set of material pieces managed by mass, an estimated quantity of a subset of the material pieces obtained by separating the set at the separation candidate position; a separation position determination unit that determines a position at which a predetermined amount of the material pieces are separated from the set as a separation position based on at least one of the estimated quantities; A weighing system comprising:

2. an insertion position determination unit that determines an insertion position of a support for removing the predetermined amount or more of the material pieces from the material pieces based on an area image of the material pieces accumulated in a given area; the quantity estimation unit calculates the estimated quantity for each of the at least one separation candidate position based on the set image capturing the set taken out by the support inserted into the material piece group from the insertion position; 2. The weighing system according to claim 1 .

3. The insertion position determination unit is establishing a virtual path for the support through the stack of material; Calculating the amount of the set taken out by the support passing through the virtual path as a take-out amount; determining the insertion position based on the amount of removal; The weighing system of claim 2.

4. The insertion position determination unit is Identifying a shape of an upper surface of the group of material pieces based on the area image; determining the insertion position based on the shape of the upper surface; The weighing system of claim 2.

5. An image processing unit that sets a plurality of voxels for the set captured in the set image, The quantity estimator, For each of the plurality of voxels, estimating a volume of the piece of material corresponding to the voxel as a voxel volume; calculating the estimated quantity based on a plurality of the voxel quantities; A weighing system according to any one of claims 1 to 4.

6. the at least one separation candidate location includes a plurality of separation candidate locations; the separation position determination unit determines, from the plurality of separation candidate positions, the separation candidate position that minimizes an error between the estimated amount and the predetermined amount as the separation position. A weighing system according to any one of claims 1 to 4.

7. the quantity estimator sets the plurality of separation candidate positions along one coordinate axis of a coordinate system indicating a real space in which the set is located; The separation position determination unit determines the separation position from the plurality of separation candidate positions. The weighing system of claim 6.

8. A density estimation unit that estimates a density of the collection based on the collection image, the quantity estimator calculates the estimated quantity based on the density and the volume of the subset; A weighing system according to any one of claims 1 to 4.

9. the aggregate image is a depth image, The quantity estimation unit calculates the volume based on a distance in a depth direction obtained from the depth image. The weighing system of claim 8.

10. the density estimation unit estimates the density based on a mass reduction amount of the material pieces when the collection is taken out from the material pieces accumulated in a given area and a volume of the taken out collection. The weighing system of claim 8.

11. An image processing unit that sets a plurality of voxels for the set captured in the set image, the density estimation unit calculates, for each of the plurality of voxels, a density of the material piece in the voxel as a voxel density; The quantity estimator, calculating, for each of the plurality of voxels, an amount of the material piece in the voxel as a voxel amount based on the voxel density; calculating the estimator based on a plurality of the voxel quantities; The weighing system of claim 8.

12. the density estimation unit calculates the voxel density for each of the plurality of voxels using a machine learning model that receives an input of the set image and outputs the voxel density; The weighing system of claim 11.

13. The weighing system according to claim 12 , further comprising a learning unit that performs machine learning based on an error between an estimator of the subset and a correct answer corresponding to the estimator to generate the machine learning model.

14. the density estimator estimates, for each of the plurality of voxels, a class value indicating whether the material piece is present in the voxel; the quantity estimator calculates the voxel quantity for each of the plurality of voxels based on the class value and the voxel density; The weighing system of claim 11.

15. the density estimation unit calculates the class value for each of the plurality of voxels using a machine learning model that receives an input of the set image and outputs the class value; 15. The weighing system of claim 14.

16. a correction unit that corrects the estimated amount based on a relationship between a sample group of the estimated amount previously calculated by the amount estimation unit and a sample group of an actual amount of the material pieces actually separated in the past at the separation position corresponding to the estimated amount, the separation position determination unit determines the separation position based on the corrected plurality of estimated quantities. A weighing system according to any one of claims 1 to 4.

17. a path setting unit that sets a virtual path for a support passing through a group of material pieces that are managed by mass and that are accumulated in a given area based on an area image of the group of material pieces; an insertion position determination unit that determines an insertion position of the support for removing a set of material pieces managed by mass from the group of material pieces based on the virtual path; A support control system comprising:

18. an image processing unit that sets a plurality of voxels for a set of material pieces managed by mass based on a set image of the set; a density estimation unit that calculates, for each of the plurality of voxels, a density of the material piece in the voxel as a voxel density; a voxel amount calculation unit that calculates, for each of the plurality of voxels, an amount of the material piece in the voxel as a voxel amount based on the voxel density; a quantity estimator that calculates an estimated quantity of a subset of the material pieces obtained by separating the set based on a plurality of the voxel quantities; A weighing system comprising:

19. 1. A weighing method performed by a weighing system comprising at least one processor, comprising: calculating, for each of at least one candidate separation location based on a collection image of the collection of material pieces managed by mass, an estimate of a subset of the material pieces obtained by separating the collection at the candidate separation location; determining a separation location for separating a predetermined amount of the piece of material from the collection based on at least one of the estimated quantities; A weighing method including:

20. calculating, for each of at least one candidate separation location based on a collection image of the collection of material pieces managed by mass, an estimate of a subset of the material pieces obtained by separating the collection at the candidate separation location; determining a separation location for separating a predetermined amount of the piece of material from the collection based on at least one of the estimated quantities; A weighing program that causes a computer to execute the following.

Images