System

Abstract

To estimate or grip a desired gripping amount for a gripping object.SOLUTION: A gripping estimation device has an acquisition part and an estimation part. The acquisition part acquires observation data of a gripping object. The estimation part estimates a gripping amount by which the gripping object is gripped by a gripping device.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION An embodiment of the present invention relates to a system. [Background technology]

[0002] Traditionally, in food factories that produce bento boxes, a specified target amount of food is picked from a food tray and packed into a bento box. There are many types of food to be packed, and the types of food handled change frequently. Given this background, it is difficult to create dedicated machines for most foods, and the packing work is done manually.

[0003] From the perspective of gripping ability and food size, packed foods are classified into foods that are managed by number (3 pieces, 10 pieces, etc.), and small pieces or granular foods that are managed by weight (60g, 100g, etc.). Small pieces or granular foods are difficult to handle, for example, due to their irregular shape. Gripping devices for gripping small pieces or granular foods often use specialized tools for each type of food to grip a fixed amount. In this case, a great deal of effort, such as design changes, is required to accommodate various foods and different gripping amounts. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-151843 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the invention is to estimate or grasp a desired amount of grasping of a grasp object. [Means for solving the problem]

[0006] The system according to the embodiment includes a food gripper having a first member and a second member supported by a base portion that face each other and can be opened and closed, a gripping device equipped with the gripper, and a control means for controlling the operation of the gripping device so that the gripper inserts the gripper vertically into the food from above to grasp a portion of the food and releases the grasped food at a target location. The gripper includes the first member and the second member having surface shapes that form a space for containing the food when the gripper is closed, the first member and the second member having tip portions that face each other in the closed state and are positioned vertically below the base portion, and the first surface shape of the first member has a shape that has a surface that is inclined from the base portion to the tip portion toward the second member with respect to the vertical direction when in the closed state. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a grip estimation system including a grip estimation device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the appearance of the gripping device according to the embodiment. [Figure 3] FIG. 3 is a perspective view of the parallel gripper in an open state according to the embodiment. [Figure 4] FIG. 4 is a front view of the parallel gripper in the open state according to the embodiment. [Figure 5] FIG. 5 is a perspective view of the parallel gripper in a closed state according to the embodiment. [Figure 6] FIG. 6 is a front view of the parallel gripper in the closed state according to the embodiment. [Figure 7] FIG. 7 is a perspective view of the parallel gripper according to the embodiment, seen from a different direction. [Figure 8] FIG. 8 is a flowchart illustrating an example of a procedure for a gripping process according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of observation data according to the embodiment. [Figure 10]FIG. 10 is a diagram showing an example of the estimated amount and uncertainty in a partial region of the photographing range of the tray when the target amount is 60 g according to the embodiment. [Figure 11] FIG. 11 is a flowchart illustrating an example of a procedure for the estimation model generation process according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a procedure for the evaluation model generation process according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of an outline of an uncertainty assessment model executed as an RND model according to the embodiment. [Figure 14] FIG. 14 is a flowchart illustrating an example of a procedure for a model generation process according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of an outline of a gripping amount uncertainty estimation model according to the embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of an outline of a gripping amount uncertainty estimation model realized by an ensemble model according to the embodiment. [Figure 17] FIG. 17 is a block diagram showing an example of the hardware configuration of each device in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a gripping estimation device, a model generation method, a model learning device, and a gripper according to embodiments will be described with reference to the drawings. Note that the configurations of the embodiments described below are merely examples and are not limited to the contents described below. Furthermore, the actions and effects brought about by the configurations are also merely examples and are not limited to the contents described below.

[0009] Fig. 1 is a block diagram showing an example of the configuration of a gripping estimation system 1 having a gripping estimation device 17. As shown in Fig. 1, the gripping estimation system 1 has a gripping device 11, an observation device 13, a weighing device 15, and the gripping estimation device 17. Fig. 2 is a diagram showing an example of the appearance of the gripping device 11.

[0010] The gripping device 11 includes an end effector 111 that grips a target 133, multiple links 113, joints 115 provided between two of the multiple links, and multiple driving devices (motors, etc.) that drive the joints 115 and the end effector 111, respectively. The end effector 111 is realized, for example, by a gripper that opens and closes. The open-close gripper 111 has, for example, multiple surfaces that form a space in which food is contained in a closed state. In other words, the gripping device 11 has an open-close gripper 111 for gripping food that, in a closed state, forms a space capable of holding the target 133 with multiple surfaces. Note that the surfaces may be formed of a rigid, non-deformable member, or may be formed of a deformable member such as an elastic body. For the sake of concreteness, the following description will be given assuming that the gripper that realizes the end effector 111 is a parallel gripper 111. However, the gripper may also be a gripper that has one or more rotation axes and opens and closes by the rotational movement of a member. The multiple links 113 and multiple joints 115 provide the gripping device 11 with multiple degrees of freedom.

[0011] 3 to 7 are diagrams illustrating an example of a parallel gripper 111. FIG. 3 illustrates a perspective view of the parallel gripper 111 in an open state. FIG. 4 illustrates a front view of the parallel gripper 111 in an open state. FIG. 5 illustrates a perspective view of the parallel gripper 111 in a closed state. FIG. 6 illustrates a front view of the parallel gripper 111 in a closed state. As indicated by dotted lines in FIGS. 3 to 6, a force sensor may be installed, for example, at the base portion 121 of the parallel gripper 111. In this case, the force sensor detects the weight of the object 133 gripped by the parallel gripper 111. The force sensor outputs the weight of the object 133 to the gripping estimation device 17. FIG. 7 illustrates a perspective view of the parallel gripper 111 as viewed from a different direction. As illustrated in FIG. 7, the parallel gripper 111 is formed of a plurality of surface shapes 123. As a result, as shown in FIGS. 5 and 6, the parallel grippers 111 in the closed state form a space 125 in which the food is contained.

[0012] The observation device 13 is realized by, for example, various cameras. The camera is realized by, for example, an RGB (Red, Green, Blue) camera. The RGB camera is an optical camera that captures images of multiple objects 133 to be grasped placed on the tray 131. In this case, the observation data corresponds to an RGB image. Note that a monochrome camera may be used instead of the RGB camera. Furthermore, the observation device 13 may acquire a type of observation data different from the above-mentioned observation data. In this case, the observation device 13 is realized by a three-dimensional measurement camera (hereinafter referred to as a D (Depth) camera). Furthermore, the observation device 13 may be realized by an RGB camera and a D camera (hereinafter referred to as an RGB-D camera). In the following, for concrete explanation, it is assumed that the observation device 13 is realized by an RGB-D camera. The capture range of the RGB-D camera is a range that includes the entire area of ​​the tray 131 on which the objects 133 to be grasped are placed.

[0013] The weighing device 15 is disposed, for example, on the underside of the tray 131. The weighing device 15 corresponds, for example, to a scale that measures the total weight of the tray 131 and the gripping target 133 placed on the tray 131. The weighing device 15 outputs measurement data resulting from weighing the tray 131 to the gripping estimation device 17. The measurement data output from the weighing device 15 is used, for example, in generating (learning) a gripping amount estimation model and an uncertainty assessment model, which will be described later. Therefore, once the generation of the gripping amount estimation model and the uncertainty assessment model is complete, the weighing device 15 does not need to use the gripping estimation system 1. Furthermore, if a force sensor is provided in the parallel gripper 111, the weighing device 15 does not need to be disposed on the underside of the tray 131.

[0014] The grip estimation device 17 includes a controller 171. As shown in FIG. 1, the controller 171 includes an acquisition unit 173, an estimation unit 175, an evaluation unit 177, a selection unit 179, and a control unit 181.

[0015] The acquisition unit 173 acquires observation data of the object 133 to be held. As described above, the observation data includes image data of the object 133 to be held. The observation data also includes distance data from the acquisition position of the observation data to the surface of the object 133 to be held. That is, the distance data is data indicating the distance from the installation position of the D camera, which is the acquisition position of the observation data, to the surface of the object 133 to be held. The object 133 to be held is, for example, various beans such as coffee beans or granular food such as rice. Note that the acquisition unit 173 may acquire observation data of a type different from the observation data. Specifically, the acquisition unit 173 acquires an RGB image and a D image. More specifically, the acquisition unit 173 acquires observation data for each of a plurality of partial regions in the distribution region of the object 133 to be held. The distribution region of the object 133 corresponds, for example, to a region in the tray 131 where the object 133 can be placed. Each of the plurality of partial regions is an area of ​​a predetermined size and corresponds to an area (cut area: patch) cropped from the RGB image and the D image. The partial region may be referred to as a gripping point. When generating each model described later, the acquisition unit 173 acquires measurement data from the weighing device 15 or acquires the weight of the gripping target 133 gripped by the end effector 111 from a force sensor.

[0016] The estimation unit 175 estimates the gripping amount of the gripping target 133 gripped by the gripping device 11 based on the observation data. The estimated gripping amount is called an estimated amount. Specifically, the estimation unit 175 estimates the gripping amount in each of the plurality of partial regions in the observation data. For example, the estimation unit 175 inputs the observation data into a trained gripping amount estimation model that has been trained to input observation data of the gripping target 133 and output an estimated amount, and estimates the gripping amount. More specifically, the estimation unit 175 inputs each of the plurality of partial regions in the observation data into the trained gripping amount estimation model, and estimates the gripping amount in each of the plurality of partial regions.

[0017] The grip amount estimation model is generated by learning the first learning model using, for example, the actual measurement value of the grip amount of the gripping target 133 gripped by the gripping device 11 as training data for the first learning model and observation data of the gripping target 133 before being gripped by the gripping device 11 as input data for the first learning model. The procedure for generating the grip amount estimation model by learning the first learning model will be described later.

[0018] The evaluation unit 177 evaluates the uncertainty that the gripping device 11 will grasp only the grip amount estimated by the trained grip amount estimation model based on the observation data. The index of uncertainty (uncertainty index) may be, for example, an uncertainty degree, which is a degree (or percentage) indicating the uncertainty that the gripping device 11 will grasp only the estimated grip amount. In other words, the uncertainty index is an index for evaluating whether the actual grip amount is close to the estimated amount. Note that, instead of the uncertainty degree, the uncertainty index may be a degree (or percentage) indicating the certainty that the gripping device 11 will grasp only the estimated grip amount, i.e., the reliability that the gripping device 11 will grasp the estimated grip amount.

[0019] Hereinafter, for the sake of simplicity, the term "uncertainty" may be appropriately interpreted as the degree of uncertainty (degree of uncertainty) or an uncertainty index. The evaluation unit 177 may also evaluate the uncertainty of only the estimated quantity being grasped by the grasping device 11 based on a type of observation data different from the observed data. Specifically, the evaluation unit 177 evaluates the uncertainty in each of a plurality of partial regions based on the observed data. More specifically, the evaluation unit 177 inputs the observed data into a trained uncertainty evaluation model that has been trained to input the observed data of the grasp target 133 and output an uncertainty index, and evaluates the uncertainty.

[0020] The uncertainty assessment model is generated by self-supervised learning using observation data of the grasped object 133. This self-supervised learning inputs the observation data into a learning model, outputs data based on the internal parameters of the model (e.g., weights and biases in a neural network), generates training data (correct answer data) from the observation data, and updates the internal parameters of the model based on the output data and training data. The procedure for generating the uncertainty assessment model will be explained later.

[0021] The selection unit 179 selects from the plurality of partial regions a region where the object to be grasped is to be grasped by the grasping device 11, based on the uncertainty evaluated for each of the plurality of partial regions. For example, the selection unit 179 selects from the plurality of partial regions a position to be grasped by the grasping device 11 (hereinafter referred to as a grasping position) based on the estimated quantity and uncertainty corresponding to each of the plurality of partial regions and a predetermined condition. The predetermined condition is, for example, a predetermined weight range (hereinafter referred to as a predetermined range) relative to a target grasping amount (hereinafter referred to as a target amount) set by the user. The predetermined range is, for example, a range from +0.5 g to −0.5 g. The predetermined condition (e.g., the predetermined range) is stored in a memory in the grasping estimation device 17. The predetermined condition may include a predetermined uncertainty range (e.g., an uncertainty of 0.1 or less).

[0022] Specifically, the selection unit 179 selects, from among a plurality of estimated quantities corresponding to a plurality of partial regions, a plurality of estimated quantities that fall within a predetermined range based on a target quantity. Next, the selection unit 179 identifies, from among a plurality of uncertainties corresponding to the selected plurality of estimated quantities, an estimated quantity with the smallest degree of uncertainty, i.e., the lowest uncertainty. The selection unit 179 selects, as the gripping position to be gripped by the gripping device 11, the partial region for the identified estimated quantity with the lowest uncertainty. Note that, when reliability is used as the uncertainty index, the selection unit 179 identifies, from among a plurality of reliabilities corresponding to the selected plurality of estimated quantities, a partial region for the estimated quantity with the highest degree of reliability, i.e., the lowest uncertainty.

[0023] The method for selecting a gripping position by the selection unit 179 is not limited to the above. The selection unit 179 may select a gripping position based on the target amount, the estimated amount, and the uncertainty of the estimated amount. For example, the gripping position may be selected taking into account the balance between the difference between the target amount and the estimated amount and the uncertainty. The selection unit 179 may compare a partial region where the difference between the target amount and the estimated amount is equal to or less than a threshold but the uncertainty exceeds the threshold with a partial region where the difference between the target amount and the estimated amount exceeds a threshold but the uncertainty is equal to or less than the threshold, and select the latter. In this case, when the estimated amount is less than the target amount, the currently gripped target may be released at the target position, and then a second gripping operation may be performed with the difference between the estimated amount and the target amount as a new target amount. When the estimated amount is greater than the target amount, the excess amount of the currently gripped target may be dropped and then released at the target position (described below). In this way, variations in gripping strategies may be prepared that take into account the evaluated gripping uncertainty.

[0024] The control unit 181 controls the operations of the gripping device 11 (the gripping operation, moving operation, and releasing operation of the gripping target 133). Specifically, the control unit 181 controls the gripping device 11 so as to grip the gripping target 133 at the selected gripping position. As a result, the gripping device 11 grips the gripping target 133 at the gripping position. The control unit 181 controls the gripping device 11 so as to move the gripped gripping target 133 to a predetermined target position and release the gripping target 133 at this target position. Known controls are used as appropriate for controlling the moving operation and the releasing operation, and therefore a description thereof will be omitted.

[0025] The above describes the configuration of the grip estimation system 1. Below, we will explain the processing related to the gripping motion (hereinafter referred to as gripping processing) performed by the grip estimation system 1. Fig. 8 is a flowchart showing an example of the procedure of the gripping processing.

[0026] (Grip processing) (Step S801) The acquisition unit 173 acquires observation data for each of a plurality of partial regions of a predetermined size related to the gripping target 133 from the observation device 13. The predetermined size is stored in advance in the memory of the gripping estimation device 17 as a size that can be gripped by the parallel gripper 111. FIG. 9 is a diagram showing an example of the observation data. As shown in FIG. 9, the acquisition unit 173 acquires, for example, an RGB image and a D image of a granular food item, which is the gripping target 133, placed on the tray 131, for each of the plurality of partial regions.

[0027] (Step S802) The estimation unit 175 inputs the observation data for each partial region into the grip amount estimation model, and estimates the grip amount of the gripping target 133 grasped by the gripping device 11. As a result, the estimation unit 175 outputs a plurality of estimated amounts corresponding to a plurality of partial regions.

[0028] (Step S803) The evaluation unit 177 inputs the observation data for each partial region into an uncertainty evaluation model and outputs an uncertainty index from the uncertainty evaluation model. That is, the evaluation unit 177 evaluates the uncertainty of whether the grasp target 133 will be grasped by the estimated amount. In this way, the evaluation unit 177 evaluates the uncertainty of the estimated amount for each partial region.

[0029] (Step S804) The selection unit 179 identifies an area where the estimated gripping amount is within a predetermined range (for example, within ±0.5 g) from the target amount. FIG. 10 is a diagram showing an example of the estimated amount and uncertainty in a portion of the imaging range of the tray 131 when the target amount is 60 g. The dotted circles in FIG. 10 indicate areas for multiple estimated amounts. The solid squares in FIG. 10 indicate multiple areas identified by the selection unit 804 and the uncertainty in those areas. Note that if there is no area with an estimated amount within a predetermined range from the target amount, the control unit 181 may notify an error, or the control unit 181 may control the gripping device 111 to level the gripping target 133 placed on the tray 131 with the gripping device 111 before returning the process to step S801.

[0030] (Step S805) The selection unit 179 selects the region with the lowest uncertainty (i.e., the region with the lowest uncertainty or the highest reliability) from the identified regions. In FIG. 10, the region with the lowest uncertainty MR corresponds to a region with an uncertainty of 0.1. If the predetermined condition includes a predetermined uncertainty range (e.g., an uncertainty of 0.1 or less), a region with an uncertainty within this uncertainty range is selected. If there is no region with uncertainty within this uncertainty range, the control unit 181 may notify an error via the output device, or the control unit 181 may control the gripping device 111 to level the gripping target 133 placed on the tray 131 with the gripping device 111 and then return the process to step S801.

[0031] (Step S806) The control unit 181 controls the gripping device 11 to grip the gripping target 133 from the selected region, thereby completing the gripping process.

[0032] The grip estimation device 17 according to the embodiment described above acquires observation data of the gripping target 133, which is, for example, a granular food item, and estimates the gripping amount of the gripping target 133 gripped by the gripping device 11 based on the acquired observation data. Specifically, the grip estimation device 17 inputs the acquired observation data into a trained gripping amount estimation model that has been trained to estimate the gripping amount using the observation data of the gripping target 133 as input, and estimates the gripping amount. As a result, the grip estimation device 17 can estimate the gripping amount desired by the user from the observation data.

[0033] The grip estimation device 17 also evaluates the uncertainty of the estimated grip amount being grasped by the grip device 11 based on the observation data. The grip estimation device 17 also acquires a type of observation data different from the observation data and evaluates the uncertainty of the estimated grip amount being grasped by the grip device 11 based on the acquired type of observation data. For example, the grip estimation device 17 inputs the observation data of the gripping target 133 into a trained uncertainty assessment model that has been trained to output an uncertainty index using the input observation data, and evaluates the uncertainty. The observation data in the grip estimation device 17 includes image data of the gripping target 133 or distance data from the acquisition position of the observation data to the surface of the gripping target 133. Specifically, the grip estimation device 17 according to the embodiment acquires observation data for each of a plurality of partial regions in a distribution region of the gripping target 133, estimates the grip amount for each of the plurality of partial regions, evaluates the uncertainty for each of the plurality of partial regions, and selects a region to be grasped by the grip device 11 from the plurality of partial regions based on the grip amount and the uncertainty corresponding to each of the plurality of partial regions and predetermined conditions. From these facts, the grip estimation device 17 can grasp the desired grip amount for the gripping object 133 in a region where the degree of uncertainty is low among the estimated grip amounts.

[0034] Furthermore, the gripping estimation device 17 of this embodiment has a parallel gripper 111 that, in a closed state, forms a space capable of holding a gripping target 133 using a plurality of surface shapes 123. For example, the parallel gripper 111 for gripping food according to this embodiment has a plurality of surface shapes 123 that form a space for containing food in a closed state. Furthermore, the parallel gripper 111 for gripping food according to this embodiment is made of a non-deformable member, for example, realized by a parallel gripper. As a result, the parallel gripper 111 for gripping food according to this embodiment can appropriately grip various foods that are irregularly shaped and deformable. Therefore, the parallel gripper 111 does not require a dedicated tool or an end effector for gripping a fixed amount, thereby reducing the considerable effort required for design changes and the like.

[0035] From the above, the gripping estimation device 17 and the parallel gripper 111 for gripping food in this embodiment can estimate or grip a desired gripping amount for the object to be gripped 133.

[0036] The process of generating a grip amount estimation model and the process of generating an uncertainty assessment model will be described below. The process of generating a grip amount estimation model and the process of generating an uncertainty assessment model each correspond to a learning process. In this case, the grip estimation system 1 or the grip estimation device 17 shown in FIG. 1 functions as a model learning device. First, the process of generating a grip amount estimation model will be described, and then the process of generating an uncertainty assessment model will be described. FIG. 11 is a flowchart showing an example of the procedure for generating a grip amount estimation model (hereinafter referred to as an estimation model generation process).

[0037] (Estimation model generation process) (Step S111) The acquisition unit 173 acquires observation data for a partial region of a predetermined size related to the grasp target 133 from the observation device 13. The acquisition unit 173 outputs the position of the partial region to the control unit 181. The acquisition unit 173 also stores the position of the partial region in memory.

[0038] (Step S112) The estimation unit 175 inputs the observation data relating to the partial region into the first learning model, and outputs (estimates) the gripping amount of the gripping target 133 gripped by the gripping device 11 as a first output value.

[0039] (Step S113) The control unit 181 controls the gripping device 11 so as to grip the gripping target 133 in the partial region. Under the control of the control unit 181, the gripping device 11 grips the gripping target 133 in the partial region.

[0040] (Step S114) The acquisition unit 173 acquires an actual measurement value of the gripping amount by the gripping device 11, i.e., the actual gripping amount, based on the output from the weighing device 15 or the force sensor. When calculating the actual measurement value based on measurement data that is output from the weighing device 15, the acquisition unit 173 acquires the actual measurement value, for example, by calculating the difference between the measurement data before and after the gripping device 11 grips the gripping target 133. Note that when a force sensor is mounted on the parallel gripper 111, the acquisition unit 173 acquires the actual measurement value by converting the output from the force sensor into weight.

[0041] (Step S115) The estimation unit 175 calculates a loss based on the difference between the first output value (estimate) and the actual measurement value. For example, the estimation unit 175 calculates the loss (squared error) by applying the difference to a loss function. Next, the estimation unit 175 updates the parameters in the first learning model so as to reduce the loss using a known method such as backpropagation or stochastic gradient descent.

[0042] (Step S116) If the number of parameter updates is the predetermined number (Yes in step S116), the process of step S118 is executed. If the number of parameter updates is not the predetermined number (No in step S116), the process of step S117 is executed. The predetermined number is set in advance by a user or the like. Note that the process in this step is not limited to comparing the number of parameter updates with the predetermined number. For example, the processes of steps S111 to S116 may be repeated until learning for the first learning model converges. Convergence of learning for the first learning model refers to, for example, when the difference between the estimated amount and the actual measurement value becomes equal to or less than a predetermined value. The predetermined number or the predetermined value is stored in advance in a memory in the grip estimation device 17.

[0043] (Step S117) The acquisition unit 173 sets another partial area in the observation data that is different from the partial area in step S111. The acquisition unit 173 outputs the position of the other partial area to the control unit 181.

[0044] (Step S118) The estimation unit 175 sets the first learning model as the grip amount estimation model. The estimation unit 175 stores the grip amount estimation model in the memory of the grip estimation device 17. This completes the estimation model generation process.

[0045] According to the estimation model generation process of the embodiment described above, the grip amount estimation model used in the grip estimation device 17 is generated by learning the first learning model using the actual measured value of the grip amount of the gripping object 133 gripped by the gripping device 11 as training data for the first learning model and the observation data before the gripping object 133 is gripped by the gripping device 11 as input data for the first learning model.

[0046] The process of generating an uncertainty assessment model will be described below. Fig. 12 is a flowchart showing an example of the procedure for processing to generate an uncertainty assessment model (hereinafter referred to as evaluation model generation processing).

[0047] (Evaluation model generation process) (Step S121) The acquisition unit 173 acquires observation data for a partial region of a predetermined size related to the grasp target 133 from the observation device 13. The acquisition unit 173 also stores the position of the partial region in a memory.

[0048] (Step S122) The evaluation unit 177 inputs the observation data for the partial region into the second learning model and outputs a second output value. As will be understood from the following explanation, the evaluation unit 177 estimates the data (output value) output by a reference model (a fixed weight model, a target net, described later) different from the second learning model based on the observation data as the second output value output from the second learning model.

[0049] (Step S123) The evaluation unit 177 inputs the observation data for the partial region into the fixed weight model and outputs a third output value. The fixed weight model corresponds to, for example, a model with fixed weights in a neural network model. The fixed weight model is a reference neural network model and is called a target net. The fixed weight model and the second learning model are different neural network models, differing in hyperparameters, such as the number of intermediate layers. Preferably, the fixed weight model and the second learning model are models with different neural network structures, such as an image net. The fixed weight model may be configured so that its statistical properties are similar to those of the observed data, even if the values ​​themselves differ from those obtained from the actual observed data, by artificially adding minute noise within a predetermined range to the observed data input to the fixed weight model or the output obtained from the fixed weight model, or by adjusting the statistics, such as the mean and variance of the values ​​output from the fixed weight model, so that they match the observed data.

[0050] (Step S124) The evaluation unit 177 calculates the difference between the second output value and the third output value. Next, the evaluation unit 177 calculates the loss (squared error) based on the calculated difference and the loss function. This difference corresponds to the uncertainty index output by the uncertainty evaluation model.

[0051] (Step S125) The evaluation unit 177 updates the parameters in the second learning model so as to reduce the loss by a known method such as backpropagation or stochastic gradient descent.

[0052] (Step S126) If the number of parameter updates is the predetermined number (Yes in step S126), the process of step S128 is executed. In this case, the second learning model corresponds to a learned model, and is called, for example, a Predictor Net. Learning for the second learning model corresponds to self-supervised learning. If the number of parameter updates is not the predetermined number (No in step S126), the process of step S127 is executed. Note that the process in this step is not limited to comparing the number of parameter updates with the predetermined number. For example, the processes of steps S121 to S126 may be repeated until learning for the first learning model converges. Learning for the second learning model converges when, for example, the difference between the second output value and the third output value is equal to or less than a predetermined value.

[0053] (Step S127) The acquisition unit 173 sets another partial area in the observation data that is different from the partial area in step S121. The acquisition unit 173 outputs the position of the other partial area to the control unit 181.

[0054] (Step S128) The evaluation unit 177 sets the predictor net and the target net together as an uncertainty evaluation model. In this case, the uncertainty evaluation model corresponds to an RND (Random Network Distillation) model. In this case, the uncertainty evaluation model corresponds to a model that evaluates whether input observation data is close to the data used during learning, i.e., the degree of similarity. The evaluation unit 177 stores the uncertainty evaluation model in the memory of the grip estimation device 17. In this case, the output from the uncertainty evaluation model is a value indicating the difference between the output from the fixed weight model and the output from the trained uncertainty evaluation model, and is expressed, for example, in the range from 0 (no difference) to 1 (maximum difference). When the output from the uncertainty evaluation model is 0, the degree of uncertainty corresponds to 0 and the reliability is maximum. On the other hand, when the output from the uncertainty evaluation model is 1, the degree of uncertainty corresponds to 0 and the reliability is maximum. This completes the evaluation model generation process.

[0055] Fig. 13 is a diagram showing an example of an outline of an uncertainty assessment model executed as an RND model. As shown in Fig. 13, in the RND model, an RGB image (cropped RGB) in a subregion and a D image (cropped depth) in the subregion are input to a predictor net and a target net, respectively. The RND model outputs the difference between the output from the predictor net and the output from the target net as an uncertainty index (estimation error).

[0056] According to the evaluation model generation process of the embodiment described above, the uncertainty evaluation model used in the grip estimation device 17 is generated by training the second learning model so that when observation data is input, the second learning model outputs the same data as the data output by the target net based on the observation data.

[0057] Although an example has been described in which the estimation model generation process and the evaluation model generation process are executed separately, the gripping amount estimation model and the uncertainty evaluation model can also be generated simultaneously. In this case, the gripping amount estimation model and the uncertainty evaluation model are collectively referred to as the gripping amount uncertainty estimation model (see FIG. 15). The process of generating the gripping amount uncertainty estimation model (hereinafter referred to as the model generation process) will be described below. FIG. 14 is a flowchart showing an example of the procedure for the model generation process. The processes of steps S141, S142, S144, S145, S148, and S149 in FIG. 14 are similar to the processes of steps S111, S112, S113, S114, S116, and S117 in FIG. 11, respectively, and therefore will not be described again.

[0058] (Model generation process) (Step S143) The evaluation unit 177 inputs the observation data related to the partial region into the third learning model and outputs the error amount of the estimated amount of the gripping target 133 gripped by the gripping device 11. The error amount corresponds to the difference between the actual measurement value of the gripping amount of the gripping target 133 actually gripped by the gripping device 11 in the partial region and the estimated amount output by the gripping amount estimation model. In other words, the third learning model estimates the error of the estimated amount of the gripping target 133 gripped from the partial region (grasping amount estimation error) based on the observation data related to the partial region. This error (error amount) corresponds to the uncertainty (uncertainty index).

[0059] (Step S146) The estimation unit 175 calculates the difference (hereinafter referred to as the first difference) between the first output value (estimated amount) and the actual measurement value. The evaluation unit 177 calculates the difference (hereinafter referred to as the second difference) between the amount of error and the first difference.

[0060] (Step S147) The estimation unit 175 calculates a first loss (squared error) by, for example, applying the first difference to a loss function. Then, the estimation unit 175 updates parameters in the first learning model so as to reduce the first loss by a known method such as backpropagation or stochastic gradient descent. The evaluation unit 177 calculates a second loss (squared error) by, for example, applying the second difference to the loss function. Then, the evaluation unit 177 updates parameters in the third learning model so as to reduce the second loss by a known method such as backpropagation or stochastic gradient descent.

[0061] (Step S150) The estimation unit 175 sets the trained first learning model as the grip amount estimation model. The evaluation unit 177 sets the trained third learning model as the uncertainty evaluation model. At this time, the uncertainty evaluation model corresponds to an error estimation model that estimates the estimation error (uncertainty) of the grip amount estimated by the grip amount estimation model. Specifically, the estimation unit 175 and the evaluation unit 177 combine the trained first learning model and the trained third learning model and set them as the grip amount uncertainty estimation model. The estimation unit 175 and the evaluation unit 177 store the set grip amount uncertainty estimation model in the memory of the grip estimation device 17. At this time, the processes of steps S802 and S803 in the grip processing shown in FIG. 8 can be summarized as follows. That is, in the processes of steps S802 and S803, for each partial region, the observation data is input into the grip amount uncertainty estimation model, and an estimated amount and uncertainty (uncertainty index) are output. Learning the third learning model in the model generation process corresponds to self-supervised learning.

[0062] Fig. 15 is a diagram showing an example of an outline of a grip amount uncertainty estimation model. As shown in Fig. 15, an RGB image (cropped RGB) of a partial region and a D image (cropped depth) of the partial region are input to the grip amount uncertainty estimation model. The grip amount uncertainty estimation model outputs an estimated amount and a degree of uncertainty (uncertainty index).

[0063] The grip amount uncertainty estimation model described above is a combination of separate models, namely the first learning model (grip amount estimation model) and the third learning model (uncertainty assessment model). This provides the following advantages: The grip amount estimation model and the uncertainty assessment model can be trained independently, allowing for separate parameter tuning for each model. Furthermore, the combination of the grip amount estimation model and the uncertainty assessment model can be freely selected as appropriate, and the RND model may be combined instead of the error estimation model described above, or both models may be combined.

[0064] However, on the other hand, the grip amount uncertainty estimation model can also be configured as a single model. That is, a single model (preferably a single neural network model) that receives observation data related to a subregion and outputs an estimate of the grip amount and an error of that estimate can be trained, and this trained model can be set as the grip amount uncertainty estimation model. Benefits of training using a single model include a shorter training time and a smaller memory capacity for storing the model. This model training is performed, for example, as follows: The estimation unit 175 and the evaluation unit 177 input the observation data into the model and output an estimate of the grip amount and an error of that estimate. The estimation unit 175 and the evaluation unit 177 calculate a first loss related to the output estimate and a second loss related to the output error, and then add the calculated first loss and second loss to calculate a total loss. The estimation unit 175 and the evaluation unit 177 update the parameters of the model to reduce the total loss using known methods such as backpropagation or stochastic gradient descent. The model trained in this way is obtained as the grasp amount uncertainty estimation model.

[0065] In the above description, the uncertainty assessment model has been described as an RND model or an error estimation model. The uncertainty assessment model is not limited to an RND model or an error estimation model, and can be realized by other methods. For example, the uncertainty assessment model may be realized by an ensemble using multiple grip amount estimation models. An ensemble uncertainty assessment model (hereinafter referred to as an ensemble model) uses multiple grip amount estimation models, each of which has a different neural network model structure to be trained or different training data, and uses the variation (variance) of the output (estimated amount) from each grip amount estimation model as an uncertainty index (e.g., uncertainty).

[0066] FIG. 16 is a diagram illustrating an example of an outline of a grip amount uncertainty estimation model realized by an ensemble model. As illustrated in FIG. 16, the ensemble model has multiple grip amount estimation models. The evaluation unit 177 inputs an RGB image (cropped RGB) of a partial region and a D image (cropped depth) of the partial region to each of the multiple grip amount estimation models. The evaluation unit 177 calculates an uncertainty index based on outputs (estimations) from the multiple grip amount estimation models in the ensemble model. The calculated uncertainty index corresponds to, for example, a standard deviation or variance normalized based on the multiple estimates. As a result, the evaluation unit 177 outputs the uncertainty index. Note that the estimation unit 175 may calculate an average (average estimate) of the multiple estimates output from the multiple grip amount estimation models in the ensemble model and output the average estimate as the estimate in step S802.

[0067] According to the ensemble model generation process of the embodiment described above, the ensemble model used in the grip estimation device 17 uses multiple trained grip amount estimation models, and the variation in the output (estimated amount) from each grip amount estimation model can be used as an uncertainty index.

[0068] According to the model learning device of this embodiment, observation data of the gripping target 133 is acquired, and the actual measured value of the gripping amount of the gripping target 133 gripped by the gripping device 11 is used as training data for the first learning model. The observation data is used as input data for the first learning model to train the first learning model, generating a gripping amount estimation model that inputs the observation data and outputs a gripping amount (estimated amount). The first model generation unit that generates the gripping amount estimation model is realized by, for example, the estimation unit 175. Furthermore, according to this learning device, a learning model that receives the observation data and outputs an uncertainty index is trained by self-supervised learning, thereby generating an uncertainty assessment model that outputs an uncertainty index indicating that only the estimated gripping amount (estimated amount) is grasped from the input observation data. Examples of training data for this self-supervised learning are "the difference between the gripping amount (estimated amount) output when the observation data is input to the gripping amount estimation model and the actual measured value" or "the difference between the first output value output when the observation data is input to a predetermined model and the second output value output when the observation data is input to the learning model." The second model generation unit that generates the uncertainty assessment model is realized by, for example, the evaluation unit 177. Either the first model generation unit or the second model generation unit, or both together, may be referred to as the model generation unit. The first model generation unit and the second model generation unit may be provided separately in the grip estimation device 17. Furthermore, with regard to model learning, the model learning device does not necessarily need to generate both the grip amount estimation model and the uncertainty assessment model, and may generate either one of the required models.

[0069] From the above, with the gripping estimation device 17 according to this embodiment, even if the reliability of the output results from the gripping amount estimation model becomes low for a granular food object 133 that is easily deformed, has large interactions between individual particles, and is difficult to simulate or model accurately, for example, for which it is difficult to collect training data by repeatedly using the same food due to factors such as food deterioration, damage due to gripping, and moisture, i.e., it is difficult to collect sufficient training data, in order to deal with the incompleteness of the estimated gripping amount, the uncertainty of only the estimated amount being gripped is modeled using an uncertainty evaluation model through self-supervised learning, and the gripping object 133 can be gripped at the most reliable gripping position among those that are closest to the gripping amount (target amount) specified by the user.

[0070] That is, according to the grip estimation device 17 of this embodiment, the most reliable grip position among the output results from the grip amount estimation model can be selected based on the output from the uncertainty assessment model. Therefore, even if learning data is collected in a shorter time than conventional methods and the grip amount estimation model, uncertainty assessment model, and grip amount uncertainty estimation model are learned in a shorter time, the target amount of gripping of the grip object 133 can be achieved with high accuracy.

[0071] When the technical ideas of the estimation model generation process, the uncertainty model generation process, and the model generation process are realized by a model generation method (which may also be called a model manufacturing method), the model generation method generates a grip amount estimation model and an uncertainty assessment model by a processing procedure similar to that of the embodiment. Furthermore, the model generation method is executed by a model learning device.

[0072] As described above, according to this embodiment, a target amount of a target object is grasped by a gripper after estimating the grasped amount and evaluating the uncertainty of the estimation. However, there are cases where the actual amount grasped by the gripper exceeds the target amount. To prepare for such cases, the grasp estimation device 17 according to this embodiment may achieve grasping of the target amount by dropping the excess amount of the actually grasped target object from the gripper. As an example of a method for dropping the excess amount, a method of opening the closed gripper by an opening time x (e.g., 0.5 seconds) and a gap width y (e.g., 0.5 mm) will be described.

[0073] (Method 1) The estimation unit 175 learns a learning model (preferably a neural network model) that inputs the gripper's actual gripping amount p, gripper release time x, and gap width y and outputs the amount z of the gripped object that falls through the gap, and stores the learning model in memory in the gripping estimation device 17. This learning involves inputting the inputs p, x, and y to the learning model to obtain the output z, calculating a loss based on the output z and the amount of the gripped object that actually falls from the gripper (fall amount) z' when the gap width y is opened for the release time x when the amount actually gripped by the gripper is gripping amount p, and updating the parameters of the learning model in a direction that reduces the loss using known methods such as backpropagation or stochastic gradient descent based on the loss. The estimation unit 175 then uses this trained model stored in memory to determine the gripper control amounts x and y for dropping the surplus amount from the gripper and setting the gripping amount to the target amount. Specifically, the estimation unit 175 (1) sets values ​​for the candidate control amounts x' and y', (2) inputs x' and y' and the current gripping amount p into the trained model to estimate the drop amount z, and determines whether this drop amount z is close to the difference between the current gripping amount p and the target amount (i.e., whether this drop amount z is within a predetermined range (e.g., within ±0.5 g) of this difference). (3) If they are close, the estimation unit 175 passes these x' and y' to the control unit 181 as the gripper control amounts. (4) If they are not close, the estimation unit 175 sets a different value for one or both of x' and y' and returns to process (2). The control unit 181 controls the release of the gripper according to the passed control amount, thereby dropping the excess amount of the gripped object from the gripper and achieving the target amount of gripping.

[0074] (Method 2) In Method 1, the output z of the learning model was the amount of drop, but it may also be the remaining amount of the object to be grasped remaining in the gripper. Even in this case, the model can be learned in the same framework as Method 1. In this case, the estimation unit 175 replaces the amount of drop in Method 1 with the remaining amount z, and determines whether the remaining amount z is close to the target amount in the determination of (2).

[0075] (Method 3) While a trained model is used to estimate the drop amount in Method 1, a table having items such as grip amount p, release time x, gap width y, and drop amount z may also be used. In this case, the values ​​of each pair of p, x, y, and z in the table may be experimentally determined in advance or may be determined by interpolating between experimentally determined values. When dropping the excess amount, the estimation unit 175 may estimate the drop amount z by searching the table using x' and y' and the current grip amount p as keys in estimating the drop amount z in Method 1 (2). Alternatively, the estimation unit 175 may consider the difference between the current grip amount p and the target amount as the drop amount z, search the table using the current grip amount p and drop amount z as keys to identify the control amounts x and y, and pass these x and y to the control unit 181 as the gripper control amounts.

[0076] (Method 4) Instead of the drop amount z in the table of Method 3, the remaining amount z remaining in the gripper may be used as an item. In this case, a table can be prepared as in Method 3. In this case, the estimation unit 175 may replace the estimation of the drop amount z described in Method 3 with the estimation of the remaining amount z, and determine whether the estimated remaining amount z is close to the target amount in the determination. Alternatively, the estimation unit 175 may regard the target amount as the remaining amount z, search the table using the current gripping amount p and the remaining amount z as keys to identify the control amounts x and y, and pass these x and y to the control unit 181 as the control amounts of the gripper.

[0077] The uncertainty of the drop amount z or remaining amount z may also be evaluated for the above methods 1 to 4. In this case, when selecting the control amounts x and y to obtain the desired drop amount z or remaining amount z, the control amounts x and y with small uncertainty should be selected.

[0078] Some or all of the devices (such as the grip estimation device 17 and / or the controller 171) in the above-described embodiments may be configured as hardware, or may be configured as software (program) information processing executed by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). When configured as software information processing, software that realizes at least some of the functions of the devices (such as the grip estimation device 17 and / or the controller 171) in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), or a USB (Universal Serial Bus) memory, and the software information processing may be executed by loading the software into a computer. Alternatively, the software may be downloaded via a communication network. Furthermore, the software may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), thereby executing the information processing by hardware.

[0079] The type of storage medium that stores the software is not limited. The storage medium is not limited to removable media such as magnetic disks or optical disks, but may be fixed storage media such as hard disks or memory. The storage medium may be provided inside the computer or outside the computer.

[0080] 17 is a block diagram showing an example of the hardware configuration of each device (such as the grip estimation device 17 and / or the controller 171) in the above-described embodiment. Each device may be realized as a computer 81 including, for example, a processor 811, a main storage device 812 (memory), an auxiliary storage device 813 (memory), a network interface 814, and a device interface 815, which are connected via a bus 816.

[0081] Although the computer 81 in FIG. 17 includes one of each component, it may also include multiple of the same component. Although FIG. 17 shows one computer 81, the software may be installed on multiple computers, and each of the multiple computers may execute the same or different parts of the software. In this case, a distributed computing configuration may be used in which each computer communicates with the other computers via a network interface 814 or the like to execute the processing. That is, each device in the above-described embodiment (such as the grip estimation device 17 and / or the controller 171) may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize its functions. Furthermore, the system may be configured such that information transmitted from a terminal is processed by one or more computers provided on a cloud, and the processing results are transmitted to the terminal.

[0082] The various calculations of each device (such as the gripping estimation device 17 and / or the controller 171) in the above-described embodiments may be executed in parallel using one or more processors, or using multiple computers via a network. Furthermore, the various calculations may be distributed to multiple processing cores in a processor and executed in parallel. Furthermore, some or all of the processes, means, etc. of the present disclosure may be executed by at least one of a processor and a storage device provided on a cloud that can communicate with the computer 81 via a network. In this way, each device in the above-described embodiments may be implemented in the form of parallel computing using one or more computers.

[0083] The processor 811 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, or the like) including a computer control device and arithmetic device. The processor 811 may also be a semiconductor device or the like including a dedicated processing circuit. The processor 811 is not limited to an electronic circuit using electronic logic elements, and may be realized by an optical circuit using optical logic elements. The processor 811 may also include an arithmetic function based on quantum computing.

[0084] The processor 811 performs arithmetic processing based on data and software (programs) input from each device, etc. configured inside the computer 81, and can output the arithmetic results and control signals to each device, etc. The processor 811 may control each component constituting the computer 81 by executing the OS (Operating System) of the computer 81, applications, etc.

[0085] Each device (such as the grip estimation device 17 and / or the controller 171) in the above-described embodiment may be realized by one or more processors 811. Here, the processor 811 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the respective electronic circuits may communicate with each other via wire or wirelessly.

[0086] The main memory device 812 is a memory device that stores instructions executed by the processor 811, various data, etc., and information stored in the main memory device 812 is read by the processor 811. The auxiliary memory device 813 is a memory device other than the main memory device 812. Note that these memory devices refer to any electronic component that can store electronic information, and may be semiconductor memories. The semiconductor memories may be either volatile memories or non-volatile memories. The memory devices for saving various data in each device (such as the gripping estimation device 17 and / or the controller 171) in the above-described embodiments may be realized by the main memory device 812 or the auxiliary memory device 813, or may be realized by an internal memory built into the processor 811. For example, the memory unit 105 in the above-described embodiments may be realized by the main memory device 812 or the auxiliary memory device 813.

[0087] A plurality of processors may be connected (coupled) to one storage device (memory), or a single processor may be connected. A plurality of storage devices (memories) may be connected (coupled) to one processor. When each device (such as the grip estimation device 17 and / or the controller 171) in the above-described embodiments is configured with at least one storage device (memory) and a plurality of processors connected (coupled) to this at least one storage device (memory), a configuration in which at least one of the plurality of processors is connected (coupled) to at least one storage device (memory) may be included. This configuration may also be realized by storage devices (memories) and processors included in a plurality of computers. Furthermore, a configuration in which a storage device (memory) is integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache) may be included.

[0088] The network interface 814 is an interface for connecting to the communication network 82 wirelessly or via a wire. The network interface 814 may be an appropriate interface, such as one that conforms to an existing communication standard. Information may be exchanged with an external device 83a connected via the communication network 82 via the network interface 814. The communication network 82 may be any one of a WAN (Wide Area Network), a LAN (Local Area Network), a PAN (Personal Area Network), etc., or a combination thereof, as long as information is exchanged between the computer 81 and the external device 83a. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0089] The device interface 815 is an interface such as a USB that directly connects to the external device 83b.

[0090] The external device 83a is a device connected to the computer 81 via a network, and the external device 83b is a device connected directly to the computer 81.

[0091] The external device 83a or the external device 83b may be, for example, an input device. The input device is, for example, a device such as a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, or a touch panel, and provides acquired information to the computer 81. Alternatively, the external device 83a or the external device 83b may be a device including an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0092] Furthermore, the external device 83a or the external device 83b may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), or an organic EL (Electro Luminescence) panel, or may be a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0093] The external device 83a or 83b may be a storage device (memory). For example, the external device 83a may be a network storage or the like, and the external device 83b may be a storage such as an HDD.

[0094] Furthermore, the external device 83a or the external device 83b may be a device having some of the functions of the components of each device (such as the gripping estimation device 17 and / or the controller 171) in the above-described embodiments. That is, the computer 81 may transmit or receive some or all of the processing results of the external device 83a or the external device 83b. For example, the external device 83a or the external device 83b may be at least one of the gripping device 11, the observation device 13, the weighing device 15, the gripping estimation device 17, etc. in the above-described embodiments.

[0095] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0096] In this specification (including the claims), when expressions such as "using data as input / based on / according to / in response to" (including similar expressions) are used, unless otherwise specified, this includes cases where various data itself is used as input, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is used as input. Furthermore, when a statement is made that a result is obtained "based on / according to / in response to data," this includes cases where the result is obtained based solely on the data in question, as well as cases where the result is obtained as a result of being influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when a statement is made that "data is output," this includes cases where various data itself is used as output, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is output, unless otherwise specified.

[0097] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that encompass any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0098] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor or dedicated arithmetic circuit, it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0099] When used in this specification (including the claims), terms implying containing or possessing (e.g., "comprising / including" and "having") are intended to be open-ended terms that include containing or possessing things other than the object designated by the object of the term. When the object of such a term implies no quantity or a singular number (e.g., an article such as "a" or "an"), the expression should be construed as not being limited to a specific number.

[0100] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0101] In this specification, when a particular advantage / result is described as being obtained from a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained from one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, etc., and that the effect is not necessarily obtained by the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or states, etc. are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same or a similar configuration.

[0102] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that hardware may include an electronic circuit, a device including an electronic circuit, etc.

[0103] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices (memories) may store only a portion of the data, or may store the entire data.

[0104] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention derived from the content defined in the claims and their equivalents. For example, in all of the above-described embodiments, when numerical values ​​or formulas are used in the explanation, they are shown as examples and are not limited to these. Furthermore, the order of each operation in the embodiments is shown as an example and is not limited to these.

Claims

1. a food gripper having a first member and a second member supported by a base portion so as to be opposed to each other and openable and closable; a gripping device including the gripper; a control means for controlling the operation of the gripping device so that the gripper inserts the food from above in a vertical direction to grip a part of the food, and releases the gripped food at a target position; The gripper is the first member and the second member have surface shapes that form a space in which the food is contained when the gripper is in a closed state, the first member and the second member face each other in the closed state and have a tip portion located below the base portion in the vertical direction, the first surface shape of the first member has a surface that is inclined from the base portion to the tip portion toward the second member with respect to the vertical direction in the closed state; The gripper system.

2. the control means determines an area in which the food is to be gripped within the area in which the food is spread, and controls the operation of the gripping device so that the gripper is inserted into the determined area to grip the food. The system of claim 1 .

3. The control means determines to grab the food from an area where the state of the food satisfies a predetermined condition among the areas where the food is spread.

3. The system according to claim 1 or 2.

4. The control means acquires the state of the food in a plurality of areas among the areas in which the food is spread, and determines to grab the food from an area where the state of the food satisfies the predetermined condition. The system of claim 3.

5. a measuring device for acquiring the state of the food in the area where the food spreads; 5. The system according to claim 3 or 4.

6. the predetermined condition includes a condition regarding uncertainty of the amount of the food product grasped by the gripper; 6. A system according to any one of claims 3 to 5.

7. the control means determines a position where the gripper is to be inserted from above in a vertical direction to grip the portion of the food product, and controls the operation of the gripping device so that the gripper is inserted into the determined position and grips the portion. The system of claim 1 .

8. The control means determines the insertion position depending on the state of the food. The system of claim 7.

9. The state of the food product includes information related to the surface irregularities of the area where the food product is spread.

9. The system of claim 3, 4, or 8.

10. the inclined surface of the first surface shape is a plane.

10. A system according to any one of claims 1 to 9.

11. the tip of the first member is an edge of the inclined surface; 11. A system according to any one of claims 1 to 10.

12. the inclined surface of the first surface shape is configured to be inclined with respect to a direction in which the gripper is inserted into the food product while the gripper is inserted into the food product to grasp the portion of the food product.

12. A system according to any one of claims 1 to 11.

13. The tip portion is formed with a linear edge.

13. A system according to any one of claims 1 to 12.

14. The gripper is the second surface shape of the second member has a shape having a surface that is inclined toward the first member with respect to the vertical direction from the base portion toward the tip portion in the closed state; The gripper 14. A system according to any one of claims 1 to 13.

15. the inclined surface of the second surface shape is a plane. The system of claim 14.

16. the tip of the second member is an edge of the inclined surface; 16. A system according to claim 14 or 15.

17. the inclined surface of the second surface shape is configured to be inclined with respect to a direction in which the gripper is inserted into the food product while the gripper is inserted into the food product to grasp the portion of the food product.

17. A system according to any one of claims 14 to 16.

18. The food product, the portion of which is to be gripped by the gripper being inserted vertically from above, is contained in a container, the control means controls the operation of the gripping device so as to insert the gripper vertically into the food contained in the container from above to grab a portion of the food from the container, and release the grabbed food at the target position.

18. A system according to any one of claims 1 to 17.

19. the control means controls the operation of the gripping device to insert the gripper into the food from above in a vertical direction and grip a portion of the food so that a target amount of the food is gripped by the gripper.

19. A system according to any one of claims 1 to 18.

Images