Arithmetic device, control device, processing system, search method, and program

Abstract

This calculation device is a searching means which finds a solution while finding solution candidates for an optimal problem having constraints including variables, wherein, when the found solution candidates do not satisfy the constraints, the searching means changes the values of the variables pertaining to the constraints, which are stricter than the predetermined constraints, to values that can be assumed by random variables, or changes the values pertaining to the constraints to any values that can be taken by the variables included in a first area in which the values of the variables pertaining to the constraints are included and a second area that does not have a common area with the first area, and re-obtains the solution candidates after the change.

Description

Arithmetic device, control device, processing system, search method, and recording medium

[0001] The present disclosure relates to an arithmetic device, a control device, a processing system, a search method, and a recording medium.

[0002] Robots are used in various fields, such as logistics. Some robots operate autonomously. Patent Document 1 discloses a related technology relating to a device that generates a trajectory plan for the tip of a robot arm to move from a start point to an end point.

[0003] Japanese Patent Application Laid-Open No. 2021-079482

[0004] Incidentally, in order to execute appropriate processing, it is necessary to obtain an appropriate plan for processing when considering the control of moving an object by a robot as described in Patent Document 1, heat flow control that controls temperature by turning on or off either heating or cooling, and a travel problem with constraints on the order in which locations should be visited at a certain time, etc. Therefore, there is a demand for a technology that can obtain an appropriate plan for processing.

[0005] One of the objectives of each aspect of the present disclosure is to provide an arithmetic device, a control device, a processing system, a search method, and a recording medium that can solve the above-mentioned problems.

[0006] In order to achieve the above object, according to one aspect of the present disclosure, a computing device includes a search means for searching for a solution while searching for solution candidates for an optimization problem with constraints including variables, and when the constraints are not satisfied for the searched solution candidate, the search means changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a possible value of a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the possible values ​​of the variable included in a second region that does not have a region in common with the first region, and searches for the solution candidate again after the change.

[0007] To achieve the above object, according to another aspect of the present disclosure, a control device controls a controlled object based on the solution obtained by the above arithmetic device.

[0008] To achieve the above object, according to another aspect of the present disclosure, a processing system includes the above control device and a robot that is the object of control.

[0009] In order to achieve the above object, according to another aspect of the present disclosure, a search method includes: obtaining a solution while obtaining candidate solutions for an optimization problem with constraints including variables; and, if the constraints are not satisfied for the obtained candidate solutions, changing the value of the variable related to the constraints that are stricter than predetermined constraints to a possible value of a random variable; or changing the value of the variable related to the constraints from a first region that includes the value of the variable related to the constraints to one of the possible values ​​of the variable included in a second region that does not have a region in common with the first region; and then re-obtaining candidate solutions after the change.

[0010] In order to achieve the above object, according to another aspect of the present disclosure, a recording medium stores a program that causes a computer to execute the following steps: find a solution to an optimization problem with constraints including variables while finding solution candidates; if the constraints are not satisfied for the found solution candidates, change the value of the variable related to the constraints that are stricter than predetermined constraints to a possible value of a random variable; or change the value of the variable related to the constraints from a first region that includes the value of the variable related to the constraints to one of the possible values ​​of the variable included in a second region that does not have a region in common with the first region; and find solution candidates again after the change.

[0011] According to aspects of the present disclosure, an appropriate plan for treatment can be determined.

[0012] 1 is a diagram illustrating an example of the configuration of a processing system according to an embodiment of the present disclosure; FIG. 2 is a diagram illustrating an example of the configuration of a computing device according to an embodiment of the present disclosure; FIG. 3 is a diagram illustrating an example of the configuration of a generation unit according to an embodiment of the present disclosure; FIG. 4 is a diagram illustrating an example of position coordinates of a robot hand and position coordinates of an object according to an embodiment of the present disclosure; FIG. 5 is a diagram illustrating an example of each process and a movement path of an object according to an embodiment of the present disclosure; FIG. 6 is a first diagram illustrating an image of a solution obtained by using a gradient descent method according to an embodiment of the present disclosure; FIG. 7 is a second diagram illustrating an image of a solution obtained by using a gradient descent method according to an embodiment of the present disclosure; FIG. 8 is a diagram illustrating an example of a sequence of a plan generated by a generation unit according to an embodiment of the present disclosure; FIG. 9 is a diagram illustrating an example of the configuration of a control device according to an embodiment of the present disclosure; FIG. 10 is a diagram illustrating an example of a control signal of a plan generated by a control unit according to an embodiment of the present disclosure; FIG. 11 is a diagram illustrating an example of the configuration of a computing device with a minimum configuration according to an embodiment of the present disclosure; FIG. 12 is a diagram illustrating an example of the processing flow of a processing system according to an embodiment of the present disclosure; FIG. 13 is a diagram illustrating an example of the configuration of a computing device with a minimum configuration according to an embodiment of the present disclosure; FIG. 14 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment.

[0013] Hereinafter, an embodiment will be described in detail with reference to the drawings. <Embodiment> A processing system 1 according to an embodiment of the present disclosure is a system that can appropriately execute a process by determining an appropriate plan for the process and executing the process based on the determined plan. Examples of the processing system 1 include a robot system, a heat flow control system that controls temperature by turning on or off either heating or cooling, and a system that determines the order of visits, i.e., which locations to visit at a certain time. Below, a specific example will be described in which the processing system 1 is a robot system that grasps and moves an object.

[0014] (Configuration of Processing System) FIG. 1 is a diagram illustrating an example of the configuration of a processing system 1 according to an embodiment of the present disclosure. As shown in FIG. 1, the processing system 1 includes a calculation device 10, a control device 20, a robot 30, and an imaging device 40. Note that FIG. 1 illustrates a floor surface F, an object M, a tray T, and a cardboard box C. Below, the processing system 1 will be described using an example in which the processing system 1 obtains a plan for moving the object M from the tray T to the cardboard box C, and controls the movement of the object M from the tray T to the cardboard box C based on the obtained plan.

[0015] As shown in FIG. 1 , the robot 30 includes a robot arm 301, a base 302, and a robot hand 303. The robot arm 301 is connected to the base 302. The robot hand 303 is connected to the end opposite to the end where the robot arm 301 is connected to the base 302. The robot hand 303 includes, for example, two or more artificial fingers that resemble the fingers of a human or an animal, or a vacuum. The robot hand 303 grasps the object M in response to a control signal output by the control device 20. The robot arm 301 moves the object M from a source to a destination in response to the control signal output by the control device 20.

[0016] In each embodiment of the present disclosure, "grasping" includes "suction" in which an object M is sucked in by a vacuum or the like, and "clamping" in which an object is pinched between two or more pseudo-fingers that mimic the fingers of a human or animal.

[0017] The image capturing device 40 captures an image of the state of the object M. The image capturing device 40 is, for example, a depth camera, and is capable of identifying the state (i.e., the position and posture) of the object M. The image captured by the image capturing device 40 is represented, for example, by colored point cloud data, and includes three-dimensional information of the captured object. The image capturing device 40 outputs the captured image to the generation unit 102.

[0018] 2 is a diagram illustrating an example of the configuration of the arithmetic device 10 according to an embodiment of the present disclosure. As shown in FIG. 2, the arithmetic device 10 includes an input unit 101 and a generation unit 102.

[0019] The input unit 101 inputs task goals and constraint conditions to the generation unit 102. Examples of task goals include information indicating the type of object M, the number of objects M to be moved, the source of the object M, and the destination of the object M. Examples of constraint conditions include a no-entry area when moving the object M, an area that deviates from the range of motion of the robot 30, and further conditions on the surface of the object M related to gripping the object M, releasing the grip of the object M, or changing the object M. Note that the input unit 101 may accept an input from a user as a task goal, such as "move three items A from a cardboard box C to a tray T," and specify that the type of object M to be moved is item A, the number of items M to be moved is three, the source of the object M is cardboard box C, and the destination of the object M is tray T, and input the specified information to the generation unit 102. Alternatively, the position of the object M identified in the image captured by the image capture device 40 may be set as the origin of movement of the object M. The input unit 101 may, for example, receive from the user the position of an obstacle along the path of the object M from the origin to the destination as a constraint condition indicating a no-entry area, and input the information to the generation unit 102. Alternatively, a file indicating the constraint conditions may be stored in a storage device, and the input unit 101 may input the constraint conditions indicated by the file to the generation unit 102, or a fourth processing unit 102d (described later) of the generation unit 102 may read the constraint conditions directly from the file, or both. In other words, any acquisition method may be used as long as the generation unit 102 can acquire the necessary task goal and the necessary constraint conditions. Details of how to define the constraint conditions will be described later.

[0020] 3 is a diagram illustrating an example of the configuration of the generation unit 102 according to an embodiment of the present disclosure. As illustrated in FIG. 3, the generation unit 102 includes a first processing unit 102a, a second processing unit 102b, a third processing unit 102c, a fourth processing unit 102d, and a fifth processing unit 102e (an example of a search unit, an example of a determination unit).

[0021] The first processing unit 102a recognizes the robot 30. For example, the first processing unit 102a recognizes a robot model using CAD (Computer Aided Design) data. This CAD data includes information indicating the shape of the robot 30 and information indicating the range of motion, such as the reach range of the robot arm 301. The shape includes dimensions. The CAD data is, for example, drawing data designed using CAD.

[0022] The first processing unit 102a also recognizes the environment around the robot 30. For example, the first processing unit 102a acquires images captured by the camera device 40. The images captured by the camera device 40 include information captured by the camera and depth information. This depth information corresponds to the colored point cloud data described above. The first processing unit 102a recognizes the position and shape of obstacles from the acquired images. Here, obstacles refer to all objects present within the camera range of the camera device 40, other than the target object M that the robot 30 is to move to its destination. As described above, the camera device 40 is capable of acquiring three-dimensional information of objects within the camera range. Therefore, the first processing unit 102a can recognize the environment around the robot 30, including the position and shape of obstacles. Note that the first processing unit 102a is not limited to recognizing the environment around the robot 30 from images captured by the camera device 40. For example, the first processing unit 102a may recognize the environment around the robot 30 using a three-dimensional occupancy map (Octomap), CAD data, AR (Augmented Reality) markers, etc. This CAD data includes information indicating the shape of obstacles. The shape includes dimensions.

[0023] The first processing unit 102a also recognizes the release position at the destination of the object M. For example, when the destination is a container (e.g., a tray T), the first processing unit 102a recognizes the release position by machine learning using model-based matching. Model-based matching is a technique for determining the position and orientation of an object by using image data obtained from a camera or the like and shape and structural data of the object (in this case, a container) whose position and orientation are to be acquired, and matching the shape and structural data with the object extracted from the image. Note that the first processing unit 102a is not limited to a device that recognizes the release position by machine learning using model-based matching. For example, the first processing unit 102a may also recognize the release position using an AR marker.

[0024] The second processing unit 102b also recognizes a pedestal 302 (described later) of the robot 30. For example, the second processing unit 102b recognizes the pedestal 302 by acquiring CAD data. This CAD data includes information indicating the shape of the pedestal 302. The shape includes dimensions. This allows the second processing unit 102b to recognize the Z coordinate of the top surface of the pedestal 302 in the coordinate system as the height of the pedestal 302.

[0025] The third processing unit 102c recognizes the state (i.e., position and posture) of the object M. For example, the third processing unit 102c recognizes the position of the object M by performing machine learning using model-based matching. Furthermore, the third processing unit 102c recognizes the posture of the object M by using a technique for generating a bounding box, such as an AABB (Axis Aligned Bounding Box) or an OBB (Oriented Bounding Box), for the object M whose position has been identified. Note that the third processing unit 102c may classify the object M using clustering, which is a machine learning technique, for the image captured by the image capture device 40, and identify the state of the object M by using a technique for generating a bounding box.

[0026] Furthermore, the third processing unit 102c acquires the height of the object M. For example, the third processing unit 102c recognizes the object M by acquiring CAD data. This CAD data includes information indicating the shape of the object M. The shape includes dimensions. This allows the third processing unit 102c to recognize the Z coordinate of the object M in that coordinate system as the height of the object M. Note that the third processing unit 102c may recognize the height of the object M by subtracting the Z coordinate of the base 302 from the Z coordinate of the top surface of the object M.

[0027] The fourth processing unit 102d acquires various constraint conditions. Then, the fourth processing unit 102d sets the acquired various constraint conditions. Here, the method for defining the constraint conditions will be described in detail.

[0028] 4 is a diagram illustrating an example of the position coordinates of the robot hand 303 and the position coordinates of the object M according to an embodiment of the present disclosure. First, the position coordinates of the robot hand 303 at time t and the position coordinates of the object M at time t are defined using the notation in FIG. 4. That is, the position coordinate x(robo, t) of the robot hand 303 at time t is expressed as in Equation (1). Furthermore, the position coordinate x(obj, t) of the object M at time t is expressed as in Equation (2).

[0029]

[0030]

[0031] Note that the position coordinates x(robo, t) and the position coordinates x(obj, t) respectively represent the positions of the robot hand 303 and the object M at time t in three-dimensional space in which the robot 30 including the robot hand 303 operates and the object M moves.

[0032] In addition, to define the constraints, we introduce a switch variable s(P&P, i, t) shown in equation (3).

[0033]

[0034] When the switch variable s(P&P, i, t) is 0, it indicates that the object M is not being grasped by the robot hand 303. When the switch variable s(P&P, i, t) is 1, it indicates that the object M is being grasped by the robot hand 303.

[0035] Here, consider a motion plan for a robot in which the robot hand 303 grasps an object M and moves the object M over the shortest distance. In this case, the goal is to minimize the difference between the position of the robot hand 303 at time t and the position of the robot hand 303 at time t+1 (i.e., the distance traveled by the robot hand 303 from time t to time t+1). Therefore, the objective function f can be expressed as in equation (4).

[0036]

[0037] The constraints can be expressed as in equations (5) to (7).

[0038]

[0039]

[0040]

[0041] The first factor on the left side of equation (5) is 0 when the robot hand 303 is gripping the object M. Furthermore, the second factor on the left side of equation (5) is 0 when the position of the object M at time t and the position of the object M at time t+1 are the same. Therefore, equation (5) is a constraint that always holds when the robot hand 303 is gripping the object M, regardless of the position of the object M (i.e., regardless of whether the robot hand 303 is moving or not). Furthermore, equation (5) is a constraint that always holds when the position of the object M at time t and the position of the object M at time t+1 are the same (i.e., when the object M is not moving), regardless of whether the robot hand 303 is gripping the object M.

[0042] The first factor on the left side of equation (6) is 0 when the robot hand 303 is not gripping the object M. Furthermore, the second factor on the left side of equation (6) is 0 when the position of the object M at time t and the position of the robot hand 303 at time t are the same. Therefore, equation (6) is a constraint that always holds when the robot hand 303 is not gripping the object M, regardless of whether the position of the object M at time t and the position of the robot hand 303 at time t are the same. Furthermore, equation (6) is a constraint that always holds when the position of the object M at time t and the position of the robot hand 303 at time t are the same, regardless of whether the robot hand 303 is gripping the object M.

[0043] The first factor on the left side of formula (7) is 0 when the robot hand 303 is not grasping the object M. Furthermore, the second factor on the left side of formula (7) is 0 when the robot hand 303 is grasping the object M. Therefore, formula (7) is a constraint that always holds true in either the case where the robot hand 303 is not grasping the object M or the case where the robot hand 303 is not grasping the object M. Note that the constraint according to formula (7) is set because the robot hand 303 is either grasping the object M or not.

[0044] The fourth processing unit 102d sets the constraint conditions described above. As shown in equations (5) to (7), a constraint condition that generally requires A(x)=0 or B(x)=0 for a variable x can be expressed as A(x)B(x)=0. Note that when a constraint other than the above-described constraint conditions is to be set, the fourth processing unit 102d may set the constraint conditions by adding additional constraint conditions.

[0045] The fifth processing unit 102e generates a sequence indicating the flow of motion of the robot 30 based on task targets determined by processing by the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c and constraint conditions set by processing by the fourth processing unit 102d. For example, the fifth processing unit 102e acquires task targets from the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c. The fifth processing unit 102e also acquires constraint conditions from the fourth processing unit 102d. The fifth processing unit 102e adds the constraint conditions acquired from the fourth processing unit 102d to the constraint conditions input from the input unit 101. Then, based on the acquired work goal and constraint conditions, the fifth processing unit 102e generates information indicating each state of the robot 30 at each time step from the state at the source of the object M to the state at the destination of the object M, which is necessary for the control unit 201 of the control device 20, which will be described later, to generate a control signal to control the robot 30 (including, for example, the type of object M, the position and posture of the robot 30, the strength of the grip of the object M, the operation of the robot 30 (e.g., the approach operation to approach the object M (equivalent to the processing of the approach process in Figure 5, which will be described later), the pick operation to grip the object M (equivalent to the processing of the pick process in Figure 5), the carry operation to move the arm to correctly move the gripped object M to the destination (equivalent to the processing of the carry process in Figure 5), and the place operation to release the grip of the object M (equivalent to the processing of the place process in Figure 5)). In other words, the sequence is information indicating each state of the robot 30 at each time step from the state at the origin of the object M to the state at the destination of the object M, which is necessary for the control unit 201 described later to generate a control signal to control the robot 30.

[0046] 5 is a diagram illustrating an example of each process and a movement path of the object M according to an embodiment of the present disclosure. When a task goal and various constraints are set, the process and the movement path of the object M in the sequence for moving the object M shown in FIG. 5 are determined by the fifth processing unit 102e executing a simulation, such as reducing the amount of energy consumed by the robot 30 as much as possible using an objective function f, shortening the trajectory of the movement of the robot hand 303 as much as possible using an objective function f, or shortening the movement path of the object M as much as possible using an objective function f. As shown in FIG. 5 , examples of processes for moving the object M from the source to the destination include an approach process in which the robot hand 303 approaches the object M, a pick process in which the robot hand 303 grasps the object M, a carry process in which the robot hand 303 moves the object M, and a place process in which the robot hand 303 releases its grasp of the object M.

[0047] Here, a method will be described in which the fifth processing unit 102e determines the trajectory of the movement of the robot hand 303 for each time step through simulation. For simplicity, variables (e.g., x, y) that affect the trajectory of the movement of the robot hand 303 are defined, and a function f(x, y) that represents the trajectory of the movement of the robot hand 303 is used as an objective function, and a method will be described in which the objective function f(x, y) is minimized. Furthermore, for example, a constraint condition such as x+y=0 is set due to a restriction on the movable area of ​​the robot hand 303. In such a case, the Lagrangian function L is given as shown in Equation (8) using a positive constant λ.

[0048]

[0049] The fifth processing unit 102e may then use gradient descent to find a solution. Examples of gradient descent include simulated annealing and a gradient method known as a primal-dual interior point method. FIG. 6 is a first diagram illustrating an image of a solution found using gradient descent in an embodiment of the present disclosure. In this case, the fifth processing unit 102e identifies a desired solution indicated by a star in FIG. 6 at which the objective function f(x, y) is minimized by repeatedly searching for a region that satisfies the constraint condition x + y = 0 and that has a local minimum value by differentiating the Lagrangian function L. Note that the Lagrangian function L expressed by equation (8) is merely an example, and any Lagrangian function generally used in continuous optimization may be used. For example, when using an optimization algorithm based on a gradient method known as a primal-dual interior point method, the Lagrangian function L has a value of zero in a region where the constraint condition is not violated and takes on a value of infinity as soon as the region where the constraint condition is violated is entered, which is called a barrier function.

[0050] However, when a solution is found using gradient descent with the derivative of a function under certain constraints, as described above, a problem commonly referred to as a "non-convex constraint" may occur, in which the searchable range of solutions is limited by the constraints, making it impossible to find a local minimum. In other words, when solving an optimization problem in which the objective function f is minimized (or maximized) using only gradient descent, the solution candidates may converge to an extreme value and be unable to escape from that extreme value due to the constraints. As a result, it may be impossible to find a solution that minimizes (maximizes) the objective function f. More specifically, for example, when solving an optimization problem in which the objective function f is minimized (or maximized) using a simulated annealing technique, the temperature is raised, the search range for solution candidates is widened (relaxed), and solution candidates are found within the searchable range while the temperature is gradually lowered. However, if the searchable range is divided while the temperature is gradually lowered, an appropriate solution candidate cannot be found unless an appropriate search range is searched before the division. Furthermore, if an inappropriate extreme solution candidate is found using the simulated annealing technique, the temperature is raised again to a high temperature, and then gradually lowered while the extreme solution candidate is found again within the searchable range.If an appropriate search range cannot be found before it is divided, then no matter how many times the search is repeated, an appropriate solution cannot be found using only the simulated annealing technique.

[0051] The fifth processing unit 102e according to an embodiment of the present disclosure prevents candidate solutions from converging to extreme values ​​and becoming stuck when solving an optimization problem, thereby efficiently obtaining an appropriate solution. Here, a method for preventing candidate solutions from converging to extreme values ​​and becoming stuck when solving an optimization problem by the fifth processing unit 102e and efficiently obtaining an appropriate solution will be described. Here, it is assumed that the fifth processing unit 102e obtains the extreme value using a gradient descent method. However, for ease of understanding, the method for efficiently obtaining an appropriate solution by the fifth processing unit 102e will be described using one variable instead of multiple variables.

[0052] Here, we consider minimizing the objective function f(θ) for the objective function f(θ) expressed by equation (9) and the constraints expressed by equations (10) and (11). That is, we consider that the fifth processing unit 102e minimizes the objective function f(θ) when the Lagrangian function L is expressed by equation (12). Here, λ and μ are positive constants.

[0053]

[0054]

[0055]

[0056]

[0057] Here, the function max(a, b) is a function that returns the larger value of two numbers a and b.

[0058] 7 is a second diagram illustrating an image of a solution obtained by using the gradient descent method according to an embodiment of the present disclosure. For example, for the objective function f and constraints expressed by equations (9) to (11), the fifth processing unit 102e searches for a solution θ using the minimum descent method to find the minimum of the objective function f(θ), and the solution candidate converges to the extreme value of θ = 0 shown in FIG. 7. In this case, the fifth processing unit 102e cannot obtain the correct solution, θ = 1, by searching using only the gradient descent method.

[0059] In this way, the solution candidate may converge to an inappropriate extreme value and be unable to escape from that extreme value. Therefore, for example, the fifth processing unit 102e is a search unit that searches for a solution candidate while searching for a solution candidate for an optimization problem with constraints including variables. If the constraints are not satisfied for the searched solution candidate (e.g., the inappropriate extreme value), the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than the predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not share a region with the first region, and then searches for the solution candidate again after the change. Specifically, the fifth processing unit 102e performs processing using the first algorithm shown below to determine whether the constraint condition expressed by A(x)B(x)=0 (note that in the specific example shown here, the first factor θ on the left side of Equation (10) corresponds to A(x), and the second factor (1-θ) on the left side corresponds to B(x)) is a highly discrete constraint condition. That is, the fifth processing unit 102e sets parameters r, N, M, and K for a constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0. Parameter r is a parameter for setting a criterion for determining the strength of the discreteness of A(x) for which A(x)=0 and B(x) for which B(x)=0. Parameter r takes a value greater than 0 and less than or equal to 1. The closer parameter r is to 1, the stronger the discreteness of A(x) for which A(x)=0 and B(x) for which B(x)=0. Parameter N is a parameter for setting a criterion for the proportion of detections of points that do not satisfy A(x)B(x)=0. Parameter M is a parameter for setting the number of times to repeat detection of points that do not satisfy A(x)B(x)=0. Parameter K is a parameter representing the number of times that the detection of points that do not satisfy A(x)B(x)=0 is repeated M times and the number of times is equal to or greater than the criterion parameter N. The initial value of the parameter K is zero.

[0060] The fifth processing unit 102e determines positive values ​​for each of the parameters r, N, and M. The fifth processing unit 102e then samples xA, which satisfies A(x) = 0, and xB, which satisfies B(x) = 0. The fifth processing unit 102e then sets a step width α by dividing the line segment between the sampled xA and xB into a predetermined number of equal parts (e.g., N = 100). Each time the fifth processing unit 102e moves from one of xA or xB to the other by the step width α, the fifth processing unit 102e determines whether A(x)B(x) = 0 is satisfied. Note that in the specific example shown here, θ can only take on the values ​​0 or 1. Therefore, in the specific example shown here, the fifth processing unit 102e determines that A(x)B(x) = 0 is satisfied for the start point and end point, but that A(x)B(x) = 0 is not satisfied for the other points. The specific example shown here is an example of the most discrete constraint, in which only the starting point and the ending point satisfy A(x)B(x)=0. If the fifth processing unit 102e determines N or more times that A(x)B(x)=0 is not satisfied, it adds 1 to the value of K. The fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 M-1 times. Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each sampled line segment of xA and xB each time the fifth processing unit 102e moves by a step width α, and calculates the cumulative total of the value of K. The fifth processing unit 102e then determines that the constraint expressed by A(x)B(x)=0, which satisfies K / M≧r, is a highly discrete constraint (an example of a constraint stricter than a predetermined constraint). In this way, the fifth processing unit 102e determines whether or not the constraint condition expressed by A(x)B(x)=0 is a highly discrete constraint condition by performing processing using the first algorithm.

[0061] Specifically, the fifth processing unit 102e may determine whether the constraint expressed by A(x)B(x)=0 is a highly discrete constraint by performing processing using the second algorithm described below. That is, the fifth processing unit 102e sets parameters N, M, and K for a constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0. The parameter N is a parameter for setting a criterion for the proportion of detection of points that do not satisfy A(x)B(x)=0. The parameter M is a parameter for setting the number of times to repeat detection of points that do not satisfy A(x)B(x)=0. The parameter K is a parameter representing the number of times that the number of times the detection of points that do not satisfy A(x)B(x)=0 is repeated M times will be equal to or greater than the criterion parameter N. The initial value of the parameter K is 0.

[0062] The fifth processing unit 102e determines positive values ​​for each of the parameters N and M. For example, the fifth processing unit 102e samples xA that satisfies A(x) = 0 and xB that satisfies B(x) = 0. The fifth processing unit 102e then sets a step width α by dividing the line segment between the sampled xA and xB into a predetermined number of equal parts (e.g., N = 100), for example. Each time the fifth processing unit 102e moves from one of xA or xB to the other by the step width α, the fifth processing unit 102e determines whether A(x)B(x) = 0 is satisfied. If the fifth processing unit 102e determines that A(x)B(x) = 0 is not satisfied N or more times, it adds 1 to the value of K. The fifth processing unit 102e samples xA that satisfies A(x) = 0 and xB that satisfies B(x) = 0 M-1 times. Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each of the sampled line segments xA and xB, each time the fifth processing unit 102e moves by the step width α, and calculates the cumulative total of the value of K. The fifth processing unit 102e then determines that the constraint expressed by A(x)B(x)=0, which satisfies K / M=1, is a highly discrete constraint (an example of a constraint that is stricter than a predetermined constraint). In the specific example shown here, the fifth processing unit 102e determines that equation (10) is a highly discrete constraint that satisfies K / M=1. In this way, the fifth processing unit 102e may determine whether the constraint expressed by A(x)B(x)=0 is a highly discrete constraint by performing processing using the second algorithm.

[0063] Next, the fifth processing unit 102e finds a solution candidate using a gradient descent method. After finding the solution candidate, the fifth processing unit 102e determines whether the solution candidate satisfies the constraints. If the fifth processing unit 102e determines that the solution candidate satisfies all constraints, the fifth processing unit 102e designates the solution candidate as the final solution and terminates the search. The fifth processing unit 102e then outputs the final solution (i.e., the trajectory of the movement of the robot hand 303 for each time step) to the control device 20. If the fifth processing unit 102e determines that the solution candidate does not satisfy at least one constraint, it identifies variables associated with the constraint represented by A(x)B(x)=0, which it determines to be a highly discrete constraint. The fifth processing unit 102e then changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from a first region containing the value of the variable to one of the possible values ​​of variables contained in a second region that does not share a value of the variable with the first region.

[0064] After changing the variable values, the fifth processing unit 102e re-finds a solution candidate using the gradient descent method. The fifth processing unit 102e determines whether the found solution candidate satisfies the constraints. If the fifth processing unit 102e determines that the solution candidate satisfies all constraints, it designates the solution candidate as the final solution and terminates the search. The fifth processing unit 102e then outputs the final solution (i.e., a sequence representing the trajectory of the robot hand 303 for each time step) to the control device 20. If the fifth processing unit 102e determines that the solution candidate does not satisfy at least one constraint, it identifies variables associated with the constraint represented by A(x)B(x)=0, which it determines to be a highly discrete constraint. The fifth processing unit 102e then changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from a first region containing the value of the variable to one of the possible values ​​of a variable included in a second region that does not share a value of the variable with the first region.

[0065] After changing the variable values, the fifth processing unit 102e re-finds solution candidates using the gradient descent method. The fifth processing unit 102e repeats the above process for the obtained solution candidates until it determines that all constraints are satisfied. The fifth processing unit 102e generates a plan sequence in this manner.

[0066] 8 is a diagram illustrating an example of a sequence TBL1 of a plan generated by the generator 102 according to an embodiment of the present disclosure. For example, the sequence TBL1 of a plan generated by the generator 102 is a sequence indicating each state of the robot 30 for each n time step from the origin to the destination of the object M, as shown in FIG.

[0067] 9 is a diagram illustrating an example of the configuration of the control device 20 according to an embodiment of the present disclosure. As shown in FIG. 9 , the control device 20 includes a control unit 201. The control unit 201 generates a control signal for controlling the robot 30 based on the sequence generated by the generation unit 102. That is, the control signal generates a control signal that realizes the posture of the object M and the movement path of the object M according to the sequence generated by the calculation device 10. The control unit 201 outputs the generated control signal to the robot 30.

[0068] 10 is a diagram illustrating an example of the control signal Cnt of the plan generated by the control unit 201 according to an embodiment of the present disclosure. For example, as shown in FIG. 10 , the control signal Cnt of the plan generated by the control unit 201 is each control signal for controlling the robot 30 for each n time steps from the origin to the destination of the object M.

[0069] 11 is a diagram illustrating an example of a processing flow of the processing system 1 according to an embodiment of the present disclosure. Here, the processing performed by the processing system 1 to generate a sequence and control the robot 30 will be described with reference to FIG. 11. It is assumed here that each of the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c performs the above-described processing.

[0070] The fourth processing unit 102d acquires various constraint conditions. Then, the fourth processing unit 102d sets the acquired various constraint conditions (step S1). For example, the fourth processing unit 102d acquires the constraint conditions expressed by equations (5) to (7). Then, the fourth processing unit 102d sets the constraint conditions expressed by the acquired equations (5) to (7).

[0071] The fifth processing unit 102e generates a sequence indicating the flow of motion of the robot 30 based on the task goals determined by the processing by the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c and the constraint conditions set by the processing by the fourth processing unit 102d (step S2). For example, the fifth processing unit 102e acquires the task goals from the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c. The fifth processing unit 102e also acquires the constraint conditions from the fourth processing unit 102d. The fifth processing unit 102e adds the constraint conditions acquired from the fourth processing unit 102d to the constraint conditions input from the input unit 101. Then, based on the acquired work goal and constraint conditions, the fifth processing unit 102e generates information indicating each state of the robot 30 at each time step from the state at the source of the object M to the state at the destination of the object M, which is necessary for the control unit 201 of the control device 20, which will be described later, to generate a control signal to control the robot 30 (including, for example, the type of object M, the position and posture of the robot 30, the strength of the grip of the object M, the operation of the robot 30 (e.g., the approach operation to approach the object M (equivalent to the processing of the approach process in Figure 5), the pick operation to grip the object M (equivalent to the processing of the pick process in Figure 5), the carry operation to move the arm to correctly move the gripped object M to the destination (equivalent to the processing of the carry process in Figure 5), and the place operation to release the grip of the object M (equivalent to the processing of the place process in Figure 5)).

[0072] For example, the fifth processing unit 102e determines the posture and movement path of the object M for each time step through simulation. Specifically, the fifth processing unit 102e sets parameters r, N, M, and K for a constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0 (step S201). The fifth processing unit 102e determines positive values ​​for each of the parameters r, N, and M (step S202). The fifth processing unit 102e then samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 (step S203). The fifth processing unit 102e sets the step size α by dividing the line segment between the sampled xA and xB into, for example, a predetermined number of equal N parts (e.g., N=100) (step S204). Each time the fifth processing unit 102e moves from one of xA and xB to the other by a step size α, it determines whether A(x)B(x)=0 is satisfied. If the fifth processing unit 102e determines that A(x)B(x)=0 is not satisfied N or more times, it adds 1 to the value of K (step S205). The fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 M-1 times (step S206). Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each sampled line segment of xA and xB each time the fifth processing unit 102e moves by a step size α, and calculates the cumulative total of the value of K (step S207). The fifth processing unit 102e then determines that the constraint expressed by A(x)B(x)=0 that satisfies K / M≧r is a highly discrete constraint (step S208). In this way, the fifth processing unit 102e determines whether the constraint condition represented by A(x)B(x)=0 is a highly discrete constraint condition by performing processing using the first algorithm. Note that the specific example of determining whether the constraint condition shown here is a highly discrete constraint condition is the processing using the first algorithm described above. However, the processing for determining whether the constraint condition is a highly discrete constraint condition is not limited to the processing using the first algorithm. For example, the processing for determining whether the constraint condition is a highly discrete constraint condition may be the processing using the second algorithm described above.

[0073] For example, the first factor on the left side of equation (7) is 0 when the robot hand 303 is not gripping the object M, and the second factor on the left side is 0 when the robot hand 303 is gripping the object M. In other words, there is no common area between A(x) indicated by the first factor on the left side and B(x) indicated by the second factor on the left side. Therefore, the fifth processing unit 102e determines that the constraint condition in equation (7) is the most discrete of the constraint conditions in equations (5) to (7).

[0074] Next, the fifth processing unit 102e finds a solution candidate using the gradient descent method (step S209). After finding the solution candidate, the fifth processing unit 102e determines whether the solution candidate satisfies the constraints (step S210). If the fifth processing unit 102e determines that the solution candidate satisfies all constraints (YES in step S210), the fifth processing unit 102e treats the solution candidate as the final solution and terminates the search. The fifth processing unit 102e outputs the final solution (i.e., a sequence that is the trajectory of the robot hand 303 for each time step) to the control device 20 (step S211). Then, the fifth processing unit 102e terminates the processing.

[0075] Furthermore, if the fifth processing unit 102e determines that at least one constraint is not satisfied for the solution candidate (NO in step S210), it identifies variables associated with the constraint determined to be a highly discrete constraint, i.e., the constraint represented by A(x)B(x)=0 (step S212). The fifth processing unit 102e then changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from the first region containing the variable's value to one of the possible values ​​of a variable included in a second region that does not share a variable value with the first region (step S213). For example, in the case of the constraints of equations (5) to (7), the fifth processing unit 102e identifies s(P&P, i, t) in equation (7) as the variable associated with the constraint determined to be a highly discrete constraint. The fifth processing unit 102e then changes s(P&P, i, t) from 0 to 1. Note that s(P&P, i, t) in the exemplified formula (7) can only take on the value 0 or 1, and therefore s(P&P, i, t) can only be changed from 0 to 1 or from 1 to 0. The fifth processing unit 102e returns to the processing of step S210.

[0076] (Advantages) The processing system 1 according to an embodiment of the present disclosure has been described above. In the arithmetic device 10 of the processing system 1, the fifth processing unit 102e (an example of a search means) is a processing unit that searches for a solution to an optimization problem with constraints including variables while searching for solution candidates, and if the constraints are not satisfied for the searched solution candidate, changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not share a region with the first region, and then searches for the solution candidate again after the change.

[0077] This allows for an appropriate plan for treatment to be determined.

[0078] In another embodiment of the present disclosure, the control device 20 may comprise a computing device 10 .

[0079] In another embodiment of the present disclosure, the robot 30 may include at least one of the control device 20 and the computing device 10 .

[0080] Next, a minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure will be described. FIG. 12 is a diagram illustrating an example of the configuration of a minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure. In the minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure, as shown in FIG. 12, a fifth processing unit 102e (an example of a search means) is a processing unit that searches for a solution candidate for an optimization problem with constraints including variables. If the constraints are not satisfied for the searched solution candidate, the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a possible value of a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the possible values ​​of the variable included in a second region that does not share a region with the first region, and then searches for a solution candidate again after the change. The fifth processing unit 102e can be realized, for example, using the functions of the fifth processing unit 102e illustrated in FIG. 2.

[0081] Next, processing of the arithmetic device 10 with the minimum configuration according to the embodiment of the present disclosure will be described. Fig. 13 is a diagram showing an example of a processing flow of the arithmetic device 10 with the minimum configuration according to the embodiment of the present disclosure. Here, processing of the arithmetic device 10 with the minimum configuration will be described with reference to Fig. 13.

[0082] The fifth processing unit 102e (an example of a search unit) searches for a solution candidate for an optimization problem with constraints including variables (step S101). If the constraints are not satisfied for the searched solution candidate, the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a possible value of a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the possible values ​​of the variable included in a second region that does not share a region with the first region, and searches for a solution candidate again after the change (step S102).

[0083] The above has described the minimum configuration of the arithmetic device 10 according to the embodiment of the present disclosure. This arithmetic device 10 can obtain an appropriate plan for processing.

[0084] The order of the processes in the embodiments of the present disclosure may be changed as long as the processes are performed appropriately.

[0085] Although the embodiments of the present disclosure have been described, the processing system 1, the arithmetic device 10, the control device 20, the robot 30, the imaging device 40, and other control devices may have a computer device inside. The above-described processing steps are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes this program to perform the above processing. Specific examples of computers are shown below.

[0086] FIG. 14 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in FIG. 14 , the computer 5 includes a CPU (Central Processing Unit) 6, a main memory 7, a storage 8, and an interface 9. For example, the processing system 1, the arithmetic device 10, the control device 20, the robot 30, the imaging device 40, and other control devices described above are each implemented in the computer 5. The operation of each of the processing units described above is stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8, loads it into the main memory 7, and executes the above-described processing in accordance with the program. The CPU 6 also allocates storage areas in the main memory 7 corresponding to each of the storage units described above in accordance with the program.

[0087] Examples of storage 8 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 8 may be an internal medium directly connected to the bus of computer 5, or an external medium connected to computer 5 via interface 9 or a communication line. Furthermore, if the program is distributed to computer 5 via a communication line, computer 5 that receives the program may load the program into main memory 7 and execute the above-described processing. In at least one embodiment, storage 8 is a non-transitory tangible storage medium.

[0088] The program may also implement some of the functions described above. Furthermore, the program may be a file that can implement the functions described above in combination with a program already stored in the computer device, a so-called differential file (differential program).

[0089] Although several embodiments of the present disclosure have been described, these embodiments are merely examples and do not limit the scope of the disclosure. Various additions, omissions, substitutions, and modifications may be made to these embodiments without departing from the spirit of the disclosure.

[0090] Note that part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0091] (Supplementary Note 1) A computing device comprising: a search means for searching for a solution while searching for solution candidates for an optimization problem with constraint conditions including variables, wherein, if the constraint conditions are not satisfied for the searched solution candidate, the search means changes the value of the variable related to the constraint condition stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and searches for the solution candidate again after the change.

[0092] (Supplementary Note 2) The computing device according to Supplementary Note 1, wherein the search means changes the value of the variable related to the constraint condition only for the variable that contributes to the violation of the constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and re-searches for solution candidates after the change.

[0093] (Supplementary Note 3) The arithmetic device according to Supplementary Note 1 or Supplementary Note 2, further comprising: a determination unit that determines whether the constraint condition is stricter than a predetermined constraint condition.

[0094] (Supplementary Note 4) A control device that controls a control target based on the solution obtained by the arithmetic device according to any one of Supplementary Note 1 to Supplementary Note 3.

[0095] (Supplementary Note 5) A processing system comprising: the control device according to Supplementary Note 4; and a robot that is an object to be controlled.

[0096] (Supplementary Note 6) A searching method comprising: obtaining a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables; and, if the constraint conditions are not satisfied for the obtained solution candidate, changing the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changing the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by the variable included in a second region that does not have a region in common with the first region, and then obtaining the solution candidate again after the change.

[0097] (Supplementary Note 7) A recording medium storing a program that causes a computer to execute the following steps: (1) obtaining a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, changing the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changing the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and then obtaining solution candidates again after the changes.

[0098] According to aspects of the present disclosure, an appropriate plan for treatment can be determined.

[0099] REFERENCE SIGNS LIST 1 Processing system 5 Computer 6 CPU 7 Main memory 8 Storage 9 Interface 10 Arithmetic unit 20 Control unit 30 Robot 40 Imaging device 101 Input unit 102 Generation unit 102a First processing unit 102b Second processing unit 102c Third processing unit 102d Fourth processing unit 102e Fifth processing unit 201 Control unit 301 Robot arm 302 Base 303 Robot hand C Cardboard F Floor M Object T Tray

Claims

1. a search means for searching for a solution while searching for solution candidates for an optimization problem with constraint conditions including variables, and when the constraint conditions are not satisfied for the searched solution candidates, changing the value of the variable related to the constraint conditions stricter than a predetermined constraint condition to a possible value of a random variable, or changing the value of the variable related to the constraint conditions from a first region including the value of the variable related to the constraint conditions to any of the possible values ​​of the variable included in a second region not sharing a region with the first region, and searching again for the solution candidates after the change; A computing device comprising:

2. The searching means includes: changing the value of the variable related to the constraint condition only for the variable that contributes to the violation of the constraint condition to a possible value of a random variable, or changing the value of the variable related to the constraint condition from a first region including the value of the variable related to the constraint condition to any of the possible values ​​of the variable included in a second region that does not have a common region with the first region, and re-obtaining solution candidates after the change; The computing device of claim 1 .

3. A determination means for determining whether the constraint condition is stricter than a predetermined constraint condition; The computing device of claim 1 .

4. A control device that controls a controlled object based on the solution obtained by the arithmetic unit of claim 1.

5. A control device according to claim 4; A robot to be controlled; A processing system comprising:

6. A computing device comprising: obtaining a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidates, changing the value of the variable related to the constraint conditions stricter than a predetermined constraint condition to a possible value of a random variable, or changing the value of the variable related to the constraint conditions from a first region including the value of the variable related to the constraint conditions to any of the possible values ​​of the variable included in a second region not sharing a region with the first region, and re-obtaining the solution candidates after the change; A search method including:

7. obtaining a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidates, changing the value of the variable related to the constraint conditions stricter than a predetermined constraint condition to a possible value of a random variable, or changing the value of the variable related to the constraint conditions from a first region including the value of the variable related to the constraint conditions to any of the possible values ​​of the variable included in a second region not sharing a region with the first region, and re-obtaining the solution candidates after the change; A program that causes a computer to execute the following.