Arithmetic device, control device, processing system, search method, and program

Abstract

This calculation device is a searching means which finds a solution while finding solution candidates for an optimal problem having constraints including variables, wherein, when the found solution candidates do not satisfy the constraints, the searching means changes the values of the variables pertaining to the constraints, which are stricter than the predetermined constraints, to values that can be assumed by random variables, or changes the values pertaining to the constraints to any values that can be taken by the variables included in a first area in which the values of the variables pertaining to the constraints are included and a second area that does not have a common area with the first area, and re-obtains the solution candidates after the change.

Description

[Technical Field]

[0001] The present disclosure relates to a computing device, a control device, a processing system, a search method, and program Regarding. [Background technology]

[0002] Robots are used in various fields, including logistics. Some robots operate autonomously. Patent Document 1 discloses a related technology relating to a device that generates a trajectory plan for the tip of a robot arm to move from a start point to an end point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-079482 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in order to be able to execute appropriate processing, it is necessary to obtain an appropriate plan for processing when considering the control of moving an object by a robot as described in Patent Document 1, heat flow control that controls temperature by turning on or off either heating or cooling, and a travel problem with constraints on the order in which locations should be visited at a certain time, etc. Therefore, there is a demand for a technology that can obtain an appropriate plan for processing.

[0005] The aspects of the present disclosure provide a computing device, a control device, a processing system, a search method, and program One of the aims is to provide [Means for solving the problem]

[0006] In order to achieve the above object, according to one aspect of the present disclosure, a computing device includes: a search means for searching for a solution while searching for solution candidates for an optimization problem with constraint conditions including variables, and when the constraint conditions are not satisfied for the searched solution candidate, changing the value of the variable related to the constraint condition, which has stronger discreteness than a predetermined constraint condition, to a value that can be taken by a random variable, or changing the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and searching again for the solution candidate after the change; .

[0007] To achieve the above object, according to another aspect of the present disclosure, a control device controls a controlled object based on the solution obtained by the above arithmetic device.

[0008] To achieve the above object, according to another aspect of the present disclosure, a processing system includes the above control device and a robot that is the object of control.

[0009] In order to achieve the above object, according to another aspect of the present disclosure, a searching method includes: The calculation device obtains a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, changes the value of the variable related to the constraint condition that is more discrete than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and obtains the solution candidate again after the change. .

[0010] To achieve the above object, according to another aspect of the present disclosure, a program includes: a computer is caused to execute the following: obtaining a solution while obtaining solution candidates for an optimization problem with constraint conditions including variables; if the constraint conditions are not satisfied for the obtained solution candidate, changing the value of the variable related to the constraint condition, which has stronger discreteness than a predetermined constraint condition, to a value that can be taken by a random variable; or changing the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region; and obtaining a solution candidate again after the change. . [Effects of the Invention]

[0011] According to aspects of the present disclosure, an appropriate plan for treatment can be determined. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of a configuration of a processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of a configuration of a computing device according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a diagram illustrating an example of a configuration of a generation unit according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of position coordinates of a robot hand and position coordinates of an object according to an embodiment of the present disclosure. [Figure 5] FIG. 2 is a diagram illustrating an example of each process and a moving path of an object according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a first diagram showing an image of a solution obtained by using the gradient descent method in one embodiment of the present disclosure. [Figure 7]FIG. 2 is a second diagram showing an image of a solution obtained by using the gradient descent method in one embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of a sequence of plans generated by a generation unit according to an embodiment of the present disclosure. [Figure 9] FIG. 2 is a diagram illustrating an example of a configuration of a control device according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating an example of a planned control signal generated by a control unit according to an embodiment of the present disclosure. [Figure 11] FIG. 1 is a diagram illustrating an example of a processing flow of a processing system according to an embodiment of the present disclosure. [Figure 12] FIG. 1 is a diagram illustrating an example of a configuration of a calculation device with a minimum configuration according to an embodiment of the present disclosure. [Figure 13] FIG. 2 is a diagram illustrating an example of a processing flow of a calculation device 10 having a minimum configuration according to an embodiment of the present disclosure. [Figure 14] FIG. 1 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the embodiments will be described in detail with reference to the drawings. <Embodiment> A processing system 1 according to an embodiment of the present disclosure is a system that can appropriately execute a process by determining an appropriate plan for performing a certain process and executing the process based on the determined plan. Examples of the processing system 1 include a robot system, a heat flow control system that controls temperature by turning either heating or cooling on or off, and a system that determines the order of visits, i.e., which locations to visit at a certain time. Below, a specific example will be described in which the processing system 1 is a robot system that grasps and moves an object.

[0014] (Processing system configuration) FIG. 1 is a diagram illustrating an example of the configuration of a processing system 1 according to an embodiment of the present disclosure. As shown in FIG. 1, the processing system 1 includes a calculation device 10, a control device 20, a robot 30, and an imaging device 40. Note that FIG. 1 illustrates a floor surface F, an object M, a tray T, and a cardboard box C. Below, the processing system 1 will be described using an example in which the processing system 1 determines a plan for moving the object M from the tray T to the cardboard box C, and performs control to move the object M from the tray T to the cardboard box C based on the determined plan.

[0015] As shown in FIG. 1, the robot 30 includes a robot arm 301, a base 302, and a robot hand 303. The robot arm 301 is connected to the base 302. The robot hand 303 is connected to the end opposite to the end where the robot arm 301 is connected to the base 302. The robot hand 303 includes, for example, two or more artificial fingers that resemble the fingers of a human or an animal, or a vacuum. The robot hand 303 grasps an object M in response to a control signal output by the control device 20. The robot arm 301 moves the object M from a source to a destination in response to the control signal output by the control device 20.

[0016] In each embodiment of the present disclosure, "grasping" includes "suction" in which an object M is sucked in by a vacuum or the like, and "clamping" in which an object is pinched between two or more pseudo-fingers that mimic the fingers of a human or animal.

[0017] The image capturing device 40 captures an image of the state of the object M. The image capturing device 40 is, for example, a depth camera, and can identify the state (i.e., the position and posture) of the object M. The image captured by the image capturing device 40 is represented, for example, by colored point cloud data, and includes three-dimensional information of the captured object. The image capturing device 40 outputs the captured image to the generation unit 102.

[0018] FIG. 2 is a diagram illustrating an example of the configuration of the arithmetic device 10 according to an embodiment of the present disclosure. As shown in FIG. 2, the calculation device 10 includes an input unit 101 and a generation unit 102.

[0019] The input unit 101 inputs task goals and constraint conditions to the generation unit 102. Examples of task goals include information indicating the type of object M, the number of objects M to be moved, the source of the object M, and the destination of the object M. Examples of constraint conditions include no-entry areas when moving the object M, areas that deviate from the range of motion of the robot 30, and conditions on the surface of the object M related to gripping the object M, releasing the grip of the object M, or changing the object M. Note that the input unit 101 may accept an input from a user as a task goal, such as "move three items A from a cardboard box C to a tray T," and specify that the type of object M to be moved is item A, the number of items M to be moved is three, the source of the object M is cardboard box C, and the destination of the object M is tray T, and input the specified information to the generation unit 102. Alternatively, the position of the object M identified in the image captured by the image capture device 40 may be set as the origin of movement of the object M. The input unit 101 may receive, for example, the positions of obstacles along the path of the object M from the origin to the destination as constraint conditions indicating no-entry areas from the user, and input the information to the generation unit 102. Alternatively, a file indicating the constraint conditions may be stored in a storage device, and the input unit 101 may input the constraint conditions indicated by the file to the generation unit 102, or a fourth processing unit 102d (described later) of the generation unit 102 may read the constraint conditions directly from the file, or both. In other words, any method may be used as long as the generation unit 102 can acquire the necessary task goal and the necessary constraint conditions. Details of how to define the constraint conditions will be described later.

[0020] 3 is a diagram illustrating an example of the configuration of the generation unit 102 according to an embodiment of the present disclosure. As shown in FIG. 3, the generation unit 102 includes a first processing unit 102a, a second processing unit 102b, a third processing unit 102c, a fourth processing unit 102d, and a fifth processing unit 102e (an example of a search unit, an example of a determination unit).

[0021] The first processing unit 102a recognizes the robot 30. For example, the first processing unit 102a recognizes a robot model using CAD (Computer Aided Design) data. This CAD data includes information indicating the shape of the robot 30 and information indicating the range of movement, such as the reach range of the robot arm 301. The shape includes dimensions. The CAD data is, for example, drawing data designed using CAD.

[0022] The first processing unit 102a also recognizes the environment around the robot 30. For example, the first processing unit 102a acquires images captured by the camera device 40. The images captured by the camera device 40 include information captured by the camera and depth information. This depth information corresponds to the colored point cloud data described above. The first processing unit 102a recognizes the position and shape of obstacles from the acquired images. Here, obstacles refer to all objects present within the image capture range of the camera device 40, other than the target object M that the robot 30 is to move to its destination. As described above, the camera device 40 is capable of acquiring three-dimensional information of objects within the image capture range. Therefore, the first processing unit 102a can recognize the environment around the robot 30, including the position and shape of obstacles. Note that the first processing unit 102a is not limited to recognizing the environment around the robot 30 from images captured by the camera device 40. For example, the first processing unit 102a may recognize the environment around the robot 30 using a three-dimensional occupancy map (Octomap), CAD data, AR (Augmented Reality) markers, etc. The CAD data includes information indicating the shape of an obstacle. The shape includes dimensions.

[0023] Furthermore, the first processing unit 102a recognizes a release position at the destination of the object M. For example, when the destination is a container (e.g., a tray T), the first processing unit 102a recognizes the release position by machine learning using model-based matching. Model-based matching is a technique for determining the position and orientation of an object by using image data obtained from a camera or the like and shape and structure data of the object (in this case, a container) whose position and orientation is to be acquired, and matching the shape and structure data with the object extracted from the image. Note that the first processing unit 102a is not limited to a unit that recognizes the release position by machine learning using model-based matching. For example, the first processing unit 102a may recognize the release position by using an AR marker.

[0024] Furthermore, the second processing unit 102b recognizes the pedestal 302 of the robot 30, which will be described later. For example, the second processing unit 102b recognizes the pedestal 302 by acquiring CAD data. This CAD data includes information indicating the shape of the pedestal 302. The shape includes dimensions. This allows the second processing unit 102b to recognize the Z coordinate of the top surface of the pedestal 302 in that coordinate system as the height of the pedestal 302.

[0025] The third processing unit 102c recognizes the state (i.e., the position and posture) of the object M. For example, the third processing unit 102c recognizes the position of the object M by performing machine learning using model-based matching. Furthermore, the third processing unit 102c recognizes the posture of the object M by using a technique for generating a bounding box, such as an axis-aligned bounding box (AABB) or an oriented bounding box (OBB), for the object M whose position has been identified. Note that the third processing unit 102c may classify the object M using clustering, which is one of machine learning techniques, for the image captured by the image capture device 40, and identify the state of the object M by using a technique for generating a bounding box.

[0026] Furthermore, the third processing unit 102c acquires the height of the object M. For example, the third processing unit 102c recognizes the object M by acquiring CAD data. This CAD data includes information indicating the shape of the object M. The shape includes dimensions. This allows the third processing unit 102c to recognize the Z coordinate of the object M in that coordinate system as the height of the object M. Note that the third processing unit 102c may recognize the height of the object M by subtracting the Z coordinate of the base 302 from the Z coordinate of the top surface of the object M.

[0027] The fourth processing unit 102d acquires various constraint conditions. Then, the fourth processing unit 102d sets the acquired various constraint conditions. Here, the method for defining the constraint conditions will be described in detail.

[0028] 4 is a diagram illustrating an example of the position coordinates of the robot hand 303 and the position coordinates of the object M according to an embodiment of the present disclosure. First, the position coordinates of the robot hand 303 at time t and the position coordinates of the object M at time t are defined using the notation in FIG. 4. That is, the position coordinate x(robo, t) of the robot hand 303 at time t is expressed as in equation (1). Furthermore, the position coordinate x(obj, t) of the object M at time t is expressed as in equation (2).

[0029]

number

[0030]

number

[0031] Note that the position coordinates x(robo,t) and x(obj,t) respectively represent the positions of the robot hand 303 and the object M at time t in the three-dimensional space in which the robot 30 including the robot hand 303 operates and the object M moves.

[0032] In addition, to specify the constraints, we introduce a switch variable s(P&P,i,t) shown in equation (3).

[0033]

number

[0034] When the switch variable s(P&P,i,t) is 0, it indicates that the object M is not being grasped by the robot hand 303. When the switch variable s(P&P,i,t) is 1, it indicates that the object M is being grasped by the robot hand 303.

[0035] Here, consider a motion plan for a robot in which the robot hand 303 grasps an object M and moves the object M over the shortest distance. In this case, the goal is to minimize the difference between the position of the robot hand 303 at time t and the position of the robot hand 303 at time t+1 (i.e., the distance traveled by the robot hand 303 from time t to time t+1). Therefore, the objective function f can be expressed as in equation (4).

[0036]

number

[0037] The constraints can be expressed as in equations (5) to (7).

[0038]

number

[0039]

number

[0040]

number

[0041] The first factor on the left side of equation (5) is 0 when the robot hand 303 is gripping the object M. Furthermore, the second factor on the left side of equation (5) is 0 when the position of the object M at time t and the position of the object M at time t+1 are the same. Therefore, equation (5) is a constraint that always holds when the robot hand 303 is gripping the object M, regardless of the position of the object M (i.e., regardless of whether the robot hand 303 is moving or not). Furthermore, equation (5) is a constraint that always holds when the position of the object M at time t and the position of the object M at time t+1 are the same (i.e., when the object M is not moving), regardless of whether the robot hand 303 is gripping the object M.

[0042] The first factor on the left side of equation (6) is 0 when the robot hand 303 is not gripping the object M. Furthermore, the second factor on the left side of equation (6) is 0 when the position of the object M at time t and the position of the robot hand 303 at time t are the same. Therefore, equation (6) is a constraint that always holds when the robot hand 303 is not gripping the object M, regardless of whether the position of the object M at time t and the position of the robot hand 303 at time t are the same. Furthermore, equation (6) is a constraint that always holds when the position of the object M at time t and the position of the robot hand 303 at time t are the same, regardless of whether the robot hand 303 is gripping the object M.

[0043] The first factor on the left side of equation (7) is 0 when the robot hand 303 is not grasping the object M. Furthermore, the second factor on the left side of equation (7) is 0 when the robot hand 303 is grasping the object M. Therefore, equation (7) is a constraint that always holds true in either the case where the robot hand 303 is not grasping the object M or the case where the robot hand 303 is not grasping the object M. Note that the constraint according to equation (7) is set because the robot hand 303 is either grasping the object M or not.

[0044] The fourth processing unit 102d sets the constraint conditions described above. As shown in equations (5) to (7), a constraint condition that generally requires A(x)=0 or B(x)=0 for a variable x can be expressed as A(x)B(x)=0. When a constraint other than the above-mentioned constraint conditions is to be set, the fourth processing unit 102d may set it by adding a further constraint condition.

[0045] The fifth processing unit 102e generates a sequence indicating the flow of motion of the robot 30 based on task targets determined by processing by the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c, and constraint conditions set by processing by the fourth processing unit 102d. For example, the fifth processing unit 102e acquires task targets from the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c. The fifth processing unit 102e also acquires constraint conditions from the fourth processing unit 102d. The fifth processing unit 102e adds the constraint conditions acquired from the fourth processing unit 102d to the constraint conditions input from the input unit 101. Then, based on the acquired work goal and constraints, the fifth processing unit 102e generates information indicating each state of the robot 30 at each time step from the state at the source of the object M to the state at the destination of the object M, which is necessary for the control unit 201 of the control device 20, which will be described later, to generate a control signal to control the robot 30 (including, for example, the type of object M, the position and posture of the robot 30, the strength of the grip on the object M, the operation of the robot 30 (e.g., the approach operation to approach the object M (equivalent to the processing of the approach process in Figure 5, which will be described later), the pick operation to grip the object M (equivalent to the processing of the pick process in Figure 5), the carry operation to move the arm to correctly move the gripped object M to the destination (equivalent to the processing of the carry process in Figure 5), and the place operation to release the grip on the object M (equivalent to the processing of the place process in Figure 5)). In other words, the sequence is information indicating each state of the robot 30 at each time step from the state at the origin of the object M to the state at the destination of the object M, which is necessary for the control unit 201 described later to generate a control signal to control the robot 30.

[0046] 5 is a diagram illustrating an example of each process and a movement path of the object M according to an embodiment of the present disclosure. When a task goal and various constraints are set, the process and the movement path of the object M in the sequence for moving the object M shown in FIG. 5 are determined by the fifth processing unit 102e executing a simulation to reduce the amount of energy consumed by the robot 30 as much as possible using the objective function f, to make the trajectory of the robot hand 303 as short as possible using the objective function f, or to make the movement path of the object M as short as possible using the objective function f. As shown in FIG. 5, examples of processes for moving the object M from the origin to the destination include an approach process in which the robot hand 303 approaches the object M, a pick process in which the robot hand 303 grasps the object M, a carry process in which the robot hand 303 moves the object M, and a place process in which the robot hand 303 releases its grasp of the object M.

[0047] Here, a method will be described in which the fifth processing unit 102e determines the trajectory of the movement of the robot hand 303 for each time step through simulation. For simplicity, variables (e.g., x, y) that affect the trajectory of the movement of the robot hand 303 are defined, and a function f(x, y) that represents the trajectory of the movement of the robot hand 303 is set as an objective function, and a method will be described in which the objective function f(x, y) is minimized. Also, for example, a constraint condition such as x+y=0 is set due to a restriction on the movable area of ​​the robot hand 303. In such a case, the Lagrangian function L is given as shown in Equation (8) using a positive constant λ.

[0048]

number

[0049] The fifth processing unit 102e may then use gradient descent to find a solution. Examples of gradient descent include simulated annealing and a gradient method called a primal-dual interior point method. FIG. 6 is a first diagram illustrating an image of a solution found using gradient descent in an embodiment of the present disclosure. In this case, the fifth processing unit 102e identifies a desired solution indicated by a star in FIG. 6 where the objective function f(x, y) is minimized by repeatedly searching for a region that satisfies the constraint condition x + y = 0 and that has a local minimum value by differentiating the Lagrangian function L. Note that the Lagrangian function L expressed by equation (8) is just an example, and any Lagrangian function generally used in continuous optimization may be used. For example, when using an optimization algorithm based on a gradient method called a primal-dual interior point method, the Lagrangian function L has a value of zero in a region where the constraint condition is not violated and takes on a value of infinity as soon as it enters a region where the constraint condition is violated, which is called a barrier function.

[0050] However, when using gradient descent to find a solution using the derivative of a function under certain constraints, as described above, a problem commonly known as a "non-convex constraint" may occur, in which the searchable range of solutions is limited by the constraints, making it impossible to find a local minimum. In other words, when solving an optimization problem in which the objective function f is minimized (or maximized) using only gradient descent, the solution candidates may converge to an extreme value and be unable to escape from that extreme value due to the constraints. As a result, it may be impossible to find a solution that minimizes (maximizes) the objective function f. More specifically, for example, when solving an optimization problem in which the objective function f is minimized (or maximized) using simulated annealing, the temperature is raised to a high temperature, the search range for solution candidates is widened (relaxed), and solution candidates are found within the searchable range while the temperature is gradually lowered. However, if the searchable range is divided while the temperature is gradually lowered, an appropriate solution candidate cannot be found unless an appropriate search range is searched before the division. Furthermore, if an inappropriate extreme solution candidate is found using the simulated annealing technique, the temperature is raised again to a high temperature, and then gradually lowered while the extreme solution candidate is found again within the searchable range.If an appropriate search range cannot be found before it is divided, then no matter how many times the search is repeated, an appropriate solution cannot be found using only the simulated annealing technique.

[0051] The fifth processing unit 102e according to an embodiment of the present disclosure prevents solution candidates from converging to extreme values ​​and becoming stuck when solving an optimization problem, thereby efficiently obtaining an appropriate solution. Here, a method for preventing solution candidates from converging to extreme values ​​and becoming stuck when solving an optimization problem by the fifth processing unit 102e and efficiently obtaining an appropriate solution will be described. Here, it is assumed that the fifth processing unit 102e obtains the extreme value using a gradient descent method. However, for ease of understanding, the method for efficiently obtaining an appropriate solution by the fifth processing unit 102e will be described using a single variable instead of multiple variables.

[0052] Here, we consider minimizing the objective function f(θ) for the objective function f(θ) expressed by equation (9) and the constraints expressed by equations (10) and (11). That is, we consider that the fifth processing unit 102e minimizes the objective function f(θ) when the Lagrangian function L is expressed by equation (12). Here, λ and μ are positive constants.

[0053]

number

[0054]

number

[0055]

number

[0056]

number

[0057] Here, the function max(a, b) is a function that returns the larger value of two numbers a and b.

[0058] 7 is a second diagram illustrating an image of a solution obtained by using the gradient descent method according to an embodiment of the present disclosure. For example, for the objective function f and constraint conditions expressed by equations (9) to (11), the fifth processing unit 102e searches for a solution θ using the minimum descent method to obtain the minimum of the objective function f(θ), and the solution candidate converges to the extreme value of θ=0 shown in FIG. 7. In this case, the fifth processing unit 102e cannot obtain the correct solution, θ=1, by searching using only the gradient descent method.

[0059] In this way, the solution candidate may converge to an inappropriate extreme value and be unable to escape from that extreme value. Therefore, for example, the fifth processing unit 102e is a search unit that searches for a solution candidate while searching for a solution candidate for an optimization problem with constraints including variables. If the constraints are not satisfied for the searched solution candidate (e.g., the inappropriate extreme value), the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by the variable included in a second region that does not share a region with the first region, and then searches for the solution candidate again after the change. Specifically, the fifth processing unit 102e performs processing using the first algorithm shown below to determine whether the constraint condition expressed by A(x)B(x)=0 (in the specific example shown here, the first factor θ on the left side of Equation (10) corresponds to A(x), and the second factor (1-θ) on the left side corresponds to B(x)) is a highly discrete constraint condition. That is, the fifth processing unit 102e sets parameters r, N, M, and K for a constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0. The parameter r is a parameter for setting a criterion for determining the strength of the discreteness of A(x) for which A(x)=0 and B(x) for which B(x)=0. The parameter r takes a value greater than 0 and less than or equal to 1. The closer the parameter r is to 1, the stronger the discreteness of A(x) for which A(x)=0 and B(x) for which B(x)=0. The parameter N is a parameter for setting a criterion for the proportion of detection of points that do not satisfy A(x)B(x)=0. The parameter M is a parameter for setting the number of times to repeat detection of points that do not satisfy A(x)B(x)=0. The parameter K is a parameter that indicates the number of times that the detection of points that do not satisfy A(x)B(x)=0 is repeated M times and the result is equal to or greater than the criterion parameter N. The initial value of the parameter K is 0.

[0060] The fifth processing unit 102e determines positive values ​​for each of the parameters r, N, and M. Then, the fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0. The fifth processing unit 102e then sets a step size α by dividing the line segment between the sampled xA and xB into a predetermined number of equal parts (e.g., N=100), for example. Each time xA or xB moves from one to the other by the step size α, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied. Note that in the specific example shown here, θ can only take the value 0 or 1. Therefore, in the specific example shown here, the fifth processing unit 102e determines that A(x)B(x)=0 is satisfied for the start point and the end point, but that A(x)B(x)=0 is not satisfied for the other points. The specific example shown here is an example of the most discrete constraint, where only the start point and the end point satisfy A(x)B(x)=0. If the fifth processing unit 102e determines N or more times that A(x)B(x)=0 is not satisfied, it adds 1 to the value of K. The fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 M−1 times. Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each of the sampled line segments xA and xB each time the fifth processing unit 102e moves by a step width α, and calculates the cumulative total of the value of K. Then, the fifth processing unit 102e determines that the constraint expressed by A(x)B(x)=0 that satisfies K / M≧r is a highly discrete constraint (an example of the constraint stricter than a predetermined constraint). In this way, the fifth processing unit 102e performs processing using the first algorithm to determine whether or not the constraint condition expressed by A(x)B(x)=0 is a constraint condition with strong discreteness.

[0061] Specifically, the fifth processing unit 102e may determine whether the constraint expressed by A(x)B(x)=0 is a highly discrete constraint by performing processing using a second algorithm described below. That is, the fifth processing unit 102e sets parameters N, M, and K for the constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0. The parameter N is a parameter for setting a criterion for the proportion of detection of points that do not satisfy A(x)B(x)=0. The parameter M is a parameter for setting the number of times to repeat detection of points that do not satisfy A(x)B(x)=0. The parameter K is a parameter representing the number of times that the number of times that the detection of points that do not satisfy A(x)B(x)=0 is repeated M times will be equal to or greater than the criterion parameter N. The initial value of the parameter K is 0.

[0062] The fifth processing unit 102e determines positive values ​​for each of the parameters N and M. For example, the fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0. The fifth processing unit 102e then sets the step size α by dividing the line segment between the sampled xA and xB into a predetermined number of equal parts (e.g., N=100), for example. Each time the fifth processing unit 102e moves from one of xA or xB to the other by the step size α, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied. If the fifth processing unit 102e determines that A(x)B(x)=0 is not satisfied N or more times, it adds 1 to the value of K. The fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 M−1 times. Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each of the sampled line segments xA and xB, each time the fifth processing unit 102e moves by the step width α, and calculates the cumulative total of the value of K. The fifth processing unit 102e then determines that the constraint condition expressed by A(x)B(x)=0, which satisfies K / M=1, is a highly discrete constraint condition (an example of a constraint condition that is stricter than a predetermined constraint condition). In the specific example shown here, the fifth processing unit 102e determines that Equation (10) is a highly discrete constraint condition that satisfies K / M=1. In this way, the fifth processing unit 102e may determine whether the constraint condition expressed by A(x)B(x)=0 is a highly discrete constraint condition by performing processing using the second algorithm.

[0063] Next, the fifth processing unit 102e finds a solution candidate using a gradient descent method. After finding the solution candidate, the fifth processing unit 102e determines whether the solution candidate satisfies the constraints. If the fifth processing unit 102e determines that the solution candidate satisfies all the constraints, it treats the solution candidate as the final solution and terminates the search. Then, the fifth processing unit 102e outputs the final solution (i.e., the trajectory of the movement of the robot hand 303 for each time step) to the control device 20. If the fifth processing unit 102e determines that the solution candidate does not satisfy at least one constraint, it identifies variables associated with the constraint condition determined to be a highly discrete constraint, where the constraint condition represented by A(x)B(x)=0 is a highly discrete constraint. Then, the fifth processing unit 102e changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from a first region containing the value of the variable to one of the possible values ​​of a variable included in a second region that does not have a value of the variable common to the first region.

[0064] After changing the variable values, the fifth processing unit 102e again obtains solution candidates using the gradient descent method. The fifth processing unit 102e determines whether the obtained solution candidates satisfy the constraints. If the fifth processing unit 102e determines that the solution candidates satisfy all the constraints, the fifth processing unit 102e designates the solution candidates as the final solution and terminates the search. The fifth processing unit 102e then outputs the final solution (i.e., the sequence of the trajectories of the robot hand 303 for each time step) to the control device 20. If the fifth processing unit 102e determines that the solution candidates do not satisfy at least one constraint, it identifies variables associated with the constraint condition determined to be a highly discrete constraint, where the constraint condition represented by A(x)B(x)=0 is a highly discrete constraint. The fifth processing unit 102e then changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from a first region containing the value of the variable to one of the possible values ​​of variables contained in a second region that does not have a common variable value with the first region.

[0065] After changing the variable values, the fifth processing unit 102e again obtains candidate solutions by gradient descent. The fifth processing unit 102e repeats the above process for the obtained candidate solutions until it determines that all constraints are satisfied. The fifth processing unit 102e generates a plan sequence in this manner.

[0066] 8 is a diagram illustrating an example of a sequence TBL1 of a plan generated by the generating unit 102 according to an embodiment of the present disclosure. For example, the sequence TBL1 of the plan generated by the generating unit 102 is a sequence indicating each state of the robot 30 for each n time step from the origin to the destination of the object M, as shown in FIG.

[0067] 9 is a diagram illustrating an example of the configuration of the control device 20 according to an embodiment of the present disclosure. As shown in FIG. 9, the control device 20 includes a control unit 201. The control unit 201 generates a control signal for controlling the robot 30 based on the sequence generated by the generation unit 102. That is, the control signal generates a control signal that realizes the posture of the object M and the movement path of the object M according to the sequence generated by the calculation device 10. The control unit 201 outputs the generated control signal to the robot 30.

[0068] 10 is a diagram illustrating an example of the control signal Cnt of the plan generated by the control unit 201 according to an embodiment of the present disclosure. For example, as shown in FIG. 10, the control signal Cnt of the plan generated by the control unit 201 is each control signal for controlling the robot 30 for each n time steps from the origin to the destination of the object M.

[0069] 11 is a diagram showing an example of a processing flow of the processing system 1 according to an embodiment of the present disclosure. Here, the processing performed by the processing system 1 to generate a sequence and control the robot 30 will be described with reference to FIG. 11. It is assumed here that each of the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c performs the above-described processing.

[0070] The fourth processing unit 102d acquires various constraint conditions. Then, the fourth processing unit 102d sets the acquired various constraint conditions (step S1). For example, the fourth processing unit 102d acquires constraint conditions expressed by equations (5) to (7). Then, the fourth processing unit 102d sets the constraint conditions expressed by the acquired equations (5) to (7).

[0071] The fifth processing unit 102e generates a sequence indicating the flow of motion of the robot 30 based on the task target determined by the processing by the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c and the constraint conditions set by the processing by the fourth processing unit 102d (step S2). For example, the fifth processing unit 102e acquires the task target from the first processing unit 102a, the second processing unit 102b, and the third processing unit 102c. The fifth processing unit 102e also acquires the constraint conditions from the fourth processing unit 102d. The fifth processing unit 102e adds the constraint conditions acquired from the fourth processing unit 102d to the constraint conditions input from the input unit 101. Then, based on the acquired work goal and constraint conditions, the fifth processing unit 102e generates information indicating each state of the robot 30 at each time step from the state at the source of the object M to the state at the destination of the object M, which is necessary for the control unit 201 of the control device 20, described later, to generate a control signal to control the robot 30 (including, for example, the type of object M, the position and posture of the robot 30, the strength of the grip on the object M, the operation of the robot 30 (e.g., the approach operation to approach the object M (equivalent to the processing of the approach process in Figure 5), the pick operation to grip the object M (equivalent to the processing of the pick process in Figure 5), the carry operation to move the arm to correctly move the gripped object M to the destination (equivalent to the processing of the carry process in Figure 5), and the place operation to release the grip on the object M (equivalent to the processing of the place process in Figure 5)).

[0072] For example, the fifth processing unit 102e determines the posture and movement path of the object M for each time step through simulation. Specifically, the fifth processing unit 102e sets parameters r, N, M, and K for a constraint that can be expressed as A(x)B(x)=0, which requires A(x)=0 or B(x)=0 (step S201). The fifth processing unit 102e determines positive values ​​for each of the parameters r, N, and M (step S202). Then, the fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 (step S203). The fifth processing unit 102e sets a step size α by dividing the line segment between the sampled xA and xB into, for example, a predetermined number of equal N parts (e.g., N=100) (step S204). Each time the fifth processing unit 102e moves from one of xA or xB to the other by a step size α, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied. If the fifth processing unit 102e determines that A(x)B(x)=0 is not satisfied N or more times, it adds 1 to the value of K (step S205). The fifth processing unit 102e samples xA that satisfies A(x)=0 and xB that satisfies B(x)=0 M−1 times (step S206). Similarly, the fifth processing unit 102e determines whether A(x)B(x)=0 is satisfied for each of the sampled line segments xA and xB each time the fifth processing unit 102e moves by a step size α, and calculates the cumulative total of the value of K (step S207). Then, the fifth processing unit 102e determines that the constraint expressed by A(x)B(x)=0 that satisfies K / M≧r is a highly discrete constraint (step S208). In this way, the fifth processing unit 102e determines whether or not the constraint condition expressed by A(x)B(x)=0 is a highly discrete constraint condition by performing processing using the first algorithm. Note that the specific example of determining whether or not the constraint condition shown here is a highly discrete constraint condition is the processing using the first algorithm described above. However, the processing for determining whether or not the constraint condition is a highly discrete constraint condition is not limited to the processing using the first algorithm. For example, the processing for determining whether or not the constraint condition is a highly discrete constraint condition may be the processing using the second algorithm described above.

[0073] For example, the first factor on the left side of equation (7) is 0 when the robot hand 303 is not gripping the object M, and the second factor on the left side is 0 when the robot hand 303 is gripping the object M. That is, there is no common area between A(x) indicated by the first factor on the left side and B(x) indicated by the second factor on the left side. Therefore, the fifth processing unit 102e determines that the constraint condition in equation (7) is the most discrete of the constraint conditions in equations (5) to (7).

[0074] Next, the fifth processing unit 102e obtains a solution candidate by gradient descent (step S209). After obtaining the solution candidate, the fifth processing unit 102e determines whether the solution candidate satisfies the constraint conditions (step S210). If the fifth processing unit 102e determines that the solution candidate satisfies all the constraint conditions (YES in step S210), the fifth processing unit 102e treats the solution candidate as the final solution and ends the search. The fifth processing unit 102e outputs the final solution (i.e., a sequence that is the trajectory of the movement of the robot hand 303 for each time step) to the control device 20 (step S211). Then, the fifth processing unit 102e ends the processing.

[0075] Furthermore, when the fifth processing unit 102e determines that at least one constraint is not satisfied for the solution candidate (NO in step S210), it identifies a variable associated with the constraint determined to be a highly discrete constraint, where the constraint represented by A(x)B(x)=0 is a highly discrete constraint (step S212). The fifth processing unit 102e then changes the value of the identified variable to another possible value of a random variable, or changes the value of the identified variable from a first domain containing the value of the variable to one of the possible values ​​of a variable included in a second domain that does not share a variable value with the first domain (step S213). For example, in the case of the constraints of equations (5) to (7), the fifth processing unit 102e identifies s(P&P,i,t) in equation (7) as a variable associated with the constraint determined to be a highly discrete constraint. The fifth processing unit 102e then changes s(P&P,i,t) from 0 to 1. Note that s(P&P, i, t) in the exemplified formula (7) can only take on the value 0 or 1, and therefore s(P&P, i, t) can only be changed from 0 to 1 or from 1 to 0. The fifth processing unit 102e returns to the processing of step S210.

[0076] (advantage) The processing system 1 according to an embodiment of the present disclosure has been described above. In the arithmetic device 10 of the processing system 1, the fifth processing unit 102e (an example of a search means) is a processing unit that searches for solution candidates for an optimization problem with constraints including variables, and if the constraints are not satisfied for the searched solution candidates, changes the value of the variable related to the constraints that are stricter than predetermined constraints to a value that can be taken by a random variable, or changes the value of the variable related to the constraints from a first region that includes the value of the variable related to the constraints to one of the values ​​that can be taken by the variable included in a second region that does not share a region with the first region, and searches for solution candidates again after the changes.

[0077] This allows for an appropriate plan for treatment to be determined.

[0078] In another embodiment of the present disclosure, the control device 20 may comprise a computing device 10 .

[0079] In another embodiment of the present disclosure, the robot 30 may include at least one of the control device 20 and the computing device 10.

[0080] Next, a minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure will be described. FIG. 12 is a diagram illustrating an example of the configuration of a minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure. As shown in FIG. 12, the minimum-configuration arithmetic device 10 according to an embodiment of the present disclosure includes a fifth processing unit 102e (an example of a search unit) that searches for a solution candidate for an optimization problem with constraints including variables. If the constraints are not satisfied for the searched solution candidate, the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not share a region with the first region, and then searches for a solution candidate again after the change. The fifth processing unit 102e can be realized, for example, using the functions of the fifth processing unit 102e illustrated in FIG. 2.

[0081] Next, a description will be given of processing by the arithmetic device 10 with a minimum configuration according to an embodiment of the present disclosure. Fig. 13 is a diagram showing an example of a processing flow of the arithmetic device 10 with a minimum configuration according to an embodiment of the present disclosure. Here, the processing by the arithmetic device 10 with a minimum configuration will be described with reference to Fig. 13.

[0082] The fifth processing unit 102e (an example of a search means) searches for a solution candidate for an optimization problem with constraints including variables (step S101). If the constraints are not satisfied for the searched solution candidate, the fifth processing unit 102e changes the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not share a region with the first region, and searches for a solution candidate again after the change (step S102).

[0083] The above has described the minimum configuration of the arithmetic device 10 according to the embodiment of the present disclosure. This arithmetic device 10 can obtain an appropriate plan for processing.

[0084] The order of the processes in the embodiments of the present disclosure may be changed as long as the processes are performed appropriately.

[0085] Although the embodiments of the present disclosure have been described, the processing system 1, the arithmetic device 10, the control device 20, the robot 30, the photographing device 40, and other control devices may have a computer device inside. The above-described processing steps are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes this program to perform the above processing. Specific examples of computers are shown below.

[0086] FIG. 14 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in FIG. 14, the computer 5 includes a CPU (Central Processing Unit) 6, a main memory 7, a storage 8, and an interface 9. For example, the processing system 1, the arithmetic device 10, the control device 20, the robot 30, the imaging device 40, and other control devices described above are each implemented in the computer 5. The operation of each of the processing units described above is stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8, loads it into the main memory 7, and executes the above-mentioned processing in accordance with the program. The CPU 6 also allocates storage areas in the main memory 7 corresponding to each of the storage units described above in accordance with the program.

[0087] Examples of storage 8 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 8 may be an internal medium directly connected to the bus of computer 5, or an external medium connected to computer 5 via interface 9 or a communication line. In addition, when this program is distributed to computer 5 via a communication line, computer 5 that receives the program may load the program into main memory 7 and execute the above-mentioned processing. In at least one embodiment, storage 8 is a non-transitory tangible storage medium.

[0088] The program may also implement some of the functions described above. Furthermore, the program may be a file that can implement the functions described above in combination with a program already stored in the computer device, a so-called differential file (differential program).

[0089] Although several embodiments of the present disclosure have been described, these embodiments are merely examples and do not limit the scope of the disclosure. Various additions, omissions, substitutions, and modifications may be made to these embodiments without departing from the spirit of the disclosure.

[0090] Note that part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0091] (Appendix 1) a search means for searching for a solution while searching for solution candidates for an optimization problem with constraint conditions including variables, wherein, when the constraint conditions are not satisfied for the searched solution candidate, the search means changes the value of the variable related to the constraint condition stricter than a predetermined constraint condition to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and searches for the solution candidate again after the change; A computing device comprising:

[0092] (Appendix 2) The searching means Only for variables that contribute to the violation of the constraint condition, the values ​​of the variables related to the constraint condition are changed to values ​​that can be taken by random variables, or the values ​​of the variables related to the constraint condition are changed from a first region that includes the values ​​of the variables related to the constraint condition to any of the values ​​that can be taken by variables included in a second region that does not have a common region with the first region, and solution candidates are obtained again after the changes. 10. The computing device of claim 1.

[0093] (Appendix 3) a determination means for determining whether the constraint conditions are stricter than predetermined constraint conditions; 3. The computing device according to claim 1 or 2, comprising:

[0094] (Appendix 4) A control device that controls a control target based on the solution obtained by the arithmetic device of any one of Supplementary Note 1 to Supplementary Note 3.

[0095] (Appendix 5) a control device according to Supplementary Note 4; and a robot to be controlled; A processing system comprising:

[0096] (Appendix 6) a solution is obtained by finding solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition is changed to a value that can be taken by a random variable, or the value of the variable related to the constraint condition is changed from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and after the change, the solution candidate is obtained again; A search method including:

[0097] (Appendix 7) a solution is obtained by finding solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, the value of the variable related to the constraint condition that is stricter than a predetermined constraint condition is changed to a value that can be taken by a random variable, or the value of the variable related to the constraint condition is changed from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and after the change, the solution candidate is obtained again; A recording medium on which a program that causes a computer to execute the above is stored. [Industrial Applicability]

[0098] According to aspects of the present disclosure, an appropriate plan for treatment can be determined. [Explanation of symbols]

[0099] 1. Processing System 5. Computer 6 CPU 7. Main memory 8. Storage 9. Interface 10...Arithmetic unit 20. Control device 30. Robot 40. Imaging device 101 Input section 102...Generation section 102a... First processing section 102b... Second processing section 102c...Third processing section 102d...Fourth processing section 102e 5th processing section 201 Control unit 301···Robot arm 302···Pedestal 303 Robot Hand C···Cardboard F...Floor surface M...Object T···Tray

Claims

1. a search means for searching for a solution while searching for solution candidates for an optimization problem with constraint conditions including variables, wherein, when the constraint conditions are not satisfied for the searched solution candidate, the search means changes the value of the variable related to the constraint condition, which has stronger discreteness than a predetermined constraint condition, to a value that can be taken by a random variable, or changes the value of the variable related to the constraint condition from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and searches for the solution candidate again after the change; A computing device comprising:

2. The searching means Only for variables that contribute to the violation of the constraint condition, the value of the variable related to the constraint condition is changed to a value that can be taken by a random variable, or the value of the variable related to the constraint condition is changed from a first region that includes the value of the variable related to the constraint condition to any of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and after the change, solution candidates are obtained again. The computing device of claim 1 .

3. a determination means for determining whether the constraint condition has stronger discreteness than a predetermined constraint condition; The computing device of claim 1 .

4. A control device that controls a controlled object based on the solution obtained by the arithmetic unit of claim 1.

5. The control device according to claim 4; a robot to be controlled; A processing system comprising:

6. The computing device a solution is obtained by finding solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, the value of the variable related to the constraint condition, which is more discrete than a predetermined constraint condition, is changed to a value that can be taken by a random variable, or the value of the variable related to the constraint condition is changed from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and after the change, the solution candidate is obtained again; A search method including:

7. a solution is obtained by finding solution candidates for an optimization problem with constraint conditions including variables, and if the constraint conditions are not satisfied for the obtained solution candidate, the value of the variable related to the constraint condition, which is more discrete than a predetermined constraint condition, is changed to a value that can be taken by a random variable, or the value of the variable related to the constraint condition is changed from a first region that includes the value of the variable related to the constraint condition to one of the values ​​that can be taken by a variable included in a second region that does not have a region in common with the first region, and after the change, the solution candidate is obtained again; A program that causes a computer to execute the following.

Images