Object recognition device, object recognition method, and program

Abstract

An object recognition device 100 according to the present disclosure comprises: a recognition unit 121 that recognizes an object from an image obtained by imaging a storage space; a target generation unit 122 that generates a movement target of the object on the basis of the status of the recognized object; a planning unit 123 that performs planning processing for planning an operation plan of a robot so as to move the object by means of the robot on the basis of the movement target; and a change unit 124 that changes the movement target in accordance with the planning processing.

Description

Object recognition device, object recognition method, and program

[0001] The present disclosure relates to an object recognition device, an object recognition method, and a program.

[0002] In recent years, robots have been introduced in various situations, and work tasks have been automated. For example, in warehouses where a large number of objects are stored, the objects must be managed at all times, and robots are used for such management. As an example, in Patent Document 1, objects are recognized from images captured by a camera, and unrecognized objects are moved by a robot, and the objects are recognized and managed from the images after the movement.

[0003] Special table 2021-503374 publication

[0004] However, with the above-mentioned technology, the robot may not always be able to move the object, and in that case, the robot may not be able to recognize the object. As a result, the robot may not be able to accurately recognize the object stored in the storage space.

[0005] Therefore, one of the objects of the present disclosure is to solve the problem of not being able to accurately recognize objects contained in the storage space.

[0006] An object recognition device according to one aspect of the present disclosure includes: a recognition unit that recognizes an object from an image of a storage space; a target generation unit that generates a movement target for the object based on a status of the recognized object; a planning unit that performs planning processing to plan an operation plan for the robot so that the robot moves the object based on the movement target; and a modification unit that changes the movement target in accordance with the planning processing. An object recognition method according to one aspect of the present disclosure includes: recognizing an object from an image of a storage space; generating a movement target for the object based on a status of the recognized object; performing planning processing to plan an operation plan for the robot so that the robot moves the object based on the movement target; and modifying the movement target in accordance with the planning processing. A program according to one aspect of the present disclosure includes: recognizing an object from an image of a storage space; generating a movement target for the object based on a status of the recognized object; performing planning processing to plan an operation plan for the robot so that the robot moves the object based on the movement target; and modifying the movement target in accordance with the planning processing.

[0007] With the above-described configuration, the present disclosure can accurately recognize an object contained in a storage space.

[0008] FIG. 1 is a diagram illustrating an overall configuration of a system according to the present disclosure. FIG. 2 is a block diagram illustrating a configuration of an object recognition device according to the present disclosure. FIG. 3 is a diagram illustrating a processing state by an object recognition device according to the present disclosure. FIG. 4 is a diagram illustrating a processing state by an object recognition device according to the present disclosure. FIG. 5 is a flowchart illustrating an operation of an object recognition device according to the present disclosure. FIG. 6 is a block diagram illustrating a hardware configuration of an object recognition device according to the present disclosure. FIG. 7 is a block diagram illustrating a configuration of an object recognition device according to the present disclosure.

[0009] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings can be referenced in any embodiment.

[0010] [Configuration] The system of this embodiment is used to recognize the status of objects in a storage space where the objects are stored and manage the objects. Specifically, this embodiment is directed to a system that is installed in a warehouse, which is a storage space where shelves R are installed, and that recognizes and manages the status of packages B (objects), which are boxes stored on the shelves R, as shown in FIG. 1 . The system of this embodiment includes a camera 20 that photographs the warehouse interior, i.e., the shelves R, and a robot 30 that can move by carrying the packages B. The camera 20 is mounted on the robot 30. The system also includes an object recognition device 10, which is an information processing device connected to the camera 20 and the robot 30 so as to be able to communicate with them. The object recognition device 10 recognizes the packages B stored on the shelves R using images acquired from the camera 20, and further has the function of controlling the operation of the robot 30 to move the packages B so that the packages B can be easily recognized. The configuration of the system will be described in detail below.

[0011] First, a warehouse will be described, which serves as a storage space for cargo B (objects) and in which the system will be introduced. Within the warehouse, shelves R are installed at predetermined locations. Each shelf R is composed of, for example, pillars and a platform on which cargo B is placed. As a result, the positions of the pillars and platforms constituting the shelf R within the warehouse are also predetermined. Furthermore, codes serving as identification information, such as barcodes or two-dimensional codes, are affixed to the predetermined positions on the pillars and platforms of the shelf R. As will be described later, by recognizing the codes that appear in an image captured within the warehouse, the positions in the image can be recognized and correlated with positions within the warehouse. The position and shape of the shelf R within the warehouse, i.e., information on the shape and position of the pillars and platforms of the shelf R, are stored in the management information storage unit 16 of the object recognition device 10, as will be described later. At this time, the identification information of the codes is also stored in the management information storage unit 16 of the object recognition device 10 in association with the positions of the pillars and platforms of the shelf R to which they are affixed.

[0012] In this embodiment, the robot 30 is configured as a robot arm, and is provided with a hand at the tip of the arm that grasps and holds the package B. The robot 30 is controlled to move while holding the package B in response to commands from the object recognition device 10, as will be described later, and to move and place the package B at a target location. Therefore, the position and shape of the robot 30, i.e., the position of the robot 30 itself within the warehouse, the joint angles and positions of the arm, and the position and opening of the hand, are recognized by the object recognition device 10. Note that the hand of the robot 30 is not limited to grasping and holding the package B, and may be configured to hold the package B by suction, for example, and may hold the package B in any structure.

[0013] A range of motion is set in advance for the robot 30, and information about this range of motion is stored in the management information storage unit 16 of the object recognition device 10. The range of motion of the robot 30 includes, for example, the range of movement of the robot 30 within the warehouse, the angle range of the joints of the arm unit, and the opening range of the hand unit.

[0014] As will be described later, the camera 20 takes an image of a shelf R in the warehouse, i.e., an image of a package B stored on the shelf R, in response to a command from the object recognition device 10. In this embodiment, the camera 20 is attached near the tip of the arm of the robot 30 and is capable of taking images of an area that can be operated by the hand of the robot 30. For example, the camera 20 takes a distance image that enables measurement of the shape and distance of the package B. However, the camera 20 may take any image that enables measurement of the shape and distance of an object. Furthermore, the camera 20 is not necessarily limited to being attached to the robot 30, and may be attached anywhere in the warehouse, such as on a wall or pillar.

[0015] The object recognition device 10 is composed of one or more information processing devices each including a calculation device and a storage device. As shown in FIG. 2 , the object recognition device 10 includes a motion control unit 11, a recognition unit 12, a target generation unit 13, a planning unit 14, and a modification unit 15. The functions of the motion control unit 11, the recognition unit 12, the target generation unit 13, the planning unit 14, and the modification unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device. The object recognition device 10 also includes a management information storage unit 16 and a motion plan storage unit 17. The management information storage unit 16 and the motion plan storage unit 17 are each composed of a storage device. Each component will be described in detail below.

[0016] The management information storage unit 16 stores warehouse information, robot information, and movement constraint information as information used to manage packages B in the warehouse. The warehouse information includes location information within the warehouse, i.e., information on the positions and shapes of the columns and platforms of the shelves R within the warehouse. The warehouse information also includes identification information of codes affixed to the shelves R as described above, which are associated with the positions of the columns and platforms of the shelves R to which the codes are affixed. The warehouse information also includes information on packages B stored in the warehouse. For example, the warehouse information includes the size, shape, and type of pre-registered packages B that may be stored in the warehouse. The information on packages B included in the warehouse information also includes information on the size, shape, type, and number of packages B associated with the positions of the shelves R, i.e., the platforms. Information such as the number of packages B associated with the positions of the shelves R, i.e., the platforms, is stored according to the recognition results of the packages B by the object recognition device 10, as described below.

[0017] The robot information stored in the management information storage unit 16 also includes information about the structure of the robot 30. For example, the robot information includes information about the size and shape of the robot arm, as well as information about the range of motion of the robot 30, such as the range of movement of the robot 30 within the warehouse, the range of angles over which the joints of the arm section can move, and the range of opening angles over which the hand section can move.

[0018] Furthermore, the movement constraint information stored in the management information storage unit 16 includes information regarding constraints imposed when setting a movement target for the luggage B. Here, the luggage B is moved by the robot 30 to a position and posture that will be the movement target, as will be described later, and the movement constraint information includes an upper limit value for the amount of change from the position of the luggage B before movement to the position at the movement target, and an upper limit value for the amount of change from the posture of the luggage B before movement to the posture at the movement target. For example, the amount of change in position is expressed by a distance in each direction, and the amount of change in posture is expressed by a rotation angle in each rotation direction.

[0019] The motion plan storage unit 17 stores motion plan information for the robot 30. For example, the motion plan information is information representing a motion plan for the robot 30 that is planned by the object recognition device 10 based on the current position and posture of the package B and the position and posture of a movement target, as will be described later, and includes a motion trajectory and motion procedures for the robot 30. As an example, the motion plan information is made up of a series of motion procedures, and for each motion procedure, includes a motion trajectory such as the position of the robot 30 itself, the joint angles of the arm unit, and the opening degree of the hand unit.

[0020] As will be described later, the operation control unit 11 controls the operation of the robot 30 based on the operation plan information that has been planned and stored in the operation plan storage unit 17, and performs an operation to move the robot while holding the package B. The operation control unit 11 also controls the operation of the camera 20 mounted on the robot 30, and operates the camera 20 to capture images. When capturing images with the camera 20, the operation control unit 11 refers to the warehouse information described above and controls the camera 20 to capture images within the range of the image capture.

[0021] The recognition unit 12 performs a process of recognizing the package B shown in the image captured by the camera 20. Specifically, the recognition unit 12 estimates the contour of the package B shown in the image and, based on the estimated contour, recognizes whether the package B is of a type registered in the warehouse information. As an example, the recognition unit 12 performs edge processing on the image to estimate the contour and recognizes the type of package B in the image by template matching with the size, shape, and other characteristics of the package B registered in the warehouse information. Additionally, the recognition unit 12 identifies the position of the platform on the shelf R on which the recognized package B is located, based on the image and the control position of the robot 30. The recognition unit 12 then determines that the recognized package B is located at the identified platform position on the shelf R, and associates the shelf R, i.e., the platform position, with the size, shape, and type of the recognized package B, and stores this information as warehouse information. At this time, the recognition unit 12 also recognizes the number of packages B located in the same location, i.e., on the platform on the same shelf R, and stores this information in association with the number of packages as warehouse information.

[0022] As described above, the recognition unit 12 may use a code attached to the shelf R shown in the image when specifying the position of the shelf R shown in the image. For example, the recognition unit 12 can specify the position of the shelf R shown in the image by recognizing the code shown in the image and acquiring the position information of the shelf R corresponding to the code from the warehouse information.

[0023] Here, when the recognition unit 12 recognizes the package B from the image as described above, there may be cases where the recognition unit 12 insufficiently recognizes the package B in the image. For example, the left diagram of FIG. 3 shows an example of an image captured by the camera 20. As shown in this diagram, the first package B1 and the second package B2 are captured in the image, but they overlap in the depth direction of the image, resulting in insufficient recognition of a portion of the contour (edge ​​of the outer shape) of the second package B2. It should be noted that the insufficient recognition of the contour of package B can be determined, for example, by utilizing the fact that the image is a distance image and detecting that the second package B2 in the back is located within a portion of the contour of the first package B1 in the front. Alternatively, if the template matching described above is performed using the contour of the second package B2 in the back, but the result does not match the registered package, it can be determined that the recognition of the contour of the second package B2 in the back is insufficient. In this way, when the recognition of the contour of package B is insufficient, package B is moved to eliminate the overlap of package B in the image.

[0024] As described above, when it is determined that the recognition of the contour of the luggage B is insufficient, the target generation unit 13 generates a target for moving the luggage B. Specifically, when multiple luggage B1 and B2 are overlapping one another, as shown in the left diagram of FIG. 3 , the target generation unit 13 generates a movement target for moving the first luggage B1 located in the front, as an example. At this time, the target generation unit 13 estimates the hidden contour of the second luggage B2, as shown by the dotted line in the left diagram of FIG. 3 , from the recognized contour or from a registered template of luggage B. The target generation unit 13 then generates a movement target representing the post-movement position and orientation of the first luggage B1 so that the entire contour of the second luggage B2 is not hidden by the first luggage B1 and is displayed in the image. For example, the movement target is a position and orientation (such as a rotation angle) in a three-dimensional space transformed using a range image. In the example of FIG. 3 , as shown in the right diagram of FIG. 3 , a movement target is generated representing the post-movement position and orientation of the first luggage B1, which is moved in the direction of arrow Y.

[0025] The target generator 13 may generate movement targets for moving multiple luggage B in a situation such as that shown in the left diagram of Fig. 4, in which the contours of the two luggage B at the back, the second luggage B2 and the third luggage B3, are insufficiently recognized because they are hidden by the first luggage B1 at the foreground in the image. For example, as shown in the right diagram of Fig. 4, a movement target after movement may be generated in which the first luggage B1 is moved as indicated by arrow Y1 and the third luggage B3 is further moved as indicated by arrow Y2. However, in this case, a movement target for moving only the first luggage B1 may be generated.

[0026] Here, the target generation unit 13 is not necessarily limited to generating a movement target for the luggage B when the contour of the luggage B is not sufficiently recognized on the image. Even when the entire contour of the luggage B can be recognized, the target generation unit 13 may generate a movement target for moving the luggage B when the position or posture of the luggage B does not satisfy a preset condition. For example, as shown in the left diagram of FIG. 5, when the posture of the first luggage B1 does not satisfy a preset condition, a movement target after movement may be generated such that the first luggage B1 is moved by rotating it as indicated by arrow Y3 as shown in the right diagram of FIG. In this way, the target generation unit 13 may generate a movement target for moving the luggage B in order to adjust the position or posture of the luggage B according to the situation in which the luggage B is recognized.

[0027] When generating a movement target as described above, the target generator 13 may take into consideration the movement constraint information stored in the management information storage unit 16. For example, the target generator 13 may generate the post-movement position and posture of the luggage B, which is the movement target, so as not to exceed an upper limit value of the amount of change from the position and posture of luggage B before movement, which is registered as movement constraint information, to the position and posture at the movement target.

[0028] The planner 14 performs planning processing to plan an operation plan for the robot 30 so that the robot 30 moves the load B to a target position and posture before movement. Specifically, the planner 14 plans an operation plan consisting of an operation trajectory and operation procedures for the robot 30 to move the load B from its current position and posture to a target position and posture. As an example, the operation plan consists of a series of multiple operation procedures, and each operation procedure is planned to include an operation trajectory such as the position of the robot 30 itself, the joint angles of the arm unit, and the opening degree of the hand unit.

[0029] The planner 14 performs planning processing to plan an operation plan for the robot 30, taking into account warehouse information and robot information stored in the management information storage unit 16. For example, the planner 14 plans the operation plan taking into account the positions of the pillars and platforms of the shelf R, the shape of the robot 30, and the ranges of movement of the arm and hand. The planner 14 then simulates the operation plan from the current position and posture of the package B to the position and posture of the movement target, and determines whether the operation of the robot according to the operation plan satisfies preset conditions. For example, the planner 14 determines whether the operation conditions are met, such as whether the robot 30 is movable within the set range of movement and whether the robot 30 will not collide with fixed objects such as pillars and platforms of the shelf R. If the planned operation plan does not feasibly allow the robot 30 to move beyond the range of movement, or if the robot 30 will collide with the shelf R, the planner 14 determines that the operation conditions are not met and notifies the change unit 15 of this fact. Furthermore, when it is determined by the above-described simulation of the motion plan that it is impossible to generate the motion plan in the first place, the planner 14 also notifies the change unit 15 of this fact. Note that cases in which it is impossible to generate the motion plan include not only cases in which the simulation cannot be executed, but also cases in which the range of motion of the robot 30 is exceeded as described above, or cases in which the robot 30 collides with the shelf R. On the other hand, when the planner 14 determines that it is possible to generate the motion plan and that the operation conditions are satisfied, it stores the generated motion plan information in the motion plan storage unit 17.

[0030] As described above, the modification unit 15 modifies the movement target when the planner 14 notifies the user that it is impossible to generate a movement plan or that the movement plan does not satisfy the movement conditions. At this time, the modification unit 15 modifies the position and orientation of the movement target before the change so as not to exceed the upper limit of the change amount between the position and orientation of the package B before the movement and the position and orientation at the movement target, which is registered as movement constraint information. In addition, the modification unit 15 modifies the movement target according to the shape of the shelf R and the position of the robot 30. For example, the modification unit 15 may modify the position of the movement target before the change so that it is farther from the column of the shelf R and closer to the position of the robot 30. This makes it possible to plan a movement plan that prevents the hand of the robot 30 from colliding with the column of the shelf R and improves the efficiency of the movement of the robot 30. However, the modification unit 15 may modify the movement target without being subject to the modification restrictions described above.

[0031] Furthermore, when the planning unit 14 notifies the change unit 15 that it is impossible to generate an operation plan or that the operation plan does not satisfy the operation conditions, the change unit 15 may change the movement target by changing the luggage B to be moved and generating a movement target for the luggage B. For example, as shown in FIG. 3 , in a case where a movement target for a first luggage B1 is generated and an operation plan is generated, but it is impossible to generate the operation plan, the change unit 15 may set a second luggage B2 as the luggage B to be moved and generate a movement target for the second luggage B2.

[0032] Then, the change unit 15 notifies the planner 14 of the changed movement targets as described above. The planner 14 performs planning processing to plan an operation plan for the robot 30 in accordance with the changed movement targets so notified. The planner 14 and the change unit 15 repeat the above processing until an operation plan that satisfies the operation conditions is generated.

[0033] [Operation] Next, the operation of the object recognition device 10 described above will be described mainly with reference to the flowchart of FIG.

[0034] The object recognition device 10 controls the operation of the camera 20 mounted on the robot 30 to capture an image of the warehouse interior (step S1). The object recognition device 10 then performs a process to recognize the package B shown in the image (step S2). At this time, the object recognition device 10 determines whether it is necessary to move the package B depending on the result of recognizing the package B (step S3). For example, the object recognition device 10 determines whether it is necessary to move the position or posture of the package B depending on the circumstances under which the package B is recognized, such as if the contour of the package B shown in the image is insufficient or if it is necessary to adjust the position or posture of the package B.

[0035] Then, when the object recognition device 10 determines that movement of the luggage B is necessary (Yes in step S3), it generates a movement target for moving the luggage B (step S4). For example, when multiple luggage B1 and B2 are stacked one on top of the other, as shown in the left diagram of Fig. 3, the object recognition device 10 generates a movement target for moving the first luggage B1 located in the front, as an example. Furthermore, when the posture of the first luggage B1 does not satisfy a preset condition, as shown in the left diagram of Fig. 5, it generates a post-movement movement target for rotating and moving the first luggage B1 as shown by arrow Y3, as shown in the right diagram of Fig. 5.

[0036] Next, the object recognition device 10 performs a planning process to plan an operation plan for the robot 30 so as to move the luggage B from its pre-movement position and posture to the generated movement target position and posture (step S5). At this time, the object recognition device 10 determines whether the operation plan can be generated and whether the generated operation plan satisfies operation conditions, such as not exceeding the range of motion of the robot 30 (steps S6 and S7). If the object recognition device 10 can generate an operation plan in the operation planning process (Yes in step S6) and the generated operation plan satisfies operation conditions, such as not exceeding the range of motion of the robot 30 (Yes in step S7), the object recognition device 10 controls the operation of the robot 30 based on the generated operation plan to move the luggage B to the movement target position and posture (step S8).

[0037] On the other hand, if the object recognition device 10 is unable to generate an operation plan in the operation planning process (No in step S6) or if the generated operation plan exceeds the movable range of the robot 30 and does not satisfy the operation conditions (No in step S7), it performs a process of changing the movement target of the package B (step S9). At this time, the object recognition device 10 may, for example, change the position and posture of the movement target so that the amount of change from the position and posture of the package B before movement to the position and posture at the movement target does not exceed a preset upper limit, or may change the position of the movement target so that it is farther from the pillar of the shelf R and closer to the position of the robot 30. Furthermore, the object recognition device 10 may change the movement target by changing the package B to be moved and generating a movement target for the package B.

[0038] Thereafter, the object recognition device 10 performs a planning process to plan an operation plan for the robot 30 in the same manner as described above, in accordance with the changed moving target (step S5), and repeats the above-described process until an operation plan that satisfies the operation conditions is generated.

[0039] As described above, according to this embodiment, even if an object such as package B stored in a storage space such as a warehouse is difficult to recognize due to overlapping on the image, a movement target is generated that enables the operation plan of the robot 30 for moving the object to be realized, and the object can be moved more reliably. As a result, the object can be recognized with high accuracy using the image.

[0040] <Embodiment 2> Next, a second embodiment of the present disclosure will be described with reference to Fig. 7 and Fig. 8. Fig. 7 and Fig. 8 are block diagrams showing the configuration of an object recognition device in embodiment 2. Note that this embodiment shows an outline of the configuration of the object recognition device described in the above-mentioned embodiment.

[0041] First, the hardware configuration of the object recognition device 100 of this embodiment will be described with reference to Fig. 7. The object recognition device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example: - CPU (Central Processing Unit) 101 (arithmetic unit) - ROM (Read Only Memory) 102 (storage device) - RAM (Random Access Memory) 103 (storage device) - Programs 104 loaded into the RAM 103 - Storage device 105 storing the programs 104 - Drive device 106 for reading and writing from / to a storage medium 110 external to the information processing device - Communication interface 107 for connecting to a communication network 111 external to the information processing device - Input / output interface 108 for inputting and outputting data - Bus 109 for connecting the various components

[0042] 7 shows an example of the hardware configuration of the information processing device that is the object recognition device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as not including the drive device 106. Furthermore, instead of the above-described CPU, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.

[0043] The object recognition device 100 can be equipped with a recognition unit 121, a goal generation unit 122, a planner 123, and a change unit 124 shown in FIG. 8 by having the CPU 101 acquire and execute the group of programs 104. The group of programs 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The group of programs 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the programs and supply them to the CPU 101. However, the above-mentioned recognition unit 121, the goal generation unit 122, the planner 123, and the change unit 124 may be constructed using dedicated electronic circuits for realizing such means.

[0044] The recognition unit 121 recognizes an object from an image captured of the storage space, the target generation unit 122 generates a movement target for the object based on the recognized state of the object, the planning unit 123 performs planning processing to plan an operation plan for the robot so that the robot moves the object based on the movement target, and the change unit 124 changes the movement target in accordance with the planning processing.

[0045] With the above-described configuration, the present disclosure generates a moving target that enables the robot to realize an operation plan for moving an object even when the object stored in the storage space is difficult to recognize due to overlapping on the image, thereby enabling the object to be moved more reliably. As a result, the object can be recognized with high accuracy using the image.

[0046] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0047] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that are understandable to those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, at least one or more of the functions of the recognition unit 121, goal generation unit 122, planning unit 123, and change unit 124 described above may be executed by an information processing device installed and connected anywhere on a network, that is, may be executed by so-called cloud computing.

[0048] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of an information processing device, an information processing method, and a program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) An object recognition device comprising: a recognition unit that recognizes an object from an image captured of a storage space; a target generation unit that generates a movement target for the object based on a state of the recognized object; a planning unit that performs planning processing to plan an operation plan of the robot so that the robot moves the object based on the movement target; and a modification unit that modifies the movement target in accordance with the planning processing. (Supplementary Note 2) The object recognition device according to Supplementary Note 1, wherein the modification unit modifies the movement target when the operation plan cannot be planned in the planning processing. (Supplementary Note 3) The object recognition device according to Supplementary Note 1, wherein the modification unit modifies the movement target when the operation of the robot in the operation plan planned in the planning processing does not satisfy a preset condition. (Supplementary Note 4) The object recognition device according to Supplementary Note 3, wherein the change unit changes the movement target when movement of the robot in the operation plan planned in the planning process exceeds a movement range set for the robot. (Supplementary Note 5) The object recognition device according to Supplementary Note 3, wherein the change unit changes the movement target when a collision of the robot occurs due to movement of the robot in the operation plan planned in the planning process. (Supplementary Note 6) The object recognition device according to Supplementary Note 1, wherein the change unit changes the movement target so that an amount of change in a position or posture of an object in the movement target from before movement does not exceed a preset upper limit value. (Supplementary Note 7) The object recognition device according to Supplementary Note 1, wherein the change unit changes the movement target based on a position of the robot. (Supplementary Note 8) The object recognition device according to Supplementary Note 7, wherein the change unit changes the movement target so that a position of an object in the movement target becomes closer to a position of the robot.(Supplementary Note 9) The object recognition device according to Supplementary Note 1, wherein the change unit changes the movement target based on the position of a fixed object in the storage space. (Supplementary Note 10) The object recognition device according to Supplementary Note 9, wherein the change unit changes the movement target so that its position becomes farther away from the position of the fixed object in the storage space. (Supplementary Note 11) The object recognition device according to Supplementary Note 1, wherein the change unit changes the movement target by changing an object to be moved by the robot. (Supplementary Note 12) The object recognition device according to Supplementary Note 1, wherein the target generation unit generates the movement target so that multiple objects do not overlap on the image. (Supplementary Note 13) An object recognition method comprising: recognizing an object from an image captured of a storage space; generating a movement target for the object based on a state of the recognized object; performing a planning process to plan an operation plan for the robot so that the robot moves the object based on the movement target; and changing the movement target in accordance with the planning process. (Supplementary Note 14) A computer-readable storage medium storing a program that causes a computer to execute the following processes: recognizing an object from an image taken of a storage space; generating a movement target for the object based on the status of the recognized object; performing a planning process that plans an operation plan for the robot to move the object based on the movement target; and changing the movement target in accordance with the planning process.

[0049] REFERENCE SIGNS LIST 10 Object recognition device 11 Operation control unit 12 Recognition unit 13 Target generation unit 14 Planning unit 15 Change unit 16 Management information storage unit 17 Operation plan storage unit 100 Object recognition device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Recognition unit 122 Target generation unit 123 Planning unit 124 Change unit

Claims

1. An object recognition device comprising: a recognition unit that recognizes an object from an image of an accommodation space; a target generation unit that generates a movement target of the object based on the recognized situation of the object; a planning unit that performs planning processing for planning an operation plan of the robot to move the object by the robot based on the movement target; and a change unit that changes the movement target according to the planning processing.

2. The object recognition device according to claim 1, wherein the change unit changes the movement target when the operation plan cannot be planned in the planning processing.

3. The object recognition device according to claim 1, wherein the change unit changes the movement target when the operation of the robot in the operation plan planned in the planning processing does not satisfy a preset condition.

4. The object recognition device according to claim 3, wherein the change unit changes the movement target when the movement of the robot in the operation plan planned in the planning processing exceeds the movable range set for the robot.

5. The object recognition device according to claim 3, wherein the change unit changes the movement target when a collision of the robot occurs due to the movement of the robot in the operation plan planned in the planning processing.

6. The object recognition device according to claim 1, wherein the change unit changes the movement target so that a change amount of a position or posture of the object in the movement target from before movement does not exceed a preset upper limit value.

7. The object recognition device according to claim 1, wherein the change unit changes the movement target based on the position of the robot.

8. The object recognition device according to claim 7, wherein the change unit changes the movement target so that the position of the object in the movement target becomes closer to the position of the robot.

9. The object recognition device according to claim 1, wherein the change unit changes the movement target based on the position of a fixed object in the accommodation space.

10. The object recognition device according to claim 9, wherein the change unit changes the movement target so that the position of the movement target becomes farther from the position of the fixed object in the accommodation space.

11. The object recognition device according to claim 1, wherein the changing unit changes an object to be moved by the robot to change the movement target.

12. The object recognition device according to claim 1, wherein the target generation unit generates the movement target so that a plurality of objects do not overlap on the image.

13. An object recognition method for recognizing an object from an image of an accommodation space, generating a movement target of the object based on the recognized situation of the object, performing a planning process for planning an operation plan of the robot to move the object by the robot based on the movement target, and changing the movement target according to the planning process.

14. A computer-readable storage medium storing a program for causing a computer to execute a process of recognizing an object from an image of an accommodation space, generating a movement target of the object based on the recognized situation of the object, performing a planning process for planning an operation plan of the robot to move the object by the robot based on the movement target, and changing the movement target according to the planning process.

Images