Information processing device, information processing method, and program

Abstract

To provide an information processing device that can achieve both security protection and work execution.SOLUTION: An information processing device (202) includes: first acquisition means (206) that acquires measurement information obtained by measuring a target of a work and the surroundings thereof or recognition information recognized based on the measurement information; second acquisition means (205) that acquires a content of the work as work information; determination means (207) that determines a control method for carrying out the work based on confidential information contained in the measurement information or the recognition information and the work information; and control means (208) that executes the control method based on the measurement information or the recognition information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, autonomous mobile objects that operate in environments such as factories and logistics warehouses, such as automated guided vehicles (for example, AGVs) and autonomous mobile robots (for example, AMRs), have been known.

[0003] Patent Document 1 discloses a technology for protecting privacy and confidentiality by masking sensed data while preventing the masking from interfering with the robot's intended use. Specifically, the technology discloses a technology for adaptively switching masking processes such as image processing, changing the camera direction, or changing the path of the moving object depending on the intended use (stationary, talking, or moving) and the positional relationship with the prohibited area for photography. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-62308 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology disclosed in Patent Document 1, depending on the work content of the robot, it may not be possible to maintain confidentiality and perform the work at the same time.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an information processing device that can maintain confidentiality while allowing work to be carried out. [Means for solving the problem]

[0007] An information processing device as one aspect of the present invention has a first acquisition means for acquiring measurement information obtained by measuring a work object and its surroundings, or recognition information recognized based on the measurement information, a second acquisition means for acquiring the content of the work as work information, a decision means for deciding a control method for performing the work based on confidential information contained in the measurement information or the recognition information and the work information, and a control means for executing the control method based on the measurement information or the recognition information.

[0008] Other objects and features of the present invention are illustrated in the following examples. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an information processing device that can achieve both confidentiality and work execution. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a usage scenario of an information processing device according to each embodiment. [Figure 2] FIG. 2 is a block diagram showing a module configuration of a system according to each embodiment. [Figure 3] FIG. 2 is a block diagram showing a hardware configuration of an information processing device according to each embodiment. [Figure 4] 10 is a flowchart illustrating a process executed by an information processing device in each embodiment. [Figure 5] 10 is a correspondence table between work information, confidential information, and control method types in the first embodiment. [Figure 6] 10 is a correspondence table between work information, confidential information, and control method types in the second embodiment. [Figure 7] FIG. 13 is a diagram showing a display screen as an example of a notification means in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following describes in detail the embodiments of the present invention with reference to the drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. [Example]

[0012] First, an overview of an environment in which an information processing device according to a first embodiment of the present invention is used will be described. In this embodiment, an example in which the information processing device is applied in a situation in which a task execution device linked with a vision system executes a predetermined task on an object (task target) whose appearance includes confidential information will be described.

[0013] The vision system is a device that recognizes and measures surrounding objects and measures its own position and orientation (hereinafter referred to as "position and orientation") based on information acquired from sensors. The vision system also determines the content of the work to be performed by the work performing device and issues work instructions. The work performing device is, for example, a robot that performs a predetermined work. In this embodiment, the work performing device performs the work based on information output by the vision system.

[0014] The information measured by the sensor contains confidential information. Therefore, the information processing device of the vision system processes the information so that it does not contain confidential information according to the content (type) of work and limits the information measured by the sensor so that the work can be carried out. This makes it possible to maintain confidentiality while also carrying out the work.

[0015] An example of a usage scenario of the information processing device according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of a usage scenario of the information processing device according to this embodiment. Fig. 1 shows a situation (work environment 101) in which a task execution device 103 equipped with a vision system 102 according to this embodiment performs pick-and-place work (predetermined work) while traveling along a route 104. A specific pick-and-place work involves grabbing a camera prototype 106 on a desk 105 with a finger hand 107 and placing the prototype 106 on an adjacent shelf 108.

[0016] Next, the module configuration of a system 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the module configuration of a system 200 according to this embodiment. The system 200 includes a measurement / recognition device 201, an information processing device 202, and an operation execution device 103.

[0017] The measurement and recognition device 201 comprises an image sensor 203 and recognition means 204. The task execution device 103 comprises task information transmission means 210, and an actuator 209 used to execute the task. The information processing device 202 comprises task information acquisition means (second acquisition means) 205, sensor information acquisition means (first acquisition means) 206, control method determination means (determination means) 207, and control means 208. The information processing device 202 can communicate with both the task execution device 103 and the measurement and recognition device 201. Note that in this embodiment, the measurement and recognition device 201 and the information processing device 202 are included in the vision system 102.

[0018] The work information acquisition means 205 acquires the content of the work and the content of the confidential information as work information. The recognition means 204 acquires information that recognizes a specific target based on the information acquired from the image sensor 203 and the work information acquisition means 205. The recognized information includes, but is not limited to, information regarding the position and orientation of the vision system 102 and the work performing device 103, or other information that changes depending on the content of the work.

[0019] The sensor information acquisition means 206 acquires information about the work target and its surroundings from the image sensor 203 or the recognition means 204. The control method determination means 207 determines a control method for carrying out the work while protecting confidential information, based on the work information acquired by the work information acquisition means 205. The method for determining the control method will be described in detail later. The control means 208 executes the control method determined by the control method determination means 207, based on the information acquired by the sensor information acquisition means 206.

[0020] Next, the hardware configuration of the information processing device 202 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the hardware configuration of the information processing device 202 according to this embodiment. That is, the information processing device 202 in Fig. 2 has the hardware configuration shown in Fig. 3.

[0021] A CPU (Central Processing Unit) 301 controls various devices connected to a bus 308 and executes information processing in this embodiment. A ROM (Read Only Memory) 302 stores a BIOS (Basic Input / Output System) program and a boot program. A RAM (Random Access Memory) 303 is used as the main storage device of the CPU 301. An external memory 304 stores programs processed by the information processing device 202.

[0022] The input unit 305 is a keyboard or mouse, and performs processing related to the input of information. The display unit 306 outputs the calculation results of the information processing device 202 to a display device (notification means) in accordance with instructions from the CPU 301. The display device may be of any type, such as a liquid crystal display device, a projector, or an LED (Light Emitting Diode) indicator. The bus 308 connects the CPU 301, RAM 303, ROM 302, and external memory 304 so that they can communicate with each other. The I / O (Input / Output) 307 is connected to the image sensor 203, recognition means 204, actuator 209, and work information transmission means 210.

[0023] Next, a process (information processing method) executed by the information processing device 202 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the process executed by the information processing device 202. The process of the flowchart in Fig. 4 starts simultaneously with the start of travel of the work execution device 103 within the work environment 101.

[0024] First, in step S401, the control means 208 initializes the information handled by the information processing device 202. During the initialization, for example, the control means 208 loads data of a correspondence table between work information, confidential information, and control methods (described later) from the external memory 304. Next, in step S402, the work information acquisition means 205 acquires information on the content of the work (work information) and the confidentiality protection target as work information. The acquired information changes depending on the work information.

[0025] Next, in step S403, the sensor information acquisition means 206 acquires sensing information (measurement information) obtained by measuring information about the work object and its surroundings, or information recognized based on the sensing information (recognition information). The detailed sensing information or recognition information acquired varies depending on the work information. The sensing information is, for example, but is not limited to, information about captured images used in performing the work or the three-dimensional shape of the surrounding environment. The recognition information is, for example, but is not limited to, the position and posture, shape, and color information of the work object or surrounding obstacles, or the position and posture of the vision system 102.

[0026] Next, in step S404, the control method determination means 207 determines a control method for performing the work while protecting the confidential information, based on the work information acquired by the work information acquisition means 205. The method for determining a control method based on the work information will be described later. Next, in step S405, the control means 208 executes the control method determined by the control method determination means 207, based on the information acquired by the sensor information acquisition means 206.

[0027] Next, a method for determining a control method will be described in detail with reference to FIG. 5. FIG. 5 is a correspondence table between work information, confidential information, and control methods, and is used in the processing of step S404 in FIG. 4. The table in FIG. 5 consists of "work content," "work details," "confidential information," and "control method," and shows that the "control method" is determined based on the contents of the "work content," "work details," and "confidential information." "Work content" indicates the work content (e.g., pick-and-place, unscrewing, dusting, etc.) performed by the work performing device 103. "Work details" indicates supplementary information for the work content. "Confidential information" is information that is subject to confidentiality protection. "Control method" indicates a method for processing information provided to the work performing device 103 so that it does not contain confidential information while ensuring the execution of the work. "Control method" is broadly classified into control types consisting of at least one of "reducing the resolution," "reducing the dimensions," "limiting the range of measurement values ​​(limiting the measurement range (spatial range))," or "replacing with abstract information" of the output information. However, the processing method is not limited to these, and other methods may be used as long as they do not include specific confidential information.

[0028] In step S404 of FIG. 4, first, a "control method" is determined from the correspondence table based on the "work content" and "confidential information" acquired by the work information acquisition means 205. Then, the control method determination means 207 determines specific control content as needed, depending on the control type associated with the determined "control method." For example, if the control type is "reduce resolution," the method for reducing the resolution is determined. Also, if the control type is "reduce dimensions," the method for reducing the dimensions is determined. Also, if "limit the range of measurement values," the control means 208 determines what information to limit and to what range. Also, if "replace with abstract information," the control means 208 determines what information to abstract and how.

[0029] The specific contents determined by the flow of this embodiment are as follows.

[0030] If the task is "pick and place," the task details are "shape with number of corners of the object to be grasped exceeding a threshold," and the confidential information is "shape of the object to be grasped," then the control method for reducing the resolution is determined as "leaving the shape in the grasping direction unchanged, and reducing the resolution of the shape in other directions." If the number of corners of the object to be grasped is greater than the threshold, a resolution value is determined and set that makes it impossible to detect the position and angle of edges and faces, which are shape information, except in the grasping direction. This is because, in order to stably grasp an object with an arm, it is necessary to accurately know the contact point of the arm that will not wobble when grasped. On the other hand, even if the resolution of the shape in other directions is reduced, as long as the resolution of the shape in the grasping direction is high, the grasping task can still be performed, so the above control method is determined.

[0031] If the task is "pick and place," the task details are "shape with number of corners of the grasped object below threshold," and the confidential information is "shape of the grasped object," the control method for reducing dimensions is determined to "provide the position and posture of the finger hand and the distance between the finger and the object." As a result, if the shape of the grasped object has number of corners below threshold, the task can be performed without handing over the shape.

[0032] If the task is "pick and place," the task details are "performed by suction hand," and the confidential information is "shape of the object to be grasped," then the control method of "providing the coordinates of the center of gravity of a group of planar areas" is selected as a dimensional reduction method. This planar area is a planar area that is larger than the size of the suction cup of the suction hand. If the coordinates of the center of gravity of this planar area are known, it becomes possible to grasp the object to be grasped even if the shape of the object does not exist, so the above control method is selected.

[0033] If the task is "pick and place," the task details are other than the three patterns above, and the confidential information is the "shape of the object to be grasped," then the control method of replacing the information with abstract information, "providing the hand position, hand direction, and hand opening width," is determined. This is because if the hand position, orientation, and opening width at the time of grasping are known, grasping is possible even without shape information, so the above control method can be determined.

[0034] If the task is to "remove screws" and the confidential information is "the shape and color of the object, although the shape is not confidential if it is only within 3 cm of the screw," then the control method is determined to be "providing information on the position and orientation of the screw, as well as a grayscale image of the local area near the screw," which is a combination of reducing dimensions and replacing it with abstract information. This means that if the arm can move close to the position of the screw, and only an image of the area around the screw is passed, the shape of the object becomes unnecessary for the task. Furthermore, the movement of the arm to the screw position (without error) required for unscrewing the screw is possible through feedback control even without color information by converting it to a grayscale image, so the above control method is determined.

[0035] If the task is "dusting" and the confidential information is "the shape of the object to be cleaned," the control method for reducing the resolution is determined to be "approximating the 3D shape of the object with a bounding box." In the case of dusting, the above control method can be determined because the task can be performed even if the distance to the object to be cleaned is rough.

[0036] If the task is "picking up something that has fallen on the floor," the task details are "the dropped object is below a certain height," and the confidential information is "the shape of surrounding objects," then the control method for limiting the range of measurement values ​​is determined to be "limiting measurement values ​​to only the task area near the floor." The area to be limited is measurement values ​​below a certain height from the floor, and any measurement values ​​above that height are deleted and provided. The above control method is determined because the task of picking up something that has fallen on the floor requires only information about the area near the floor.

[0037] If the task is "mopping the floor," the task details are "the floor is wet," and the confidential information is "the color of the work environment without location specification," the control method "providing the position and orientation of the task execution device and a grayscale image" is determined. This corresponds to a combination of the method of reducing dimensions and the method of replacing with abstract information. The above control method is determined because determining the degree of wetness is possible using a grayscale image even without color information.

[0038] If the task is "suction cleaning regardless of material" and the confidential information is "the color of the work environment without specifying the location," the control method of "providing a grayscale image" is selected as a way to reduce the dimension. In the case of suction cleaning without distinguishing between tatami mats and carpets, the cleaning environment can be grasped even without floor color information, so the above control method is selected.

[0039] If the task is "moving" and the confidential information is "the shape of surrounding objects," the control method for replacing the information with abstract information, "providing information on the position and orientation of the task performing device and information on the placement of surrounding objects," is determined. The above control method is determined because, as long as the current position of the task performing device and the positions of surrounding obstacles are known, movement is possible even if the shapes of surrounding obstacles are not known.

[0040] If the work content is "packaging work" and the confidential information is "shape and color of areas outside the work area," the control method for limiting the range of measurement values ​​is determined to be "providing information on shape and color only within the work area." This is because as long as the shape and color within the work area are known, information outside the work area is not necessary, so the above control method is determined.

[0041] According to this embodiment, it is possible to maintain confidentiality while still being able to carry out the work. <Variation 1> Next, a first modified example of this embodiment will be described. In the above-described embodiment, the vision system 102 is mounted on the task execution device 103, and is a device that estimates its own position, determines the content of the task to be performed by the task execution device 103, and issues task instructions. The self-position estimation can be performed, for example, by Visual SLAM (Simultaneous Localization and Mapping).

[0042] However, this embodiment is not limited to this, and other forms are also acceptable as long as it is possible to measure the position and orientation of the task execution device 103, determine the content of the task to be performed by the task execution device 103, and issue task instructions. For example, it may be an autonomously traveling mobile body that is physically different from the task execution device 103, or a network camera fixed to the task execution location. In other words, it may be configured so that at least some of the functions of the vision system 102 are provided in a device separate from the task execution device 103, and information is exchanged via wireless communication or the like. For example, it may be configured so that at least some of the functions of the information processing device 202 are executed by an external device such as cloud computing.

[0043] This allows the control method to be implemented to protect confidentiality regardless of the form of the vision system 102. <Variation 2> Next, a second modification of this embodiment will be described. In the above embodiment, the control method is determined using confidential information, but this is not limited to this. Even if there is no area designation of specific confidential information, if there is confidential information to be dealt with, the control method may be determined without specifying the confidential information. For example, the control method may be determined in a state where it is known that the color of the surrounding environment is to be omitted.

[0044] In the above-described embodiment, confidential information related to the work content is acquired from the work information acquisition means 205, but the present invention is not limited to this. As long as the confidential information to be dealt with can be acquired, the confidential information may be acquired from a means other than the work information acquisition means 205. For example, the information processing device 202 may further include confidential identification information acquisition means for acquiring specific confidential information, and may acquire the confidential information to be dealt with from the confidential identification information acquisition means.

[0045] This makes it possible to maintain confidentiality and carry out work at the same time, regardless of the method used to acquire the confidential information. [Example]

[0046] Next, a second embodiment of the present invention will be described. In the first embodiment, a control method for processing information to be output to the work execution device 103 was described. On the other hand, in this embodiment, a method for adjusting (determining) the operation of the work execution device 103 according to the processing of information will be described. Specifically, the operation timing is advanced or the operation speed is slowed down. This adjustment allows the work to be performed more safely.

[0047] The usage of the information processing device 202 according to this embodiment is the same as that in Fig. 1, and therefore the description thereof will be omitted. The configuration of the system according to this embodiment is the same as that in Fig. 2, and therefore the description thereof will be omitted. The hardware configuration of the information processing device 202 according to this embodiment is the same as that in Fig. 3, and therefore the description thereof will be omitted.

[0048] The processing of the information processing device 202 according to this embodiment will be described with reference to Figures 4 to 6. In this embodiment, as described above, the output information to the work execution device 103 is processed, and then a method for controlling the work execution device 103 is determined according to changes in the information caused by the processing of the output information. Note that the method for determining the control method for processing the information is the same as in Example 1, and therefore a description thereof will be omitted.

[0049] Fig. 6 is an explanatory diagram of a method for determining the control method of the work performing device 103, and is a table used in the process of determining the second control method, which is performed in step S404 of Fig. 4. The table in Fig. 6 consists of "work content," "first control method," "reduced information," and "second control method," and shows that the "reduced information" is determined in association with the "first control method," and the "second control method" is determined according to the "reduced information" and the "work content." The "second control method" is a control method of the work performing device for performing the work content while compensating for the "reduced information" resulting from the "first control method."

[0050] In the process of determining the second control method, first, the "information to be reduced" is determined from the correspondence table based on the "first control method." Next, the "second control method" is determined from the correspondence table based on the "task content" and the "information to be reduced." The specific contents determined by the above flow are as follows:

[0051] In a situation where the task is "moving," if the first control method is a control method of "providing an image with reduced resolution of the shape of the surrounding objects," it is determined that the information will reduce the "accuracy of distance to the surrounding objects." This is because the accuracy of detecting the distance to the surrounding objects based on the image will decrease due to the image having reduced resolution.

[0052] When the task is "moving" and the information that is being reduced is "the accuracy of the distance to surrounding objects," the second control method is determined, which is to "advance the timing of stopping movement and turning in front of surrounding objects." The timing to be determined is set to the inverse of the reduced resolution. This is because the accuracy of the distance to the target object becomes unclear due to the reduced resolution, and in order to prevent collisions with surrounding objects, the timing of stopping movement and turning is advanced according to the reduced resolution. For this purpose, the control method of the task execution device is determined.

[0053] In a situation where the task is "picking and placing a deformable object," if the first control method is "providing an image with reduced resolution of the object to be grasped," it is determined that the "position and orientation of the grasped object" is information that is reduced. This is because the detection accuracy of the position and orientation of the grasped object decreases in images with reduced resolution. If the task is "picking and placing a deformable object" and the information that is reduced is "the position and orientation of the grasped object," it is determined that the second control method is "shortening the finger feedback cycle." The feedback cycle is set to a value that shortens the cycle by the proportion of the reduced resolution. This control method for the task execution device is determined to prevent the grasp position from becoming coarse due to the coarse distance accuracy between the grasped object and the fingers, resulting in the fingers missing or over-grasping the object and deforming the grasped object.

[0054] When the task is "pick and place," if the first control method is a control method that "reduces resolution only in directions other than the gripping direction," it is determined that the information on the "shape of the grasped object in directions other than the gripping direction" is decreasing. If the task is "pick and place" and the information that is decreasing is "shape of the grasped object in directions other than the gripping direction," it determines a second control method that "slows the lifting speed of the hand." The determined operating speed is set to a speed at which the arm can immediately stop moving. This is because the shape information on the grasped object in directions other than the gripping direction becomes coarse, and the control method for the task execution device described above is determined to prevent collisions during lifting.

[0055] In a situation where the task is "pick and place," if the first control method is a control method that "provides the position and posture of the finger hand and the distance between the finger and the object," it determines that the information is decreasing "the distance between the finger and the object during the grasping operation." If the task is "pick and place" and the decreasing information is "the distance between the finger and the object during the grasping operation," it determines a second control method that "shortens the finger feedback cycle." The task execution device performs the grasping operation based on the distance between the finger and the object, but if the position and posture of the finger changes due to the grasping operation, the new finger position and distance between the object are required. Therefore, by shortening the feedback cycle, the grasping operation can be performed while the finger is constantly aware of the distance between the object and the object. In a situation where the work content is "pick and place," if the first control method is a control method of "providing the coordinates of the center of gravity of a planar area," it is determined that the information is that "the distance to the peripheral parts of the group of planar areas" decreases.

[0056] If the task is "pick and place" and the information being reduced is "distance to the peripheral parts of the group of flat areas," the second control method is determined, which is to "slow the finger movement speed while shortening the feedback period." This is because if only the position of the flat area is given to the task performing device in the first control method, information about the periphery of the flat area will not be passed on. For this reason, the control method for the task performing device is determined in order to prevent the hand from colliding with peripheral objects in the flat area during the gripping operation.

[0057] In a situation where the work content is "pick and place," if the first control method is a control method that "provides the hand position, hand direction, and hand opening width," it is determined that the information is that "the distance between the finger and the object during the grasping operation" decreases.

[0058] If the task is "pick and place" and the decreasing information is "the distance between the fingers and the object during the gripping operation," the second control method of "shortening the finger feedback cycle" is determined. The task execution device 103 operates based on the finger opening degree during the gripping operation. When the finger opening degree changes due to the gripping operation, the next finger opening degree becomes necessary. Therefore, by shortening the feedback cycle, the gripping operation can be performed while constantly understanding the finger opening degree over time.

[0059] In a situation where the task is "loosening a screw," if the first control method is a control method of "providing information on the position and orientation of the screw and a grayscale image of the local area near the screw," it is determined that the information is "decreasing the distance between the hand and the screw while approaching the screw position." If the task is "loosening a screw" and the information that is decreasing is "the distance between the hand and the screw while approaching the screw position," it is determined that the second control method is "slowing the movement speed when approaching the screw position." This is because there is no information on the distance between the hand and the screw while the arm is moving to the vicinity of the screw position or during feedback control based on the grayscale image, and so the control method for the task execution device 103 is determined in order to safely perform the gripping task without the screw and the hand accidentally colliding with each other.

[0060] In a situation where the task is "dusting," if the first control method is a control method of "approximating the three-dimensional shape of the object with a bounding box," it is determined that this information will decrease the "accuracy of the distance to the object to be cleaned." If the task is "dusting" and the decreasing information is "accuracy of the distance to the object to be cleaned," it determines a second control method of "shortening the stroke of the dusting motion." If the accuracy of the distance to the object to be cleaned decreases due to approximating the three-dimensional shape of the object to be cleaned with a bounding box, it determines the control method of the task execution device to clean more safely so as not to collide with the object to be cleaned during the dusting motion.

[0061] In a situation where the task is "cleaning by picking up items that have fallen on the floor," if the first control method is a control method that "limits the measurement values ​​only to the task target area near the floor," it is determined that the information is decreasing "distance to objects above a certain height." Then, if the task is "cleaning by picking up items that have fallen on the floor" and the decreasing information is "distance to objects above a certain height," it determines a second control method that "slows the operating speed of the task performing device." By limiting the measurement values ​​only to the task target area near the floor, the information on the distance to objects above a certain height decreases, so the control method for the task performing device is determined in order to clean more safely without interfering with objects above a certain height.

[0062] In a situation where the task is "mopping the floor," if the first control method is a control method of "providing the position and orientation of the task-performing device and a grayscale image," it is determined that the information to be reduced is "the position and orientation of an object with a similar color to the floor when it is encountered." If the task is "mopping the floor" and the information to be reduced is "the position and orientation of an object with a similar color to the floor when it is encountered," it is determined that the second control method is "slowing down the operating speed of the task-performing device." By providing a grayscale image, the position and orientation and distance information of objects with a similar color to the floor is reduced. This determines the control method for the task-performing device to avoid interference with objects with a similar color to the floor during cleaning work and to clean more safely.

[0063] In a situation where the task is "suction cleaning without regard to materials," if the first control method is a control method of "providing grayscale images," it is determined that the information to be reduced is "the position and orientation of objects with similar colors to the floor when such objects are present." If the task is "suction cleaning without regard to materials" and the information to be reduced is "the position and orientation of objects with similar colors to the floor when such objects are present," it is determined that the second control method is "slowing down the operating speed of the task-performing device." This is because by providing grayscale images, the position and orientation and distance information to objects with similar colors to the floor is reduced. As a result, the control method for the task-performing device is determined in order to avoid interference with objects with similar colors to the floor during cleaning work and to clean more safely.

[0064] In a situation where the task content is "packing work," if the first control method is a control method of "providing shape and color information only within the work area," it is determined that the information that will decrease is the "position and orientation of objects outside the work area." If the task content is "packing work" and the information that will decrease is the "position and orientation of objects outside the work area," it is determined that the second control method will "slow the operating speed of the task performing device." By providing shape and color information only within the work area, the information on the position and orientation of objects outside the work area is reduced. As a result, the control method for the task performing device is determined in order to avoid the task performing device and objects outside the work area during packing work and work more safely.

[0065] According to this embodiment, even in a situation where a task cannot be performed by processing information alone, it is possible to maintain confidentiality and perform the task at the same time by adjusting the operation of the task performing device. [Example]

[0066] Next, a third embodiment of the present invention will be described. In this embodiment, a method for achieving both confidentiality and execution of work without processing information provided to the work execution device 103 will be described.

[0067] In this embodiment, a method for controlling the work execution device is determined and implemented so that the work execution device 103 removes peripheral objects to be classified from its sensing range (so that objects to be classified are outside the sensing range of the sensor) while ensuring the execution of the work. This makes it possible to achieve both confidentiality protection and the execution of the work even in situations that cannot be resolved by processing all information. A specific method for determining a control method will be described later.

[0068] The usage scenario of the information processing device 202 according to this embodiment is the same as that in Fig. 1, and therefore the description thereof will be omitted. Also, the configuration of the system 200 according to this embodiment is the same as that in Fig. 2, and therefore the description thereof will be omitted.

[0069] The processing (information processing method) of the information processing device 202 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the processing executed by the information processing device 202 in this embodiment, and the description common to the first and second embodiments will be omitted.

[0070] In step S404 of FIG. 4, the following content is determined. If the work content is "serving food," the work details are other than those mentioned above, and the confidential information is "the shape and color of a specific object in the vicinity," a control method is determined to "change the route of the work performing device to one that does not sense the specific object." The route determined here is a movement route of the work performing device 103 that does not include a specific object that is a confidential subject within the sensing field of view of the work performing device 103. In this case, it is assumed that the work performing device 103 is equipped with an omniwheel mechanism. This allows work to be performed while traveling on a route that does not measure the specific object that is confidential if the movement route is directly controlled, and therefore the above control method is determined.

[0071] According to this embodiment, even in a situation where information processing is prohibited, it is possible to maintain confidentiality while still allowing work to be carried out, and this can be applied only to the area of ​​confidentiality. <Variation 3> Next, a modified example (Modification 3) of this embodiment will be described. In the above-described embodiment, the control method "changing the path of the task executing device so that the specific object is not sensed" was determined. However, if the task executing device has a pan-tilt mechanism for the sensor, another control method may be determined. For example, if the task content is "serving food," the task details are "the task executing device is equipped with a pan-tilt mechanism," and the confidential information is "the shape and color of a specific object in the vicinity," the control method for the task executing device 103 may be determined to "changing the pan-tilt value so that the specific object is not sensed." Specifically, the horizontal prospect angle of the specific object and the prospect angles at both ends excluding the horizontal prospect angle are calculated from the area size of the specific object to be classified in the captured image acquired from the sensor information acquisition means 206. Then, the sum of the horizontal prospect angle of the specific object and the smaller of the other prospect angles is determined as the pan-tilt angle at which the specific object does not fall within the sensing range.

[0072] As a result, in the case of a work execution device equipped with a pan-tilt mechanism, by applying the pan-tilt angle to the work execution device 103, work can be performed without measuring a specific object that is confidential. [Example]

[0073] Next, a fourth embodiment of the present invention will be described. In this embodiment, a method for notifying a visual system management user of the implementation content of control for protecting confidential information according to the work content when the work execution device 103 and the visual system cooperate with each other according to the first to third embodiments will be described. In this embodiment, the use scene of the information processing device 202 is the same as in Fig. 1, and therefore the description thereof will be omitted.

[0074] The configuration of a system 200 according to this embodiment will be described with reference to Figures 2 and 7. An information processing device 202 according to this embodiment has a display unit (notification means) in addition to the configuration shown in Figure 2. Since the other configurations are the same as those of the other embodiments, detailed description will be omitted. The display unit notifies the results of at least one of the sensor information acquisition means 206, the work information acquisition means 205, the control method determination means 207, the control means 208, and the confidential identification information acquisition means.

[0075] 7 is a display screen 701 of the display unit (notification means) according to this embodiment, which is an example of the display screen in the usage form of Example 1. The display screen 701 shows the result of determining a control method of "providing the position and posture of the finger hand and the distance between the finger and the object" when the work content is "pick and place", the work details are "the object to be grasped has a simple shape", and the confidential information is "the shape of the object to be grasped".

[0076] The display screen 701 has a unique information display section 702. The unique information display section 702 displays the work content, the work target, and confidential information. The display screen 701 has a control content display section 703. The control content display section 703 displays the sensor information measured by the vision system 102 on the left side. Then, at least one of the content for processing the information to be provided to the work performing device 103 determined by the control method determination means 207, the content for changing the sensing range of the work performing device 103, or the control content to be performed by the control means 208 is displayed on the right side.

[0077] According to this embodiment, it is possible to notify the visual system management user of information that allows for both confidentiality and execution of work. <Variation 4> Next, a modified example (Modification 4) of this embodiment will be described. In the above-described embodiment, the notification of the implementation details of control to protect confidential information according to the work content is in GUI format. However, this embodiment is not limited to this, and other formats may be used as long as the implementation details of control to protect confidential information according to the work content can be notified to the visual system management user. For example, the content may be notified in text format.

[0078] This makes it possible to notify the implementation details of control to protect confidential information according to the work content described in the fourth embodiment, regardless of whether the display unit is GUI display compatible or CUI display compatible hardware. <Variation 5> Next, another modified example (Modified Example 5) of this embodiment will be described. The notification contents in the above-described embodiment are the display contents of the unique information display section 702 and the control content display section 703. However, this embodiment is not limited to this, and other information may be additionally notified as long as it is information related to determining the control content. For example, intermediate products used in determining the control content or the correspondence table of FIG. 5 or FIG. 6 may be additionally displayed. This makes it possible to notify various information related to balancing confidentiality protection and work execution.

[0079] In each embodiment, confidential information is information that the user does not want to leak to the outside. The confidential information can be determined in advance by the user. The user can also change the confidential information depending on the situation, such as the content of the work. The confidential information can be stored in memory in advance, or can be set appropriately when used.

[0080] Furthermore, in each embodiment, the control method determination means 207 determines a control method for performing a task based on confidential information included in the measurement information or recognition information and task information (task content), but is not limited to this. Even if some information (specific information) included in the measurement information or recognition information may not strictly be confidential information, the user may desire to minimize the information transmitted to the task execution device 103. For this reason, the device may be configured to process at least some of the information included in the measurement information or recognition information in accordance with the task information, regardless of whether the information (specific information) included in the measurement information or recognition information is confidential information specified by the user.

[0081] That is, the information processing device 202 may be configured to transmit only the information necessary to perform the work to the work execution device 103 according to the work information (i.e., for each work content or type). The information necessary to perform the work may be set in advance according to the work content, or may be determined by the user according to the work content. Here, the information necessary to perform the work is preferably information that is essential to perform the work, but is not limited to this, and it is sufficient if it does not include at least a portion of the information included in the measurement information or recognition information that is unnecessary for performing the work.

[0082] That is, regardless of whether specific information included in the measurement information or recognition information is confidential information specified by the user, the specific information may be processed (corrected) depending on the content of the work. Here, processing of specific information means processing the specific information such as deleting, correcting, or blurring (reducing resolution). However, processing is not limited to these, and may also include, for example, deforming the shape of an object included in the measurement information or recognition information or changing the color of the object.

[0083] As a result, even if there is a possibility that some information (specific information) included in the measurement information or recognition information is not confidential information in the strict sense, it is possible to prevent the transmission of information unnecessary for the work by processing the information to be transmitted to the work execution device 103 according to the content of the work. Even in this form, it is possible to substantially achieve both confidentiality and the execution of the work. <Other Examples> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0084] According to each embodiment, it is possible to provide an information processing device, an information processing method, and a program that can achieve both confidentiality protection and work execution.

[0085] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) a first acquisition means for acquiring measurement information obtained by measuring a work target and the surrounding area of ​​the work target, or recognition information recognized based on the measurement information; A second acquisition means for acquiring the content of the work as work information; a determination means for determining a control method for performing the work based on confidential information included in the measurement information or the recognition information and the work information; a control means for executing the control method based on the measurement information or the recognition information; An information processing device comprising: (Configuration 2) the determining means determines a processing method for processing the measurement information or the recognition information, The processing method is at least one of reducing the resolution, reducing the dimension, limiting the spatial range, or replacing it with abstract information. 2. The information processing device according to configuration 1, (Configuration 3) the determination means determines the operation of the work execution device that executes the work in accordance with the processing method. 3. The information processing device according to configuration 2. (Configuration 4) the determining means determines the operation of the task execution device so as not to measure the confidential information. 4. The information processing device according to configuration 3. (Configuration 5) the determination means determines the operation of the task execution device so that the confidential information is outside a sensing range of a sensor. 5. The information processing device according to configuration 4. (Configuration 6) Further comprising a notification means for notifying a result of at least one of the first acquisition means, the second acquisition means, the determination means, and the control means. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) a first acquisition means for acquiring measurement information obtained by measuring a work target and the surrounding area of ​​the work target, or recognition information recognized based on the measurement information; A second acquisition means for acquiring the content of the work as work information; a determining means for determining a processing method for processing the measurement information or the recognition information based on the work information; a control means for controlling the work execution device based on the measurement information or the recognition information processed by the processing method; An information processing device comprising: (Method 1) An information processing method executed by a computer installed in an information processing device, a sensor information acquisition step of acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a work information acquisition step of acquiring the content of the work as work information; a control method determination step of determining a control method for performing the work based on confidential information included in the measurement information or the recognition information and the work information; a control step of executing the control method based on the measurement information or the recognition information; An information processing method comprising: (Configuration 8) A program that causes a computer to execute each step of the information processing method described in Method 1. (Method 2) An information processing method executed by a computer installed in an information processing device, a first acquisition step of acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a second acquisition step of acquiring the content of the work as work information; a determination step of determining a processing method for processing the measurement information or the recognition information based on the work information; a control step of controlling an operation execution device based on the measurement information or the recognition information processed by the processing method; An information processing method comprising: (Configuration 9) A program that causes a computer to execute each step of the information processing method described in Method 2.

[0086] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and changes are possible within the scope of the present invention. The embodiments may also be combined as appropriate. [Explanation of symbols]

[0087] 202 Information processing equipment 206 sensor information acquisition means (first acquisition means) 205 Work information acquisition means (second acquisition means) 207 Control information determination means (determination means) 208 Control Means

Claims

1. a first acquisition means for acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a second acquisition means for acquiring the content of the work as work information; a determination means for determining a control method for performing the work based on confidential information included in the measurement information or the recognition information and the work information; a control means for executing the control method based on the measurement information or the recognition information; An information processing device comprising:

2. the determining means determines a processing method for processing the measurement information or the recognition information, The processing method is at least one of reducing the resolution, reducing the dimension, limiting the spatial range, or replacing it with abstract information.

2. The information processing apparatus according to claim 1, wherein:

3. the determination means determines the operation of the work execution device that executes the work in accordance with the processing method.

3. The information processing apparatus according to claim 2, wherein:

4. the determining means determines the operation of the task execution device so as not to measure the confidential information.

4. The information processing apparatus according to claim 3,

5. the determination means determines the operation of the task execution device so that the confidential information is outside a sensing range of a sensor.

5. The information processing apparatus according to claim 4,

6. The apparatus further includes a notification unit that notifies a result of at least one of the first acquisition unit, the second acquisition unit, the determination unit, and the control unit.

6. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

7. a first acquisition means for acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a second acquisition means for acquiring the content of the work as work information; a determining means for determining a processing method for processing the measurement information or the recognition information based on the work information; a control means for controlling the work execution device based on the measurement information or the recognition information processed by the processing method; An information processing device comprising:

8. An information processing method executed by a computer installed in an information processing device, a sensor information acquisition step of acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a work information acquisition step of acquiring the content of the work as work information; a control method determination step of determining a control method for performing the work based on confidential information included in the measurement information or the recognition information and the work information; a control step of executing the control method based on the measurement information or the recognition information; An information processing method comprising:

9. A program for causing a computer to execute each step of the information processing method according to claim 8.

10. An information processing method executed by a computer installed in an information processing device, a first acquisition step of acquiring measurement information obtained by measuring a work target and its surroundings, or recognition information recognized based on the measurement information; a second acquisition step of acquiring the content of the work as work information; a determination step of determining a processing method for processing the measurement information or the recognition information based on the work information; a control step of controlling an operation execution device based on the measurement information or the recognition information processed by the processing method; An information processing method comprising:

11. A program for causing a computer to execute each step of the information processing method according to claim 10.

Images