Information processing device and information processing program

Abstract

To provide an information processing device and an information processing program which can suppress the occurrence of excess or deficiency of processing in a video that has been shot even when a region occupied by a work target changes.SOLUTION: An information processing device includes a processor. The processor acquires an image obtained by imaging an object, specifies a first region occupied by a work target being the target of the work in the objects in the image, generates a processed image applied with processing of making a second region indicating a region other than the first region in the image invisible, further generates a processed image in which the second region following the application is made to be invisible by applying a deformation region being a region determined by the shape of the work target after deformation that has been previously registered instead of the first region if the first region changes following deformation of the work target, and transmits the processed image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Patent Document 1 discloses a maintenance support system comprising: a wearable terminal equipped with an imaging means and worn by a maintenance worker; a first identification means for identifying a predetermined first three-dimensional area including the maintenance target and / or a predetermined second three-dimensional area excluding the maintenance target based on a predetermined reference point in an image acquired by the imaging means in an initial state; a second identification means for identifying a mask pixel area excluding an effective pixel area corresponding to the first three-dimensional area and / or a mask pixel area corresponding to the second three-dimensional area in the image acquired by the imaging means in a post-movement state in which the wearable terminal has moved; a processed image generation means for generating a processed image of the image acquired by the imaging means in the post-movement state in which the mask pixel area identified by the second identification means is made invisible; and a communication means for transmitting the processed image generated by the processed image generation means to a maintenance supporter terminal, wherein the reference point does not need to be included in the image acquired by the imaging means in the post-movement state.

[0003] Patent Document 2 discloses a remote IT operation support system that supports operation of an IT system, comprising: a first mobile terminal of a first worker who works on a local IT system; a second mobile terminal of a second worker who works remotely; and a server. Each device in the IT system is provided with an ID medium bearing an ID, and the system has setting information including information relating to the device and the ID, the user ID of the first worker, the user ID of the second worker, and information on the second worker's access authority for the device indicated by the ID. The server detects the ID on the ID medium from video of the device taken by the first worker using a camera, and, based on the setting information, confirms whether the second worker has access authority for the device indicated by the detected ID. The server then performs a masking process on the video, designating image portions of the device for which the second worker has access authority as non-masked areas and image portions of the device for which the second worker does not have access authority as masked areas, thereby generating a masked image. The masked image is then provided to the second mobile terminal for display on a display screen.

[0004] Patent document 3 discloses a method for excluding portions of a scene from a video signal in a local augmented reality (AR) environment for compliance with data capture and transmission, the method including the steps of: making a pointing direction of a video camera follow a user's movement; capturing a video signal within the field of view (FOV) of the video camera of an object in the local scene; receiving hand gestures from a remote location for manipulating the object; overlaying the hand gestures on the video signal and displaying an augmented reality for the user to instruct the user on manipulating the object; determining, before capturing the video signal, whether the pointing direction of the video camera satisfies a positioning condition with respect to the marker in the local scene so that the FOV of the video camera is within a user-defined acceptable FOV with respect to the marker; and, if the positioning condition is not satisfied, controlling the camera to exclude at least portions of the camera's FOV that are outside the user-defined acceptable FOV from being captured in the video signal. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6748793 [Patent Document 2] Japanese Patent Application Publication No. 2018-36812 [Patent Document 3] Special Publication No. 2020-537230 Summary of the Invention [Problem to be solved by the invention]

[0006] A technology has been proposed in which video footage taken by on-site workers is sent to a terminal operated by a remote support person, allowing the on-site workers and the remote support person to share the work situation and provide support for the work.

[0007] However, the captured video may contain information other than the work object that the work is being performed on. Therefore, the area occupied by the work object at the worker's position may be specified in advance, only the area occupied by the work object may be captured, and the background area other than the pre-registered work object may be processed before transmitting the captured video to the terminal, thereby preventing the leakage of confidential information and other information.

[0008] However, when the area occupied by the work object changes due to tasks such as opening a door of the work object or removing a part of the work object, a part of the work object may be recognized as the background, and the background may be captured without being processed. In other words, the captured image may be over- or under-processed.

[0009] The present invention aims to provide an information processing device and an information processing program that can prevent over- or under-processing of captured video even when the area occupied by the work object changes. [Means for solving the problem]

[0010] A first aspect of the information processing device has a processor, which acquires an image of an object, identifies a first area of the object in the image that is occupied by a work object that is the object on which work will be performed, generates a processed image that has been processed to make a second area indicating an area other than the first area in the image invisible, and if the first area changes due to a deformation of the work object, applies a deformation area that is an area determined by the shape of the work object after deformation that has been registered in advance in place of the first area, further generates a processed image in which the second area resulting from the application is made invisible, and transmits the processed image.

[0011] In a second aspect of the information processing device, in the information processing device according to the first aspect, the processor further acquires spatial information, which is information on a three-dimensional space corresponding to an image and including an object, detects feature points indicating the work object from the image, and uses the feature points to identify a first space corresponding to a first region in the spatial information and a second space corresponding to a second region.

[0012] In a third aspect of the information processing device, in the information processing device relating to the second aspect, the processor sets invisibility information indicating information to be made invisible for the second space, and uses the invisibility information to generate a processed image in which a second area in the image corresponding to the second space is made invisible.

[0013] In the information processing device of the fourth aspect, in the information processing device of the third aspect, the processor further acquires position information indicating the position of the photographing unit that photographs an image in spatial information and the position of the work object, and directional information indicating the direction in which the photographing unit photographs, and generates a processed image in which the second area is made invisible according to the position information and directional information.

[0014] The information processing device of a fifth aspect is the information processing device according to the fourth aspect, and estimates position information and direction information from the amount of change in the feature points.

[0015] An information processing device of a sixth aspect is an information processing device according to any one of the first to fifth aspects, in which the processor stores a plurality of transformation areas and accepts designation of one of the plurality of transformation areas.

[0016] An information processing device of a seventh aspect is an information processing device relating to any one of the first to sixth aspects, in which the processor detects feature points indicating the work object from the image, and when it detects a deformation of the work object from the amount of change in the feature points, it notifies the user that the first area will be switched to a deformation area.

[0017] An information processing device of the eighth aspect is an information processing device according to the seventh aspect, in which, when the detected deformation of the work object is a deformation that expands the first area, the processor applies the corresponding deformation area of the work object after the deformation to generate a processed image.

[0018] An information processing device of a ninth aspect is an information processing device relating to any one of the first to eighth aspects, in which when the processor receives an instruction to switch to a deformation area that narrows the first area, it applies the deformation area according to the instruction to generate a processed image.

[0019] The information processing program of the tenth aspect causes a computer to acquire an image of an object, identify a first area of the object in the image occupied by a work object that is the object on which work will be performed, generate a processed image that has been processed to make a second area indicating an area other than the first area in the image invisible, and if the first area changes due to a deformation of the work object, apply a deformation area that is an area determined by the shape of the work object after deformation that has been registered in advance in place of the first area, further generate a processed image in which the second area resulting from the application is made invisible, and transmit the processed image. [Effects of the Invention]

[0020] According to the information processing device of the first aspect and the information processing program of the tenth aspect, even if the area occupied by the work object changes, it is possible to prevent over- or under-processing of the captured video.

[0021] According to the information processing device of the second aspect, even if the relative positions of the object and the image capturing unit change, the first area and the second area can be identified.

[0022] According to the information processing device of the third aspect, even if the relative positions of the object and the imaging unit change, the second region can be processed.

[0023] According to the information processing device of the fourth aspect, the second region can be processed in consideration of the relative positional relationship and the relative direction.

[0024] According to the information processing device of the fifth aspect, it is possible to easily obtain information on position and direction without using a sensor or the like for detecting position and direction.

[0025] According to the information processing device of the sixth aspect, it is possible to apply a first region having a shape desired by the user at a time desired by the user.

[0026] According to the information processing device of the seventh aspect, it is possible to recognize that the shape of the first area is changing.

[0027] According to the information processing device of the eighth aspect, the burden of switching to the modification area can be reduced.

[0028] According to the information processing device of the ninth aspect, it is possible to prevent unintended insufficient processing from occurring. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a configuration of an information processing system according to each embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device according to each embodiment. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device according to each embodiment. [Figure 4] 1A and 1B are schematic diagrams illustrating an example of three-dimensional space information used to explain SLAM according to each embodiment. [Figure 5] 10A and 10B are schematic diagrams illustrating an example of three-dimensional space information used to explain a space occupied by a work object according to each embodiment. [Figure 6] FIG. 2 is a schematic diagram illustrating an example of an image according to each embodiment. [Figure 7] 10A to 10C are schematic diagrams illustrating examples of processed images according to the embodiments. [Figure 8] 10A and 10B are diagrams illustrating an example of an object space for explaining switching accompanying deformation of a work object according to each embodiment; [Figure 9] 6 is a flowchart showing an example of processing for generating a processed image according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a target space for explaining deformation detection according to the second embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a target space for explaining deformation detection according to the second embodiment. [Figure 12] 10 is a flowchart showing an example of processing for generating a processed image according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] [First embodiment] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described in detail with reference to the accompanying drawings. Fig. 1 is a schematic diagram showing an example of the configuration of an information processing system 1 according to this embodiment.

[0031] 1, an information processing system 1 includes an information processing device 10 operated by a worker and a terminal 50 operated by a supporter. The information processing device 10 and the terminal 50 are connected to each other via a network N.

[0032] The information processing device 10 is a terminal such as a tablet equipped with a monitor 16 and a camera 18, which will be described later. The information processing device 10 acquires an image of an object to be worked on (hereinafter referred to as "work object") and transmits the acquired image to the terminal 50. When transmitting the image to the terminal 50, if the image contains objects other than the work object and background, the information processing device 10 processes the image to make the objects other than the work object and background invisible. The information processing device 10 also acquires information related to work support provided by a supporter (hereinafter referred to as "support information") from the terminal 50 and presents it to the worker.

[0033] The terminal 50 acquires the image from the information processing device 10 and presents it to the supporter. The terminal 50 also transmits support information input by the supporter to the information processing device 10.

[0034] In the information processing system 1, the information processing device 10 transmits an image taken by the worker and presents it to the terminal 50, and the terminal 50 transmits support information input by the supporter and presents it to the information processing device 10. The information processing system 1 enables the worker to receive support information from a remote supporter via the information processing device 10 and perform work on the work target.

[0035] In this embodiment, a description will be given of a mode in which an image is captured, but the present invention is not limited to this mode and a video may also be captured.

[0036] Next, the hardware configuration of the information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the information processing device 10 according to this embodiment.

[0037] 2, the information processing device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a monitor 16, a communication interface (communication I / F) 17, and a camera 18. The CPU 11, the ROM 12, the RAM 13, the storage 14, the input unit 15, the monitor 16, the communication I / F 17, and the camera 18 are connected to each other via a bus 19. Here, the CPU 11 is an example of a processor.

[0038] The CPU 11 supervises and controls the entire information processing device 10. The ROM 12 stores various programs, including the information processing program used in this embodiment, and data. The RAM 13 is a memory used as a work area when various programs are executed. The CPU 11 loads the program stored in the ROM 12 into the RAM 13 and executes it, thereby performing processing on an image to generate a processed image. The storage 14 is, for example, a hard disk drive (HDD), a solid state drive (SSD), or flash memory. The storage 14 may also store information processing programs. The input unit 15 is, for example, a touch panel and a keyboard that accepts character input. The monitor 16 displays characters and images. The communication I / F 17 transmits and receives data. The camera 18 is an imaging device that captures an image of a work target. Here, the camera 18 is an example of an "imaging unit."

[0039] Next, the functional configuration of the information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the information processing device 10 according to this embodiment.

[0040] 3 , the information processing device 10 includes an acquisition unit 21, a detection unit 22, an estimation unit 23, a setting unit 24, a specification unit 25, a generation unit 26, a storage unit 27, a reception unit 28, and a notification unit 29. The CPU 11 executes an information processing program, thereby functioning as the acquisition unit 21, the detection unit 22, the estimation unit 23, the setting unit 24, the specification unit 25, the generation unit 26, the storage unit 27, the reception unit 28, and the notification unit 29.

[0041] The acquisition unit 21 acquires an image including an object to be worked on, captured by the camera 18. The acquisition unit 21 also acquires a designation of a work object, which is a target for work, from among the objects included in the image. Note that in this embodiment, the work object is described as being designated by reading a QR (Quick Response) code attached to the surface of the object. However, this is not limiting. The identified object may be displayed on the monitor 16, and the user may select a work object from the objects.

[0042] The detection unit 22 detects feature points that indicate an object from the acquired image. Here, feature points are points that indicate the edges, corners, etc. of the object included in the image.

[0043] The estimation unit 23 estimates the position and direction of the worker (camera 18) using the detected feature points 31, and estimates the target space 33 at the worker's position and direction. Specifically, the estimation unit 23 uses SLAM (Simultaneous Localization And Mapping) technology to estimate position information indicating the worker's position, direction information indicating the direction, and the target space 33.

[0044] For example, the worker starts capturing an image by reading a QR code (registered trademark) attached to the work object 32, and the detection unit 22 detects feature points 31 included in the captured image. The estimation unit 23 compares the feature points 31 included in multiple images captured over time, and estimates the position and direction in which the worker (camera 18) is capturing images from the amount of change in the feature points 31.

[0045] As an example, as shown in FIG. 4 , by tracking feature points 31 included in a captured image, it is possible to estimate the worker's position information, direction information, and target space 33. In the present embodiment, the position of the worker at the start of image capture is estimated by reading a QR code (registered trademark) attached to the work object 32. However, this is not limited to this. For example, image capture of the work object 32 may be started at a predetermined position. In the present embodiment, the position information and direction information of the worker are estimated from the amount of change in the feature points 31. However, this is not limited to this. For example, a feature point map in which feature points 31 of the space where the worker is located is created in advance, and the position information and direction information of the worker may be estimated by comparing the feature points 31 included in the captured image with the feature point map.

[0046] The setting unit 24 sets three-dimensional space information using the acquired image and the detected feature points. Specifically, the setting unit 24 sets the three-dimensional space information shown in FIG. 5 as an example. As shown in FIG. 5, the setting unit 24 identifies the space and each object included in the image using feature points 31. The setting unit 24 sets, as the three-dimensional space information, a space (hereinafter referred to as "target space") 33 occupied by a work target 32 designated by the user among the objects, and a space (hereinafter referred to as "non-target space") 34 occupied by objects other than the work target (other objects, background, etc.). The target space 33 is an example of a "first space," and the non-target space 34 is an example of a "second space."

[0047] Furthermore, the setting unit 24 sets information (hereinafter referred to as "invisibility information") to make the non-target space 34 invisible as a background that is not desired to be seen. That is, the three-dimensional space information is a three-dimensional coordinate space in which feature points 31 indicating objects are set, and is composed of a target space 33 occupied by a specified work target 32, and a non-target space 34 which is a space excluding the target space 33.

[0048] In addition, when the setting unit 24 holds three-dimensional spatial information, it uses the acquired image and the detected feature points to set the three-dimensional spatial information, and aligns the target space 33 and the non-target space 34 in the worker's position information and direction information.

[0049] As shown in FIG. 6 as an example, the identification unit 25 identifies, in an acquired image 35, an area (hereinafter referred to as a "target area") 36 occupied by a work target 32 among objects included in the image 35. Specifically, the identification unit 25 compares the acquired image 35 with the three-dimensional space information, and identifies, in the image 35, the target area 36 corresponding to a target space 33 in the three-dimensional space information. The identification unit 25 also identifies, as a background to be processed, an area (hereinafter referred to as a "non-target area") 37 other than the target area 36 corresponding to a non-target space 34 in the three-dimensional space information. Here, the target area 36 is an example of a "first area," and the non-target area 37 is an example of a "second area."

[0050] 7, the generation unit 26 generates an image 38 in which a non-target area 37 corresponding to the non-target space 34 for which invisibility information is set is processed to be invisible in the acquired image 35 (hereinafter referred to as a "processed image") 38. Here, the processing to make the non-target area 37 invisible includes, for example, blurring the outlines of objects and characters, and blacking out the non-target area 37.

[0051] The storage unit 27 stores the shape of the work object after transformation.

[0052] When the target area 36 changes due to deformation of the work object 32, the receiving unit 28 receives from the user an instruction to switch to an area (hereinafter referred to as the "deformation area") defined by the shape of the work object 32 after deformation, which has been registered in advance, instead of the target area 36, and the shape of the work object after deformation.

[0053] When the notification unit 29 detects a deformation of the work object 32 from the amount of change in the feature point 31, it notifies the user that the area should be switched to a deformation area.

[0054] Next, before describing the operation of the information processing device 10, a method for switching the target area 36 to a transformation area will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a target space 33 used to explain switching accompanying the transformation of the work object 32.

[0055] For example, if the work object 32 is an image forming device, the door of the image forming device may be opened or closed, causing the shape of the image forming device to change. When the work object 32 is changed, in the target space 33 set by the setting unit 24 before the change, a part of the work may be made invisible, or an area that should be made invisible may not be made invisible.

[0056] Therefore, the information processing device 10 according to this embodiment stores the shape of the work object 32 after deformation in the memory unit 27 in advance for each work object 32, and receives from the user a designation of a shape from the pre-stored shapes that corresponds to the shape of the work object 32 after deformation.

[0057] The receiving unit 28 receives from the user a designation of the shape of the work object 32 after transformation, and the setting unit 24 sets a space (hereinafter referred to as "transformed space") corresponding to the received transformed shape of the work object 32. As shown in Fig. 8, the setting unit 24 switches the pre-transformed object space 33 to the transformed space in response to the user's designation, and sets it in the three-dimensional space information.

[0058] Here, the deformation space is set to three-dimensional space information taking into consideration the estimated position information and direction information. For example, if the shape of the work object 32 after deformation stored in the storage unit 27 is the shape of the work object 32 when viewed from the front, a deformation space corresponding to the estimated position information and direction information (e.g., the shape of the work object 32 after deformation when viewed from the side) is generated. This associates the shape of the work object 32 after deformation stored in the storage unit 27 with the shape of the work object 32 when viewed from the worker's current location.

[0059] The identification unit 25 compares the acquired image 35 with the three-dimensional space information, and identifies a deformation area corresponding to the deformation space of the three-dimensional space information in the image 35. The identification unit 25 also identifies an area other than the deformation area as a non-target area 37 to be processed as a background.

[0060] The generating unit 26 generates a processed image 38 in which the non-target area 37 in the image 35 is made invisible.

[0061] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of processing for generating a processed image 38 according to this embodiment. The image generation processing shown in Fig. 9 is executed by the CPU 11 reading and executing an information processing program from the ROM 12 or the storage 14. The image generation processing shown in Fig. 9 is executed, for example, when an instruction to generate a processed image 38 is input from the user.

[0062] In step S101, the CPU 11 acquires an image 35 of an object including the work target 32.

[0063] In step S102, the CPU 11 acquires the work object 32 designated by the user.

[0064] In step S103, the CPU 11 detects feature points 31 of the object from the acquired image .

[0065] In step S104, the CPU 11 estimates the position information and direction information of the worker from the detected feature points 31 using the SLAM technique.

[0066] In step S105, the CPU 11 sets the target space 33 and the non-target space 34 in the three-dimensional space information. Here, in the three-dimensional space information, invisibility information is set for the non-target space 34. Furthermore, as time passes, when a new image 35 is acquired and a feature point 31 is detected, the target space 33 and the non-target space 34 are set in the three-dimensional space information using the feature point 31.

[0067] In step S106, the CPU 11 determines whether or not it has received from the user a designation of the shape of the work object 32 after deformation. If a designation of the shape of the work object 32 after deformation has been received (step S106: YES), the CPU 11 proceeds to step S107. On the other hand, if a designation of the shape of the work object 32 after deformation has not been received (step S106: NO), the CPU 11 proceeds to step S109.

[0068] In step S107, the CPU 11 receives an instruction to switch to the deformation area and the shape of the work object 32 after deformation.

[0069] In step S108, the CPU 11 sets a modification space using the accepted shape of the work object 32 after modification, identifies a modification area corresponding to the modification space, and applies the modification area in place of the target area .

[0070] In step S109, the CPU 11 compares the three-dimensional space information with the acquired image 35 to identify the target region 36 in the image 35.

[0071] In step S110, the CPU 11 generates a processed image 38 by processing the non-target region 37.

[0072] In step S111, the CPU 11 transmits the generated processed image 38 to the terminal 50.

[0073] In step S112, the CPU 11 determines whether or not to end the process. If the process is to be ended (step S112: YES), the CPU 11 ends the process of generating the processed image 38. On the other hand, if the process is not to be ended (step S112: NO), the CPU 11 proceeds to step S113.

[0074] In step S113, the CPU 11 acquires a new image 35 in which an object including the work target 32 is photographed.

[0075] In step S114, the CPU 11 detects feature points 31 of the object from the acquired image 35, proceeds to step S104, and uses the detected feature points 31 to align the target space 33 and the non-target space 34, and sets three-dimensional space information.

[0076] As described above, according to this embodiment, even if the area occupied by the work object 32 changes, it is possible to prevent over- or under-processing of the captured image.

[0077] [Second embodiment] In the first embodiment, a configuration was described in which a user's designation of the shape of the work object 32 after deformation was accepted. In the present embodiment, a configuration in which the shape of the work object 32 after deformation is detected will be described.

[0078] The configuration of the information processing system according to this embodiment (see FIG. 1), the hardware configuration of the information processing device 10 (see FIG. 2), and the functional configuration of the information processing device 10 (see FIG. 3) are the same as those in the first embodiment, and therefore descriptions thereof will be omitted. Also, examples of three-dimensional space information according to this embodiment (see FIGS. 4 and 5), examples of a captured image 35 and a processed image 38 (see FIGS. 6 and 7) are the same as those in the first embodiment, and therefore descriptions thereof will be omitted.

[0079] As an example, as shown in FIG. 10 , the detection unit 22 of the information processing device 10 compares the detected feature points 31 and detects spaces where the amount of change in the feature points 31 is large and spaces where the amount of change in the feature points 31 is small. For example, as shown in FIG. 10 , the detection unit 22 detects spaces where the amount of change in the feature points 31 is large and spaces where the amount of change in the feature points 31 is small. When the notification unit 29 detects a change that narrows the target space 33, it specifies the shape of the work object 32 after the change and notifies the user that the target space 33 will be switched. Here, upon receiving the notification, the user specifies the shape of the work object 32 after the change and then performs work on the work object (work to narrow the target space 33). This prevents the area that was the work object 32 before the deformation from appearing in the image due to the deformation of the work object 32.

[0080] 11, when the detection unit 22 detects a change that expands the target space 33, it detects the shape of the work object 32 after the deformation using the detected feature points 31, and the setting unit 24 sets the target space 33 corresponding to the shape of the work object 32 after the deformation. This prevents the work object 32 from becoming invisible even if the user does not specify the shape of the work object 32 after the deformation. Furthermore, when the change that expands the target space 33 stops, the setting unit 24 sets the target space 33 corresponding to the shape of the work object 32 after the deformation, thereby preventing background areas from appearing in the image while the user is working.

[0081] The operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of processing for generating a processed image 38 according to this embodiment. The image generation processing shown in Fig. 12 is executed by the CPU 11 reading and executing an information processing program from the ROM 12 or the storage 14. The image generation processing shown in Fig. 12 is executed, for example, when an instruction to generate a processed image 38 is input from the user. Note that in Fig. 12, the same steps as those in the image generation processing shown in Fig. 9 are assigned the same reference numerals as those in Fig. 9, and their description will be omitted.

[0082] In step S115, CPU 11 determines whether or not deformation of work object 32 has been detected from the amount of change in the feature points. If deformation of work object 32 has been detected (step S115: YES), CPU 11 proceeds to step S116. On the other hand, if deformation of work object 32 has not been detected (step S115: NO), CPU 11 proceeds to step S109.

[0083] In step S116, the CPU 11 determines whether the detected deformation is a deformation that reduces the target space 33. If it is a deformation that reduces the target space 33 (step S116: YES), the CPU 11 proceeds to step S117. On the other hand, if it is not a deformation that reduces the target space 33 (a deformation that expands the target space 33) (step S116: NO), the CPU 11 proceeds to step S119.

[0084] In step S117, the CPU 11 notifies the user that the area will be switched to the modification area.

[0085] In step S118, the CPU 11 receives an instruction from the user to switch to the deformation area and the shape of the work object 32 after deformation.

[0086] In step S119, the CPU 11 detects the shape of the work object 32 after deformation from the amount of change in the detected feature points 31.

[0087] In step S120, the CPU 11 sets a modification space using the shape of the work object 32 after modification, identifies a modification area corresponding to the modification space, and applies the modification area in place of the target area .

[0088] As described above, according to this embodiment, it is possible to prevent unintended insufficient machining from occurring.

[0089] In the above embodiment, the information processing device 10 is described as a terminal carried by the worker. However, the present invention is not limited to this. The information processing device 10 may be a server. For example, a server equipped with the information processing device 10 may acquire an image 35 and a task target 32 from a terminal carried by the worker, process a non-target area 37 in the image 35 so that it is invisible, and transmit a processed image 38 to a terminal carried by a supporter.

[0090] Although the present invention has been described above using the embodiments, the present invention is not limited to the scope of the embodiments. Various modifications and improvements can be made to the embodiments without departing from the spirit of the present invention, and such modifications and improvements are also included in the technical scope of the present invention.

[0091] In the above embodiment, the term "processor" refers to a processor in a broad sense, and includes, for example, a general-purpose processor (e.g., a CPU: Central Processing Unit) and a dedicated processor (e.g., a GPU: Graphics Processing Unit, an ASIC: Application Specific Integrated Circuit, an FPGA: Field Programmable Gate Array, a programmable logic device, etc.).

[0092] Furthermore, the operations of the processor in each of the above embodiments may not only be performed by a single processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

[0093] Although the present embodiment has been described with reference to a configuration in which the information processing program is installed in storage, the present invention is not limited to this. The information processing program according to the present embodiment may be provided in a format recorded on a computer-readable storage medium. For example, the information processing program according to the present invention may be provided in a format recorded on an optical disc such as a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The information processing program according to the present embodiment may be provided in a format recorded on a semiconductor memory such as a USB (Universal Serial Bus) memory or a memory card. The information processing program according to the present embodiment may also be acquired from an external device via a communication line connected to the communication I / F. [Explanation of symbols]

[0094] 1. Information Processing Systems 10. Information processing equipment 11 CPU 12 ROM 13 RAM 14. Storage 15 Input section 16 monitors 17 Communication I / F 18 Camera 19 Bus 21 Acquisition Department 22 Detection unit 23 Estimation part 24 Setting section 25 Specific section 26 Generation part 27 Memory section 28 Reception Department 29 Notification Department 31 feature points 32 Work Object 33 Target Space 34 Non-target space 35 images 36 Target Areas 37 Excluded Areas 38 Edited Images 50 devices

Claims

1. a processor, the processor comprising: Acquire an image of the object, Identifying a first area of the object in the image, the first area being occupied by a work object on which work is to be performed; generating a processed image in which a second region indicating a region other than the first region in the image is made invisible; Accepting designation of a shape corresponding to the post-deformation shape of the work object from among pre-stored post-deformation shapes of the work object; When the first area changes due to the deformation of the work object, a deformation area that is an area defined by the shape of the work object after the received deformation is applied in place of the first area, and a processed image is further generated in which the second area associated with the application is made invisible; Send the processed image Information processing device.

2. The processor: Further acquiring spatial information corresponding to the image and being information on a three-dimensional space including the object; Detecting feature points indicative of the work object from the image; Using the feature points, a first space corresponding to the first region in the spatial information and a second space corresponding to the second region are identified. The information processing device according to claim 1 .

3. The processor: setting invisibility information indicating information to be made invisible for the second space; Using the invisibility information, a processed image is generated in which the second region corresponding to the second space in the image is made invisible. The information processing device according to claim 2 .

4. The processor: Further acquiring position information indicating the position of an image capturing unit that captures the image in the spatial information and the position of the work object, and direction information indicating the direction in which the image capturing unit captures the image; generating the processed image in which the second region is made invisible according to the position information and the direction information; The information processing device according to claim 3 .

5. The position information and the direction information are estimated from the amount of change in the feature points. The information processing device according to claim 4 .

6. The processor: storing a plurality of said deformation regions; Accepting the designation of one of the plurality of transformation areas The information processing device according to any one of claims 1 to 5.

7. The processor: Detecting feature points indicative of the work object from the image; When a deformation of the work object is detected from the amount of change in the feature points, the first region is Notify to switch to shape area The information processing device according to any one of claims 1 to 6.

8. The processor: If the detected deformation of the work object is a deformation that expands the first area, Apply the corresponding deformation area of the object to generate a processed image. The information processing device according to claim 7 .

9. The processor: When an instruction to switch to a deformation area that narrows the first area is acquired, The modified image is generated by applying the modified area. The information processing device according to any one of claims 1 to 8.

10. On the computer, Acquire an image of the object, Identifying a first area of the object in the image, the first area being occupied by a work object on which work is to be performed; generating a processed image in which a second region indicating a region other than the first region in the image is made invisible; Accepting designation of a shape corresponding to the post-deformation shape of the work object from among pre-stored post-deformation shapes of the work object; When the first area changes due to the deformation of the work object, a deformation area that is an area defined by the shape of the work object after the received deformation is applied in place of the first area, and a processed image is further generated in which the second area associated with the application is made invisible; Send the processed image An information processing program for executing tasks.

Images