IMAGE PROCESSING SYSTEM, IMAGE PROCESSING PROGRAM, AND IMAGE PROCESSING METHOD

The image processing system uses visible and distance images to generate a masking image that accurately separates objects, addressing the challenge of overlapping or similarly colored objects, thereby maintaining image quality for work reports.

JP7594723B2Active Publication Date: 2024-12-05TOKYO ELECTRIC POWER CO HOLDINGS INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2020147567
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-09-02
Publication Date
2024-12-05
Estimated Expiration
2040-09-02

AI Technical Summary

Technical Problem

Existing image processing techniques struggle to accurately separate objects from images when they overlap or have similar colors, leading to difficulties in maintaining image quality suitable for work reports by masking unnecessary parts.

Method used

An image processing system that utilizes a visible image and a distance image to identify the contours of objects, generating a masking image that masks regions outside the object's outline, using a combination of edge detection, distance information, and correction modes to enhance accuracy.

Benefits of technology

The system effectively masks unnecessary portions while preserving image quality, ensuring clarity and accuracy in work reports by reliably identifying and masking non-target equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594723000001
    Figure 0007594723000001
  • Figure 0007594723000002
    Figure 0007594723000002
  • Figure 0007594723000003
    Figure 0007594723000003
Patent Text Reader

Abstract

To provide an image processing system which generates a masking image having such an image quality that an unnecessary part is reliably masked and a target object is worth a work report.SOLUTION: An image processing system has: an image input unit for receiving an input of a visible image obtained by capturing an object and a distance image having distance information of a subject in the visible image; an effective distance range designating unit (an effective distance range setting bar 21, and a distance range limiting unit 32) for receiving an input for designating a range of an effective region of an image to determine an image region with the designated range based on the distance information; an object selecting unit (an object selecting tool 22, and an object detecting unit 33) for receiving an input for selecting an object in the image region with the designated range to specify a contour of the object in the selected image region based on the visible image and the distance image; and a masking processing unit 34 for generating a masking image obtained by masking a region of the visible image corresponding to an area outside of the contour of the object in the specified image region to display it on a display.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an image processing system, an image processing program, and an image processing method, and more particularly to an image processing system, an image processing program, and an image processing method for generating a masked image by masking a part of a visible image. [Background technology]

[0002] At various facilities, on-site photographs are taken at each stage of maintenance and inspection work and when the work is completed, and are attached to work reports. For example, in facilities where confidential information is managed, such as power plants, thorough information management of important equipment is required, and photographs must be taken so that important equipment that is not the target subject is not photographed at the same time as the target subject, or photographs that include important equipment must be excluded from the report. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5819856 [Patent Document 2] Patent No. 5871862 Summary of the Invention [Problem to be solved by the invention]

[0004] In this case, if important equipment that is not the target subject can be easily identified and masked as unnecessary parts from the captured site photograph, it is possible to reduce failures in capturing site photographs and unnecessary capturing. Methods such as edge detection and similar color area selection are known as methods for extracting an arbitrary subject from an image captured on the site. Edge detection is a technique for detecting as an edge a part where the change in the luminance value of pixels constituting an image is greater than a threshold, and similar color area selection is a technique for identifying pixels whose color has a similarity to the color of a specified pixel that is greater than a threshold as the same area.

[0005] However, techniques such as edge detection and similar color area selection have the problem that it is difficult to separate objects when objects overlap in the image or when the object and the background have similar colors. In addition, it is difficult to determine the threshold value, because a small threshold value makes the edges more visible, but if the threshold value is too small, the object becomes difficult to see.

[0006] As a method for extracting an arbitrary subject in an image, in addition to the above-mentioned method using a visible image, a method using a depth image having the distance from the camera to the subject as depth information is also known.

[0007] In the technique of Patent Document 1, for example, different depth values ​​of pixels in selected areas or points of the depth image are compared to determine edges, each target or object in the depth image is filled, and each target or object is compared to a known pattern to determine whether the depth image contains a human target, thereby enabling a motion capture system to distinguish between multiple people.

[0008] In the method described in Patent Document 1, edges are determined only from the depth image, so accurate edge identification is not possible. Since the purpose is to capture the movements of humans as targets in advance, there is no problem as long as the part to be compared with a known pattern can be extracted, and there is no need to accurately capture the edges of the object. Furthermore, Patent Document 1 does not mention image quality, etc.

[0009] Patent Document 2 describes a method of determining a distance based on depth information, determining a blur factor, and adding image blur to a subject based on the determined blur factor. The method described in Patent Document 2 does not intend to accurately identify an area having a boundary based on depth information.

[0010] As described above, conventional techniques have not yet achieved image quality that is sufficient for operational reporting and the ability to mask unnecessary portions at the same time.

[0011] The present invention has been made in consideration of the above-mentioned conventional problems, and aims to provide an image processing system capable of generating a masking image in which unnecessary parts are reliably masked while maintaining image quality for the target subject that is sufficient for work reporting. [Means for solving the problem]

[0012] An image processing system according to a representative embodiment of the present invention is characterized in that it comprises an image input unit that accepts input of a visible image of an object and a distance image having distance information of the subject in the visible image, an effective distance range designation unit that accepts input specifying an effective area of ​​the image and determines the specified image area based on the distance information, an object selection unit that accepts input to select an object within the specified image area and identifies the contour of the object in the selected image area based on the visible image and the distance image, and a masking image display unit that generates a masking image that masks an area of ​​the visible image corresponding to the outside of the contour of the object in the specified image area, and displays the masked image on a display. Effect of the Invention

[0013] According to the image processing system of the present invention, it is possible to generate a masking image in which unnecessary parts are reliably masked while maintaining image quality of the target subject sufficient for work reporting. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram showing the external configuration of an image processing system according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of an image processing unit; [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of functional blocks of the image processing system according to the present embodiment. [Figure 4] 10A to 10C are diagrams illustrating the processing of the object detection unit in a correction mode according to the embodiment. [Diagram 5] FIG. 4 is a flowchart showing a processing example in the image processing system according to the embodiment. [Figure 6] 3A and 3B are diagrams illustrating examples of input and output in the image processing system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] 1. Overview of the embodiment First, a typical embodiment of the invention disclosed in this application will be outlined. In the following description, as an example, reference numerals in the drawings corresponding to components of the invention are given in parentheses.

[0016] [1] An image processing system (1) according to a representative embodiment includes an image input unit that accepts input of a visible image of an object and a distance image having distance information of the subject in the visible image, an effective distance range designation unit (21, 32) that accepts input designating an effective area of ​​the image and determines the designated image area based on the distance information, an object selection unit (22, 33) that accepts input selecting an object within the designated image area and identifies the contour of the object in the selected image area based on the visible image and the distance image, and a masking image display unit (34) that generates a masking image that masks the area of ​​the visible image that corresponds to the outside of the contour of the object in the identified image area, and displays the masked image on a display.

[0017] [2] In the above [1], the device may further include a masking image storage unit that stores the masking image displayed on the masking image display unit.

[0018] [3] In the above [1] or [2], the object selection unit may extract the contour of the object in the selected image area by edge processing in the visible image, and identify the contour of the object in the selected image area by correcting the extracted contour by distance information in the distance image.

[0019] [4] In any one of [1] to [3] above, the device may further include a visible image camera that captures the visible image and an infrared camera that captures the distance image, and the captured visible image and distance image may be input to the image input unit.

[0020] [5] In any one of [1] to [4] above, a pre-processing unit may be provided that corrects the distance information of the distance image received by the image input unit by performing an expansion process using a predetermined size as a threshold.

[0021] [6] An image processing program according to a representative embodiment causes a computer to function as an image input unit that accepts input of a visible image of an object and a distance image having distance information of the subject in the visible image, an effective distance range designation unit that accepts input designating an effective area of ​​the image and determines the designated image area based on the distance information, an object selection unit that accepts input selecting an object within the designated image area and identifies the contour of the object in the selected image area based on the visible image and the distance image, and a masking image display unit that generates a masking image that masks an area of ​​the visible image that corresponds to the outside of the contour of the object in the identified image area, and displays the masked image on a display.

[0022] [7] An image processing method according to a representative embodiment is an image processing method executed on a computer, and includes an image input step of accepting input of a visible image of an object and a distance image having distance information of the subject in the visible image, an effective distance range designation step of accepting input designating an effective area of ​​the image and determining the designated image area based on the distance information, an object selection step of accepting input to select an object within the designated image area and identifying the contour of the object in the selected image area based on the visible image and the distance image, and a masking image display step of generating a masking image in which an area of ​​the visible image corresponding to the outside of the contour of the object in the identified image area is masked, and displaying the masking image on a display.

[0023] 2. Specific examples of embodiments Hereinafter, specific examples of the embodiments of the present invention will be described with reference to the drawings. In the following description, components common to each embodiment will be given the same reference numerals, and repeated description will be omitted. It should be noted that the drawings are schematic, and the dimensional relationships and ratios of each element may differ from the actual situation. The drawings may also include parts with different dimensional relationships and ratios.

[0024] FIG. 1 is a diagram showing the external configuration of an image processing system according to this embodiment.

[0025] The image processing system 1 includes a camera 10 and a display 20, and further includes an image processing unit connected to the camera 10 and the display 20, although this is not shown in FIG.

[0026] The camera 10 has a visible image camera 11 that captures visible images, and a distance image camera 12 that captures distance images. A distance image is an image that has distance information of a subject that indicates the distance from the camera to the subject (hereinafter also referred to as "distance"). A distance image can have a distance information value for each pixel that constitutes the image, for example. A visible image is an image that can be captured by a general visible light camera. Unlike a distance image, a visible image has a value that indicates the brightness, color, etc. of the subject for each pixel.

[0027] The visible image camera 11 is a camera having an image sensor such as a CCD or CMOS. The distance image camera 12 is a camera having a means for measuring distance, such as an infrared camera.

[0028] The camera 10 may be configured as one unit with the image processing system 1 as shown in Fig. 1, or may be configured separately from the image processing system 1. For example, a visible image and a distance image taken at a location other than the image processing system 1 can be input to an image processing unit of the image processing system 1 via a communication means. The communication means may be either wired or wireless. However, the visible image camera 11 and the distance image camera 12 are configured so as to be able to take images of the same subject under substantially the same conditions such as the angle of view.

[0029] The output contents controlled by the image processing unit can be displayed on the display 20. As shown in Fig. 1, the display 20 can display an effective distance range setting bar 21, an object selection tool 22, a setting menu 23, and a shooting button 24. Note that, in addition to being displayed on the display 20, the effective distance range setting bar 21, the setting menu 23, and the shooting button 24 may also be provided alongside the display 20 as a hardware interface.

[0030] The effective distance range setting bar 21 is an interface for receiving an operational input for designating the effective area in the visible image. For example, a bar having far and near at both ends can be displayed to allow input of the effective range. Also, for example, the magnitude of the numerical value indicating the distance can be adjusted with a slider.

[0031] The object selection tool 22 is an interface for receiving an operation input for selecting an object from among the subjects in the range-specified visible image. For example, it can be in the form of an arrow pointing to the object or a range-specifying frame surrounding the object. Here, the object refers to an object that is not to be masked.

[0032] The setting menu 23 is an interface for receiving an operational input for selecting whether to enable or disable a correction mode, which is a mode for executing an object detection process.

[0033] The shooting button 24 is an interface for receiving an operational input regarding the timing of image capture.

[0034] Fig. 2 is a block diagram showing an example of the hardware configuration of the image processing unit. As shown in Fig. 2, the image processing unit 3 has hardware elements such as a CPU 301, a RAM 302, a ROM 303, a communication circuit 304, and a peripheral circuit 305, and is configured by a program processing device (for example, a microcontroller: MCU) in which the components are connected to each other via a bus or a dedicated line. The peripheral circuit 305 includes an A / D conversion circuit, an input / output I / F circuit, a clock circuit, etc.

[0035] In the image processing unit 3, a CPU 301 functioning as a processor is stored in a ROM 303, performs various calculations according to programs called up by a RAM 302, and controls an A / D conversion circuit and an input / output I / F circuit, thereby realizing the configuration of each functional unit of the image processing unit described later.

[0036] 3 is a diagram showing an example of the configuration of functional blocks of the image processing system according to this embodiment. In the image processing system, a visible image captured by a visible image camera 11 and a distance image captured by a distance image camera 12 are input to an image processing unit 3, and the image processing unit 3 executes image processing while controlling the display of a display 20 to display a desired image on the display 20, and the image data is output to a storage medium 4 (an example of a masking image storage unit). The storage medium 4 may be a part of the configuration of the image processing system 1, or may be a memory detachable from the image processing system 1, or an external memory or server connected to the image processing system 1 via a wired or wireless communication line.

[0037] The image processing unit 3 includes an image input unit (not shown) that accepts input of a visible image of an object and a distance image having distance information of the subject in the visible image, a distance image pre-processing unit 31, a distance range limiting unit 32 (an example of an effective distance range designation unit), an object detection unit 33 (an example of an object selection unit), and a masking processing unit 34 (an example of a masking image display unit).

[0038] The image input unit can receive input of a visible image and a distance image by acquiring a frame image (hereinafter also referred to as an "image") from the video data input from the visible image camera 11 and the distance image camera 12. The image input unit may acquire a frame image from the video data using, for example, an operation input to a shooting button 24 displayed on the display 20 as a trigger.

[0039] The distance image preprocessing unit 31 performs preprocessing of the input distance image. For example, it can perform preprocessing to smooth the distance information of the distance image. In addition, in an infrared distance image, there are pixels for which distance information cannot be obtained due to reflection or blocking of infrared rays. For this reason, for an infrared distance image, a filling process using nearby distance pixels can be used as a preprocessing. The distance image preprocessing unit 31 can also perform preprocessing based on multiple distance images (moving images) that are consecutive in time to the distance image to be processed. In this case, the moving images are input from the image input unit to the distance image preprocessing unit 31 for preprocessing. If it is determined that a sufficient frame rate is ensured in the moving images and that there is little blurring per frame, the distance image preprocessing unit 31 improves the accuracy of the value of the distance information for each pixel by taking a scan average across multiple images for each pixel.

[0040] The distance range limiting unit 32 executes distance range limiting processing for limiting the effective range on the preprocessed distance image, and can generate a masking area image showing an image area that has been determined as an image area designated as an effective range. Furthermore, the generated masking area image is passed to the object detection unit 33. The masking area image is an image showing an image area that has been designated as an effective range or an image area that has not been designated as an effective range. The effective range refers to a distance range that is not subject to masking.

[0041] When an effective distance range is input by an operation input to the effective distance range setting bar 21 displayed on the display 20, the distance range limiting unit 32 performs a process of limiting the effective range of the image to be processed according to the input effective distance range. Specifically, for example, when an operation input is made to set the effective distance to less than 7 on the effective distance range setting bar 21, the distance range limiting unit 32 can accept an input of the effective distance range "effective distance range is less than 7". The distance range limiting unit 32 can perform a raster scan of the distance image based on the input "effective distance range is less than 7", determine pixel parts with distance information less than 7, and generate a masking region image having information of the determined pixel parts with distance information less than 7 as the image region with the range specified. Note that the masking region image may have information other than the pixel parts with the determined distance information less than 7 as the image region with the range specified, instead of having information of the pixel parts with the determined distance information less than 7 as the image region with the range specified. In addition, the distance range limiting unit 32 may not be limited to operational input to the effective distance range setting bar 21 displayed on the display 20, but may also set as the effective range the vicinity of the values ​​of the distance information of the pixels in the area of ​​the distance image corresponding to the area of ​​the object in the visible image specified by the object selection tool 22.

[0042] The object detection unit 33 can execute an object detection process for identifying the contour of an object on the visible image and the distance image, and generate a masking area image showing the image area where the contour of the object is identified. Furthermore, the generated masking area image is passed to the masking processing unit 34. The object detection unit 33 can limit the processing range by using the masking area image from the distance range limiting unit 32 during the object detection process, thereby reducing the processing load. For example, due to discoloration of the contour of the object or the way the light hits it, it may be difficult to distinguish the background from the object using only the visible image, and accurate detection of the contour may not be possible. In this case, by using the masking area image from the distance range limiting unit 32, the background and the object in front of it are separated in advance, so accurate detection of the contour can be easily performed.

[0043] The object detection unit 33 performs object detection processing by accepting an input for selecting an object to be detected by an operation input to the object selection tool 22 displayed on the display 20. For example, it is possible to display a visible image and the object selection tool 22 on the display 20, identify an area of ​​the visible image surrounded by the object selection tool 22 as an area including a "pixel position within the object" and a "pixel position near the object", detect an edge process for the visible image and an outline of an object in the distance image within the identified area, and specify the pixel position of the object to be detected.

[0044] In the image processing system of this embodiment, the object detection unit 33 detects the contour of an object based on a visible image and a distance image. In conventional contour detection (edge ​​detection) based only on a visible image, the contour of the image becomes unclear when the light reflection is not uniform, and the contour of the object cannot be captured well by edge detection. In addition, capturing the contour of an object based only on a distance image depends largely on the accuracy of the camera that captures the distance image, and a general-purpose camera for capturing distance images is not suitable for contour detection due to its poor resolution. However, it has become clear that accurate contour detection is possible by combining the detection results based on a visible image and a distance image.

[0045] For example, the GrabCut algorithm is known as a method for separating background and objects from a visible image. In this algorithm, the user usually roughly sets the background area (pixel set) and the object area (pixel set) in advance by selecting a rectangle or the like. The GrabCut algorithm calculates color statistics from the given background and object pixel sets, and determines whether each pixel is background or object along with its edge-likeness. When detecting the edges of an object using this GrabCut algorithm, by using a distance image, it is possible to easily set more accurate initial values ​​for the background and object areas, enabling accurate contour detection.

[0046] It is also common to use a fill-in algorithm with similar colors to detect the edges of objects. When detecting the edges of objects using a fill-in algorithm with similar colors, the user does not have to select the similar color directly from the image, but can specify the similar color from the pixel position of the object selected by specifying the distance, which enables accurate detection of the contour.

[0047] Furthermore, in edge extraction processing using the difference between adjacent pixel values, patterns on the surface of an object are also captured as edges. When detecting the edges of an object using the difference between adjacent pixel values, accurate contour detection is possible by removing from the object boundary, among the edges detected by image processing, edges that divide areas that are approximately equidistant on the range image.

[0048] The object detection unit 33 can also control whether or not to execute the object detection process by the object detection unit 33 based on an operation input to the setting menu 23 (see FIG. 1) displayed on the display 20. For example, the object detection unit 33 can control the correction mode to be set in the setting menu 23, so that the object detection process is executed when the correction mode is selected, and the object detection process is not executed when the correction mode is not selected. For example, when there is no non-target object within a distance range close to the distance of the target object, the object detection unit 33 can be set not to execute the object detection process ("no correction mode"). When "no correction mode" is selected, the object detection unit 33 passes the masking area image input from the distance range limiting unit to the masking processing unit 34. When the correction mode is selected, the object detection unit 33 passes the masking area image after the object detection process to the masking processing unit 34.

[0049] By being set to the correction mode, if there is an object that you do not want to mask in an image area that is not specified as a valid range in the masking area image from the distance range limiting unit 32, the object detection unit 33 can select the object and exclude it from being masked.

[0050] FIG. 4 is a diagram for explaining the processing of the object detection unit 33 in the correction mode. For example, as shown in FIG. 4(a), consider a case where a desk and a clock are captured in an image as objects that do not need to be masked, and a caution sign and a cylindrical object are captured as objects that need to be masked. In this image, if the distance range limiting unit 32 specifies a distance in front of the object to be masked as an effective range, and the masking process is performed without processing in the object detection unit 33, a masked image in which the desk is cut in half is generated as shown in FIG. 4(b). On the other hand, if the object detection unit 33 can select the desk and the clock as objects to be detected in the correction mode, a masked image can be generated in which the desk and the clock can be displayed without being masked, while the sign and the cylindrical object are masked, as shown in FIG. 4(c).

[0051] When selecting a desk and a clock as objects to be detected by the object detection unit 33 in the correction mode, it is difficult for the user to select the desk and the clock if the masked image shown in Fig. 4(b) is used. To avoid this, the image displayed on the display 20 returns to the original image of Fig. 4(a), and the desk and the clock can be selected in an unmasked state. This allows the user to accurately select the object to be detected and more accurate contour detection, resulting in the generation of a masked image as shown in Fig. 4(c).

[0052] The masking processing unit 34 executes a masking process on the visible image based on the masking region image. Using the masking region image input from the object detection unit 33, a masking process is executed to generate a masked image in which a region of the visible image corresponding to the outside of the contour of the object in the specified image region is masked. Masking the region of the visible image is image processing to such an extent that the image of the region of the visible image cannot be identified, and the method may be a solid image or a mosaic process. The masking processing unit 34 can display the masked image generated by the masking process on the display 20 or store it in the storage medium 4.

[0053] FIG. 5 is a flow diagram showing an example of processing in the image processing system according to this embodiment.

[0054] When the software of the image processing system is started, the image input section of the image processing section 3 acquires frame images from the images captured by the visible image camera 11 and the distance image camera 12 (step S101).

[0055] Distance image preprocessing unit 31 preprocesses the input distance image (step S ), and passes the preprocessed distance image to distance range limiting unit 32.

[0056] Distance range limiting unit 32 performs distance range limiting processing on the distance image received from distance image preprocessing unit 31 (step S103), and passes the masking region image after the distance range limiting processing to object detection unit 33.

[0057] When the object detection unit 33 receives the masking region image, it determines whether or not the correction mode is set (step S104), and if the correction mode is not set (step S104: No), the object detection unit 33 does nothing to the masking region image and passes it to the masking processing unit 34. The correction mode is a mode in which the object detection unit 33 executes object detection processing.

[0058] If the object detection unit 33 is set to the correction mode (step S104: Yes), it performs an object detection process on the visible image and the distance image using the masking area image received from the distance range limiting unit 32 to identify the contours of objects in the image area (step S105), and passes the masking area image generated by the object detection process to the masking processing unit 34.

[0059] The masking processing unit 34 performs a masking process using the masking area image input from the object detection unit 33 to generate a masking image in which the area of ​​the visible image corresponding to the outside of the contour of the object in the identified image area is masked (step S106).

[0060] The masking process 34 displays the masked image on the display 20 and stores the masked image in the storage medium 4 (step S107). For example, when an operation input is performed using a shooting button or the like displayed on the display 20, it is determined that the correction is completed (step S108: Yes), and the process returns to step S101 to execute processing for another image frame.

[0061] FIG. 6 is a diagram showing an example of input and output in the image processing unit of the image processing system according to this embodiment.

[0062] In the image processing system according to this embodiment, when a subject is photographed by the camera 10, a visible image and a distance image including the subject of interest, as shown on the left side of Fig. 5, are input to the image processing unit 3 (see Figs. 2 and 3). The subject of interest in this image is a series of box-like devices and an operation panel on the right side.

[0063] When the image processing unit 3 performs the processing shown in Figure 5 based on these visible images and distance images, an image is displayed on the display 20 in which the target object is clearly depicted and the other areas are masked, as shown on the right side of Figure 6.

[0064] In this way, according to the image processing system of this embodiment, it is possible to generate a masking image in which unnecessary parts are reliably masked while maintaining image quality for the target subject that is sufficient for work reports.

[0065] (Modification of the embodiment) In the above embodiment, an example has been described in which an image captured by a camera 10 attached to the image processing unit 3 is input and output to a display 20 and a storage medium 4 attached to the image processing unit 3, but this is not limited to this and any specified processing can be performed.

[0066] Furthermore, the above configurations of the camera 10 having the visible image camera 11 and the distance image camera 12, the image processing section 3, and the display 20 are also merely examples, and the specific configurations are not limited to these.

[0067] Furthermore, the above-mentioned flowchart shows an example for explaining the operation, and is not limited to this. That is, the steps shown in each figure of the flowchart are specific examples, and are not limited to this flow. For example, the order of some processes may be changed, other processes may be inserted between each process, and some processes may be performed in parallel.

[0068] In this embodiment, the correction mode can be set to be enabled or disabled, but this is not limiting. The correction mode may not be enabled by setting, and only the correction mode may be executed by default.

[0069] In the above embodiment, the case where one object or multiple objects in a close distance range are targeted has been described as an example, but multiple objects in distant distance ranges, such as in the foreground and background, may also be targeted. In that case, for example, the distance range limiting unit 32 can set an effective distance range different from the effective distance range initially set in an area other than the area of ​​the visible image selected by the object selection tool 22. By selecting an object other than the object initially selected by the object selection tool 22 outside the initially set effective distance range, a different effective distance range can be locally specified, and the foreground and background objects can be individually selected. [Explanation of symbols]

[0070] 1 image processing system, 10 camera, 11 visible image camera, 12 distance image camera, 20 display, 21 effective distance range setting bar, 22 object selection tool, 23 setting menu, 24 shooting button, 3 image processing section, 31 distance image preprocessing section, 32 distance range limiting section, 33 object detection section, 34 masking processing section, 301 CPU, 302 RAM, 303 ROM, 304 communication circuit, 304 peripheral circuit

Claims

1. an image input unit that receives an input of a visible image of an object and a distance image having distance information of the object in the visible image; an effective distance range designation unit that accepts an input designating a distance as an effective area of ​​the image as an effective distance range, determines an image area not designated in the range based on the distance information, and generates a masking area image having information on the image area not designated by the effective distance range; an object selection unit that uses the masking area image to accept an input for selecting an object not to be masked within an image area not designated by the effective distance range, and identifies an outline of the object not to be masked within the selected image area based on the visible image and the distance image; a masking image display unit that generates a masking image by masking an area of ​​the visible image corresponding to the outside of the contour of the object not to be masked within the specified image area, and displays the masking image on a display; An image processing system comprising:

2. 2. The image processing system according to claim 1, Further, a masking image storage unit is provided for storing the masking image displayed on the masking image display unit. Image processing system.

3. 3. The image processing system according to claim 1, the object selection unit extracts contours of objects not to be masked in the selected image region by edge processing in the visible image, and corrects the extracted contours by distance information in the distance image, thereby identifying contours of objects not to be masked in the selected image region. Image processing system.

4. 4. The image processing system according to claim 1, a visible image camera that captures the visible image and an infrared camera that captures the distance image, and the captured visible image and distance image are input to the image input unit. Image processing system.

5. 5. The image processing system according to claim 1, a pre-processing unit that performs expansion processing on distance information of the distance image received by the image input unit using a predetermined size as a threshold value to correct the distance information of the distance image, Image processing system.

6. Computer, an image input unit that receives an input of a visible image of an object and a distance image having distance information of the object in the visible image; an effective distance range designation unit that accepts an input designating a distance as an effective area of ​​the image as an effective distance range, determines an image area not designated in the range based on the distance information, and generates a masking area image having information on the image area not designated by the effective distance range; an object selection unit that uses the masking area image to accept an input for selecting an object not to be masked within an image area not designated by the effective distance range, and identifies an outline of the object not to be masked within the selected image area based on the visible image and the distance image; a masking image display unit that generates a masking image by masking an area of ​​the visible image corresponding to the outside of the contour of the object not to be masked within the specified image area, and displays the masking image on a display; An image processing program that acts as a

7. 1. A computer-implemented image processing method, comprising: an image input step of receiving an input of a visible image of an object and a distance image having distance information of a subject in the visible image; an effective distance range designation step of accepting an input for designating a distance as an effective area of ​​the image as an effective distance range, determining an image area not designated based on the distance information, and generating a masking area image having information on the image area not designated by the effective distance range; an object selection step of accepting an input for selecting an object not to be masked in an image area not designated by the effective distance range using the masking area image, and identifying an outline of the object not to be masked in the selected image area based on the visible image and the distance image; a masking image display step of generating a masking image by masking an area of ​​the visible image corresponding to the outside of the contour of the object not to be masked within the specified image area, and displaying the masking image on a display; An image processing method comprising:

Citation Information

Patent Citations

  • Alkaline storage battery

    JP1983019856A

  • Preparation of powdery miso

    JP1983071862A

  • Method of correcting contour of grayscale image, and device therefor

    JP2008243184A

  • Rectangular table detection using RGB and depth measurement hybrid camera sensor

    JP2011523742A

  • Information processor, information processing method, and program

    JP2014236896A