Background removal device and background removal method

Abstract

To enable background removal from an image without using a background image.SOLUTION: A background removal apparatus includes: a point group data acquisition unit 201 that acquires image indicating point group data obtained by an imaging apparatus 1; a template acquisition unit 203 that acquires a template which is image indicating point group data of a three-dimensional shape of an object model; a background determination processing unit 204 that specifies point group data corresponding to a background portion among the image indicating the point group data by shape recognition based on the image indicating the point group acquired by the point group data acquisition unit 201 and the template acquired by the template acquisition unit 203; and a background removal processing unit 205 that removes point group data corresponding to the background portion from the image indicating the point group data based on the image indicating the point group data acquired by the point group data acquisition unit 201 and a determination result by the background determination processing unit 204.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a background removal device and a background removal method for removing a background from an image showing point cloud data.

Background Art

[0002] Conventionally, a method of removing (separating) a background from an image using a background image has been known (see, for example, Patent Documents 1 to 4). In the method disclosed in Patent Document 1, a background image is prepared and the background is removed from the image using a background differentiator. Also, in the method disclosed in Patent Document 2, a foreground image and a background image are used to extract an image parallax. Also, Patent Document 3 shows a method for improving the detection accuracy of an object while suppressing an increase in load such as an increase in the size and complexity of the device. Also, Patent Document 4 shows a method of performing feedback processing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the methods disclosed in the above-mentioned patent documents require background images. For example, in the methods disclosed in Patent Documents 1 and 2, the number of background image captures is small if the same designated space is maintained, but if the same space cannot be maintained, the effort of capturing background images before the actual event increases. Furthermore, for example, the methods disclosed in Patent Documents 3 and 4 use a threshold value for the background image to perform background separation processing based on the threshold value between the latest image and the background image. In this case as well, there is the extra step of preparing a reference background image.

[0005] On the other hand, for example, in the process of removing weld beads on a metalworking line, the size of the beads to be removed is not necessarily constant. Furthermore, when detecting these beads to be removed using image processing, conventional methods require updating the background image depending on the size, even if the beads to be removed have the same shape. In this case, updating the background image takes time, which in turn takes time to identify the extent of the beads to be removed.

[0006] This disclosure was made to solve the above-mentioned problems and aims to provide a background removal device that can remove the background from an image without using a background image. [Means for solving the problem]

[0007] The background removal device according to this disclosure is characterized by comprising: a point cloud data acquisition unit that acquires an image showing point cloud data obtained by a shooting device; a template acquisition unit that acquires a template which is an image showing point cloud data of the three-dimensional shape of an object model; a background determination processing unit that identifies point cloud data corresponding to the background portion of the image showing the point cloud data by shape recognition based on the image showing the point cloud data acquired by the point cloud data acquisition unit and the template acquired by the template acquisition unit; and a background removal processing unit that removes point cloud data corresponding to the background portion from the image showing the point cloud data based on the image showing the point cloud data acquired by the point cloud data acquisition unit and the determination result by the background determination processing unit. [Effects of the Invention]

[0008] According to this disclosure, the above configuration makes it possible to remove the background from an image without using a background image. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of the background removal system according to Embodiment 1. [Figure 2] This figure shows an example of the configuration of the background removal device according to Embodiment 1. [Figure 3] This flowchart shows an example of the operation of the background removal device according to Embodiment 1. [Figure 4] Figures 4A and 4B show examples of the operation of the point cloud data acquisition unit and the template acquisition unit in Embodiment 1. Figure 4A shows an example of an image showing point cloud data acquired by the point cloud data acquisition unit, and Figure 4B shows an example of a template acquired by the template acquisition unit. [Figure 5] Figures 5A and 5B show examples of the operation of the range selection processing unit in Embodiment 1. Figure 5A shows an example of an image showing point cloud data acquired by the point cloud data acquisition unit (image before extraction by the range selection processing unit), and Figure 5B shows an example of an image showing point cloud data extracted by the range selection processing unit. [Figure 6] Figures 6A to 6D are diagrams showing examples of the operation of the background determination processing unit in Embodiment 1. Figure 6A is a diagram showing an example of an image showing point cloud data before changing the magnification, Figure 6B is a diagram showing an example of an image showing point cloud data with the magnification changed to 30%, Figure 6C is a diagram showing an example of an image showing point cloud data with the magnification changed to 50%, and Figure 6D is a diagram showing an example of an image showing point cloud data with the magnification changed to 75%. [Figure 7]A diagram showing an operation example of the background removal processing unit in Embodiment 1. FIG. 7A is a diagram showing the case where background removal is performed using the image shown in FIG. 6B, FIG. 7B is a diagram showing the case where background removal is performed using the image shown in FIG. 6C, and FIG. 7C is a diagram showing the case where background removal is performed using the image shown in FIG. 6D.

Embodiments for Carrying out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the drawings. Embodiment 1. FIG. 1 is a diagram showing a configuration example of a background removal system according to Embodiment 1. As shown in FIG. 1, the background removal system includes a photographing device 1 and a background removal device 2.

[0011] The photographing device 1 obtains an image showing point cloud data in a predetermined three-dimensional space by photographing the three-dimensional space. Note that an object of interest exists in the three-dimensional space. Examples of the object of interest include a welding bead. The image showing the point cloud data obtained by the photographing device 1 is output to the background removal device 2. Examples of the photographing device 1 include a three-dimensional camera.

[0012] The background removal device 2 obtains an image showing the point cloud data of the object of interest by removing the point cloud data corresponding to the background portion from the image showing the point cloud data obtained by the photographing device 1. The image showing the point cloud data of the object of interest obtained by the background removal device 2 is output to the outside.

[0013] As shown in FIG. 2, the background removal device 2 includes a point cloud data acquisition unit 201, a range specification processing unit 202, a template acquisition unit 203, a background determination processing unit 204, and a background removal processing unit 205.

[0014] The background removal device 2 is realized by a processing circuit such as a system LSI (Large Scale Integration), or a CPU (Central Processing Unit) that executes a program stored in a memory or the like.

[0015] The point cloud data acquisition unit 201 acquires an image showing the point cloud data obtained by the imaging device 1. In the image showing the point cloud data acquired by the point cloud data acquisition unit 201, in addition to the point cloud data corresponding to the target object, the point cloud data corresponding to the background other than the target object is included.

[0016] The range specification processing unit 202 extracts an image showing the point cloud data within a predetermined range from the image showing the point cloud data acquired by the point cloud data acquisition unit 201. At this time, the range specification processing unit 202 extracts, for example, an image showing the point cloud data within the range specified by the user.

[0017] The template acquisition unit 203 acquires a template. The template is an image showing the point cloud data of the three-dimensional shape of the target object model. The target object model is a model having the same shape (including the meaning of substantially the same) as the target object. Note that the template acquisition unit 203 may acquire, as a template, an image showing the point cloud data of the three-dimensional shape of the target object model set in advance. Alternatively, the template acquisition unit 203 may acquire, as an image showing the point cloud data of the three-dimensional shape of the target object model, an image showing the point cloud data of the target object obtained by the background removal processing unit 205.

[0018] The background determination processing unit 204 specifies the background portion in the image showing the point cloud data by shape recognition based on the image showing the point cloud data extracted by the range specification processing unit 202 and the template acquired by the template acquisition unit 203.

[0019] In this process, the background determination processing unit 204 first changes the magnification of the template acquired by the template acquisition unit 203 and compares it with the image showing the point cloud data extracted by the range specification processing unit 202, and detects point cloud data from the point cloud data shown in the image whose shape matches (including approximate matches). The background determination processing unit 204 then determines that the point cloud data other than the point cloud data detected above, among the point cloud data shown in the image extracted by the range specification processing unit 202, corresponds to the background portion.

[0020] Furthermore, the background determination processing unit 204 may change the magnification of the image showing the point cloud data extracted by the range specification processing unit 202. The background determination processing unit 204 may change the magnification of the image showing the point cloud data by multiple different values. In this case, the background determination processing unit 204 identifies the background portion by performing shape recognition on the image showing the point cloud data after the magnification has been changed, using a template. If there are multiple images showing the point cloud data after the magnification has been changed, the background determination processing unit 204 performs shape recognition on each of these images using a template.

[0021] The background removal processing unit 205 removes the background portion from the image showing the point cloud data extracted by the range selection processing unit 202, and the result of the background determination processing unit 204. In other words, the background removal processing unit 205 removes the point cloud data that the background determination processing unit 204 has determined to be the background portion from the image showing the point cloud data extracted by the range selection processing unit 202. As a result, the background removal device 2 can obtain an image showing the point cloud data of the target object.

[0022] Figure 2 shows the case where the background removal device 2 is equipped with a range selection processing unit 202. However, the range selection processing unit 202 is not a mandatory component of the background removal device 2, and the background removal device 2 does not need to be equipped with a range selection processing unit 202. In other words, in this case, the background determination processing unit 204 identifies the background portion of the image showing the point cloud data by shape recognition, based on the image showing the point cloud data acquired by the point cloud data acquisition unit 201 and the template acquired by the template acquisition unit 203. Furthermore, the background removal processing unit 205 removes the background portion from the image showing the point cloud data acquired by the point cloud data acquisition unit 201, and the background determination processing unit 204, based on the determination result.

[0023] Next, an example of the operation of the background removal device 2 according to Embodiment 1 shown in Figures 1 and 2 will be explained with reference to Figure 3. The imaging device 1 obtains an image showing point cloud data in a predetermined three-dimensional space by imaging that space. The target object exists in this three-dimensional space. An example of the target object is a weld bead. The image showing the point cloud data obtained by the imaging device 1 is then output to the background removal device 2.

[0024] In the example of operation of the background removal device 2 according to Embodiment 1 shown in Figures 1 and 2, as shown in Figure 3, first, the point cloud data acquisition unit 201 acquires an image showing the point cloud data obtained by the imaging device 1 (step ST301).

[0025] Figure 4A is a diagram showing an example of an image displaying point cloud data acquired by the point cloud data acquisition unit 201. As shown in Figure 4A, the image showing the point cloud data acquired by the point cloud data acquisition unit 201 includes not only the point cloud data corresponding to the target object, but also point cloud data corresponding to the background other than the target object. Note that in Figure 4A, for the sake of simplicity, the image showing the point cloud data is shown as a two-dimensional image. In Figure 4A, reference numeral 41 indicates the point cloud data corresponding to the target object, and reference numeral 42 indicates the point cloud data corresponding to the background.

[0026] Next, the range specification processing unit 202 extracts an image showing point cloud data within a predetermined range from the image showing point cloud data acquired by the point cloud data acquisition unit 201 (step ST302). At this time, the range specification processing unit 202 extracts an image showing point cloud data within a range specified by the user, for example.

[0027] For example, the user roughly outlines the area containing the target object from an image showing point cloud data acquired by the point cloud data acquisition unit 201, and specifies the area within that image to be subject to background removal. The range specification processing unit 202 then extracts the image of the specified range from the image showing point cloud data acquired by the point cloud data acquisition unit 201.

[0028] Figure 5 shows an example of the operation of the range selection processing unit 202. Figure 5A shows an example of an image showing point cloud data acquired by the point cloud data acquisition unit 201 (image before extraction by the range selection processing unit 202), and Figure 5B shows an example of an image showing point cloud data extracted by the range selection processing unit 202. Note that in Figure 5, for the sake of simplicity, the image showing the point cloud data is shown as a two-dimensional image. In Figure 5, reference numeral 51 indicates, for example, the area (a predetermined range) to be removed by the user.

[0029] Furthermore, the template acquisition unit 203 acquires a template (step ST303). The template is an image showing point cloud data of the 3D shape of the target object model. The 3D shape of the target object model only needs to be identical (including approximately identical) in shape to the 3D shape of the target object captured by the shooting device 1, and the size is irrelevant. For example, welding beads have unique shapes depending on the welding method or the skill level of the worker, so although the shape is the same, the size is usually different. Therefore, the template does not need to match the size of the target object.

[0030] The template acquisition unit 203 may acquire an image showing the point cloud data of the three-dimensional shape of a pre-set target object model as a template. Alternatively, the template acquisition unit 203 may acquire an image showing the point cloud data of the target object obtained by the background removal processing unit 205 as an image showing the point cloud data of the three-dimensional shape of the target object model. Furthermore, the acquisition of templates by the template acquisition unit 203 can be performed at any time before processing by the background determination processing unit 204.

[0031] Figure 4B shows an example of a template acquired by the template acquisition unit 203. Note that in Figure 4B, the template is shown as a two-dimensional image for simplicity of explanation.

[0032] Next, the background determination processing unit 204 identifies the background portion of the image showing the point cloud data by shape recognition, based on the image showing the point cloud data extracted by the range specification processing unit 202 and the template acquired by the template acquisition unit 203 (step ST304).

[0033] In this process, the background determination processing unit 204 first changes the magnification of the template acquired by the template acquisition unit 203 and compares it with the image showing the point cloud data extracted by the range specification processing unit 202, and detects point cloud data from the point cloud data shown in the image whose shape matches (including approximate matches). The background determination processing unit 204 then determines that the point cloud data other than the point cloud data detected above, among the point cloud data shown in the image extracted by the range specification processing unit 202, corresponds to the background portion.

[0034] Furthermore, the background determination processing unit 204 may change the magnification of the image showing the point cloud data extracted by the range specification processing unit 202. The background determination processing unit 204 may change the magnification of the image showing the point cloud data by multiple different values. In this case, the background determination processing unit 204 identifies the background portion by performing shape recognition on the image showing the point cloud data after the magnification has been changed, using a template. If there are multiple images showing the point cloud data after the magnification has been changed, the background determination processing unit 204 performs shape recognition on each of these images using a template.

[0035] Figure 6 shows an example of the operation of the background determination processing unit 204 to change the magnification. Figure 6A shows an example of an image showing point cloud data extracted by the range specification processing unit 202 (image before magnification change by the background determination processing unit 204), and Figures 6B to 6D show examples of images showing point cloud data after magnification change by the background determination processing unit 204 (Figure 6B shows the case when the magnification is set to 30%, Figure 6C shows the case when the magnification is set to 50%, and Figure 6D shows the case when the magnification is set to 75%). Note that in Figures 6A to 6D, the point cloud data is shown as a two-dimensional image for the sake of simplicity.

[0036] Figure 6 shows the case where the background determination processing unit 204 sets the horizontal and vertical widths of the image showing the point cloud data extracted by the range specification processing unit 202 to 100%, and then changes the image to magnifications of 30%, 50%, and 75%. Subsequently, the background detection processing unit 204 performs shape recognition on the images after changing the magnification to 30%, 50%, and 75% using the template.

[0037] Next, the background removal processing unit 205 removes the background portion from the image showing the point cloud data extracted by the range specification processing unit 202, and the result of the background determination processing unit 204 (step ST305). That is, the background removal processing unit 205 removes the point cloud data that the background determination processing unit 204 has determined to be the background portion from the image showing the point cloud data extracted by the range specification processing unit 202. As a result, the background removal device 2 can obtain an image showing the point cloud data of the target object.

[0038] Figure 7 shows an example of operation by the background removal processing unit 205. In Figure 7, point cloud data determined to be background is removed from the image showing the point cloud data after magnification change by the background determination processing unit 204 shown in Figures 6B to 6D, and an image showing the point cloud data of the target object is obtained. Note that in Figure 7, for the sake of simplicity, the image showing the point cloud data is presented as a two-dimensional image.

[0039] Furthermore, as shown in Figure 7, if the background removal processing unit 205 obtains images showing the point cloud data of the target object from multiple images of different magnifications, it may obtain the final image showing the point cloud data of the target object by adjusting these images showing the point cloud data of the target object to the same magnification and then superimposing them.

[0040] In this case, for example, weld beads differ in size but have the same shape. Therefore, such weld beads can be identified by performing shape recognition using a template on an image. Therefore, in the background removal device 2 according to Embodiment 1, the background portion other than the target object is identified by performing shape recognition using a template on an image showing point cloud data obtained by the imaging device 1, and the point cloud data corresponding to that background portion is removed from the image. As a result, the background removal device 2 according to Embodiment 1 can obtain an image showing the target object from an image showing point cloud data obtained by the imaging device 1, regardless of the size of the target object. Consequently, the background removal device 2 according to Embodiment 1 can shorten the extraction time of the weld bead, for example, when the target object is a weld bead that is being processed, thereby improving work efficiency.

[0041] Furthermore, images showing point cloud data may contain noise. In this case, depending on the type of noise, misrecognition may occur in the shape recognition process of the background determination processing unit 204. In contrast, the background determination processing unit 204 in Embodiment 1 makes it possible to change the magnification of the image showing the point cloud data acquired by the point cloud data acquisition unit 201 or the image showing the point cloud data extracted by the range specification processing unit 202. This makes it possible to suppress the effects of noise even if the image representing the original point cloud data contains noise, by changing the magnification to a size that makes it easier to remove the noise. As a result, the background determination processing unit 204 can perform shape recognition more accurately. The above magnification value and the number of times the magnification is changed can be set appropriately based on, for example, noise or other factors assumed to be present in the image showing the point cloud data.

[0042] Furthermore, the template acquisition unit 203 in Embodiment 1 may acquire an image showing the point cloud data of the target object obtained by the background removal processing unit 205 as an image showing the point cloud data of the 3D shape of the target object model. As a result, the background removal device 2 according to Embodiment 1 can handle point cloud data that corresponds to the actual object of the target as a template, enabling even more accurate background removal processing.

[0043] Furthermore, the above example shows the case where the background determination processing unit 204 reduces the magnification of the image showing the point cloud data (to less than 100%). However, the background determination processing unit 204 may also increase the magnification of the image showing the point cloud data (it may be greater than 100%). For example, if the target object is a very small object, increasing the magnification of the image showing the point cloud data is considered effective.

[0044] Furthermore, the above example illustrates a case where the background determination processing unit 204 performs shape recognition using only an image showing point cloud data after magnification. However, the background determination processing unit 204 is not limited to this, and may perform shape recognition using both an image showing the point cloud data before the magnification change and an image showing the point cloud data after the magnification change.

[0045] Furthermore, the above example shows the case where the target object is a weld bead. However, the target object is not limited to this; any object with the same shape (including nearly identical) as the target object model will suffice. Examples of target objects other than weld beads include burrs or solder.

[0046] As described above, according to this embodiment 1, the background removal device 2 includes a point cloud data acquisition unit 201 that acquires an image showing point cloud data obtained by the imaging device 1, a template acquisition unit 203 that acquires a template which is an image showing point cloud data of the three-dimensional shape of an object model, a background determination processing unit 204 that identifies point cloud data corresponding to the background portion of the image showing the point cloud data by shape recognition based on the image showing the point cloud data acquired by the point cloud data acquisition unit 201 and the template acquired by the template acquisition unit 203, and a background removal processing unit 205 that removes point cloud data corresponding to the background portion from the image showing the point cloud data based on the image showing the point cloud data acquired by the point cloud data acquisition unit 201 and the determination result by the background determination processing unit 204. As a result, the background removal device 2 according to embodiment 1 can remove the background from an image without using a background image.

[0047] Furthermore, the background determination processing unit 204 in Embodiment 1 may change the magnification of the image showing the point cloud data acquired by the point cloud data acquisition unit 201, and perform shape recognition on the image showing the point cloud data after the magnification change using the template acquired by the template acquisition unit 203. As a result, the background removal device 2 according to Embodiment 1 can obtain an image showing the point cloud data of the target object with high accuracy, even if the image showing the point cloud data acquired by the point cloud data acquisition unit 201 contains noise.

[0048] Furthermore, the background removal device 2 in Embodiment 1 may include a range specification processing unit 202 that extracts an image showing point cloud data within a predetermined range from an image showing point cloud data acquired by the point cloud data acquisition unit 201. This makes it possible to reduce the processing load and shorten the processing time of the background removal device 2 according to Embodiment 1.

[0049] Furthermore, any component of the embodiment can be modified, or any component of the embodiment can be omitted. [Explanation of symbols]

[0050] 1. Imaging device 2 Background remover 201 Point Cloud Data Acquisition Unit 202 Range Selection Processing Unit 203 Template Acquisition Section 204 Background detection processing unit 205 Background removal processing unit

Claims

1. A point cloud data acquisition unit acquires an image showing point cloud data obtained by a shooting device, A template acquisition unit acquires a template, which is an image representing the point cloud data of the three-dimensional shape of an object model. A background determination processing unit identifies point cloud data corresponding to the background portion of the image showing the point cloud data, based on an image showing the point cloud data acquired by the point cloud data acquisition unit and a template acquired by the template acquisition unit, by shape recognition. An image showing point cloud data acquired by the point cloud data acquisition unit, and a background removal unit that removes point cloud data corresponding to the background portion from the image showing the point cloud data based on the determination result by the background determination processing unit. A background removal device equipped with the following features.

2. The background determination processing unit changes the magnification of the image showing the point cloud data acquired by the point cloud data acquisition unit, and performs shape recognition on the image showing the point cloud data after the magnification change using the template acquired by the template acquisition unit. The background removal apparatus according to claim 1, characterized in that it is a background removal apparatus.

3. The system includes a range specification processing unit that extracts an image showing point cloud data within a predetermined range from an image showing point cloud data acquired by the point cloud data acquisition unit, The background determination processing unit uses the image showing the point cloud data extracted by the range specification processing unit to identify the point cloud data corresponding to the background portion of the image showing the point cloud data. The background removal processing unit uses the image showing the point cloud data extracted by the range specification processing unit to remove the point cloud data corresponding to the background portion from the image showing the point cloud data. The background removal device according to claim 1 or 2, characterized in that it is a background removal device.

4. The point cloud data acquisition unit acquires an image showing the point cloud data obtained by the imaging device, The template acquisition unit performs the step of acquiring a template, which is an image representing the point cloud data of the three-dimensional shape of the object model. The background determination processing unit identifies the point cloud data corresponding to the background portion of the image showing the point cloud data, based on the image showing the point cloud data acquired by the point cloud data acquisition unit and the template acquired by the template acquisition unit, by shape recognition. The background removal processing unit performs the steps of removing point cloud data corresponding to the background portion from the image showing the point cloud data acquired by the point cloud data acquisition unit, and the background determination processing unit's determination result. A background removal method having the following characteristics.

Images