Method and system for extracting contours from image data

Abstract

This invention provides an image data contour extraction method and system that can extract the contours of objects from image data more easily and accurately. [Solution] In this image data contour extraction method, image data D1 of an object P positioned in front of a background object 6 at a distance is acquired from in front of the object P together with the background object 6 by a monocular camera device 2, and the contour of the object P in this image data D1 is extracted by a processing device 3. The processing device 3 processes the image data D1 using a monocular depth estimation model 20 constructed by machine learning, thereby generating a depth map D2 in which the pixel values ​​of each pixel in the image data D1 are converted to pixel values ​​corresponding to the distance from the monocular camera device 2. Based on the pixel values ​​of each pixel in this depth map D2, the image data D1 is classified into regions, and the contour of the object P in the image data D1 is extracted.

Description

Technical Field

[0001] The present invention relates to a method and system for extracting the contour of image data, and more particularly, to a method and system for extracting the contour of an object in image data more simply and accurately.

Background Art

[0002] When evaluating the desired characteristics of an object such as vulcanized rubber or rubber products, an evaluation test device or the like corresponding to the characteristics is used. Sometimes, image data of the object is acquired during the evaluation test, and the characteristics of the object are evaluated using the image data. The image data includes not only the object but also its background.

[0003] The background is generally an object on the wall surface or the back surface of the housing of the evaluation test device, and the contrast with the object is not always clear. In addition, since the background is arranged at a distance from the object, the shadow of the object generated on the surface of the background becomes a factor that makes the contrast between the object and the background unclear. Furthermore, dirt on the surface of the background or the object also becomes a factor that makes the contrast between the two unclear. Therefore, even if the object is in various arrangement environments, it is difficult to accurately extract the contour of the object in the image data by a method based on the pixel values of each pixel of the image data.

[0004] While this invention does not extract the contours of objects from image data, a device has been proposed that utilizes a learning model to estimate the 3D coordinates of an object from image data using a feature point map with depth information as training data (see Patent Document 1). The invention proposed in Patent Document 1 assumes that the contours of objects (boundaries between objects and backgrounds) in image data generated by a monocular camera can be clearly identified through data processing. It then extracts various feature information such as the size, position, and light attenuation of the object, and estimates the 3D coordinates, including the 2D coordinates (map) of the object's feature points and the distance to the feature points (depth information), based on the extracted feature information. Therefore, simply applying this invention to image data acquired with background objects where objects are placed in various environments will not accurately extract the contours of the objects. Thus, further development is needed to extract the contours of objects from image data more easily and accurately. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2021-117130 [Overview of the project] [Problems that the invention aims to solve]

[0006] The objective of the present invention is to provide an image data contour extraction method and system that can extract the contours of objects in image data more easily and accurately. [Means for solving the problem]

[0007] The present invention provides an image data contour extraction method for achieving the above objective, which involves acquiring image data of an object positioned in front of a background object at a distance from the background object using a monocular camera device from in front of the object together with the background object, and extracting the contour of the object from the image data using the computing device, characterized in that the computing device processes the image data using a monocular depth estimation model to generate a depth map in which the pixel values ​​of each pixel in the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device, and the image data is classified into regions based on the pixel values ​​of each pixel in the depth map to extract the contour of the object.

[0008] The present invention provides an image data contour extraction system comprising: a monocular camera device that acquires image data of an object positioned in front of a background object at a distance from the background object, together with the background object, from in front of the object; and a computing device that extracts the contour of the object from the image data, wherein the computing device has a storage unit that stores a monocular depth estimation model, and is characterized by performing data processing to generate a depth map in which the pixel values ​​of each pixel of the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device using the monocular depth estimation model, and data processing to classify the image data into regions based on the pixel values ​​of each pixel of the depth map and extract the contour of the object. [Effects of the Invention]

[0009] According to the present invention, in the depth map, the degree of depth (distance from the monocular camera device) of the object and the background in the image data is represented as the difference (contrast) of the pixel values ​​of each pixel corresponding to the object and the background, respectively. In other words, in the depth map, even if the contrast between the object and the background in the image data is not clear, the contrast (luminance difference) between the object and the background becomes clear. Therefore, by classifying the image data into regions using the pixel values ​​of each pixel in this depth map, the boundary between the object and the background in the image data can be identified with high accuracy, and the contour of the object can be extracted with high accuracy.

[0010] Furthermore, according to the present invention, the image data of the object is acquired by the monocular camera device, and the contour of the object is extracted by performing predetermined data processing on the acquired image data using the computing device. Therefore, the contour of the object can be extracted more easily by significantly reducing the amount of manual work required. [Brief explanation of the drawing]

[0011] [Figure 1] This is an explanatory diagram illustrating an embodiment of an image data contour extraction system. [Figure 2] Figure 1 is an explanatory diagram illustrating the object, background, and holding mechanism as seen from a front view through the monocular camera device. [Figure 3] This is an explanatory diagram illustrating image data that includes the object in Figure 2. [Figure 4] This is an explanatory diagram illustrating processed image data obtained by applying thresholding to the image data in Figure 3. [Figure 5] This is an explanatory diagram illustrating processed image data obtained by applying a different thresholding process to the image data in Figure 3. [Figure 6] This is an explanatory diagram illustrating training data. [Figure 7] This is an explanatory diagram illustrating a monocular depth estimation model. [Figure 8]It is a flowchart exemplifying the procedure in an embodiment of a method for extracting the contour of image data. [Figure 9] It is an explanatory diagram exemplifying the depth map of the image data in FIG. 3. [Figure 10] It is an explanatory diagram exemplifying the depth map subjected to threshold processing. [Figure 11] It is an explanatory diagram exemplifying the extracted image data from which the contour of the object has been extracted. [Figure 12] It is an explanatory diagram exemplifying the image data including the tire as the object. [Figure 13] It is an explanatory diagram exemplifying the depth map of the image data in FIG. 12. [Figure 14] [[ID=​​​​​​​​​​​​​​​Using this contour extraction system 1, an embodiment of a method for extracting the contour of image data illustrated in FIG. 8 is implemented. This contour extraction method is used to extract the front view contour of the surface Pa facing the monocular camera device 2 arranged in front of the object P (for example, a test piece of vulcanized rubber). For example, in an environment of a known tensile test, image data D1 of the surface Pa of the object P is acquired by the monocular camera device 2, and the image data D1 is processed by the arithmetic unit 3 using the monocular depth estimation model 20, whereby the contour of the object P in the image data D1 is extracted.

[0015] First, each device of the object P and the contour extraction system 1 will be described in detail.

[0016] FIG. 1 shows the whole of the contour extraction system 1, and FIG. 2 shows the object P, the holding mechanism 5, and the background object 6 in a front view seen from the monocular camera device 2. In the present embodiment, the monocular camera device 2, the holding mechanism 5, and the background object 6 are arranged in order from the front to the rear in the X direction, and the lighting device 4 is arranged in front of the holding mechanism 5. That is, the object P is arranged in front (on the side of the monocular camera device 2) at an interval from the background object 6 and is interposed between the monocular camera device 2 and the background object 6.

[0017] The object P is a desired test piece (thin section) of vulcanized rubber. This object P is used in various known evaluation tests for evaluating rubber properties such as tensile tests. The object P is not limited to a test piece of vulcanized rubber, and may be, for example, rubber products such as tires, rubber belts, and hoses.

[0018] The monocular camera device 2 is arranged in front of the object P and acquires image data D1 of the surface Pa of the object P together with the background object 6. This image data D1 will be described later. The monocular camera device 2 can use various known cameras having a monocular photographing lens 2a. The photographing direction (the optical axis direction of the photographing lens 2a) of the monocular camera device 2 is preferably substantially orthogonal to the surface Pa of the object P. The image data D1 acquired by the monocular camera device 2 is input to the arithmetic unit 3 and stored in the auxiliary storage unit 9.

[0019] The computing unit 3 performs data processing using the image data D1. The computing unit 3 can be any known computer. The computing unit 3 has a processing unit (CPU) 7, a main memory unit (memory) 8, and an auxiliary storage unit (e.g., HDD) 9. The computing unit 3 is connected to an input unit 10 for inputting hyperparameters such as the number of epochs when constructing the monocular depth estimation model 20 described later, and an output unit 11 for outputting the image data D1, depth map D2, and extracted image data D3 described later. In addition to various image data, the auxiliary storage unit 9 has the monocular depth estimation model 20 and the training data 21 used to construct it pre-stored.

[0020] The lighting device 4 is positioned in front of the object P and irradiates light toward the surface Pa of the object P. The lighting device 4 can use known electric lights such as incandescent bulbs or light-emitting diodes (LEDs). The lighting device 4 should brightly illuminate the surface Pa of the object P so that the image data D1 of the surface Pa becomes clearer, and the illuminance provided by the lighting device 4 is set as appropriate.

[0021] The holding mechanism 5 is positioned in front of the background object 6 at a distance from it, and holds the object P in a desired state (extended or unextended). In Figures 1 and 2, the holding mechanism 5 holds three object P in an extended state in the Y direction. The extended state includes a state in which the object P is taut and not slack (e.g., extension rate 0%). The holding mechanism 5 has a pair of clamps 5a, and the distance between the pair of clamps 5a is adjustable. The pair of clamps 5a grip both ends of the object P, and the degree of extension (extension rate) of the object P can be adjusted by adjusting the distance between the pair of clamps 5a. The holding mechanism 5 is incorporated into an evaluation test apparatus, and any apparatus suitable for the content of the evaluation test will suffice. The number of object P held at one time and the state of the object P (degree of extension, etc.) can be arbitrarily set according to the content of the test.

[0022] Background object 6 depends on the evaluation test equipment for object P and the environment in which object P is placed, and may be the walls of the equipment's casing or an object behind object P. Therefore, the contrast between object P and background object 6 is not always clear.

[0023] The image data D1 illustrated in Figure 3 is acquired by the monocular camera device 2 and stored in the auxiliary storage unit 9 of the arithmetic unit 3. In Figure 3, a portion of the code leader lines overlapping the image data D1 is shown in white. The three gray band-like areas extending horizontally in Figure 3 represent the object P. On both the top and bottom sides of each band-like area, areas closer to black than the gray of that band-like area represent background objects 6, and areas closer to white represent noise N.

[0024] Depending on the evaluation test apparatus for the object P, various objects may be present as background objects 6. In this case, the colors of the background objects 6 vary, and the contrast between the surface Pa of the object P and the surface 6a of the background objects 6 in the image data D1 is not clear. Furthermore, since the object P is placed at a distance from the background objects 6, a shadow of the object P is cast on the surface 6a of the background objects 6, and the contrast between the two in the image data D1 is not clear. In addition, the image data D1 contains noise N caused by brightness unevenness due to reflection of light irradiated from the lighting device 4, and dirt attached to the object P and background objects 6. Below, we will describe in detail an example in which a known method of region classification based on the pixel value of each pixel in the image data D1 is applied to this image data D1.

[0025] The processed image data D1a shown in Figure 4 is obtained by applying a visually determined threshold to the image data D1 shown in Figure 3. Similarly, the processed image data D1b shown in Figure 5 is obtained by applying Otsu's binarization to the image data D1. In these processed image data D1a and D1b, the pixel values ​​of each pixel corresponding to the object P are generally converted to white, and the pixel values ​​of each pixel corresponding to the noise N superimposed on the background object 6 are also converted to white. Therefore, in these processed image data D1a and D1b, the object P and the noise N are recognized as the same object. Thus, even when using known methods such as thresholding processing like Otsu's binarization, edge detection processing like the Canny algorithm, watershed algorithms, and clustering using unsupervised machine learning on the image data D1, it is difficult to extract the contour of the object P in the image data D1.

[0026] One of the distinguishing features of the contour extraction system 1 and contour extraction method of this embodiment is that, compared to the known methods illustrated in Figures 4 and 5 described above, it uses a monocular depth estimation model 20 constructed by supervised machine learning using training data 21 for region classification of image data D1. The training data 21 and the monocular depth estimation model 20 will be described in detail below.

[0027] The training data 21 illustrated in Figure 6 is stored in the auxiliary storage unit 9 of the arithmetic unit 3. For each sample (1, 2, ..., n), the training data 21 includes training image data D4 and a training depth map D5. This training data 21 includes data acquired using a known stereo camera device, or a combination of a monocular camera device similar to the monocular camera device 2 and a known depth measurement device such as a depth sensor. The training image data D4 contains an arbitrary object (a person's face in sample 1, a building in sample 2, etc.) and a background. The training depth map D5 is image data obtained by converting the pixel values ​​of each pixel in the training image data D4 to pixel values ​​corresponding to the distance measured by the stereo camera or depth measurement device. This training depth map D5 serves as the ground truth data (labels) in machine learning.

[0028] The training data 21 is not particularly limited, but it is preferable that it contains samples in the training image data D4 that are similar to the target object P. For example, it is preferable that the training data 21 contains samples in the training image data D4 that contain vulcanized rubber test pieces similar to the target object P. This is advantageous for improving the estimation accuracy of the monocular depth estimation model 20 described later. In addition, image data D1 and depth map D2 obtained in the process of implementing the contour extraction method of this embodiment may be added to the training data 21. That is, each time this contour extraction method is repeated, the number of samples in the training data 21 increases, and the training data 21 is expanded.

[0029] The monocular depth estimation model 20 illustrated in Figure 7 is constructed before the contour extraction method described later is performed and stored in the auxiliary storage unit 9 of the computing unit 3. The monocular depth estimation model 20 is constructed using supervised machine learning with training image data D4 of the training data 21 as the explanatory variable and training depth map D5 as the target variable. The monocular depth estimation model 20 is a type of computer program that estimates a depth map D2 based on input image data D1. Various known neural networks such as convolutional neural networks (CNNs) and U-NETs, ​​which are a type of convolutional neural network, can be used in the monocular depth estimation model 20.

[0030] In the monocular depth estimation model 20, the training image data D4 and training depth map D5 are associated for each sample (1, 2, ..., n) in the training data 21. In other words, the degree of difference in the pixel values ​​of each pixel in the training image data D4 and the degree of depth of each pixel in the training depth map D5 are analyzed and linked. Therefore, this monocular depth estimation model 20 estimates the distance of each pixel from the monocular camera device 2 based on the degree of difference in the pixel values ​​of each pixel in the input image data D1, and generates a depth map D2 by converting the pixel values ​​of each pixel in the image data D1 to pixel values ​​corresponding to the estimated distance.

[0031] The monocular depth estimation model 20, once constructed, can be used repeatedly without modification. Therefore, the monocular depth estimation model 20 does not need to be reconstructed each time the contour extraction method described later is performed; it can be constructed in advance at a stage prior to performing the contour extraction method and stored in the auxiliary storage unit 9 of the computing device 3.

[0032] Next, the procedure for the contour extraction method illustrated in Figure 8 will be described in detail. In this procedure, first, the monocular camera device 2 acquires image data D1 of the object P (S110). Next, the computing device 3 performs an extraction process to extract the contour of the object P from the image data D1. In this extraction process, the monocular depth estimation model 20 is used to generate a depth map D2 from the image data D1 (S120~S140). Next, the image data D1 is classified into regions based on the pixel value of each pixel in the depth map D2 (S150), and the contour of the object P is extracted (S160). Each step (S110~S160) will be described in detail below.

[0033] In step S110, the monocular camera device 2 acquires image data D1 as illustrated in Figure 3 above. Image data D1 may be a color image, but a grayscale image is preferred. Using a grayscale image as image data D1 is advantageous for shortening the processing time and reducing the processing load for analysis.

[0034] In step S120, the arithmetic unit 3 performs data processing to apply pre-image processing to the image data D1. The pre-image processing only needs to reduce the noise N shown in white in Figure 3 above, and for example, known denoising filters and / or edge enhancement filters can be used. Examples of denoising filters include Gaussian filters, median filters, averaging filters, bilateral filters, L1 normalization, and L2 normalization. Examples of edge enhancement filters include Sobel filters and Laplacian filters. The denoising filter and edge enhancement filter can be arbitrarily selected depending on the noise situation in the image data D1, and one filter may be selected from among the denoising filter and edge enhancement filter, or multiple filters may be selected in combination. The combination of multiple filters may be a combination of denoising filters, a combination of edge enhancement filters, or a combination of a denoising filter and an edge enhancement filter.

[0035] In step S130, data processing is performed by the computing device 3 to generate a depth map D2 using the monocular depth estimation model 20 illustrated in Figure 7 above. Specifically, image data D1 is input to the monocular depth estimation model 20, and the depth map D2 generated from the monocular depth estimation model 20 is output. In this embodiment, the depth map D2 is a grayscale image, but it may also be an RGB or CMYK color image. Furthermore, the pixel values ​​corresponding to depth in the depth map D2 can be arbitrarily set as long as they differ for each depth. In this embodiment, for example, the pixel values ​​are set so that the smaller the depth (the closer to the monocular camera device 2), the larger the pixel values ​​(closer to white), and the larger the depth (the further away from the monocular camera device 2), the smaller the pixel values ​​(closer to black). Note that in the depth map D2, even if different objects are at similar depths, the pixel values ​​of each pixel corresponding to each object will be similar.

[0036] The depth map D2 illustrated in Figure 9 shows that the pixel values ​​of each pixel in the image data D1 illustrated in Figure 3 above have been converted to pixel values ​​corresponding to the depth (distance from the monocular camera device 2 to each pixel) estimated by the monocular depth estimation model 20. In other words, in this depth map D2, the degree of depth of the object P and the background object 6 in the image data D1 is represented as the contrast (luminance difference) of the pixel values ​​of the corresponding pixels. In this depth map D2, noise N is excluded, and the contrast between the object P and the background object 6 is clearer.

[0037] In Figure 9, the three roughly white bands extending horizontally represent the region R1 (the area where each pixel corresponding to object P is located) occupied by object P in image data D1, while the roughly black areas on both the top and bottom of each band represent the region R2 (the area where each pixel corresponding to background object 6 is located) occupied by background object 6 in image data D1. Note that this depth map D2 contains gray areas in parts of region R1 and region R2. These gray areas are treated as noise.

[0038] In step S140, the arithmetic unit 3 performs data processing to apply post-image processing to the depth map D2. The post-image processing only needs to reduce the noise shown in gray in Figure 9 above, and for example, a known denoising filter can be used. Examples of denoising filters include Gaussian filters, median filters, averaging filters, bilateral filters, L1 normalization, and L2 normalization. The denoising filter can be arbitrarily selected depending on the noise situation in the depth map D2, and one filter may be selected from among the denoising filters, or multiple filters may be selected in combination.

[0039] In step S150, the computing unit 3 performs data processing to segment the image data D1 based on the pixel value of each pixel in the depth map D2. In one example of this step S150, the computing unit 3 performs data processing to segment the depth map D2 and divide it into multiple regions R1 and R2. Then, the computing unit 3 performs data processing to segment the image data D1 based on each of the divided regions R1 and R2. The segmentation of the depth map D2 is not limited to threshold processing such as Otsu's binarization or adaptive binarization, but can use various known segmentation methods such as edge detection processing such as the Canny algorithm, watershed algorithms, and clustering using unsupervised machine learning.

[0040] The depth map D2a shown in Figure 10 is obtained by applying Otsu's binarization thresholding process to the depth map D2 shown in Figure 9, and is divided into multiple regions R1 and R2. In Figure 10, the three white bands extending horizontally represent region R1, and the black area represents region R2. In the thresholded depth map D2a, the contrast between region R1 and region R2 is sharper.

[0041] Since the depth of region R1 is smaller (larger pixel value) than the depth of region R2, it can be seen that region R1 is located in front of region R2 (towards the monocular camera device 2). The outer contour of this region R1 (the boundary between region R1 and region R2) roughly coincides with the contour of object P in image data D1 (the boundary between object P and background object 6). Therefore, each pixel of image data D1 corresponding to region R1 can be considered as each pixel corresponding to object P, and each pixel of image data D1 corresponding to region R2 in depth map D2a can be considered as each pixel corresponding to background object 6. Then, by comparing image data D1 and depth map D2, the regions of object P and background object 6 in image data D1 can be classified.

[0042] In step S160, the arithmetic unit 3 performs data processing to extract the contour of the object P from the image data D1. Specifically, the arithmetic unit 3 considers the outer perimeter of region R1 in the image data D1 as the contour of the object P, sets the data (pixel values) outside that contour to zero (black), and retains the data (pixel values) inside that contour, thereby outputting the extracted image data D3 to the output unit 11.

[0043] In the extracted image data D3 illustrated in Figure 11, the contour of object P from image data D1 is extracted. Image processing such as a noise reduction filter may be applied to this extracted image data D3. Alternatively, the extracted image data D3 may be extracted from image data D1 that has undergone the pre-image processing in step S120 described above.

[0044] As described above, in this embodiment, the depth map D2 represents the degree of depth (distance from the monocular camera device 2) of the object P and background object 6 in the image data D1 as the difference (contrast) of the pixel values ​​of each corresponding pixel. Furthermore, noise N present in the image data D1 is removed when the depth map D2 is generated. In other words, even if the contrast between the object P and background object 6 in the image data D1 is not clear, the contrast (luminance difference) between the object P and background object 6 becomes clear in the depth map D2. Then, by classifying the image data D1 into regions using the pixel values ​​of each pixel in this depth map D2, the boundary between the object P and background object 6 in the image data D1 can be identified with high accuracy, and the contour of the object P can be extracted with high accuracy.

[0045] Furthermore, in this embodiment, image data D1 of the object P is acquired by a monocular camera device 2, and the contour of the object P is extracted by performing predetermined data processing on the acquired image data D1 by a processing unit 3. Therefore, the contour of the object P from the image data D1 can be extracted more easily by significantly reducing manual work.

[0046] This embodiment can be used for an object P during an evaluation test, and the characteristics of the object P can be evaluated by analyzing the obtained extracted image data D3. Examples of evaluation tests include known tensile tests and ozone degradation tests. This embodiment is not limited to an object P during an evaluation test, but can also be applied to image data D1 acquired together with a background object 6 of an object P placed in a predetermined location.

[0047] The pre-image processing of image data D1 in step S120 and the post-image processing of depth map D2 in step S140 described above can be omitted as appropriate depending on the noise level in image data D1 and depth map D2. For example, if there is virtually no noise in the generated depth map D2, step S140 can be omitted.

[0048] In the region classification of image data D1 in step S150 described above, the depth map D2 can also be divided into three or more regions with different depths. For example, if a single image data D1 contains two or more objects P at different distances from the monocular camera device 2, the depth map D2 is divided into three or more regions: regions corresponding to the two or more objects P and regions corresponding to the background objects 6. This makes it possible to generate extracted image data D3 for each object P.

[0049] A special background plate can also be placed behind the object P as background object 6. This background plate can be made of a desired material such as resin, cloth, wood, or metal. When a background plate is used as background object 6, the color of its surface 6a (the surface of background object 6 facing the monocular camera device 2) is monochromatic and differs in saturation and brightness from the color of the surface Pa of the object P. The degree of saturation and brightness of the background plate should be such that the outline of the object P can be discerned when the image data D1 is viewed visually. Furthermore, it is desirable that the surface 6a of background object 6 has low reflectivity. In this way, by using a background plate with clear contrast to the object P as background object 6, the contrast between the object P and the background object (background plate) 6 in the acquired image data D1 becomes clearer, making it easier to extract the outline of the object P. However, even when using such a background plate, the shadow of the object P may be cast on the surface of the background plate, and there may be influences from dirt on the surface of the object P or the background plate, so the extraction process described above (S120~S160) to extract the outline of the object P is necessary.

[0050] If the object P has a curved shape in the front-to-back direction, the unfolded image data D4d shown in Figure 14 can be generated based on the contour of the object P extracted from the image data D1d shown in Figure 12 and the depth map D2d shown in Figure 13. This unfolded image data D4d shows the object P with its curved shape flattened in the X direction. An example of generating this unfolded image data D4d is described in detail below.

[0051] The image data D1d illustrated in Figure 12 is acquired by a monocular camera device 2 positioned in front of a tire (towards the viewer in the depth direction of the paper), with the tire being the object P, which is placed vertically on the floor. In Figure 12, the depth direction of the paper is the front-to-back direction (X direction). The image data D1d contains the tire tread surface as the surface Pa of object P. This tread surface is curved convexly toward the front, with its highest point being in the lateral center. The image data D1d also contains multiple objects as background objects 6, including the floor, another tire positioned below object P, the photographer's feet, and a rack.

[0052] The depth map D2d illustrated in Figure 13 is generated by performing the steps S120 to S140 described above on the image data D1d illustrated in Figure 12. When there are multiple objects with different depths as background objects 6, in depth map D2d, the multiple objects other than the target object P are treated as a single background object 6. Therefore, in this depth map D2d, the depths of each object other than the target object P are the same. Because the target object P is curved convexly toward the front, in region R1, the pixel value of each pixel is largest in the lateral central part, and the pixel value gradually decreases from the central part toward both ends in the lateral direction.

[0053] The planar unfolded image data D4d illustrated in Figure 14 is generated based on the contour of the object P in image data D1d and the depth map D2d, showing the object P with its curved shape flattened in the X direction. Specifically, the extracted image data from image data D1d, in which the object P is extracted, is unfolded planarly so that each depth in region R1 of the depth map D2d is approximately the same, thereby generating the planar unfolded image data D4d.

[0054] By generating such planar unfolded image data D4d, if the object P has a curved shape in the front-to-back direction, it becomes possible to grasp the contour of the object P in image data D1d over a wider area. As a result, the amount of work required to grasp the contour of the entire circumferential region of the object P, which is a tire, can be reduced.

[0055] The present invention is not limited to any particular embodiment, and various modifications and changes are possible within the scope of the gist of the invention.

[0056] This disclosure encompasses the following inventions: Invention 1: In an image data contour extraction method, image data of an object positioned in front of a background object at a distance from it is acquired from the front of the object together with the background object using a monocular camera device, and the contour of the object in the image data is extracted by the computing device, Image data contour extraction method, which involves processing the image data using a monocular depth estimation model with the aforementioned computing device to generate a depth map in which the pixel values ​​of each pixel in the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device, and then classifying the image data into regions based on the pixel values ​​of each pixel in the depth map to extract the contour of the object. Invention 2: The method for extracting the contour of an object according to Invention 1, wherein the extraction step for extracting the contour of the object is to divide the depth map into multiple regions by applying threshold processing to the depth map, and the image data is classified into regions based on each of the regions. Invention 3: The method for extracting contours from image data according to invention 1 or 2, wherein the image data is subjected to noise reduction and / or edge enhancement processing before generating the depth map. Invention 4: A method for extracting contours from image data according to any one of inventions 1 to 3, wherein the depth map is subjected to noise reduction processing before the image data is classified into regions. Invention 5: A method for extracting the contour of image data according to any one of Inventions 1 to 4, wherein, in the case where the object has a curved shape in the front-to-back direction, a planar unfolded image data is generated showing the object with its curved shape flattened in the front-to-back direction, based on the extracted contour of the object and the depth map. Invention 6: A method for extracting the contour of image data according to any one of Inventions 1 to 5, wherein when acquiring the image data, a background plate with a clear contrast with the object is used as the background object. Invention 7: An image data contour extraction system comprising a monocular camera device that acquires image data of an object positioned in front of a background object at a distance from the background object, together with the background object, from the front of the object, and a processing unit that extracts the contour of the object from the image data, The computing device has a storage unit that stores a monocular depth estimation model, and performs data processing to generate a depth map in which the pixel values ​​of each pixel of the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device using the monocular depth estimation model, and data processing to classify the image data into regions based on the pixel values ​​of each pixel of the depth map and extract the contour of the object. [Explanation of Symbols]

[0057] 1. Contour extraction system 2 Monocular camera device 3 Computing device 4. Lighting equipment 5 Retention mechanism 6 Background object 7. Arithmetic Processing Unit 8 Main memory 9 Auxiliary storage 10 Input section 11 Output section 20 Monocular depth estimation models 21 Training Data P Object D1, D1d image data D2, D2d depth map D3 Extracted Image Data D4D unfolded image data

Claims

1. In an image data contour extraction method, image data of an object positioned in front of a background object at a distance from it is acquired from the front of the object together with the background object using a monocular camera device, and the contour of the object in the image data is extracted by the computing device, Image data contour extraction method, which involves processing the image data using a monocular depth estimation model with the aforementioned computing device to generate a depth map in which the pixel values ​​of each pixel in the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device, and then classifying the image data into regions based on the pixel values ​​of each pixel in the depth map to extract the contour of the object.

2. The method for extracting the contour of an object according to claim 1, wherein in the extraction step for extracting the contour of the object, the depth map is divided into a plurality of regions by applying threshold processing to the depth map, and the image data is classified into regions based on each of the regions.

3. The method for extracting contours from image data according to claim 1, wherein the image data is subjected to noise reduction and / or edge enhancement processing before generating the depth map.

4. A method for extracting contours from image data according to any one of claims 1 to 3, wherein the depth map is subjected to noise reduction processing before the image data is classified into regions.

5. The method for extracting the contour of image data according to claim 1, wherein, in the case where the object has a curved shape in the front-to-back direction, the method generates planar unfolded image data showing the object with its curved shape flattened in the front-to-back direction, based on the extracted contour of the object and the depth map.

6. The method for extracting the contour of image data according to claim 1, wherein when acquiring the image data, a background plate with a clear contrast with the object is used as the background object.

7. An image data contour extraction system comprising a monocular camera device that acquires image data of an object positioned in front of a background object at a distance from the background object, together with the background object, from the front of the object, and a processing unit that extracts the contour of the object from the image data, The computing device has a storage unit that stores a monocular depth estimation model, and performs data processing to generate a depth map in which the pixel values ​​of each pixel of the image data are converted to pixel values ​​corresponding to the distance from the monocular camera device using the monocular depth estimation model, and data processing to classify the image data into regions based on the pixel values ​​of each pixel of the depth map and extract the contour of the object.

Images