Object confirmation device

Abstract

The object is confirmed accurately. The object confirmation device (10) comprises: a region determination unit (21) that determines a region corresponding to the photographing data in the 3D model data of the object based on a feature extracted from the photographing data obtained by photographing the object; an image generation unit (22) that generates image data of the determined region based on the 3D model data; and a difference determination unit (23) that determines a portion where there is a difference between the photographing data and the image data. There is no need to prepare a frame or the like in order to restrict the position and orientation of the imaging device 30.

Description

Technical Field

[0001] The present invention relates to an object confirmation device. Background Art

[0002] In the past, inspection of an object was sometimes performed based on photographic data captured by a photographing device. For example, Patent Document 1 discloses a technique in which a frame is arranged around an engine of an aircraft, and a photographing device is moved along the frame to photograph the engine. The photographing data is stored together with the position and orientation of the photographing device. Before photographing the engine to be inspected, a reference engine is photographed to generate a main image. Inspection can be performed by comparing photographic data captured at the same position and orientation with the main image.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2017-172984

[0004] In the technology described in Patent Document 1, the main image is not limited to being the same as the design data. If the engine that serves as the reference is not in accordance with the design data, the inspection will become inaccurate. Therefore, for example, it is desirable to develop a technology that accurately performs the process of confirming an object like product inspection. Summary of the invention

[0005] The present invention has been made in view of such a problem, and an object of the present invention is to provide an object checking device capable of accurately checking an object.

[0006] In order to solve the above-mentioned problems, the object confirmation device of the present invention comprises: an area determination unit, which determines the area in the 3D model data of the object corresponding to the shooting data based on the feature part extracted from the shooting data obtained by shooting the object; an image generation unit, which generates image data of the determined area based on the 3D model data; and a difference determination unit, which determines the part where there is a difference between the shooting data and the image data.

[0007] The device may include a difference processing unit that processes the imaging data or the image data so that a portion having a difference can be identified.

[0008] The device may further include a model processing unit that processes the 3D model data so that a region in the 3D model data where the corresponding imaging data exists and a region in the 3D model data where the corresponding imaging data does not exist can be identified.

[0009] Effects of the Invention

[0010] According to the present invention, it is possible to accurately identify an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1This is a functional block diagram schematically showing an object confirmation device.

[0012] Figure 2 A diagram for explaining the photographing situation of an object.

[0013] Figure 3 (a) is an example of image data generated by the image generating unit. Figure 3 (b) is an example of imaging data generated by the imaging device.

[0014] Figure 4 This is a diagram showing an example of processed imaging data.

[0015] Figure 5 This is a diagram showing an example of processed 3D model data.

[0016] Description of the label

[0017] P Object

[0018] 10 Object confirmation device

[0019] 21. Area Determination Department

[0020] 22 Image generation unit

[0021] 23 Difference Determination Department

[0022] 24 Differential Processing Department

[0023] 25 Model Processing Department

[0024] 40 Image data

[0025] 41 Shooting data

[0026] 42 3D model data DETAILED DESCRIPTION

[0027] Please refer to the attached Figure 1 The preferred embodiment of the present invention is described in detail. The dimensions, materials, and other specific values ​​shown in the embodiment are merely examples for easy understanding of the invention, and do not limit the present invention unless otherwise specified. In addition, in this specification and the drawings, for elements having substantially the same function and structure, repeated description is omitted by marking the same reference numerals, and for elements that are not directly related to the present invention, the drawings are omitted.

[0028] Figure 1 FIG. 1 is a functional block diagram schematically showing the object confirmation device 10. The object confirmation device 10 is composed of a computer such as a personal computer or a workstation. Figure 1As shown, the object checking device 10 is configured to include a storage unit 11 , an operation unit 12 , a display unit 13 , and a control unit 20 .

[0029] The storage unit 11 is composed of a RAM, a flash memory, a HDD, etc. The storage unit 11 stores design data of an object described later, that is, 3D model data. The operation unit 12 is composed of, for example, a keyboard and a touch panel superimposed on the display unit 13. The operation unit 12 receives operation input from an administrator. The display unit 13 is composed of a liquid crystal display, an organic EL (Electro Luminescence) display, etc.

[0030] The control unit 20 manages and controls the entire object confirmation device 10 through a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, etc. The control unit 20 also functions as a region determination unit 21, an image generation unit 22, a difference determination unit 23, a difference processing unit 24, and a model processing unit 25.

[0031] The area identification unit 21 obtains photographic data obtained by photographing the object from the photographing device 30. The photographing device 30 is composed of, for example, a digital camera, a smartphone with a camera function, etc. However, the photographing device 30 may be composed of any other device as long as it can photograph the object and generate photographic data.

[0032] Figure 2 This is a diagram for explaining the shooting of the object P. Here, as the object P, a component (web, reinforcement) used in the wing of an aircraft is taken as an example for explanation. However, the object P is not limited to this, and may be other components of the aircraft, or may be a component other than the aircraft. In addition, the object P can be any object (real object), and may be a product (finished product) or a part of a product (unfinished product).

[0033] The operator carries the camera 30 to shoot the object P. The operator continues to shoot the object P multiple times at various positions, various directions, and if there is an optical zoom function, at various angles until there is no unshot part of the object P. However, the camera 30 may be set at the shooting location instead of being carried. The camera 30 outputs the shooting data to the object confirmation device 10 through communication.

[0034] The region determination unit 21 extracts feature parts (feature points) from the captured data. The feature parts are extracted by edge extraction, spot analysis, etc. The feature parts include feature contours (edges) such as acute angles. Then, the region determination unit 21 performs pattern matching of the extracted multiple feature parts and the 3D model data stored in the storage unit 11. Specifically, the region determination unit 21 extracts an arbitrary region of the 3D model data and derives the similarity with the color, shape, positional relationship, etc. of the feature parts. The region determination unit 21 determines the region of the 3D model data with the highest similarity as the region (viewing angle) in the 3D model data of the object P that corresponds to the captured data.

[0035] The extraction of the feature portion by the region specifying unit 21 and the region specifying process by pattern matching are not limited to the above-described processes, and any process used in AR (augmented reality) technology may be used.

[0036] The image generation unit 22 generates image data (2D data) of the specified region based on the 3D model data. The generated image data has a viewing angle substantially the same as that of the imaging data.

[0037] Figure 3 (a) is an example of the image data 40 generated by the image generating unit 22 . Figure 3 (b) is an example of the imaging data 41 generated by the imaging device 30 . The image data 40 is a region corresponding to the imaging data 41 .

[0038] The difference identification unit 23 identifies a portion where there is a difference between the photographing data 41 and the image data 40. Figure 3 (a) Figure 3 In the example shown in (b), the component A present in the image data 40 does not exist in the photographic data 41. In addition, the arrangement of the component B in the image data 40 and the component B in the photographic data 41 is different.

[0039] The difference identification unit 23 extracts a portion where there is a difference between the captured data 41 and the image data 40 (a portion provisionally determined to be different) by, for example, determining the degree of similarity based on pattern matching or a difference value between binarized images.

[0040] Then, the difference specifying unit 23 specifies a component (hereinafter referred to as a target component) corresponding to the extracted portion based on the 3D model data. If the difference specifying unit 23 specifies the target component, it obtains tolerance data of the target component set in the 3D model data. Then, the difference specifying unit 23 determines whether the position of the target component in the imaging data 41 is within the tolerance range indicated by the tolerance data.

[0041] The difference determination unit 23 determines the target component that is not within the tolerance range as a portion where a difference exists (a portion that is finally determined to exist). For example, regarding component A, since it does not originally exist in the imaging data 41, it is definitely outside the tolerance range. In addition, regarding component B, it is determined whether it is within the tolerance range or outside the range based on the position of component B. For example, component B is also outside the tolerance range.

[0042] Here, the components A and B are described as an example. However, the difference determination unit 23 may determine, for example, whether processing is performed on a part of a component and whether the size and shape of a part of a component are correct. In this case, the difference determination unit 23 obtains tolerance data of a portion corresponding to the portion extracted as having a difference (hereinafter referred to as a target portion), and determines whether the shape of the target portion in the imaging data 41 is within the tolerance range indicated by the tolerance data.

[0043] The difference processing unit 24 processes the imaging data 41 or the image data 40 in a state where a portion having a difference can be identified. Here, a case where the difference processing unit 24 processes the imaging data 41 will be described as an example.

[0044] Figure 4 4 is a diagram showing an example of processed imaging data 41. Figure 4 As shown, the difference processing unit 24 superimposes the images of the components A and B on the positions where the components A and B are to be set, for example, in a manner that allows the components to be identified from the surroundings such as the original color, with respect to the photographic data 41. In addition, the photographic data 41 may also process the component B (which has been displaced) that was originally photographed in the photographic data 41 so that it can be identified from the surroundings.

[0045] In addition, the difference processing unit 24 may process the portion of the portion with the difference, which is within the tolerance range but deviates greatly to a threshold value or more, into yellow. In this case, the difference processing unit 24 processes the portion of the portion with the difference that is outside the tolerance range into a more conspicuous color (for example, red).

[0046] The difference processing unit 24 causes the display unit 13 to display the processed image data 41. The operator can recognize the portion of the object P that is not as designed through the image data 41. The operator performs work on the object P so that, for example, the components A and B are as designed.

[0047] In this way, if the object P is photographed by the photographing device 30 carried by the operator, the object P can be inspected in the object confirmation device 10. Therefore, the inspection workload can be reduced. In addition, since the 3D model data is used, a more accurate inspection can be performed compared to the case where the actual object P is photographed as the master data.

[0048] In addition, the tolerance data set in the 3D model data is used. Therefore, there will be no situation where an error is displayed even if it is within the tolerance range. As a result, it is possible to avoid the situation where useless work is generated in order to confirm whether it is within the tolerance range.

[0049] The model processing unit 25 processes the 3D model data in such a manner that it is possible to identify the area in the 3D model data where the corresponding photographing data 41 exists and the area where the corresponding photographing data 41 does not exist. That is, the model processing unit 25 processes the 3D model data in such a manner that it is possible to identify the area in the 3D model data where the photographing data 41 generated by the imaging device 30 exists and the area where the photographing data 41 is not generated.

[0050] Figure 5 4 is a diagram showing an example of processed 3D model data 42. Figure 5 As shown by the cross-hatching, the model processing unit 25 adds a primary color or the like to a region of the 3D model data 42 for which the corresponding imaging data 41 already exists so as to be distinguishable from the surroundings. The model processing unit 25 displays the processed 3D model data 42 on the display unit 13 .

[0051] Here, the model processing unit 25 adds a color to an area of ​​the 3D model data 42 for which corresponding photographic data 41 already exists. However, the model processing unit 25 may add a color such as a primary color to an area of ​​the 3D model data 42 for which corresponding photographic data 41 does not yet exist, so that the area can be distinguished from the surrounding area.

[0052] By processing the 3D model data 42 , the operator can easily grasp the region to be imaged in the object P. In addition, the possibility of forgetting to image a part of the object P can be reduced.

[0053] Please refer to the attached Figure 1 While the preferred embodiment of the present invention is described, the present invention is not limited to the embodiment. It is clear that a person skilled in the art can think of various variations or modifications within the scope of the claims, and it is clear that these variations or modifications also belong to the technical scope of the present invention.

[0054] In the above-mentioned embodiment, the case where the difference processing unit 24 is provided is described. In this case, the operator can easily understand the difference between the captured data 41 and the image data 40. However, the difference processing unit 24 is not an essential structure. The object confirmation device 10 may also include, for example, a speaker, and the names of the parts A and B, etc., are output (notified) from the speaker to convey the existence of the difference to the operator.

[0055] In addition, in the above-mentioned embodiment, the case where the pattern processing unit 25 is provided is described. However, the pattern processing unit 25 is not an essential structure.

[0056] In the above-mentioned embodiment, the case where the object confirmation device 10 is used for the inspection of the object P is described. However, the object confirmation device 10 is not limited to the inspection (inspection process), and can also be used in the autonomous confirmation operation (confirmation operation of the presence or absence of component installation, position, etc.) in the manufacturing operation of the object P.

[0057] In addition, there are provided a program for causing a computer to function as the object confirmation device 10, and a storage medium such as a floppy disk, magneto-optical disk, ROM, CD, DVD, BD, etc. on which the program is recorded and which can be read by the computer. Here, the program refers to data processing means recorded in any language and recording method.

[0058] Industrial Applicability

[0059] The present invention can be used in an object checking device.

Claims

1. An object confirmation device, comprising: a region specifying unit that specifies a region corresponding to the photographing data in the 3D model data of the object based on a feature extracted from the photographing data obtained by photographing the object; an image generating unit, which generates image data of the determined area according to the 3D model data; as well as The difference specifying unit specifies a portion where there is a difference between the photographing data and the image data, and determines whether a position of a target component corresponding to the portion where there is the difference is within a tolerance range indicated by tolerance data of the target component.

2. The object confirmation device according to claim 1, wherein: A difference processing unit is provided for processing the imaging data or the image data so that a portion where the difference exists can be identified.

3. The object checking device according to claim 1 or 2, wherein: A model processing unit is provided for processing the 3D model data so that a region in the 3D model data where the corresponding imaging data exists and a region in the 3D model data where the corresponding imaging data does not exist can be identified.

Images