Component identification device and component identification method

Abstract

To provide a component identification device capable of performing component identification processing easily and accurately.SOLUTION: A component identification device comprises a storage unit 201, an imaging data acquisition unit 202a, an image processing unit 202c, a collation unit 202e, and an input / output device 21 as an output unit. The storage unit 201 stores, for each type of component, identification information and registered point cloud data representing a contour of a shape as a point cloud in association with each other. The imaging data acquisition unit 202a acquires imaging data of an identification component being a target of identification processing. The image processing unit 202c extracts the contour of the shape of the component from the imaging data of the identification component, and generates identification point cloud data representing the extracted contour as a point cloud. The collation unit 202e acquires identification information of the type corresponding to the registered point cloud data whose degree of match with the identification point cloud data is equal to or greater than a prescribed value from the storage unit. The input / output device 21 outputs the identification information acquired by the collation unit 202e as the identification information of the type corresponding to the identification component.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a part identification device and a part identification method. [Background technology]

[0002] Industrial equipment used in a variety of fields is made up of multiple different parts. For example, an aircraft engine is made up of approximately 500 different types of small parts, such as bolts and nuts, that are assembled into one engine. Therefore, in aircraft engine maintenance factories, it is necessary to identify these parts and specify their part numbers.

[0003] In view of this, there is a technique for using a laser displacement meter to acquire point cloud data indicating the surface shape of a part to be identified, and collating the acquired point cloud data with point cloud data relating to pre-stored candidate parts to determine whether the part to be identified is this candidate part (see Patent Document 1). By using this technique, it is possible to perform the part identification process with high accuracy. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-174299 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned technology has the problem that because a laser displacement meter is used to acquire point cloud data indicating the surface shape of the part to be identified, the equipment is expensive and difficult to introduce.

[0006] Therefore, an object of the present disclosure is to provide a part identification device and a part identification method that are capable of easily and accurately executing part identification processing. [Means for solving the problem]

[0007] A part identification device according to one aspect of the present disclosure includes a memory unit that associates identification information with registered point cloud data in which an outline of the shape is indicated by a point cloud and stores them as attribute information for each type of part; an imaging data acquisition unit that acquires imaging data of an identified part to be subjected to identification processing; an image processing unit that extracts the outline of the shape of the identified part from the imaging data of the identified part and generates identification point cloud data in which the extracted outline is indicated by a point cloud; a matching unit that acquires from the memory unit registered point cloud data that has a degree of match with the identification point cloud data equal to or greater than a predetermined value and acquires identification information of a type corresponding to the acquired registered point cloud data; and an output unit that outputs the identification information acquired by the matching unit as identification information of a type corresponding to the identified part.

[0008] The matching unit may acquire, from among the registration point cloud data stored in the memory unit, registration point cloud data having a point group such that, when superimposed on the identification point cloud data, the integrated value of the distance from each point in the identification point cloud data is less than or equal to a predetermined value, as registration point cloud data having a degree of similarity with the identification point cloud data that is greater than or equal to a predetermined value.

[0009] The matching unit may determine that the two points match and calculate the integrated value if a distance from a specified point in the identification point cloud data to a specified point in the identification point cloud data is within a preset tolerance when the identification point cloud data and the registration point cloud data are overlaid.

[0010] The registration point cloud data and the identification point cloud data may be configured as two-dimensional image data.

[0011] The matching unit may obtain registered point cloud data having a degree of match with the identification point cloud data of a predetermined value or more by matching an image data portion within a predetermined distance from the contour of the identified part represented by the point cloud in the identification point cloud data with an image data portion within a predetermined distance from the contour of a specified part represented by the point cloud in the registration point cloud data.

[0012] The system may further include a weight data acquisition unit that acquires identification weight data, which is weight data of the identified parts, wherein the memory unit stores registered weight data, which is weight data for each type of the parts, in association with each corresponding identification information, and the output unit outputs the acquired type identification information by sorting the registered weight data of the types corresponding to each of these identification information in order of proximity to the identification weight data.

[0013] The system may further include a height data acquisition unit that acquires identification measurement data indicating the height of the identified parts, wherein the memory unit stores registered measurement data, which is height data for each type of the parts, in association with the corresponding identification information, and the matching unit may further acquire, from the acquired identification information, identification information in which the degree of match between the corresponding registered measurement data and the identification measurement data is equal to or greater than a predetermined value.

[0014] The system may further include a material data acquisition unit that acquires identification material data relating to the material of the identified part, wherein the memory unit stores registered material data, which is material data for each type of the part, in correspondence with the corresponding identification information, and the matching unit may further acquire, from the acquired identification information, identification information having a degree of match between the corresponding registered material data and the identification material data that is equal to or greater than a predetermined value.

[0015] The memory unit may store, for each type of part, appearance information indicating appearance features other than a contour, in association with the corresponding identification information, the matching unit may further obtain appearance information corresponding to the obtained identification information, and the output unit may output the appearance information obtained by the matching unit as appearance information corresponding to the identified part.

[0016] The imaging data acquisition unit may acquire imaging data of the identification part photographed by a plurality of camera devices, and the image processing unit may generate identification point cloud data of the identification part using the plurality of imaging data acquired by the imaging data acquisition unit.

[0017] The imaging data acquisition unit may acquire imaging data of a registered part whose model is known, and the image processing unit may further include a registration unit that extracts a contour of the shape of the registered part from the imaging data of the registered part, generates point cloud data indicating the extracted contour as a point cloud as registered point cloud data, acquires identification information of the model of the registered part, and associates the acquired identification information with the registered point cloud data generated by the image processing unit and stores it in the memory unit as the attribute information.

[0018] In one aspect of the present disclosure, a part identification method includes a part identification device having a memory unit that stores, for each part type, identification information and registered point cloud data that indicates the outline of the shape of the part in a corresponding manner, the part identification device obtains image data of the identified part to be subjected to identification processing, extracts the outline of the shape of the identified part from the image data of the identified part, generates identification point cloud data that indicates the extracted outline as a point cloud, extracts from the memory unit registered point cloud data that has a degree of match with the identification point cloud data that is equal to or greater than a predetermined value, obtains identification information of the type corresponding to the extracted registered point cloud data, and outputs the obtained identification information as the identification information of the type corresponding to the identified part. Effect of the Invention

[0019] According to the present disclosure, it is possible to provide a part identification device and a part identification method that are capable of easily and accurately executing part identification processing. [Brief description of the drawings]

[0020] [Figure 1] 1 is a side view showing a configuration of a part identifying system using a part identifying device according to a first embodiment. [Diagram 2] 1 is a block diagram showing a configuration of a part identification device according to a first embodiment. [Diagram 3] 4 is a flowchart showing a part registration process executed by the part identification device according to the first embodiment. [Figure 4] 4 is a flowchart showing a part identification process executed by the part identification device according to the first embodiment. [Diagram 5]4 is an example of display information displayed on an input / output device of the part identification device according to the first embodiment. [Figure 6] FIG. 11 is a side view showing the configuration of a part identifying system using a part identifying device according to a second embodiment. [Figure 7] FIG. 11 is a block diagram showing a configuration of a part identification device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] An exemplary embodiment of a parts identification system using a parts identification device will be described below with reference to the drawings. In this embodiment, a parts identification system that performs identification processing for parts used in three types of aircraft engines, namely, model A, model B, and model C, will be described.

[0022] First Embodiment Configuration of part identification system 1A according to the first embodiment 1 is a side view showing the configuration of a part identification system 1A according to the first embodiment. The part identification system 1A includes a table 13, lighting 14, a holder 15, a weight scale 16, and a windshield 17, which are installed on a face plate 12 on the top of a dolly 11, a camera device 19 held by a camera holder 18 fixed to the dolly 11 and installed above the table 13, a part identification device 20A placed on the dolly 11, and an input / output device 21.

[0023] As described below, registration parts to be registered or identification parts to be identified are placed on table 13. The top surface of table 13 is provided with a number of grooves to prevent the parts from rolling when placed on it. Camera device 19 operates with power supplied via DC coupler 22 and AC adapter 23, and captures images of the parts placed on table 13 from above.

[0024] The lighting 14 is installed below the table 13 and illuminates the table 13 and the parts placed on the table 13 from below with power supplied from a lighting power supply 24. The lighting 14 functions as a backlight when the camera device 19 photographs the parts placed on the table 13, as will be described later. The lighting power supply 24 supplies power to the lighting 14. The holder 15 holds the table 13 and the lighting 14.

[0025] Weighing scale 16 is installed under holder 15 and measures the weight of the parts placed on table 13. Windshield 17 is installed on face plate 12 so as to cover table 13, lighting 14, holder 15, and weighing scale 16, and prevents the measurement state from becoming unstable due to the influence of wind when weighing the parts with weighing scale 16.

[0026] Part identification device 20A is connected to weighing scale 16 via USB cable 25, to camera device 19 via HDMI (registered trademark) cable 26 and capture board 27, and to input / output device 21 via HDMI cable 28.

[0027] 2 is a block diagram showing a detailed configuration of the part identification device 20 A. The part identification device 20 A has a storage unit 201 and a CPU 202A.

[0028] The storage unit 201 stores attribute information for each model of each part generated by the CPU 202A for each aircraft engine model, as described below. The storage unit 201 has a model A information storage unit 201a that stores attribute information of parts used in aircraft engines of model A, a model B information storage unit 201b that stores attribute information of parts used in aircraft engines of model B, and a model C information storage unit 201c that stores attribute information of parts used in aircraft engines of model C.

[0029] The CPU 202A (an example of a control unit or a processing unit) is a general-purpose computer equipped with an arithmetic unit, a memory, and an input / output unit. A computer program (part identification program) for causing the CPU 202A to function as a part of the part identification device 20A is installed in the CPU 202A. By executing the computer program, the CPU 202A functions as a plurality of information processing circuits (202a, 202b, 202c, 202d, 202e, 202f, 202g, 202h) equipped in the part identification device 20A.

[0030] Alternatively, the part identification program may be stored in a computer-readable recording medium such as a HDD, an SSD, a USB memory, a CD, or a DVD. The computer-readable recording medium may be, for example, a non-transitory recording medium. The part identification program may also be distributed via a communication network.

[0031] In the following, an example will be shown in which the multiple information processing circuits (202a, 202b, 202c, 202d, 202e, 202f, 202g, 202h) included in the part identification device 20A are realized by software. However, it is also possible to prepare dedicated hardware for executing the information processing described below and configure the information processing circuits (202a, 202b, 202c, 202d, 202e, 202f, 202g, 202h). Also, the multiple information processing circuits (202a, 202b, 202c, 202d, 202e, 202f, 202g, 202h) may be configured by individual hardware.

[0032] The CPU 202A includes an imaging data acquisition unit 202a, a weight data acquisition unit 202b, an image processing unit 202c, a registration unit d, a matching unit 202e, an output information generation unit 202f, an attribute information acquisition unit 202g, and a measurement calibration unit 202h.

[0033] The imaging data acquisition unit 202a acquires two-dimensional imaging data of the part captured by the camera device 19. The weight data acquisition unit 202b acquires weight data of the part measured by the weighing scale 16.

[0034] The image processing unit 202c extracts the contour of the shape of the part from the imaging data acquired by the imaging data acquisition unit 202a, and generates point cloud data, which is two-dimensional image data indicating the extracted contour as a point cloud.

[0035] When the image processing unit 202c generates registered point cloud data, which is point cloud data of a registered part, the registration unit 202d associates this registered point cloud data with the registered weight data, which is the weight data acquired by the weight data acquisition unit 202b, and identification information for each type of part input from the input / output device 21, and stores them in the memory unit 201 as attribute information of the registered part.

[0036] When the image processing unit 202c generates identification point cloud data that is point cloud data of an identified part, the matching unit 202e matches the generated identification point cloud data with each registered point cloud data stored in the storage unit 201. The matching unit 202e acquires, from the registered point cloud data stored in the storage unit 201, registered point cloud data whose degree of match with the generated identification point cloud data is equal to or greater than a predetermined value, and acquires type identification information corresponding to the acquired registered point cloud data.

[0037] The output information generating unit 202f generates output information that presents the type identification information acquired by the collating unit 202e as a candidate for type identification information corresponding to the identified part, and transmits the output information to the input / output device 21.

[0038] When the input / output device 21 requests attribute information of a specific part, the attribute information acquisition unit 202 g acquires the attribute information of the corresponding part from the storage unit 201 and transmits it to the input / output device 21 .

[0039] As a pre-processing before the CPU 202A executes the part registration processing and the part identification processing, the measurement calibration unit 202h acquires the actual size of the subject part, which corresponds to one pixel of the imaging data generated by the camera device 19. The measurement calibration unit 202h executes this processing every time the distance between the camera device 19 and the table 13 is changed.

[0040] The input / output device 21 is configured, for example, with a touch panel, and inputs the contents of operations performed by a user to the part identifying device 20A, and outputs information transmitted from the part identifying device 20A.

[0041] <Operation of the part identification system 1A according to the first embodiment> In this embodiment, (a) part registration processing and (b) part identification processing executed by the part identification system 1A will be described. (a) Part registration processing is processing for registering attribute information, such as shape data, weight data, and model identification information, for each model of each part used in an aircraft engine. (b) Part identification processing is processing for identifying the model of the part to be identified using the attribute information of the model of each part registered in the part registration processing.

[0042] (a) Parts registration process As an example of the part registration process, a process of registering attribute information relating to the model of part X used in an aircraft engine of model A will be described.

[0043] First, the user places part X to be registered, whose model is known (hereinafter referred to as "registered part X"), on table 13, and installs windshield 17 on face plate 12 so as to cover registered part X, table 13, lighting 14, holder 15, and weighing scale 16. The user also turns on lighting power supply 24 to light lighting 14. In this way, the user prepares for part registration processing.

[0044] 3 is a flowchart showing the part registration process executed by the part identification device 20A. After the user prepares for the part registration process as described above, when the user performs an operation to instruct execution of the part registration process through the input / output device 21, information on this operation is transmitted to the part identification device 20A.

[0045] In the part identification device 20A, when information instructing the execution of part registration processing transmitted from the input / output device 21 is acquired ("YES" in S1), the imaging data acquisition unit 202a transmits an image capture instruction to the camera device 19 to cause it to perform image capture.

[0046] The camera device 19 captures an image of the registered part X on the table 13 based on an instruction from the part identification device 20A, and transmits the two-dimensional captured image data to the part identification device 20A. In the part identification device 20A, the captured image data transmitted from the camera device 19 is acquired by the captured image data acquisition unit 202a.

[0047] In addition, in the part identification device 20A, when operation information instructing the execution of part registration processing transmitted from the input / output device 21 is acquired, the weight data acquisition unit 202b transmits a measurement instruction to the weighing scale 16 to execute the weight measurement processing.

[0048] Weighing scale 16 measures the weight of registered part X on table 13 based on instructions from part identification device 20A, and transmits the measured weight data to part identification device 20A as registered weight data. In part identification device 20A, weight data acquisition unit 202b acquires the registered weight data transmitted from weighing scale 16 (S2).

[0049] The image processing unit 202c performs image analysis on the imaging data acquired by the imaging data acquisition unit 202a to extract the contour of the shape of the registered part X (S3). The image processing unit 202c may use an AI function when extracting the contour of the shape of the part from the imaging data.

[0050] The image processing unit 202c can extract the contour of the shape of registered part X from the binary image without requiring complicated parameter settings for image processing, for example by using U2Net (Reference: X. Qin, et. al., “U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection”, Pattern Recognition 106 (2020).), a machine learning model that recognizes the background and extracts the contour of an object.

[0051] Furthermore, by using such a machine learning model, the contour of the shape of registered part X can be extracted with high accuracy even when light is diffusely reflected by the lighting 14 or the lighting above the room in which part identification system 1A is installed.

[0052] Furthermore, by using U2Net, image processing unit 202c can extract the contour of registered part X with high accuracy, removing grooves and the like provided in table 13. Furthermore, by using U2Net, image processing unit 202c can extract the contour of the shape of registered part X with high accuracy even if the lighting environment changes, so it is also possible to configure part identification system 1A without lighting 14. By configuring part identification system 1A without lighting 14, the load on weighing scale 16 is reduced, and the weighing limit can be relaxed.

[0053] The image processing unit 202c generates point cloud data (hereinafter, referred to as "registered point cloud data") configured from two-dimensional image data in which the contour of the registered part X extracted from the captured data is indicated by a point cloud (S4).

[0054] The user inputs identification information of the type of registered part X from the input / output device 21. In the part identification device 20A, the registration unit 202d acquires the identification information of the type of registered part X inputted from the input / output device 21 (S5). The registration unit 202d generates attribute information of the type of registered part X by associating the registered point cloud data generated by the image processing unit 202c, the registered weight data acquired by the weight data acquisition unit 202b, and the identification information of the type acquired from the input / output device 21. The registration unit 202d stores the generated attribute information of the type of registered part X in the model A information storage unit 201a in the storage unit 201 (S6).

[0055] This completes the part registration process for the model of registered part X. Thereafter, in a similar manner, part registration processes are performed for the models of other parts used in model A, the models of various parts used in model B, and the models of various parts used in model C.

[0056] When the user performs an operation to request attribute information of a predetermined part type from the input / output device 21 at any timing, the attribute information acquisition unit 202g acquires the attribute information of the corresponding part from the storage unit 201 and outputs it from the input / output device 21. This allows the user to check the registered contents of the attribute information related to the predetermined part type.

[0057] (b) Parts Identification Processing As an example of the part identification process, a process of identifying the type of part Y will be described. Although it is known that part Y is a part used in an aircraft engine of model A, the type of part Y has not been specified.

[0058] First, the user places part Y to be identified (hereinafter referred to as "identified part Y") on the table 13, and installs the windshield 17 on the face plate 12 so as to cover the identified part Y, the table 13, the light 14, the holder 15, and the weighing scale 16. The user also turns on the light power supply 24 to light up the light 14. In this way, the user prepares for the part identification process.

[0059] 4 is a flowchart showing the part identification process executed by the part identification device 20A. After the user prepares for the part identification process as described above, when the user performs an operation to instruct execution of the part identification process using the input / output device 21, information on this operation is transmitted to the part identification device 20A.

[0060] In the part identification device 20A, when information instructing the execution of part identification processing transmitted from the input / output device 21 is acquired ("YES" in S11), the imaging data acquisition unit 202a transmits an image capture instruction to the camera device 19 to cause it to perform image capture.

[0061] The camera device 19 captures an image of the registered part X on the table 13 based on an instruction from the part identification device 20A, and transmits the two-dimensional captured image data to the part identification device 20A. In the part identification device 20A, the captured image data transmitted from the camera device 19 is acquired by the captured image data acquisition unit 202a.

[0062] In addition, in the part identification device 20A, when operation information instructing the execution of the part identification process transmitted from the input / output device 21 is acquired, the weight data acquisition unit 202b transmits a measurement instruction to the weighing scale 16 to execute the weight measurement process.

[0063] The weighing scale 16 measures the weight of the identified part Y on the table 13 based on an instruction from the part identification device 20A, and transmits the measured weight data to the part identification device 20A as identified weight data. In the part identification device 20A, the weight data acquisition unit 202b acquires the identified weight data transmitted from the weighing scale 16 (S12).

[0064] The image processing unit 202c performs image analysis on the captured image data acquired by the captured image data acquisition unit 202a to extract the contour of the shape of the identification part X (S13).

[0065] The image processing unit 202c generates point cloud data (hereinafter, referred to as "identification point cloud data") composed of two-dimensional image data in which the contour of the identification part Y extracted from the imaging data is indicated by a point cloud (S14).

[0066] When the image processing unit 202c generates the discrimination point cloud data, the matching unit 202e matches the generated discrimination point cloud data with each registered point cloud data stored in the model A information storage unit 201a of the storage unit 201. The matching unit 202e uses information acquired when the discrimination point cloud data was generated in the measurement calibration unit 202h and information acquired when the registered point cloud data was generated to match the sizes of both point cloud data and executes a matching process.

[0067] The image processing unit 202c performs matching processing using point cloud data that indicates the contours of the parts as a point cloud, and can therefore quickly execute matching processing with the identification point cloud data even if a large amount of registered point cloud data is stored in the storage unit 201. For example, even if about 500 pieces of registered point cloud data are stored in the storage unit 201, the image processing unit 202c can execute matching processing with the identification point cloud data in about a few seconds.

[0068] The matching unit 202e may also perform matching by best-fitting by matching the generated discrimination point cloud data with the predetermined registered point cloud data at a position where the sum of errors between them is the smallest. By performing matching in this way, the matching unit 202e can execute the matching process more efficiently.

[0069] The matching unit 202e may also perform matching using the Boundary-IoU method (reference: B. Cheng, et. al., “Boundary IoU: Improving Object-Centric Image Segmentation Evaluation”, Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 15334-15342 (2021)) using only the image data portion within a predetermined distance from the contour of the identified part Y shown by the point cloud in the identification point cloud data and the image data portion within a predetermined distance from the contour of the registered part shown by the point cloud in each registered point cloud data. By performing matching using this method, even if the difference in shape due to the difference in type between the identification point cloud data and the registered point cloud data is small, the matching unit 202e can accurately detect this difference. For example, even if the head shape is the same between bolts of different types but only the length differs by about several millimeters, the matching unit 202e can detect this difference and recognize that these types are different.

[0070] The matching unit 202e acquires from the memory unit 201 registered point cloud data having a point group whose integrated value of the distance from each point in the identification point cloud data is less than or equal to a predetermined value when superimposed on the identification point cloud data, as registered point cloud data whose degree of match with the identification point cloud data is greater than or equal to a predetermined value (S15).

[0071] In this case, if the distance from a specified point in the identification point cloud data to a specified point in the registration point cloud data is within a preset tolerance, the matching unit 202e may determine that these two points match and calculate the integrated value described above.

[0072] As a result, when identifying parts during maintenance work after the use of an aircraft engine, even if there is a slight change in shape in some of the parts due to wear or seizure, this change in shape can be treated as noise, making it possible to smoothly carry out the part identification process.

[0073] The matching unit 202e acquires model identification information corresponding to the acquired registered point cloud data from the storage unit 201 (S16). The output information generating unit 202f generates display information including the identification information acquired by the matching unit 202e as a candidate for information indicating the model of the identified part Y.

[0074] At this time, there are cases where there are multiple registered point cloud data whose degree of coincidence with the identification point cloud data is equal to or greater than a predetermined value, and the output information generating unit 202f acquires multiple types of identification information based on this. In such cases, the output information generating unit 202f rearranges the multiple types of identification information as a first candidate, a second candidate, etc., in order of the corresponding registered weight data being closest to the identification weight data of the identified part Y, and generates display information.

[0075] The output information generating unit 202f causes the generated display information to be displayed on the input / output device 21 (S17). Fig. 5 is an example of the display information displayed on the input / output device 21.

[0076] The display information shown in FIG. 5 includes a model designation button 211, an execution instruction button 212, an identification imaging data display area 213, a binary image display area 214, a superimposition data display area 215, a first registered imaging data display area 216, a first attribute information display area 217, a second registered imaging data display area 218, a second attribute information display area 219, a third registered imaging data display area 220, and a third attribute information display area 221.

[0077] The model designation button 211 is a button for designating a model in which the identification part Y is used. The execution instruction button 212 is a button for instructing execution of the part identification process. The identification image data display area 213 is a display area for displaying the image data of the identification part Y. The binary image display area 214 is a display area for displaying the binary image data of the identification part Y. The superimposed data display area 215 is a display area for displaying superimposed data in which the identification point cloud data of the identification part Y and the registered point cloud data of the registered part X, which is a registered part, are superimposed. The first registered image data display area 216 is a display area for displaying image data corresponding to the first candidate model. The first attribute information display area 217 is a display area for displaying information related to the first candidate model. The second registered image data display area 218 is a display area for displaying image data corresponding to the second candidate model. The second attribute information display area 219 is a display area for displaying information related to the second candidate model. The third registration imaging data display area 220 is a display area for displaying imaging data corresponding to the third candidate model. The third attribute information display area 221 is a display area for displaying information related to the third candidate model.

[0078] The first attribute information display area 217 displays, for example, the identification information of the first candidate type, the degree of agreement between the registered point cloud data and the identification point cloud data of the first candidate type, the degree of agreement between the registered weight data and the identification weight data of the first candidate type, the registered weight data of the parts of the first candidate type, etc. The second attribute information display area 219 similarly displays the identification information of the second candidate type, the degree of agreement between the registered point cloud data and the identification point cloud data of the second candidate type, the degree of agreement between the registered weight data and the identification weight data of the second candidate type, the registered weight data of the parts of the second candidate type, etc. The third attribute information display area 221 similarly displays the identification information of the third candidate type, the degree of agreement between the registered point cloud data and the identification point cloud data of the third candidate type, the degree of agreement between the registered weight data and the identification weight data of the third candidate type, the registered weight data of the parts of the third candidate type, etc.

[0079] The user can identify the type of identification part Y by referring to the displayed information.

[0080] Second Embodiment 6 is a side view showing the configuration of a part identification system 1B according to the second embodiment. The part identification system 1B further includes a height measurement sensor 29 and a magnetic sensor 30 in addition to the configuration of the part identification system 1A. The height measurement sensor 29 measures the height of the registered part or identified part to be processed. The magnetic sensor 30 identifies the material of the registered part or identified part to be processed.

[0081] 7 is a block diagram showing the configuration of a part identification device 20B in a part identification system 1B according to the second embodiment. The part identification device 20B has the same configuration as the above-mentioned part identification device 20A except that it includes a height data acquisition unit 202i and a material data acquisition unit 202j, so a detailed description of the same parts will be omitted.

[0082] The height data acquiring unit 202i acquires measurement data from the height measuring sensor 29. The material data acquiring unit 202j acquires, from the magnetic sensor 30, information on the identified material.

[0083] In the part identification system 1B configured in this way, the height measurement sensor 29 measures the height of the registered part when the part registration process is executed, and the height data acquisition unit 202i acquires this measurement data and sends it to the registration unit 202d. The registration unit 202d stores this measurement data in the storage unit 201 as registered measurement data, including it in the attribute information.

[0084] During part registration processing, the magnetic sensor 30 acquires material data indicating the characteristics of the material of the registered part, and the material data acquisition unit 202j acquires this material data and sends it to the registration unit 202d. The registration unit 202d stores this material data in the storage unit 201 as registered material data, including it in the attribute information.

[0085] When the part identification process is executed, the height measurement sensor 29 measures the height of the identified part Y, and the height data acquisition unit 202i acquires this measurement data and sends it to the collation unit 202e. In addition, the magnetic sensor 30 acquires material data indicating the characteristics of the material of the identified part Y, and the material data acquisition unit 202j acquires this material data and sends it to the collation unit 202e.

[0086] The collation unit 202e acquires the measurement data sent from the height data acquisition unit 202i and the material data sent from the material data acquisition unit 202j. After acquiring a plurality of types of identification information corresponding to the registered point cloud data acquired by image processing, the collation unit 202e further acquires identification information with a high accuracy rate from among the acquired identification information by using at least one of the measurement data and the material data.

[0087] Specifically, when measurement data is used, the matching unit 202e further acquires, from among a plurality of types of identification information corresponding to the registered point cloud data acquired by image processing, identification information having a degree of agreement between the corresponding registered measurement data and the identification measurement data that is equal to or greater than a predetermined value. When material data is used, the matching unit 202e further acquires, from among a plurality of types of identification information corresponding to the registered point cloud data acquired by image processing, identification information having a degree of agreement between the corresponding registered material data and the identification material data that is equal to or greater than a predetermined value.

[0088] Then, the output information generating unit 202f generates display information including the identification information acquired by the collating unit 202e as information indicating the type of the identification part Y.

[0089] As a result, even if multiple pieces of identification information are obtained by comparing shape data as identification information for the type of an identified part, it is possible to narrow down the identification information with a higher accuracy rate based on at least either the height measurement data or the material data of the part and present it to the user.

[0090] In the first and second embodiments described above, the storage unit 201 may store, for each part type, appearance information indicating appearance features other than the outline in association with the corresponding identification information, and the collation unit 202e may further acquire appearance information corresponding to the acquired identification information. An appearance feature other than the outline is, for example, a shape feature that does not affect the outline and is visible, such as a hole being drilled at a predetermined position of the part. When the collation unit 202e acquires the appearance information in this manner, the output information generation unit 202f generates display information so as to display the acquired appearance information in the attribute information display area corresponding to the identified part Q.

[0091] As a result, even if multiple pieces of information are obtained as identification information for the type of an identification part, the user can accurately recognize the identification information for the type of the identification part based on the visible characteristics of the exterior shape from among the multiple pieces of identification information presented.

[0092] In the above-described first and second embodiments, the part identification systems 1A and 1B are configured using one camera device 19, but the part identification systems may be configured using a plurality of camera devices, for example, two camera devices installed in the upward and lateral directions. In this case, the imaging data acquisition unit 202a acquires imaging data of the registered parts and the identified parts photographed by the plurality of camera devices, and the image processing unit 202c generates registration point cloud data and identification point cloud data using the plurality of imaging data acquired by the imaging data acquisition unit 202a.

[0093] By using a plurality of pieces of image data captured by a plurality of camera devices in this manner, it is possible to generate registration point cloud data and identification point cloud data with higher accuracy.

[0094] [Effects of the embodiment] As described above in detail, the part identification device according to the present disclosure includes a storage unit, an imaging data acquisition unit, an image processing unit, a matching unit, and an output unit. The storage unit stores, for each type of part, identification information and registered point cloud data in which the contour of the shape is represented by a point cloud in association with each other. The imaging data acquisition unit acquires imaging data of an identified part to be subjected to identification processing. The image processing unit extracts the contour of the shape of the identified part from the imaging data of the identified part, and generates identification point cloud data in which the extracted contour is represented by a point cloud. The matching unit acquires, from the storage unit, registered point cloud data whose degree of agreement with the identification point cloud data is equal to or greater than a predetermined value, and acquires identification information of the type corresponding to the acquired registered point cloud data. The output unit outputs the identification information acquired by the matching unit as identification information of the type corresponding to the identified part.

[0095] In this way, the part identification device can easily and accurately identify the identified part by identifying registered point cloud data corresponding to the identified part from the registered registered point cloud data and acquiring identification information of the part type corresponding to the identified registered point cloud data. In this case, since the registered point cloud data and the identification point cloud data are composed of point cloud data that indicates the contours of the parts as point clouds, the processing load of the registration point cloud data acquisition process by the matching unit can be reduced.

[0096] In addition, the storage unit only needs to store one piece of registered point cloud data for one part model, and there is no need to manage a huge amount of information like in existing identification devices that use AI to match registered parts, so there is an advantage that maintenance is not time-consuming. Also, when there is a change in the configuration of an aircraft engine and new registered point cloud data is added, processing can be easily performed.

[0097] In the present disclosure, the matching unit may acquire, from among the registration point cloud data stored in the storage unit, registration point cloud data having a point cloud whose integrated value of the distance from each point in the identification point cloud data is equal to or less than a predetermined value when superimposed on the identification point cloud data, as registration point cloud data whose degree of agreement with the identification point cloud data is equal to or greater than a predetermined value. This makes it possible to acquire registration point cloud data corresponding to the identification point cloud data with high accuracy.

[0098] In addition, in the present disclosure, when the identification point cloud data and the registration point cloud data are superimposed, if the distance from a specified point in the identification point cloud data to a specified point in the identification point cloud data is within a preset tolerance, the matching unit may determine that these two points match and calculate the above-mentioned integrated value.

[0099] As a result, when identifying parts during maintenance work after the use of an aircraft engine, even if there is a slight change in shape in some of the parts due to wear or seizure, this change in shape can be treated as noise, making it possible to smoothly carry out the part identification process.

[0100] In the present disclosure, the registration point cloud data and the identification point cloud data may be configured as two-dimensional image data. This allows the matching unit to perform matching processing in a short time even if a large amount of registration point cloud data is stored in the storage unit, and makes it possible to quickly provide the user with identification information of the type corresponding to the identification part.

[0101] In addition, in the present disclosure, the matching unit may obtain registered point cloud data that has a degree of match with the identification point cloud data of a predetermined value or more by matching an image data portion within a predetermined distance from the contour of an identified part represented by the point cloud in the identification point cloud data with an image data portion within a predetermined distance from the contour of a specified part represented by the point cloud in the registered point cloud data.

[0102] This allows the matching unit to accurately detect the difference between the identification point cloud data and the registered point cloud data even if the difference in shape due to the difference in model is small.

[0103] In addition, in the present disclosure, the output unit may output the acquired model identification information in order of the weight data of the model corresponding to each of the identification information being closest to the weight data of the identified part. In this way, even if multiple pieces of information are acquired as identification information of the model of the identified part, the identification information with the highest accuracy rate can be presented to the user.

[0104] In addition, the present disclosure may further include a height data acquisition unit that acquires identification measurement data indicating the height of the identified part, wherein the memory unit stores registered measurement data, which is height data for each type of part, in association with each corresponding identification information, and the matching unit further acquires, from the acquired identification information, identification information in which the degree of match between the corresponding registered measurement data and the identification measurement data is equal to or greater than a predetermined value.

[0105] As a result, even if multiple pieces of identification information are obtained by comparing shape data as identification information for the type of identified part, it is possible to narrow down the identification information with a higher accuracy rate based on the part height data and present it to the user.

[0106] In addition, the present disclosure may further include a material data acquisition unit that acquires identification material data regarding the material of the identified part, wherein the memory unit stores registered material data, which is material data for each type of part, in association with each corresponding identification information, and the matching unit may further acquire, from the acquired identification information, identification information whose corresponding registered material data and the identification material data have a degree of match equal to or greater than a predetermined value.

[0107] As a result, even if multiple pieces of identification information are obtained by comparing data related to the shape of the identified part as identification information for the type of the part, it is possible to narrow down the identification information with a higher accuracy rate based on data related to the part's material and present it to the user.

[0108] Furthermore, in the present disclosure, the memory unit may store, for each type of part, appearance information indicating appearance features other than the contour, in association with the corresponding identification information, the matching unit may further acquire appearance information corresponding to the acquired identification information, and the output unit may output the appearance information acquired by the matching unit as appearance information corresponding to the identified part.

[0109] As a result, even if a plurality of pieces of information are acquired as the identification information of the type of the identification part, the user can accurately recognize the identification information of the type of the identification part from the plurality of pieces of identification information presented.

[0110] In addition, in the present disclosure, the imaging data acquisition unit may acquire imaging data of the parts to be identified captured by a plurality of camera devices, and the image processing unit may generate identification point cloud data of the identified parts using the plurality of imaging data acquired by the imaging data acquisition unit. This makes it possible to generate identification point cloud data with even higher accuracy.

[0111] In the present disclosure, the imaging data acquisition unit may acquire imaging data of a registered part whose model is known, the image processing unit may extract an outline of the shape of the registered part from the imaging data of the registered part, generate point cloud data representing the extracted outline as a point cloud as registered point cloud data, and the CPU 202A may further include a registration unit that acquires identification information of the model of the registered part, associates the acquired identification information with the registered point cloud data generated by the image processing unit, and stores the associated information in the storage unit as attribute information. This allows both the part registration process and the part identification process to be executed by a single device.

[0112] Although some embodiments have been described, the embodiments can be modified or modified based on the above disclosure. All components of the above embodiments and all features described in the claims may be individually extracted and combined as long as they are not mutually inconsistent.

[0113] This disclosure can contribute, for example, to Goal 9 of the Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation." [Explanation of symbols]

[0114] 1A, 1B Part Identification System 11 Trolley 12 face plate 13 Tables 14. Lighting 15 Holder 16 Weight scale 17 Windshield 18 Camera holder 19 Camera Equipment 20A, 20B Parts Identification Device 21 Input / Output Devices 22 DC Coupler 23 AC adapter 24 Lighting power supply 25 USB cable 26, 28 HDMI cable 27 Capture Board 29 Height measurement sensor 30 Magnetic Sensor 201 Storage section 201a Model A information storage section 201b Model B information storage section 201c Model C information storage section 202A, 202B CPU 202a Imaging data acquisition unit 202b Weight data acquisition unit 202c Image processing unit 202d Registration Department 202e Matching section 202f Output information generation unit 202g Attribute information acquisition section 202h Measurement and Calibration Section 202i Height data acquisition unit 202j Material Data Acquisition Section 211 Model selection button 212 Execution instruction button 213 Identification imaging data display area 214 Binary image display area 215 Overlay data display area 216, 218, 220 Registered image data display area 217, 219, 221 Attribute information display area

Claims

1. a storage unit that associates identification information with registered point cloud data that indicates an outline of a shape of a part as a point cloud and stores the associated identification information as attribute information for each part type; an image data acquisition unit for acquiring image data of an identification part to be subjected to an identification process; an image processing unit that extracts an outline of a shape of the identification part from the image data of the identification part and generates identification point cloud data that indicates the extracted outline as a point cloud; a matching unit that acquires from the storage unit registered point cloud data having a degree of coincidence with the identification point cloud data equal to or greater than a predetermined value, and acquires model identification information corresponding to the acquired registered point cloud data; an output unit that outputs the identification information acquired by the collation unit as identification information of a type corresponding to the identification part; A part identification device comprising:

2. 2. The part identification device according to claim 1, wherein the matching unit acquires, from among the registered point cloud data stored in the memory unit, registered point cloud data having a point cloud such that, when superimposed on the identification point cloud data, an integrated value of distances from each point in the identification point cloud data is equal to or less than a predetermined value, as registered point cloud data having a degree of match with the identification point cloud data that is equal to or greater than a predetermined value.

3. 3. The part identification device according to claim 2, wherein when the identification point cloud data and the registered point cloud data are superimposed, if a distance from a predetermined point in the identification point cloud data to a predetermined point in the identification point cloud data is within a preset tolerance, the matching unit determines that these two points match and calculates the integrated value.

4. The registration point cloud data and the identification point cloud data are composed of two-dimensional image data.

2. The part identification apparatus of claim 1.

5. 2. The part identification device according to claim 1, wherein the matching unit obtains registered point cloud data having a degree of match with the identification point cloud data that is a predetermined value or greater by matching an image data portion within a predetermined distance from the contour of the identified part represented by the point cloud in the identification point cloud data with an image data portion within a predetermined distance from the contour of a specified part represented by the point cloud in the registration point cloud data.

6. a weight data acquisition unit for acquiring identification weight data, which is weight data of the identified part; the storage unit stores registered weight data, which is weight data for each model of the part, in association with the corresponding identification information; 2. The part identification device according to claim 1, wherein the output unit outputs the acquired model identification information in order of proximity of registered weight data of the model corresponding to each of the identification information to the identified weight data.

7. a height data acquisition unit that acquires identification measurement data indicating a height of the identified part, the storage unit stores the registered measurement data, which is height data for each model of the part, in association with the corresponding identification information; 2. The part identification device according to claim 1, wherein the collation unit further acquires, from the acquired identification information, identification information in which the degree of match between the corresponding registered measurement data and the identified measurement data is equal to or greater than a predetermined value.

8. a material data acquisition unit that acquires identification material data relating to a material of the identification part, the storage unit stores registered material data, which is material data for each type of the part, in association with corresponding identification information; 2. The part identification device according to claim 1, wherein the collation unit further acquires, from the acquired identification information, identification information in which the degree of match between the corresponding registered material data and the identification material data is equal to or greater than a predetermined value.

9. the storage unit stores, for each type of the part, appearance information indicating characteristics of the appearance other than a contour in association with corresponding identification information; The matching unit further acquires appearance information corresponding to the acquired identification information, The part identification device according to claim 1 , wherein the output unit outputs the appearance information acquired by the comparison unit as the appearance information corresponding to the identification part.

10. The imaging data acquisition unit acquires imaging data of the identification part photographed by a plurality of camera devices, The part identification device according to claim 1 , wherein the image processing unit generates identification point cloud data of the identification part by using the plurality of pieces of image data acquired by the image data acquisition unit.

11. The imaging data acquisition unit acquires imaging data of a registered part whose model is known, the image processing unit extracts a contour of a shape of the registered part from the image data of the registered part, and generates point cloud data indicating the extracted contour as a point cloud as registered point cloud data; The part identification device according to any one of claims 1 to 10, further comprising a registration unit that acquires identification information of a type of the registered part, associates the acquired identification information with the registered point cloud data generated by the image processing unit, and stores the associated identification information in the storage unit as the attribute information.

12. A part identification device including a storage unit that stores identification information and registered point cloud data that indicates a contour of a part shape by a point cloud in association with each other for each type of part, Acquire image data of a part to be identified, extracting an outline of the shape of the identification part from the image data of the identification part, and generating identification point cloud data that indicates the extracted outline as a point cloud; extracting, from the storage unit, registered point cloud data having a degree of coincidence with the identification point cloud data equal to or greater than a predetermined value, and acquiring model identification information corresponding to the extracted registered point cloud data; The part identification method includes outputting the acquired identification information as identification information of a type corresponding to the identified part.

Images