Method and apparatus for identifying an article - Patents.com

Abstract

The present invention relates to a method and apparatus (10) for recognizing an item. At least one image (14), particularly in the form of an image, of the item is created. Based on the image (14), image features (22) are identified by an image extraction algorithm (20). Based on stored item data (12), item features (28) are identified and compared with the image features (22), and correspondence information (36) is output. The present invention contemplates, among other things, providing user ratings (42) to improve both the image extraction algorithm (20) and the item extraction algorithm (26). Alternatively or additionally, the present invention contemplates, among other things, configuring both the image extraction algorithm (20) and the item extraction algorithm (26) based on a plurality of interconnected, preferably weighted, data aggregation routines.

Description

[Technical field]

[0001] 2. Background of the Invention The present invention relates to a method and an apparatus for identifying an article. The present invention further relates to the use of such a method or such an apparatus in industrial production.

[0002] In the production of articles, especially sheet metal parts, articles, especially produced sheet metal parts, are often found that cannot be assigned to a specific task. The disposal of articles is a waste of resources and, moreover, creates a risk of missing articles for a specific task. However, a full manual identification of the articles is often time-consuming and can only be successfully performed by experienced employees, who are not always present or who must be freed from other tasks in order to identify the articles. In contrast, a full machine identification of the articles often fails due to an insufficient image of the articles or an insufficient identification program. Summary of the Invention [Problem to be solved by the invention]

[0003] Problem to be solved by the invention It is therefore an object of the present invention to provide a method and an apparatus which allows for an effective matching of accidentally found articles. Furthermore, it is an object of the present invention to provide a corresponding use of the method or the apparatus. [Means for solving the problem]

[0004] Description of the Invention The above object is achieved according to the invention by a method according to claim 1, by a device according to claim 12 and by a use according to claim 16. The dependent claims indicate preferred developments.

[0005] The invention therefore relates to a method for identifying an article, which can be in the form of a substantially two-dimensional article, for example in the form of a flat sheet metal part, or in the form of a three-dimensional article, for example in the form of a deformed sheet metal part. The identification is performed by associating an image of the article with stored article data. These article data can be stored in the form of CAD data. The method comprises the following method steps: A) creating an image of an article by an image capture device, which may be in the form of a camera; B) deriving image features from the image based on an image extraction algorithm; C) deriving article features from the stored article data based on an article extraction algorithm; D) comparing the image features with the article features based on a comparison algorithm; and E) A method step of outputting information relating to the correspondence between the image and the stored item data. has.

[0006] Therefore, according to the present invention, it is intended to output at least one information regarding the correspondence between the item to be identified and the stored item data.Preferably, the information is output in the form of probability information, so that it can be seen how likely the item is to match the stored item data.The output can be implemented, for example, on a screen, on smart glasses, or as a data transmission.Advantageously, multiple correspondence information can be output together with an indication of the probability.

[0007] According to the invention, features are extracted from both the image and the stored item data, which allows a particularly reliable association of the image with the stored item data and thus a particularly high probability of identifying the item.

[0008] Preferably, the method comprises the following method steps: F) Steps for reading user ratings of associated information has.

[0009] The method is intended to read the user's evaluation of this matching information. The user's evaluation can be performed, for example, by data transmission or by an input device, such as a button, a touch screen, a voice recording, a voice recognition, or a similar input device. For example, if two or more identification hits are displayed to the user and sorted according to probability, the user can select the one that actually matches according to his / her opinion. That is, the user has given a user evaluation. This allows the matching information output in the future to be improved.

[0010] The image extraction algorithm can read data from an image, process this data based on predefined image parameters, and output image features in the form of processed data. The predefined image parameters can have so-called weighting variables, the function and determination of which is explained further below.

[0011] The method further preferably comprises the following method steps: H) A method step of modifying, in particular improving, in particular optimizing, parameters of the image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm based on the user evaluation. may have the following structure:

[0012] The parameters may have so-called weighting variables, the determination and function of which is explained further below.

[0013] Preferably, before, during and / or after method step H), the following method steps are carried out: G) A method step of storing the user evaluation of the corresponding information in a user evaluation result memory. will be implemented.

[0014] Storing user ratings allows for the collection of a large number of user ratings, which can significantly improve the image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm.

[0015] The user evaluation result memory can be configured to be cloud-based. Cloud-based in this context means, in particular, a geographically remote, preferably anonymized, memory device in which user evaluations from two or more, preferably hundreds or thousands, different users are stored. This allows different users to contribute to the optimization of the method regardless of the production location. It has been found that the described method is overwhelmingly successful, i.e., the matching information with the highest probability of correct matching is obtained only after tens of thousands, in particular hundreds of thousands, of user evaluations have been read. Such a large amount of data is often not achievable in a single production facility in one year. In that case, the method would probably have remained unattractive.

[0016] The images can be generated and captured within the wavelength range visible to the human eye. Alternatively or additionally, the images can be generated within wavelength ranges not visible to the human eye, for example within the IR range, UV range, and / or ultrasound range.

[0017] In method step A), multiple images of the article can be taken by an image capture device. In method step B), image features can be extracted from the multiple images. In method step E), correspondence information between the multiple images and the stored article can be output. It has been found that the creation and processing of multiple images significantly improves the quality of the correspondence information for the user, since, inter alia, the effect of artifacts caused by the capture position is reduced.

[0018] Preferably, the image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm have an algorithm with a plurality of data aggregation routines. The data aggregation routine can be configured to aggregate a plurality of "identified data" into one new data packet. This new data packet can have one or more numerical values ​​or vectors. The new data packet can be fully or partially supplied as "identified data" to a further data aggregation routine. The "identified data" can be, for example, image data, item data, or a data packet supplied by one of the data aggregation routines. Particularly preferably, the image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm are configured in the form of an algorithm with a plurality of connected data aggregation routines. In particular, hundreds, in particular thousands of such data aggregation routines can be interconnected. This significantly improves the quality and speed of the algorithm. The image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm can have a function with a weighted variable. One, in particular several, particularly preferably all, of the data aggregation routines can be configured to combine, in particular multiply, the several "identified data" respectively with a weighting variable, convert these "identified data" into "combined data", and then aggregate, in particular sum these "combined data" into one new data packet. Particularly preferably, a change in the weighting, in particular a change in the weighting variable, is performed based on the user evaluation. To determine the appropriate weighting variable, the image extraction algorithm, the item extraction algorithm and / or the comparison algorithm can use data, in particular image data and / or item data, respectively, whose correspondence is known. The determination of the weighting variable can be performed separately for the image extraction algorithm, the item extraction algorithm and the comparison algorithm, preferably in a first phase.

[0019] The image features and article features themselves may be data packets, in particular a plurality of structured data, in particular a data vector or data array, and these data packets, in particular a plurality of structured data, in particular a data vector or data array, may themselves be "identified data" for example for a comparison algorithm, in particular for a data aggregation routine of the comparison algorithm. The exact structure of these image features and article features may be altered, in particular improved, preferably optimized, by machine analysis of the user ratings.

[0020] By cloud-based management of weighting variables that have been modified, in particular improved and preferably optimized according to the user evaluations of a first user or a first group of users, other users can also use these weighting variables in their own algorithms and benefit from the method.

[0021] The above-mentioned algorithms or further secondary or higher-level algorithms can be configured to monitor and recognize when one or all of the algorithms output a predefined accumulation of matching information rated as bad by the user and output a negative message based thereon. The output can be implemented visually, e.g. on a screen, or in another suitable form, e.g. as a data output. The monitoring algorithm can further be configured to react to such output of a negative message by an improvement routine that modifies further properties or interactions of one or more of the above-mentioned algorithms.

[0022] In a further preferred embodiment of the method, the article characteristics are provided in the form of structural data, material data, surface property data, and / or thermal conductivity data.

[0023] The comparison algorithm may include forming a scalar product and / or a difference between the image features and the item features. The above-described procedures have been found to be particularly effective in matching image features to item features.

[0024] More preferably, the output of image matching information includes an output of multiple stored matching probabilities with various items, allowing a user to select from multiple different matching options, thereby enabling item identification to be performed with a very high probability of success.

[0025] The subject of the invention is also an apparatus for identifying an object by associating the object with stored object data, the apparatus comprising: a) an image capture device for creating an image of the item; b) an image extraction unit having an image extraction algorithm for extracting image features from the image; c) an item extraction unit having an item extraction algorithm for extracting item features from the stored item data; d) a comparison unit having a comparison algorithm for comparing the image features with the item features; having The image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm are configured to be optimizable based on user evaluations, and the apparatus further comprises: e) an output unit for outputting correspondence information between the image created by the image capture device and the stored item data; The problem is solved by an apparatus having the following features.

[0026] The apparatus according to the invention is preferably configured to carry out the methods described herein.

[0027] The image capture device may in particular be in the form of a camera for visible light.

[0028] This device is f) an input unit for reading a user evaluation regarding the association information; The present invention may further include:

[0029] This device is g) a user evaluation result memory for storing user evaluation of the corresponding information; The user evaluation result memory may further include a cloud-based configuration.

[0030] The image extraction algorithm, the item extraction algorithm, and / or the comparison algorithm may be in the form of an algorithm having multiple connected data aggregation routines.

[0031] Particularly advantageously, the method and / or the device can be used in industrial production with computer-based production control for processing reflective articles, especially sheet metal parts. Previously, images of sheet metal parts by image capture devices are often of insufficient quality for identification, since it is very difficult to distinguish between the contour and the background, and light reflections give the appearance of false contours. Thus, methods and devices that compare image data with article data without using the described algorithm have not been successful in industrial production so far. By reflective article is meant an article with a smooth surface that reflects light in such a way that, in addition to the contour, undesired light reflections from other objects may occur when creating an image. Examples of such reflective articles are metal, glass, plastic with a smooth surface, coated plates made of coated materials, such as plastic, wood, metal, glass, etc.

[0032] It is particularly advantageous if the production control is at least partly cloud-based, in which case the parameters, in particular weighting variables, can be used to modify, in particular improve, in particular optimize the algorithms from the first production facility to the other production facilities and vice versa, so that a much larger database is provided and the identification can be significantly improved for each individual production enterprise.

[0033] Further advantages of the invention will become apparent from the description and the drawings. Likewise, the above and following features can be used according to the invention either individually or in any combination. The illustrated and described embodiments should not be understood as exclusive, but rather as having exemplary features for illustrating the invention. [Brief description of the drawings]

[0034] [Figure 1] 1 is a schematic diagram of one embodiment of the method according to the invention or the device according to the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] 1 shows an apparatus 10 for identifying a found article without association. The article may be in the form of a sheet metal part. Article data 12 is stored for the article. At least one image 14 is created to associate the found article with the article data 12 and thereby identify the found article.

[0036] Preferably, the article data 12 is stored in an article database 16 in the form of CAD data, in particular in the form of a CAD database.

[0037] Image 14 is created by image capture device 18. Image capture device 18 may be in the form of a camera.

[0038] The image 14 shown in Figure 1 illustrates that matching such an image 14 can be very difficult, especially when - as shown - the image 14 shows only a portion of the article, when the image 14 is made against a cluttered background, and / or when the surface characteristics of the article make the image 14 difficult to create.

[0039] An image extraction algorithm 20 is applied to the image 14 in order to extract image features 22. This is illustrated diagrammatically in Fig. 1 based on a dashed-dotted region in the image 14. The image extraction algorithm 20 is stored in an image extraction unit 24. The image extraction algorithm 20 may comprise a number of interconnected, in particular mutually weighted, data aggregation routines.

[0040] An item extraction algorithm 26 is applied to the item data 12 in order to extract item features 28. This is illustrated diagrammatically in Fig. 1 based on a dashed-dotted region in the item data 12. The item extraction algorithm 26 is stored in an item extraction unit 30. The item extraction algorithm 26 may comprise a number of interconnected, in particular mutually weighted, data aggregation routines.

[0041] The image features 22 and the article features 28 are fed to a comparison algorithm 32. The comparison algorithm 32 is stored in a comparison unit 34. The comparison algorithm 32 may comprise a number of interconnected, and in particular mutually weighted, data aggregation routines. Preferably, the comparison algorithm 32 is configured to form a scalar product or difference between the image features 22 and the article features 28.

[0042] As a result of the comparison algorithm 32, matching information 36 is output. The output of the matching information is performed in an output unit 38. As shown in Fig. 1, a plurality of item data (here, three; not marked with reference symbols in Fig. 1) can be displayed, and for each item data, a matching probability (here, 60%, 35%, and 5%) is output. This makes it much easier for the user to match the found items with the item data.

[0043] To improve future matching, i.e. to increase the quality of future matching information, the device 10 has an input unit 40. The input unit 40 is configured to read user ratings 42. The user ratings 42 are then used to optimize the image extraction algorithm 20, the item extraction algorithm 26 and / or the comparison algorithm 32, or parameters thereof.

[0044] The user ratings 42 can be stored in a user rating result memory 44, so that the method or device 10 can be optimized with a large number of user ratings 42. Particularly preferably, the user rating result memory 44 is configured to be cloud-based, so that the user ratings 42 can be incorporated into the optimization of the method or device 10 across devices.

[0045] In other words, the present invention relates to a method and device 10 for recognizing an item. At least one image 14 of the item, particularly in the form of an image, is created. Based on the image 14, image features 22 are identified by an image extraction algorithm 20. Based on the stored item data 12, item features 28 are identified and compared with the image features 22, and matching information 36 is output. According to the present invention, it is contemplated, among other things, to provide user ratings 42 in order to improve both the image extraction algorithm 20 and the item extraction algorithm 26. Alternatively or additionally, it is contemplated, among other things, to configure both the image extraction algorithm 20 and the item extraction algorithm 26 based on a plurality of interconnected, preferably weighted, data aggregation routines. [Explanation of symbols]

[0046] 10 equipment 12 Product Data 14 images 16 Item Database 18 Image capture device 20 Image Extraction Algorithms 22 Image Features 24 Image Extraction Unit 26 Item Extraction Algorithm 28 Product Features 30 Item Extraction Unit 32 Comparison Algorithms 34 Comparison Units 36 Mapping Information 38 Output Unit 40 Input Units 42 User ratings 44 User evaluation result memory

Claims

1. 1. A method for identifying an item by matching an image (14) of the item to stored item data (12), comprising: The method comprises the following method steps: A) creating said image (14) of said article by an image capture device (18); B) the method steps of extracting image features (22) from said image (14) based on an image extraction algorithm (20); C) a method step of extracting article features (28) from said stored article data (12) based on an article extraction algorithm (26). having Said method step C) can be carried out before, during and / or after said method steps A) and B), followed by the following method steps: D) a method step of comparing said image characteristics (22) with said article characteristics (28) based on a comparison algorithm (32); E) outputting a correspondence (36) between said image (14) and said stored article data (12); F) a method step of reading user ratings (42) of said association information (36); H) a method step of optimizing said image extraction algorithm (20), said item extraction algorithm (26) and / or said comparison algorithm (32) based on said user ratings (42); was carried out, The output of the association information (36) for the image (14) includes an output of association probabilities with various stored items; In said method step A), a plurality of images (14) of said article are taken by said image capture device (18); In said method step B), said image features (22) are extracted from said plurality of images (14), In said method step E), association information (36) between said plurality of images (14) and said stored articles is output.

2. Before, during and / or after said method step H), the following method steps may be carried out: G) a method step of storing said user ratings (42) of said association information (36) in a user rating result memory (44). The method of claim 1 , wherein the following is performed:

3. The user evaluation result memory (44) is configured as a cloud-based memory. The method of claim 2.

4. The creation of the image (14) is carried out within the wavelength range visible to the human eye; 4. The method according to claim 1 ,

5. the image extraction algorithm (20), the item extraction algorithm (26), and / or the comparison algorithm (32) having a plurality of interconnected data aggregation routines; 5. The method according to any one of claims 1 to 4.

6. the article characteristics (28) are provided in the form of construction data and / or material data; 6. The method according to any one of claims 1 to 5.

7. the comparison algorithm (32) includes forming a scalar product and / or a difference between the image feature (22) and the article feature (28); 7. The method according to any one of claims 1 to 6.

8. An apparatus (10) for identifying an item by associating the item with stored item data (12), comprising: The device (10) comprises: a) an image capture device (18) for creating an image (14) of said article; b) an image extraction unit (24) having an image extraction algorithm (20) for extracting image features (22) from said image (14); c) an article extraction unit (30) having an article extraction algorithm (26) for extracting article features (28) from the stored article data; d) a comparison unit (34) having a comparison algorithm (32) for comparing said image characteristics (22) with said article characteristics (28); wherein the image extraction algorithm (20), the item extraction algorithm (26), and / or the comparison algorithm (32) are configured to be optimizable based on user ratings (42), and the apparatus further comprises: e) an output unit (38) for outputting a correspondence (36) between the image (14) created by the image capture device (18) and the stored article data (12); f) an input unit (40) for reading user ratings (42) of said association information (36), The output of the association information (36) for the image (14) includes an output of association probabilities with various stored items; The image capture device (18) captures a plurality of images (14) of the item; The image extraction unit (24) extracts the image features (22) from the plurality of images (14); The output unit (38) outputs association information (36) between the plurality of images (14) and the stored articles.

9. The device (10) comprises: g) a user evaluation result memory (44) for storing said user evaluations (42) of said association information (36); The apparatus of claim 8 further comprising:

10. the image extraction algorithm (20), the item extraction algorithm (26), and / or the comparison algorithm (32) having a plurality of interconnected data aggregation routines; 10. Apparatus according to claim 8 or 9.

Images