Method and apparatus for recognizing objects
By combining image detection and algorithm matching with user evaluation and optimization, the efficiency and accuracy issues of object recognition such as sheet metal parts have been resolved, achieving efficient and reliable object recognition, especially the recognition of reflective objects.
Patent Information
- Application Number
- CN201980059832.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-09-12
- Filing Date
- 2019-09-03
- Publication Date
- 2025-11-21
- Estimated Expiration
- 2039-09-03
AI Technical Summary
In the manufacture of objects, especially in the manufacture of sheet metal parts, existing technologies are unable to efficiently identify non-corresponding objects, resulting in waste of resources and potential dangers. At the same time, manual identification is costly and machine identification is ineffective.
Object records are created using image detection equipment. Record extraction, object extraction, and comparison algorithms are used to match the object records with stored object data, output corresponding information, and optimize algorithm parameters through user evaluation. The recognition process is improved by utilizing a cloud-based user evaluation result storage system.
It achieves efficient and reliable object recognition, reduces resource waste, and increases the success rate of recognition, especially significantly improving the recognition effect in the case of reflective objects.
Smart Images
Figure CN112703508B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and a device for identifying objects. The invention also relates to the application of this method or the device in industrial manufacturing. Background Technology
[0002] In the manufacture of objects, especially sheet metal parts, it is common to find objects that do not correspond to any specific task, particularly manufactured sheet metal parts. Removing such objects is a waste of resources and also carries the risk of the object being missing from the defined task. However, identifying such objects entirely manually is costly and can usually only be successfully performed by experienced employees who are not always present or must be relieved of other tasks to identify the object. In contrast, relying entirely on machines to identify objects often fails due to insufficient object records or inadequate identification procedures. Summary of the Invention
[0003] Therefore, the object of the present invention is to provide a method and an apparatus capable of efficiently responding to randomly discovered objects. Furthermore, the object of the present invention is to provide a corresponding application of the method or apparatus.
[0004] According to the present invention, this task is solved by a method according to the present invention for identifying an object by corresponding an object record to stored object data, an apparatus according to this aspect for identifying an object by corresponding an object to stored object data, or the application of the method or apparatus according to the present invention in industrial manufacturing. Preferred extensions are described below.
[0005] Therefore, the present invention relates to a method for identifying an object. The object may be constructed in the form of a substantially two-dimensional object (e.g., a planar sheet metal part) or a three-dimensional object (e.g., a sheet metal part with deformation). Identification is performed by correlating a record of the object with stored object data. The object data may be stored in the form of CAD data. The method comprises the following steps:
[0006] A) Create one or more records of an object using an image detection device, wherein the image detection device may be constructed in the form of a camera;
[0007] B) Extract record features from the records using the record extraction algorithm;
[0008] C) Extract object features from the stored object data using an object extraction algorithm;
[0009] D) Compare the recorded features with the object features according to the comparison algorithm; and
[0010] E) Output the correspondence information between the records and the stored object data.
[0011] Therefore, according to the present invention, at least one piece of information regarding the correspondence between the object to be identified and the stored object data is output. Preferably, this information is presented in the form of probability information, so that the probability of the object matching the stored object data can be seen. This output can be performed, for example, on a screen, on data glasses, or implemented as data transmission. Advantageously, multiple correspondence information with probability descriptions can be output.
[0012] According to the present invention, features are extracted not only from records but also from stored object data. This enables a particularly reliable correspondence between records and stored object data, thus achieving a particularly high probability of object identification.
[0013] Preferably, the method comprises the following steps:
[0014] F) Read the corresponding information for user evaluation.
[0015] The method then sets up the process of reading the user's evaluation of the corresponding information. This evaluation by the user can be performed, for example, via data transmission or via an input device (e.g., a button, touchscreen, voice recording, voice recognition, or similar input device). For instance, if the user is shown two or more identification matches ranked by probability, the user can choose the one that they perceive as truly consistent. Thus, the user provides their evaluation. This can improve the corresponding information output in the future.
[0016] Here, the record extraction algorithm reads in recorded data, processes the data according to pre-given record parameters, and outputs record features in the form of processed data. The pre-given record parameters may have so-called weighted variables. The function and acquisition of these variables are explained below.
[0017] Furthermore, preferably, the method may include the following steps:
[0018] H) Based on user feedback, change, especially improve, especially optimize the parameters of the record extraction algorithm, object extraction algorithm, and / or comparison algorithm.
[0019] The parameters can have so-called weighted variables. The determination and function of these variables are explained below.
[0020] Preferably, the following method steps are performed before, during, and / or after method step H):
[0021] G) Store the corresponding user evaluation information in the user evaluation result storage.
[0022] The storage of user evaluations enables the collection of multiple user evaluations, thereby significantly improving record extraction algorithms, object extraction algorithms, and / or comparison algorithms.
[0023] User evaluation result storage can be constructed as a cloud-based system. Here, "cloud-based" refers to a remote, preferably anonymous, storage device containing user evaluations from more than one user, advantageously from hundreds or thousands of different users. Thus, different users can contribute to the optimization of the method, regardless of the manufacturing location. The method is considered convincingly successful only when tens of thousands, especially hundreds of thousands, of user evaluations have been read, i.e., when the correspondence information with the highest probability of correct correspondence is obtained. For a single manufacturing plant, such a data volume is often impossible to achieve within a year. Therefore, this method may still remain unattractive.
[0024] The creation or recording of the record can be performed within the wavelength range visible to the human eye. Alternatively or additionally, the record can be created within the wavelength range invisible to the human eye, such as in the IR range, the UV range, and / or the ultrasound range.
[0025] In method step A), multiple records of the object can be recorded using an image detection device. In method step B), record features can be extracted from the multiple records. In method step E), the correspondence information between the multiple records and the stored object can be output. It has been shown that, for the user, the creation and processing of multiple records significantly improves the quality of the correspondence information, especially by reducing the impact of artifacts caused by record location.
[0026] Preferably, the record extraction algorithm, object extraction algorithm, and / or comparison algorithm have algorithms with multiple data aggregation routines. The data aggregation routines can be designed to aggregate multiple "demanded data" into a new data packet. The new data packet can have one or more numbers or vectors. The new data packet can be provided, in whole or in part, as "demanded data" to other data aggregation routines. The "demanded data" can be, for example, record data, object data, or a data packet provided by one of the data aggregation routines. Particularly preferably, the record extraction algorithm, object extraction algorithm, and / or comparison algorithm are constructed as algorithms with multiple interconnected data aggregation routines. In particular, hundreds, especially thousands, of such data aggregation routines can be interconnected. The quality and speed of one or more algorithms are thus significantly improved. The record extraction algorithm, object extraction algorithm, and / or comparison algorithm can have functions with weighted variables. A data aggregation routine, particularly multiple data aggregation routines, and especially preferably all data aggregation routines, can be designed to combine, particularly multiply, multiple "demanded data" with weighted variables, thereby converting the "demanded data" into "combined data," and then aggregating, particularly summing, the "combined data" into a new data package. Particularly preferably, the weighting, particularly the weighted variables, is changed based on user evaluation. To obtain suitable weighted variables, record extraction algorithms, object extraction algorithms, and / or comparison algorithms can be performed using data with known dependencies, particularly record data and / or object data. Here, preferably in the first stage, the weighted variables can be determined separately for the record extraction algorithm, object extraction algorithm, and comparison algorithm.
[0027] Here, the record features and object features themselves can be data packets, especially multiple structured data, especially data vectors or data arrays, and can also be, for example, the "requested data" for comparison algorithms, especially data aggregation routines for comparison algorithms. The precise structure of the record features and object features can be changed, especially improved, and preferably optimized through machine analysis processing evaluated by the user.
[0028] Because weighted variables that have been modified, particularly improved, and preferably optimized through user evaluations by the first user or the first group of users are managed in a cloud-based manner, other users can also use these weighted variables in their algorithms and benefit from the method.
[0029] The mentioned algorithm, or other sub- or super-algorithms, can be configured to monitor and identify when one or all of the algorithms outputs corresponding information that is evaluated as poor by the user at a pre-given cumulative level, and then output a negative report. This output can be visualized, for example, on a screen, or in other suitable forms, such as as data output. The monitoring algorithm can also be configured to respond to the output of such negative reports with improvement routines that alter other properties or interactions of one or more of the mentioned algorithms.
[0030] In another preferred configuration of the method, the object features exist in the form of structural data, material data, surface property data, and / or thermal conductivity data.
[0031] The comparison algorithm may include calculating the scalar product and / or difference between the recorded features and the object features. It has been shown that the above approach is particularly effective in mapping recorded features to object features.
[0032] Furthermore, preferably, outputting the corresponding information of the records includes outputting multiple corresponding probabilities for the stored different objects. This allows the user to select from different corresponding possibilities. Consequently, object identification can be performed with a very high probability of success.
[0033] The objective according to the invention is also achieved by a device for identifying objects by correlating them with stored object data, wherein the device has:
[0034] a) An image detection device used to create a record of an object;
[0035] b) A record-extraction unit with a record-extraction algorithm, which is used to extract record features from records;
[0036] c) Object extraction unit with an object extraction algorithm, which is used to extract object features from the stored object data;
[0037] d) A comparison unit with a comparison algorithm, which is used to compare recorded features with object features;
[0038] The record extraction algorithm, object extraction algorithm, and / or comparison algorithm are designed to be optimized based on user evaluation; and
[0039] e) Output unit, which outputs the correspondence information between the record created by the image detection device and the stored object data.
[0040] Preferably, the device according to the invention is configured to perform the methods described herein.
[0041] Image inspection equipment can be constructed in the form of a camera, especially a camera for visible light.
[0042] The device may also have:
[0043] f) Input unit, which is used to read user evaluations about the corresponding information.
[0044] In addition, the device may have:
[0045] g) A user evaluation result storage device, which stores user evaluations with corresponding information. The user evaluation result storage device may be constructed in the cloud.
[0046] Record extraction algorithms, object extraction algorithms, and / or comparison algorithms can be constructed in the form of algorithms with multiple interconnected data aggregation routines.
[0047] Particularly advantageously, this method and / or apparatus can be used in industrial manufacturing with computer-based manufacturing control devices for handling reflective objects, especially sheet metal parts. Historically, recording sheet metal parts using image detection equipment has often been insufficient for identification because the contours are very difficult to distinguish from the background, and light reflections can masquerade as false contours. Therefore, without the algorithm described above, methods and apparatus for comparing recorded data with object data have not yet been successful in industrial manufacturing. Reflective objects are those with smooth surfaces that reflect light so much that, in addition to the contours, undesirable light reflections from other objects may occur during the recording process. Examples of such reflective objects are metals, glass, plastics with smooth surfaces, and coated materials (e.g., coated panels made of plastic, wood, metal, glass, etc.).
[0048] A particular advantage is that the manufacturing control system is at least partially cloud-based. Parameters, especially weighted variables, can then be used to modify, particularly improve, and particularly optimize the algorithms in the first manufacturing plant and other manufacturing plants, and vice versa. Therefore, a much larger database exists, and the identification for each individual manufacturing operation can be significantly improved.
[0049] Other advantages of the invention are apparent from the specification and drawings. The features mentioned above and further implemented may also be used individually or in any combination of multiple features according to the invention. The illustrated and described embodiments should not be construed as exhaustive, but rather as exemplary features used to describe the invention. Attached Figure Description
[0050] Figure 1 A schematic diagram showing one configuration of the method or device according to the invention is shown. Detailed Implementation
[0051] Figure 1 A device 10 is shown for identifying a discovered object that has no corresponding relationship. The object can be constructed in the form of a sheet metal part. Object data 12 is stored about the object. In order to match the discovered object with the object data 12 and thereby identify it, at least one record 14 is created.
[0052] Preferably, the object data 12 is stored in the form of CAD data, particularly in an object database 16 in the form of a CAD database.
[0053] Record 14 is created using image detection device 18. Image detection device 18 can be constructed in the form of a camera.
[0054] exist Figure 1 The record 14 shown in the figure indicates that a correspondence to such a record 14 can be very difficult. This is especially true if, as shown, record 14 shows only a portion of the object, record 14 is made against a non-static background, and / or the surface characteristics of the object make the creation of record 14 difficult.
[0055] Record extraction algorithm 20 is applied to record 14 to extract record features 22. This is in Figure 1 The diagram illustrates the area enclosed by the dashed line in record 14. Record extraction algorithm 20 is stored in record extraction unit 24. Record extraction algorithm 20 may have interconnected, and in particular, weighted, data aggregation routines.
[0056] Object extraction algorithm 26 is applied to object data 12 to extract object features 28. This is in Figure 1 The area is schematically indicated by the dotted lines in the object data 12. The object extraction algorithm 26 is stored in the object extraction unit 30. The object extraction algorithm 26 may have interconnected, and in particular, weighted, data aggregation routines.
[0057] The record feature 22 and the object feature 28 are fed to the comparison algorithm 32. The comparison algorithm 32 is stored in the comparison unit 34. The comparison algorithm 32 may have data aggregation routines that are connected to each other, especially weighted relative to each other. Preferably, the comparison algorithm 32 is configured to calculate the scalar product or difference between the record feature 22 and the object feature 28.
[0058] As a result of comparison algorithm 32, corresponding information 36 is output. The output of the corresponding information is performed in output unit 38. For example, in... Figure 1 As shown in the image, multiple object data can be displayed (three in this case; in...). Figure 1(Not shown in the attached figures), where the corresponding probabilities for the respective object data are output (60%, 35%, and 5% in this case). This makes it easier for users to match the discovered objects with the object data.
[0059] To improve future correspondence, i.e., to enhance the quality of future correspondence information, device 10 has an input unit 40. Input unit 40 is configured to read user evaluation 42. The user evaluation 42 is then used to optimize record extraction algorithm 20, object extraction algorithm 26, and / or comparison algorithm 32, or their parameters.
[0060] User evaluations 42 can be stored in a user evaluation result storage 44, thereby enabling the optimization of the method or device 10 using a large number of user evaluations 42. Particularly preferably, the user evaluation result storage 44 is constructed based on the cloud. Thus, user evaluations 42 across devices can be incorporated into the optimization of the method or device 10.
[0061] In other words, the present invention relates to a method and apparatus 10 for identifying objects. Here, at least one record 14, particularly in photographic form, of the object is created. Record features 22 are obtained from the record 14 using a record extraction algorithm 20. Object features 28 are obtained from stored object data 12 and compared with the record features 22 to output corresponding information 36. According to the invention, a user evaluation 42 is particularly provided to improve the record extraction algorithm 20 and the object extraction algorithm 26. Alternatively or additionally, according to the invention, the record extraction algorithm 20 and the object extraction algorithm 26 are particularly configured as data aggregation routines that are interconnected, preferably weighted.
[0062] List of reference numerals
[0063] 10 Equipment
[0064] 12 Object Data
[0065] 14 Records
[0066] 16 Object Database
[0067] 18 Image Inspection Equipment
[0068] 20-Record Extraction Algorithm
[0069] 22 Recording characteristics
[0070] 24 Record-Extraction Unit
[0071] 26 Object Extraction Algorithms
[0072] 28. Object characteristics
[0073] 30 Object Extraction Unit
[0074] 32 Comparison Algorithm
[0075] 34 Comparison Units
[0076] 36 Corresponding Information
[0077] 38 Output Units
[0078] 40 Input Units
[0079] 42 User Evaluation
[0080] 44 User Evaluation Results Storage
Claims
1. A method for identifying an object by correlating an object record (14) with stored object data (12), the method comprising the following steps: A) A record (14) of the object is created using an image detection device (18); B) Extract record features (22) from the record (14) according to the record extraction algorithm (20); C) Extract object features (28) from the stored object data (12) according to the object extraction algorithm (26); in, Method step C) can be executed before, during, and / or after method steps A) and B), wherein the following method steps are then executed: D) Compare the recorded features (22) with the object features (28) according to the comparison algorithm (32); E) Output the correspondence information (36) between the record (14) and the stored object data (12); F) Read the user evaluation (42) of the corresponding information (36); H) Optimize the record extraction algorithm (20), the object extraction algorithm (26), and / or the comparison algorithm (32) based on the user evaluation (42); The record extraction algorithm (20), the object extraction algorithm (26), and the comparison algorithm (32) each have weighted data aggregation routines connected to each other, and the record extraction algorithm (20), the object extraction algorithm (26), and the comparison algorithm (32) each have functions with weighted variables, wherein the weighted variables are changed according to the user evaluation (42); The object data (12) is stored in the form of CAD data, and the object features (28) exist in the form of structural data, material data, surface property data and / or thermal conductivity data. In step A), multiple records (14) of the object are recorded using the image detection device (18). In step B), the record features (22) are extracted from the multiple records (14). In step E), the correspondence information (36) between the multiple records (14) and the stored object data is output.
2. The method according to claim 1, wherein the following method steps are performed before, during, and / or after method step H): G) Store the user evaluation (42) of the corresponding information (36) in the user evaluation result memory (44).
3. The method according to claim 2, wherein the user evaluation result storage (44) is constructed based on the cloud.
4. The method according to any one of the preceding claims, wherein the creation of the record (14) is performed within a wavelength range visible to the human eye.
5. The method according to any one of claims 1 to 3, wherein the comparison algorithm (32) comprises calculating the scalar product between the recording feature (22) and the object feature (28) and / or calculating the difference between the recording feature and the object feature.
6. The method according to any one of claims 1 to 3, wherein outputting the corresponding information (36) of the record (14) includes outputting the corresponding probability for different stored objects.
7. A device (10) for identifying objects by correlating them with stored object data (12), wherein, The device (10) has: a) an image detection device (18) for creating a record (14) of the object; b) A record extraction unit (24) having a record extraction algorithm (20) for extracting record features (22) from the record (14); c) An object extraction unit (30) has an object extraction algorithm (26) for extracting object features (28) from the stored object data; d) A comparison unit (34) having a comparison algorithm (32) for comparing the recorded features (22) with the object features (28); The record extraction algorithm (20), the object extraction algorithm (26), and / or the comparison algorithm (32) are configured to be optimized based on user evaluation (42); e) Output unit (38), the output unit is used to output the correspondence information (36) between the record (14) created by means of the image detection device (18) and the stored object data (12); f) Input unit (40), the input unit being used to read the user evaluation (42) of the corresponding information (36); The record extraction algorithm (20), the object extraction algorithm (26), and the comparison algorithm (32) each have weighted data aggregation routines connected to each other, and the record extraction algorithm (20), the object extraction algorithm (26), and the comparison algorithm (32) each have functions with weighted variables, wherein the weighted variables are changed according to the user evaluation (42); The device is configured to optimize the record extraction algorithm (20), the object extraction algorithm (26), and / or the comparison algorithm (32) based on the user evaluation (42); The object data (12) is stored in the form of CAD data, and the object features (28) exist in the form of structural data, material data, surface property data and / or thermal conductivity data. In step A), multiple records (14) of the object are recorded using the image detection device (18). In step B), the record features (22) are extracted from the multiple records (14). In step E), the correspondence information (36) between the multiple records (14) and the stored object data is output.
8. The device according to claim 7, wherein, The device (10) also has: g) User evaluation result storage (44), the user evaluation result storage is used to store the user evaluation (42) of the corresponding information (36).
9. The application of the method or apparatus according to any one of the preceding claims in industrial manufacturing.
10. The application according to claim 9, wherein, In the aforementioned industrial manufacturing process, reflective objects are handled.
11. The application according to claim 10, wherein, The reflective object is a sheet metal part.