Object recognition methods and object recognition systems
By generating multi-angle synthetic images and fusing photo feature vectors, combined with a deep learning classifier, the problem of insufficient accuracy in object recognition in existing technologies is solved, and high-precision object recognition is achieved.
Patent Information
- Application Number
- CN202080103768.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-11
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2040-09-11
AI Technical Summary
Existing object recognition methods based on 3D model data are not accurate enough in classifying factory workpieces.
By generating multi-angle synthetic images, feature vectors are extracted and fused to generate a first fused vector. Object classification is performed by combining the feature vectors of the photos, and a deep learning-based classifier is used for training and recognition.
It improves the accuracy of object recognition, reduces interference caused by different angles, and achieves high-precision object recognition.
Smart Images

Figure CN116783630B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision. Background Technology
[0002] Object recognition falls under the field of computer vision, and its main task is to identify objects in images. Currently, mainstream object recognition methods are either based on training and prediction using 2D real images or utilize 3D digital models as an auxiliary method. However, the accuracy of existing object recognition methods based on 3D model data cannot meet the requirements of factory workpiece classification. Summary of the Invention
[0003] The purpose of this invention is to provide an object recognition method with high recognition accuracy.
[0004] Another objective of this invention is to provide an object recognition system with high recognition accuracy.
[0005] Another object of the present invention is to provide a computer-readable storage medium on which the execution of code stored thereon can more accurately identify objects.
[0006] This invention provides an object recognition method, which includes:
[0007] Several composite images are generated based on a 3D digital model, and the composite images have different perspectives.
[0008] Extract feature vectors from several synthesized images respectively;
[0009] A first fusion vector is generated by fusing the feature vectors of several synthesized images;
[0010] The first fused vector is input into the classifier to train the classifier;
[0011] Acquire several photographs of the object, each photograph sharing the same viewpoint as at least a portion of the composite image;
[0012] Extract feature vectors from several photos respectively;
[0013] A second fusion vector is generated by fusing the feature vectors of several photos; and
[0014] The second fusion vector is input into the trained classifier to obtain the classification result of the object.
[0015] In this object recognition method, the perspectives of the synthesized images are different, and correspondingly, the perspectives of the individual photographs are also different, which allows for the representation of more features. Several photographs share the same perspective as at least a portion of the synthesized images, thereby reducing interference caused by differences in angle. This object recognition method exhibits high accuracy.
[0016] In another illustrative embodiment of the object recognition method, camera parameters for acquiring several photos are determined based on the viewpoints of several synthetic images, or software parameters for generating several synthetic images are determined based on the viewpoints of several photos, so that the viewpoints of several photos are the same as those of at least some of the synthetic images.
[0017] In another illustrative embodiment of the object recognition method, when several photos have the same viewpoint as all the composite images, the feature vectors of the extracted composite images are fused to generate a first fusion vector, and the feature vectors of the extracted photos are fused to generate a second fusion vector.
[0018] In another illustrative embodiment of the object recognition method, when several photos share the same viewpoint as some of the synthesized images, the feature vectors extracted from the photos are fused with auxiliary vectors to generate a second fusion vector, wherein the total number of feature vectors and auxiliary vectors from the photos equals the number of synthesized images. Then, the feature vectors extracted from the synthesized images sharing the same viewpoint as the photos are fused with auxiliary vectors to generate a first fusion vector, wherein the total number of feature vectors and auxiliary vectors from the synthesized images sharing the same viewpoint as the photos equals the number of synthesized images. Alternatively, when several photos share the same viewpoint as some of the synthesized images, the feature vectors extracted from the synthesized images sharing the same viewpoint as the photos are fused to generate a first fusion vector, and the feature vectors extracted from the photos are fused to generate a second fusion vector.
[0019] In another illustrative implementation of the object recognition method, CAD software is used to generate several composite images based on a three-dimensional digital model.
[0020] In another illustrative implementation of the object recognition method, a CNN is used to extract feature vectors from several synthetic images and feature vectors from several photographs. The classifier is a deep learning-based classifier.
[0021] In another illustrative implementation of the object recognition method, AutoML technology or neural architecture search technology is applied to determine the fusion scheme.
[0022] In another illustrative embodiment of the object recognition method, several synthetic images are first subjected to domain randomization processing, and then feature vectors of the several synthetic images are extracted respectively. Alternatively, several photos are first subjected to domain randomization processing, and then feature vectors of the several photos are extracted respectively.
[0023] This invention also provides an object recognition system, comprising a processor and an imaging mechanism. The processor integrates an image generation module, a feature extraction module, a fusion module, and a classifier module. The image generation module generates several composite images based on a three-dimensional digital model. These composite images have different viewpoints. The feature extraction module extracts feature vectors from each of the composite images. The fusion module fuses the feature vectors from the composite images to generate a first fusion vector. The classifier module is trained based on the input first fusion vector. The imaging mechanism acquires several photographs of the object. The processor controls the imaging mechanism or the image generation module to ensure that the photographs share the same viewpoint as at least some of the composite images. The feature extraction module also extracts feature vectors from each photograph. The fusion module further fuses the feature vectors from the photographs to generate a second fusion vector. The trained classifier module obtains the object classification result based on the input second fusion vector.
[0024] This object recognition system generates several synthetic images from different perspectives, and correspondingly acquires several photographs from different perspectives, allowing for the representation of more features. The processor can control the imaging mechanism or image generation module to ensure that the photographs share the same perspective as at least some of the synthetic images, thereby reducing interference caused by different angles. This object recognition system exhibits high accuracy.
[0025] In another illustrative embodiment of the object recognition system, the imaging mechanism includes a camera and a support. The camera is movably connected to the support. The object recognition system also includes a drive mechanism capable of driving the camera to move relative to the support. The processor is capable of outputting a set of control signals based on the viewpoints of several composite images. The drive mechanism is capable of controlling the movement of the camera according to the control signals to obtain several photographs, each with the same viewpoint as at least a portion of the composite images.
[0026] In another illustrative embodiment of the object recognition system, the imaging mechanism includes several cameras. The object recognition system also includes a position sensing unit. The position sensing unit is capable of detecting the spatial positions and shooting angles of the several cameras and generating a set of viewpoint signals based on the spatial positions and shooting angles of the several cameras. The processor is capable of determining the parameters used to generate several composite images based on the viewpoint signals, so that the several photographs have the same viewpoint as at least a portion of the composite images.
[0027] In another illustrative embodiment of the object recognition system, when several photos have the same viewpoint as all the composite images, the fusion module can fuse the feature vectors of the extracted composite images to generate a first fusion vector, and can fuse the feature vectors of the extracted photos to generate a second fusion vector.
[0028] In another illustrative embodiment of the object recognition system, when several photos share the same viewpoint as some of the synthesized images, the fusion module can fuse the extracted feature vectors of the several photos with auxiliary vectors to generate a second fusion vector, wherein the total number of feature vectors and auxiliary vectors of the several photos equals the number of synthesized images. It can also fuse the extracted feature vectors of the synthesized images that share the same viewpoint as the several photos with auxiliary vectors to generate a first fusion vector, wherein the total number of feature vectors and auxiliary vectors of the synthesized images that share the same viewpoint as the several photos equals the number of synthesized images. Alternatively, when several photos share the same viewpoint as some of the synthesized images, the fusion module can fuse the extracted feature vectors of the synthesized images that share the same viewpoint as the several photos to generate a first fusion vector, and can also fuse the extracted feature vectors of the several photos to generate a second fusion vector.
[0029] In another illustrative embodiment of the object recognition system, the image generation module can use CAD software to generate several composite images based on a three-dimensional digital model.
[0030] In another illustrative implementation of the object recognition system, the feature extraction module uses a CNN to extract feature vectors from several synthetic images and feature vectors from several photographs, respectively. The classifier module is a deep learning-based classifier module.
[0031] In another illustrative implementation of the object recognition system, the fusion module can apply AutoML technology or neural architecture search technology to determine the fusion scheme.
[0032] In another illustrative embodiment of the object recognition system, the feature extraction module can first perform domain randomization processing on several synthesized images, and then extract the feature vectors of the several synthesized images respectively. Alternatively, the feature extraction module can first perform domain randomization processing on several photos, and then extract the feature vectors of the several photos respectively.
[0033] The present invention also provides a computer-readable storage medium having code stored thereon for system use. When the processor executes the code, the system performs the object recognition method described above. Attached Figure Description
[0034] The following figures are for illustrative purposes only and do not limit the scope of the invention.
[0035] Figure 1 This is a flowchart illustrating one implementation of an object recognition method.
[0036] Figure 2 for Figure 1 The flowchart of step S11 of the object recognition method shown.
[0037] Figure 3 A flowchart illustrating another illustrative implementation of the object recognition method.
[0038] Figure 4 This is a schematic diagram illustrating one implementation of an object recognition system.
[0039] Figure 5 For explanation Figure 4 The diagram illustrates the working process of the object recognition system.
[0040] Figure 6 An illustrative implementation of the feature extraction module.
[0041] Figure 7 This is a schematic diagram illustrating another implementation of an object recognition system.
[0042] Label Explanation
[0043] 20 processors
[0044] 21 Image Generation Module
[0045] 22 Feature Extraction Module
[0046] 23 Fusion Module
[0047] 24 Classifier Module
[0048] 40 filming locations
[0049] 41 cameras
[0050] 42 brackets
[0051] 50 Drive mechanism
[0052] 60 position sensing units
[0053] 80 objects
[0054] M 3D digital model
[0055] Composite image of S1, S2, and S3
[0056] Feature vectors of the synthesized images Sv1, Sv2, and Sv3
[0057] Fv1 First fusion vector
[0058] Photos P1, P2, and P3
[0059] Feature vectors of Pv1, Pv2, and Pv3 photos
[0060] Fv2 Second Fusion Vector
[0061] CNN1, CNN2, CNN3 Convolutional Neural Networks
[0062] R classification results Detailed Implementation
[0063] To provide a clearer understanding of the technical features, objectives, and effects of the invention, specific embodiments of the invention are now described with reference to the accompanying drawings. In the drawings, the same reference numerals indicate components with the same or similar structures but the same function.
[0064] In this document, “illustrative” means “serving as an example, illustration or description”, and any illustration or implementation described herein as “illustrative” should not be construed as a more preferred or advantageous technical solution.
[0065] In this document, terms such as "first" and "second" do not indicate their importance or order, but are only used to distinguish them to facilitate the description of the document.
[0066] To keep the drawings simple, each drawing only schematically shows the parts related to the present invention, and they do not represent the actual structure of the product.
[0067] Figure 1 This is a flowchart illustrating one implementation of an object recognition method. (Example:) Figure 1 As shown, the object recognition method of this illustrative embodiment includes the following steps, wherein the order of the steps is not limited to the following arrangement.
[0068] S11: Generate several composite images based on the 3D digital model. The composite images have different perspectives, and preferably multiple perspectives that can reflect more features. In this paper, "several" is interpreted as "at least two".
[0069] In an illustrative embodiment, for example, CAD (Computer Aided Design) software is used to generate several composite images from a 3D digital model. CAD software such as AutoCAD, developed by Autodesk, can be used. In other illustrative embodiments, other software capable of generating composite images from a 3D digital model can also be selected, such as UG (Unigraphics NX), developed by Siemens PLM Software.
[0070] Specifically, taking CAD software as an example, such as Figure 2 As shown, step S11 includes, for example, performing the following steps sequentially in the CAD software:
[0071] S111: Rendering 3D digital models
[0072] S112: Add several virtual cameras and set the camera parameters for each virtual camera. The number of virtual cameras should match the number of composite images to be generated, and the camera parameters of the virtual cameras determine the viewpoint of the composite images.
[0073] S113: Capture images using a virtual camera to obtain a composite image.
[0074] S12: Extract the feature vectors of the several synthesized images respectively. In this illustrative embodiment, for example, CNN (Convolutional Neural Networks) is used to extract the feature vectors of the several synthesized images respectively, but it is not limited to this. In other illustrative embodiments, other methods can also be used to extract the feature vectors of the synthesized images.
[0075] Convolutional Neural Networks (CNNs) are a type of feedforward neural network that incorporates convolutional computations and has a deep structure; they are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform translation-invariant classification of input information according to their hierarchical structure, hence they are also known as "translation-invariant artificial neural networks." Using CNNs facilitates the extraction of feature vectors that reflect key characteristics, further improving the accuracy of object recognition methods.
[0076] S13: Generate a first fusion vector by fusing the feature vectors of several synthesized images. In an illustrative implementation, for example, AutoML (Automated Machine Learning) technology or neural architecture search technology is applied to determine the fusion scheme, which is beneficial to obtaining a better fusion scheme, but is not limited thereto.
[0077] AutoML combines the data preprocessing, feature selection, and algorithm selection steps of machine learning with the model architecture design and model training steps of deep learning, putting them all in a "black box." Through this "black box," we only need to input data to get the prediction results we want.
[0078] The method of "designing neural networks using neural networks" is called Neural Architecture Search (NAS), which typically uses reinforcement learning or evolutionary algorithms to design new neural network structures. NAS can automate architecture engineering; simply provide a dataset, and NAS can automatically derive the optimal architecture.
[0079] S14: Input the first fused vector into the classifier to train the classifier. In the illustrative implementation, the classifier is a deep learning-based classifier.
[0080] S21: Acquire several photographs of an object (e.g., a physical workpiece) using a camera, each photograph sharing the same viewpoint as at least a portion of the composite images. That is, the number of photographs is less than or equal to the number of composite images. If the number of composite images is 5, the number of photographs is, for example, 5 or 3. Among the composite images, there is a composite image that shares the same viewpoint as each photograph.
[0081] S22: Extract feature vectors from several photos respectively. In an illustrative implementation, for example, a CNN (Convolutional Neural Networks) is used to extract feature vectors from several photos respectively.
[0082] S23: Generate a second fused vector by fusing the feature vectors of several photos. In an illustrative implementation, for example, AutoML (Automated Machine Learning) or neural architecture search techniques are applied to determine the fusion scheme. The fusion scheme used in this step is the same as the fusion scheme used in step S13.
[0083] S30: Input the second fusion vector into the classifier trained in step S14 to obtain the classification result of the object.
[0084] In this illustrative embodiment, for example, several composite images can be generated first, and then the camera parameters (e.g., position and angle) for acquiring several photos can be determined based on the viewpoints of the composite images, so that the viewpoints of the several photos are the same as those of at least some of the composite images. Alternatively, several photos of the object can be acquired first, and then the camera parameters of the virtual camera used to generate the several composite images can be determined based on the viewpoints of the photos.
[0085] In this illustrative embodiment, when several photos have the same viewpoint as all of the composite images (i.e., the number of photos and composite images is the same, and the viewpoints correspond one-to-one), the feature vectors of the extracted composite images are fused in step S13 to generate a first fusion vector, and the feature vectors of the extracted photos are fused in step S23 to generate a second fusion vector.
[0086] Of course, it's also possible that several photos share the same viewpoint as some of the synthesized images, meaning the number of photos is less than the number of synthesized images. This situation might occur, for example, when training a classifier using five synthesized images (e.g., front view, rear view, top view, bottom view, and stereoscopic view), but when photographing an object, due to space limitations, it's impossible to obtain the same number of photos with the same angles; only photos with partially the same angles can be obtained, such as three (e.g., front view, rear view, and stereoscopic view). In this case, in step S23, the feature vectors of the extracted photos are fused with auxiliary vectors to generate a second fusion vector, where the total number of feature vectors and auxiliary vectors equals the number of synthesized images. And in step S13, the feature vectors of the synthesized images sharing the same viewpoint as the photos are fused with auxiliary vectors to generate a first fusion vector, where the total number of feature vectors and auxiliary vectors equals the number of synthesized images. For example, if the number of synthesized images is 5 and the number of photos is 3, then the number of auxiliary vectors required in both of the above steps is 2. This method does not require redefining the fusion scheme; it only needs to regenerate the first fusion vector based on the original fusion scheme and then retrain the classifier based on the regenerated first fusion vector. In this illustrative embodiment, the auxiliary vector is, for example, a unit vector (i.e., a vector with a magnitude of 1) or a zero vector.
[0087] However, this is not the only possible implementation. In other illustrative embodiments, in step S13, the feature vectors of several composite images with the same viewpoint as several photos can be fused to generate a first fusion vector, and in step S23, the feature vectors of several photos can be fused to generate a second fusion vector. Since the number of input vectors during fusion has changed, the fusion scheme needs to be redefined, and the first fusion vector is regenerated based on the new fusion scheme. Then, the classifier is retrained based on the regenerated first fusion vector.
[0088] In this object recognition method, the perspectives of the synthesized images are different, and correspondingly, the perspectives of the individual photographs are also different, which allows for the representation of more features. Several photographs share the same perspective as at least a portion of the synthesized images, thereby reducing interference caused by differences in angle. This object recognition method exhibits high accuracy.
[0089] Figure 3 This is a flowchart illustrating another illustrative implementation of the object recognition method. The object recognition method in this illustrative implementation is similar to... Figure 1The similarities to the object recognition methods shown will not be repeated here; the differences are as follows. In this illustrative embodiment, after completing step S11, step S15 is executed first: domain randomization processing is performed on several synthetic images, and then step S12 is executed. After completing step S21, step S24 is executed first: domain randomization processing is performed on several photos, and then step S22 is executed. Domain randomization can eliminate known features that cannot be used to partition objects (e.g., the environment in which the object is placed, the color of the object, etc.) in specific implementations, thereby improving the accuracy and recognition efficiency of the object recognition method.
[0090] The present invention also provides an object recognition system. Figure 4 This is a schematic diagram illustrating one implementation of an object recognition system. Figure 4 As shown, the object recognition system includes a processor 20 and an imaging mechanism 40. The processor 20 has a built-in image generation module 21, a feature extraction module 22, a fusion module 23, and a classifier module 24.
[0091] The image generation module 21 is capable of generating several composite images based on a three-dimensional digital model. These composite images have different viewpoints. In an illustrative embodiment, the image generation module 21, for example, uses CAD (Computer Aided Design) software to generate the composite images based on the three-dimensional digital model.
[0092] The feature extraction module 22 is capable of extracting feature vectors from several synthesized images. In this illustrative embodiment, the feature extraction module 22 uses, for example, a CNN (Convolutional Neural Networks) to extract feature vectors from several synthesized images, but it is not limited to this. In other illustrative embodiments, the feature extraction module 22 may also use other algorithms to extract feature vectors.
[0093] The fusion module 23 can generate a first fusion vector by fusing the feature vectors of several synthesized images. In an illustrative embodiment, the fusion module 23 may apply AutoML (Automated Machine Learning) technology or neural architecture search technology to determine the fusion scheme, which is beneficial to obtaining a better fusion scheme, but is not limited thereto.
[0094] The classifier module 24 can be trained based on the input first fusion vector. In this illustrative embodiment, the classifier module 24 is, for example, a deep learning-based classifier module 24, but is not limited thereto.
[0095] The imaging mechanism 40 is capable of acquiring several photographs of the object 80. In this illustrative embodiment, the imaging mechanism 40 includes a camera 41 and a support 42. The camera 41 is movably connected to the support 42. The object recognition system also includes a drive mechanism 50, which is capable of driving the camera 41 to move relative to the support 42. The processor 20 is capable of outputting a set of control signals based on the perspectives of several composite images. The drive mechanism 50 is capable of controlling the movement of the camera 41 according to the control signals to acquire several photographs, each with the same perspective as at least a portion of the composite images. This allows the shooting position and angle of the camera 41 to be controlled according to the perspective of the composite images, saving manpower. In this case, one camera 41 needs to change different positions and angles to acquire several photographs, but it is not limited to this. In other illustrative embodiments, the number of cameras 41 can be multiple, which can save the time required to acquire photographs.
[0096] The feature extraction module 22 can also extract feature vectors from several photos separately. The fusion module 23 can also fuse the feature vectors from several photos to generate a second fusion vector. The trained classifier module 24 can obtain the object classification result based on the input second fusion vector.
[0097] In this illustrative embodiment, when several photos have the same viewpoint as all of the composite images (i.e., the number of photos and composite images is the same, and the viewpoints correspond one-to-one), the fusion module 23 can fuse the feature vectors of the extracted composite images to generate a first fusion vector, and can fuse the feature vectors of the extracted photos to generate a second fusion vector.
[0098] Of course, it's also possible that several photos share the same viewpoint as some of the synthesized images, meaning the number of photos is less than the number of synthesized images. This can happen, for example, when training a classifier using five synthesized images (e.g., front view, rear view, top view, bottom view, and stereo view), but when photographing an object, space constraints prevent setting up a camera to obtain the same number and angles of photos; only photos with partially identical angles, such as three (e.g., front view, rear view, and stereo view), can be obtained. In this case, the fusion module 23 can combine the feature vectors of the extracted photos with auxiliary vectors to generate a second fusion vector, where the total number of feature vectors and auxiliary vectors equals the number of synthesized images. It can also combine the feature vectors of the synthesized images sharing the same viewpoint as the photos with auxiliary vectors to generate a first fusion vector, where the total number of feature vectors and auxiliary vectors equals the number of synthesized images. For example, if the number of synthesized images is five and the number of photos is three, then the number of auxiliary vectors required for both steps is two. This eliminates the need to redefine the fusion scheme; instead, it suffices to regenerate the first fusion vector based on the existing scheme and then retrain the classifier using the regenerated first fusion vector. In this illustrative embodiment, the auxiliary vector is, for example, a unit vector (i.e., a vector with a magnitude of 1) or a zero vector.
[0099] However, this is not the only possible implementation. In other illustrative embodiments, the fusion module 23 can, for example, fuse the feature vectors of several composite images with the same viewpoint as several photographs to generate a first fusion vector, and can fuse the feature vectors of several photographs to generate a second fusion vector. Since the number of input vectors during fusion has changed, the fusion module 23 needs to redetermine the fusion scheme and then re-fuse the first fusion vector according to the new fusion scheme. The classifier module 24 is then retrained based on the regenerated first fusion vector.
[0100] In the illustrative embodiment, the feature extraction module 22 can first perform domain randomization processing on several synthesized images, and then extract the feature vectors of the several synthesized images respectively. The feature extraction module 22 can first perform domain randomization processing on several photos, and then extract the feature vectors of the several photos respectively. Domain randomization can eliminate known features that cannot be used to partition objects (e.g., the environment in which the object is placed, the color of the object, etc.) in specific implementations, thereby improving the accuracy and efficiency of the object recognition method.
[0101] Figure 5 For explanation Figure 4 The illustrated object recognition system demonstrates its working process and is not intended to limit the scope of the invention. Figure 5As shown, the 3D digital model M is input to the image generation module 21, which generates composite images S1, S2, and S3 based on the 3D digital model M. The composite images S1, S2, and S3 are then input to the feature extraction module 22, which extracts feature vectors Sv1, Sv2, and Sv3. These feature vectors are then input to the fusion module 23, which fuses them to generate a first fusion vector Fv1. The first fusion vector Fv1 is then input to the classifier module 24 for training.
[0102] like Figure 5 As shown, the imaging mechanism 40 photographs object 80, acquiring photos P1, P2, and P3. Photo P1 shares the same viewpoint as the composite image S1, photo P2 shares the same viewpoint as the composite image S2, and photo P3 shares the same viewpoint as the composite image S3. Photos P1, P2, and P3 are input to the feature extraction module 22, which extracts feature vectors Pv1, Pv2, and Pv3. These feature vectors are then input to the fusion module 23, which fuses them to generate a second fusion vector Fv2. The second fusion vector Fv2 is then input to the classifier module 24 to obtain the classification result R.
[0103] like Figure 6 As shown in the illustrative embodiment, the feature extraction module 22, for example, has several convolutional neural networks, namely convolutional neural network CNN1, convolutional neural network CNN2, and convolutional neural network CNN3, which are used to process different synthetic images to obtain corresponding feature vectors. The parameters of the several convolutional neural networks can be different or the same. The fusion module 23 achieves fusion, for example, through a fusion network.
[0104] This object recognition system generates several synthetic images from different perspectives, and correspondingly acquires several photographs from different perspectives, allowing for the representation of more features. The processor can control the imaging mechanism or image generation module to ensure that the photographs share the same perspective as at least some of the synthetic images, thereby reducing interference caused by different angles. This object recognition system exhibits high accuracy.
[0105] Figure 7 This is a schematic diagram illustrating another implementation of an object recognition system. Figure 7 The object recognition system shown is Figure 4The similarities to the illustrated object recognition system will not be repeated here; the differences are described below. In this illustrative embodiment, the imaging mechanism 40 includes a plurality of cameras 41, the number of which corresponds to the number of photos to be acquired. The object recognition system also includes a position sensing unit 60. The position sensing unit 60 can detect the spatial positions and shooting angles of the plurality of cameras 41 and generate a set of viewing angle signals based on the spatial positions and shooting angles of the plurality of cameras 41. The processor 20 can determine the parameters used to generate a plurality of composite images based on the viewing angle signals, so that the viewing angles of the plurality of photos are the same as those of at least a portion of the composite images. This allows the parameters used to generate a plurality of composite images to be automatically determined based on the spatial positions and shooting angles of the cameras, saving manpower.
[0106] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.
[0107] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementation schemes or modifications made without departing from the spirit of the present invention, such as combinations, divisions or repetitions of features, should be included within the scope of protection of the present invention.
Claims
1. A method of object recognition, characterized by, include: Several composite images are generated based on a three-dimensional digital model, and the composite images have different perspectives; Extract the feature vectors of the several synthesized images respectively; A first fusion vector is generated by fusing the feature vectors of the several synthesized images; The first fusion vector is input into the classifier to train the classifier; Acquire several photographs of an object, wherein the several photographs are each taken from the same viewpoint as at least a portion of the several composite images, wherein camera parameters for acquiring the several photographs are determined based on the viewpoints of the several composite images, or software parameters for generating the several composite images are determined based on the viewpoints of the several photographs, so that the several photographs are each taken from the same viewpoint as at least a portion of the several composite images; Extract the feature vectors from the aforementioned photos respectively; A second fusion vector is generated by fusing the feature vectors of the aforementioned photos; and The second fusion vector is input into the trained classifier to obtain the classification result of the object.
2. The object recognition method as described in claim 1, characterized in that, When the perspectives of the several photos are the same as those of all the several composite images, the feature vectors of the extracted composite images are fused to generate the first fusion vector, and the feature vectors of the extracted photos are fused to generate the second fusion vector.
3. The object recognition method as described in claim 1, characterized in that, When the plurality of photos share the same viewpoint as some of the composite images, the extracted feature vectors of the plurality of photos are fused with auxiliary vectors to generate a second fusion vector, wherein the total number of feature vectors and auxiliary vectors of the plurality of photos is equal to the number of composite images. The extracted feature vectors of the composite images sharing the same viewpoint as the plurality of photos are then fused with auxiliary vectors to generate a first fusion vector, wherein the total number of feature vectors and auxiliary vectors of the composite images sharing the same viewpoint as the plurality of photos is equal to the number of composite images; or When the several photos have the same viewpoint as some of the several composite images, the feature vectors of the composite images with the same viewpoint as the several photos are fused to generate the first fusion vector, and the feature vectors of the several photos are fused to generate the second fusion vector.
4. The object recognition method as described in claim 1, characterized in that, The composite images are generated using CAD software based on the three-dimensional digital model.
5. The object recognition method as described in claim 1, characterized in that, The feature vectors of the several synthesized images and the feature vectors of the several photos are extracted using CNN; the classifier is a deep learning-based classifier.
6. The object recognition method as described in claim 1, characterized in that, The fusion scheme is determined by applying AutoML or neural architecture search techniques.
7. The object recognition method as described in claim 1, characterized in that, First, the several synthesized images are subjected to domain randomization processing, and then the feature vectors of the several synthesized images are extracted respectively; first, the several photos are subjected to domain randomization processing, and then the feature vectors of the several photos are extracted respectively.
8. An object recognition system, characterized in that, include: A processor (20) with the following built-in: An image generation module (21) is capable of generating several composite images based on a three-dimensional digital model, wherein the composite images have different viewpoints. A feature extraction module (22) is capable of extracting feature vectors from the several synthesized images respectively. A fusion module (23) is capable of generating a first fusion vector by fusing the feature vectors of the plurality of synthesized images, and A classifier module (24) capable of being trained based on the first fusion vector input; and A shooting agency (40) is capable of acquiring several photographs of an object, the several photographs having the same viewpoint as at least a portion of the several composite images, wherein camera parameters for acquiring the several photographs are determined based on the viewpoint of the several composite images, or software parameters for generating the several composite images are determined based on the viewpoint of the several photographs, so that the several photographs have the same viewpoint as at least a portion of the several composite images. The processor (20) can control the shooting mechanism (40) or the image generation module (21) to make the perspective of the plurality of photos the same as that of at least some of the plurality of composite images; the feature extraction module (22) can also extract the feature vectors of the plurality of photos respectively; the fusion module (23) can also generate a second fusion vector by fusing the feature vectors of the plurality of photos; and the trained classifier module (24) can obtain the classification result of the object based on the input second fusion vector.
9. The object recognition system as described in claim 8, characterized in that, The shooting mechanism (40) includes a camera (41) and a support (42), the camera (41) being movably connected to the support (42); the object recognition system further includes a driving mechanism (50) capable of driving the camera (41) to move relative to the support (42); the processor (20) is capable of outputting a set of control signals according to the perspective of the plurality of composite images; the driving mechanism (50) is capable of controlling the movement of the camera (41) according to the control signals to obtain the plurality of photographs, each having the same perspective as at least a portion of the plurality of composite images.
10. The object recognition system as described in claim 8, characterized in that, The shooting mechanism (40) includes several cameras (41), and the object recognition system further includes a position sensing unit (60). The position sensing unit (60) can detect the spatial position and shooting angle of the several cameras (41) and generate a set of perspective signals based on the spatial position and shooting angle of the several cameras (41). The processor (20) can determine the parameters used to generate the several composite images based on the perspective signals, so that the several photos have the same perspective as at least some of the several composite images.
11. The object recognition system as described in claim 8, characterized in that, When the perspective of each of the several photos is the same as that of all the several composite images, the fusion module (23) can fuse the feature vectors of the extracted composite images to generate the first fusion vector, and can fuse the feature vectors of the extracted photos to generate the second fusion vector.
12. The object recognition system as described in claim 8, characterized in that, When the plurality of photos have the same viewpoint as some of the plurality of composite images, the fusion module (23) can fuse the extracted feature vectors of the plurality of photos with auxiliary vectors to generate the second fusion vector, wherein the total number of feature vectors and auxiliary vectors of the plurality of photos is equal to the number of composite images, and can fuse the extracted feature vectors of the composite images with the same viewpoint as the plurality of photos with auxiliary vectors to generate the first fusion vector, wherein the total number of feature vectors and auxiliary vectors of the composite images with the same viewpoint as the plurality of photos is equal to the number of composite images; or When the several photos have the same viewpoint as some of the several composite images, the fusion module (23) can fuse the feature vectors of the several composite images with the same viewpoint as the several photos to generate the first fusion vector, and can fuse the feature vectors of the several photos to generate the second fusion vector.
13. The object recognition system as described in claim 8, characterized in that, The image generation module (21) can use CAD software to generate the several composite images based on the three-dimensional digital model.
14. The object recognition system as described in claim 8, characterized in that, The feature extraction module (22) uses CNN to extract the feature vectors of the several synthesized images and the feature vectors of the several photos respectively; the classifier module (24) is a deep learning-based classifier module (24).
15. The object recognition system as described in claim 8, characterized in that, The fusion module (23) can use AutoML technology or neural architecture search technology to determine the fusion scheme.
16. The object recognition system as described in claim 8, characterized in that, The feature extraction module (22) can first perform domain randomization processing on the several synthesized images, and then extract the feature vectors of the several synthesized images respectively; the feature extraction module (22) can first perform domain randomization processing on the several photos, and then extract the feature vectors of the several photos respectively.
17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores code for use by the system; when the processor executes the code, the system performs the object recognition method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Facial recognition using synthetic images
GB2402311A
