Information processing method and device for intelligent camera
The smart camera system addresses poor face recognition accuracy due to occlusions by adjusting posture and using a hybrid CNN-ViT model trained with unobstructed and occluded face images, achieving improved recognition results.
Patent Information
- Application Number
- CN202411125860.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-13
AI Technical Summary
Existing face recognition technologies struggle with large occlusions such as wearing masks, leading to poor recognition accuracy.
A smart camera system that adjusts its posture based on target recognition results and uses a hybrid CNN-ViT model trained with unobstructed and occluded face images to enhance recognition accuracy.
Accurately identifies occluded faces by leveraging a hybrid CNN-ViT model and adjusting camera posture, overcoming the limitations of prior face recognition systems.
Smart Images
Figure CN119131857A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to information processing technology, and in particular to an information processing method and device for a smart camera. Background Art
[0002] At present, face recognition technology is widely used in people's lives and work, such as access control systems, face payment, face punching, authority authentication, etc. Face recognition technology has developed rapidly in recent years and has made great progress.
[0003] However, in some scenarios, face recognition is often interfered with and restricted by many factors, which affects the recognition effect, such as mask occlusion, poor lighting, etc. Existing face recognition technology can handle small occlusions well, but it has poor results for large occlusions such as wearing masks, which seriously affects the accuracy of recognition results. Summary of the invention
[0004] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.
[0005] In view of this, the present invention provides an information processing method and device for a smart camera to at least solve at least one of the above-mentioned problems existing in the prior art.
[0006] One aspect of the present invention provides an information processing method for a smart camera, the method comprising: obtaining first image information, wherein the first image information includes image information of an area where the smart camera is located; performing target recognition according to the first image information, and obtaining second image information based on the target recognition result; the second image information is image information collected when the smart camera is in a first posture; and performing face recognition on the second image information using a predetermined recognition model to obtain a corresponding recognition result.
[0007] Furthermore, the method also includes: acquiring a first training data set and a second training data set; the first training data set includes multiple first training images, each first training image contains an unobstructed face; the second training data set includes at least multiple second training images, each second training image contains an occluded face; and using the first training data set and the second training data set to train a predetermined recognition model.
[0008] Furthermore, the first training data set includes multiple first training images, the multiple first training images include N groups of first training subsets, the N groups of first training subsets correspond to N predetermined users, the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
[0009] Furthermore, the second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
[0010] Furthermore, it also includes: obtaining the i-th group first training subset in advance; for each unobstructed face image in the i-th group first training subset, generating multiple obstructed face images based on a preset obstruction template as the i-th group second training subset.
[0011] Another aspect of the present invention provides an information processing device for a smart camera, the device comprising: a first acquisition unit, suitable for acquiring first image information, wherein the first image information includes image information of the area where the smart camera is located; a second acquisition unit, suitable for performing target recognition according to the first image information, and obtaining second image information based on the target recognition result; the second image information is image information collected when the smart camera is in a first posture; and a recognition unit, suitable for performing face recognition on the second image information using a predetermined recognition model to obtain a corresponding recognition result.
[0012] Furthermore, the device also includes a training unit; the training unit is suitable for: acquiring a first training data set and a second training data set; the first training data set includes multiple first training images, each first training image contains an unobstructed face; the second training data set includes at least multiple second training images, each second training image contains an obstructed face; and using the first training data set and the second training data set to train a predetermined recognition model.
[0013] Furthermore, the first training data set includes multiple first training images, the multiple first training images include N groups of first training subsets, the N groups of first training subsets correspond to N predetermined users, the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
[0014] Furthermore, the second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
[0015] Furthermore, the training unit is also suitable for: obtaining the i-th group first training subset in advance; for each unobstructed face image in the i-th group first training subset, generating multiple occluded face images based on a preset occlusion template as the i-th group second training subset.
[0016] The present invention provides an information processing method and device for an intelligent camera. The method and device can accurately identify an occluded face in the case of occlusion during face recognition, thus overcoming the shortcomings of the prior art.
[0017] These and other advantages of the present invention will become more apparent through the following detailed description of the best embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The present invention may be better understood by referring to the following description given in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout the drawings to represent the same or similar components. The accompanying drawings, together with the following detailed description, are incorporated in and form a part of this specification and are used to further illustrate the preferred embodiments of the present invention and to explain the principles and advantages of the present invention.
[0019] in:
[0020] Figure 1 is a flowchart showing an exemplary process of an information processing method for a smart camera according to an embodiment of the present invention;
[0021] Figure 2 is a schematic diagram showing a sub-process of an information processing method for a smart camera according to an embodiment of the present invention;
[0022] Figure 3 is a block diagram showing an exemplary structure of an information processing device for a smart camera according to an embodiment of the present invention.
[0023] Those skilled in the art will appreciate that the elements in the drawings are shown only for simplicity and clarity and are not necessarily drawn to scale. For example, the size of some elements in the drawings may be enlarged relative to other elements to help improve the understanding of the embodiments of the present invention. DETAILED DESCRIPTION
[0024] Exemplary embodiments of the present invention will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual implementation are described in the specification. However, it should be understood that many implementation-specific decisions must be made in the process of developing any such actual implementation in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is only a routine task for those skilled in the art who benefit from the content of the present invention.
[0025] It is also necessary to explain here that, in order to avoid obscuring the present invention due to unnecessary details, only the device structure and / or processing steps closely related to the scheme according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.
[0026] One aspect of the present invention provides an information processing method for a smart camera, the method comprising: obtaining first image information, wherein the first image information includes image information of an area where the smart camera is located; performing target recognition according to the first image information, and obtaining second image information based on the target recognition result; the second image information is image information collected when the smart camera is in a first posture; and performing face recognition on the second image information using a predetermined recognition model to obtain a corresponding recognition result.
[0027] like Figure 1 As shown, in one embodiment of the present invention, an information processing method for a smart camera includes: Figure 1 Steps S110 to S130 are shown.
[0028] In step S110, first image information is acquired, wherein the first image information includes image information of an area where the smart camera is located.
[0029] Next, in step S120, target recognition is performed according to the first image information, and second image information is obtained based on the target recognition result; the second image information is image information collected when the smart camera is in the first posture.
[0030] The area where the smart camera is located is, for example, one of a plurality of predetermined areas. In an embodiment of the present invention, for example, in security inspection, gates or other application scenarios, a group of multiple smart cameras are provided, and each smart camera can be adjusted to any posture between 360 degrees horizontally and 180 degrees vertically. That is, in each predetermined area, one or more smart cameras are configured in the area.
[0031] The smart camera collects first image information in the corresponding area in real time. After performing target recognition based on the first image information, the posture of the smart camera is adjusted based on the result of the target recognition, so that it is in a first posture corresponding to the target recognition result, so that the smart camera shoots second image information facing the target.
[0032] In an embodiment of the present invention, when multiple smart cameras are configured corresponding to each predetermined area, the first-level sub-camera is first used to collect the above-mentioned second image information. Based on template matching, it is determined whether the second image information is an occluded face image. If so, the second-level sub-camera among the multiple smart cameras is started to obtain additional image information of the user, wherein the second-level sub-camera among the multiple smart cameras includes two or more cameras, and the above-mentioned additional image information includes facial images on the left and right sides of the user (for example, including two or more image data). The above-mentioned additional image information is used to be put together into the second image information as a test image to be identified. In this way, in some cases, such as when the front of the user is blocked, several groups of user facial images can be taken to obtain more accurate recognition results.
[0033] Thus, in step S130, the second image information is subjected to face recognition using a predetermined recognition model to obtain a corresponding recognition result.
[0034] For example, the above-mentioned information processing method for smart camera may also include: Figure 2 Steps shown.
[0035] like Figure 2 As shown, in step S210, a first training data set and a second training data set are obtained; wherein the first training data set includes multiple first training images, each of which contains an unobstructed face; and the second training data set includes at least multiple second training images, each of which contains an obstructed face.
[0036] After executing step S210, in step S220, a predetermined recognition model is trained using the first training data set and the second training data set.
[0037] Among them, the preset model adopts CNN (Convolutional Neural Network) and ViT model (Vision Transformer, a deep learning model based on attention mechanism) fused by stage-level fusion. In this hybrid model, CNN is used to obtain local feature information, and ViT is used to obtain global feature information. The two are fused by stage-level fusion, for example.
[0038] For example, the first training data set includes multiple first training images, the multiple first training images include N groups of first training subsets, the N groups of first training subsets correspond to N predetermined users, the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
[0039] For example, the second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
[0040] For example, the above method also includes: obtaining the i-th group first training subset in advance; for each unobstructed face image in the i-th group first training subset, generating multiple obstructed face images based on a preset obstruction template as the i-th group second training subset.
[0041] For example, the training process of the predetermined recognition model may be processed in advance, or may be processed synchronously each time the actual image is processed.
[0042] For example, the second image information can be put into the second training data set, and then the first training data set and the second training data set currently containing the test image (i.e., the second image information) can be used to train the predetermined recognition model. In this way, while completing the training, the recognition result of the test image can also be obtained. Figure 1 A flow chart of a method for face recognition with occlusion according to the present invention is shown.
[0043] In an embodiment of the present invention, local features are extracted from each first training image, and a corresponding global vector is obtained based on sparse coding of the obtained local features. The corresponding face region and its feature descriptor are determined using the global vector to obtain a similarity representation corresponding to each first training image.
[0044] Similarly, local features are extracted from each second training image, and a corresponding global vector is obtained based on sparse coding of the obtained local features. The corresponding face region and its feature descriptor are determined using the global vector to obtain a similarity representation corresponding to each second training image.
[0045] Feature matching is performed based on the similarity representation of each first training image and each second training image, so that the trained model can match the first and second training images corresponding to the same user. During this process, for each second training image, the number of first training images of the user matched by the second training image is calculated. Assuming that the number of first training images of the user matched by the second training image is P1, and the number of all first training images of the user is P, the purpose of training is to make the ratio of P1 to P as large as possible to optimize the model parameters.
[0046] For example, a predetermined threshold may be set. If the ratio of P1 to P in the training result for a certain user is less than the predetermined threshold, the training is invalidated and the iterative calculation continues until the ratio of P1 to P calculated for each user is greater than or equal to the threshold.
[0047] In the testing phase, for example, the testing sample (ie, the occluded face image to be repaired) may be included in the second training sample set, and the above steps S110 to S180 may be repeatedly executed to complete the repair of the testing sample.
[0048] The present invention also provides an information processing device for a smart camera, the device comprising a first acquisition unit 310 , a second acquisition unit 320 , and an identification unit 330 .
[0049] like Figure 3 As shown, the first acquisition unit 310 is adapted to acquire first image information, wherein the first image information includes image information of the area where the smart camera is located.
[0050] The second acquisition unit 320 is adapted to perform target recognition according to the first image information, and obtain second image information based on the target recognition result; the second image information is image information collected when the smart camera is in the first posture.
[0051] In addition, the recognition unit 330 is adapted to perform face recognition on the second image information using a predetermined recognition model to obtain a corresponding recognition result.
[0052] For example, the device also includes a training unit 340, which is suitable for: acquiring a first training data set and a second training data set; the first training data set includes multiple first training images, each first training image contains an unobstructed face; the second training data set includes at least multiple second training images, each second training image contains an occluded face; and using the first training data set and the second training data set to train a predetermined recognition model.
[0053] For example, the first training data set includes multiple first training images, the multiple first training images include N groups of first training subsets, the N groups of first training subsets correspond to N predetermined users, the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
[0054] For example, the second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
[0055] For example, the training unit 340 is also suitable for: obtaining the i-th group first training subset in advance; for each unobstructed face image in the i-th group first training subset, generating multiple occluded face images based on a preset occlusion template as the i-th group second training subset.
[0056] The above-mentioned information processing method and device for a smart camera of the present invention can accurately identify the occluded face when there is occlusion during face recognition, thus overcoming the shortcomings of the prior art.
[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention and the beneficial effects brought about by the present invention have been described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the claims of the present invention.
Claims
1. An information processing method for a smart camera, characterized in that: The method comprises: Acquire first image information, wherein the first image information includes image information of an area where the smart camera is located; Performing target recognition according to the first image information, and obtaining second image information based on the target recognition result; the second image information is image information collected when the smart camera is in the first posture; Using a predetermined recognition model, face recognition is performed on the second image information to obtain a corresponding recognition result.
2. The information processing method for a smart camera according to claim 1, characterized in that: The method further comprises: Acquire a first training data set and a second training data set; the first training data set includes a plurality of first training images, each of which includes an unobstructed human face; the second training data set includes at least a plurality of second training images, each of which includes an obstructed human face; A predetermined recognition model is trained using the first training data set and the second training data set.
3. The information processing method for a smart camera according to claim 2, characterized in that: The first training data set includes multiple first training images, and the multiple first training images include N groups of first training subsets. The N groups of first training subsets correspond to N predetermined users, and the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
4. The information processing method for a smart camera according to claim 3, characterized in that: The second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
5. The information processing method for a smart camera according to claim 4, characterized in that: Also includes: Obtain the first training subset of the i-th group in advance; For each unobstructed face image in the i-th group of first training subsets, a plurality of obstructed face images are generated based on a preset obstruction template as the i-th group of second training subsets.
6. An information processing device for a smart camera, characterized in that: The device comprises: A first acquisition unit, adapted to acquire first image information, wherein the first image information includes image information of an area where the smart camera is located; A second acquisition unit is adapted to perform target recognition according to the first image information, and obtain second image information based on the target recognition result; the second image information is image information collected when the smart camera is in the first posture; The recognition unit is adapted to perform face recognition on the second image information using a predetermined recognition model to obtain a corresponding recognition result.
7. The information processing device for a smart camera according to claim 1, characterized in that: The device further comprises a training unit; the training unit is adapted to: Acquire a first training data set and a second training data set; the first training data set includes a plurality of first training images, each of which contains an unobstructed face; the second training data set includes at least a plurality of second training images, each of which contains an obstructed face; and use the first training data set and the second training data set to train a predetermined recognition model.
8. The information processing device for a smart camera according to claim 2, characterized in that: The first training data set includes multiple first training images, and the multiple first training images include N groups of first training subsets. The N groups of first training subsets correspond to N predetermined users, and the i-th group of first training subsets includes a group of unobstructed face images of the i-th predetermined user; N is an integer greater than 1, i=1, 2,…, N.
9. The information processing device for a smart camera according to claim 3, characterized in that: The second training data set includes multiple second training images, the multiple second training images include N groups of second training subsets, the N groups of second training subsets correspond to N predetermined users, and the i-th group of second training subsets includes a group of occluded face images of the i-th predetermined user.
10. The information processing device for a smart camera according to claim 4, characterized in that: The training unit is also adapted to: Pre-obtaining the i-th group of first training subsets; for each unobstructed face image in the i-th group of first training subsets, generating multiple obstructed face images based on a preset obstruction template as the i-th group of second training subsets.