Methods, apparatus, media, and devices for image recognition

Abstract

A method, device, storage medium and electronic equipment for image recognition are disclosed. The method comprises: determining a face image, human body region information and key point information of a human body skeleton of a to-be-recognized object from a to-be-recognized image; using a first pre-trained prediction model to predict the age of the to-be-recognized object based on the face image to obtain a predicted age of the to-be-recognized object; using a second pre-trained prediction model to predict an age interval in which the to-be-recognized object is located based on the human body region information and the key point information of the human body skeleton to obtain a predicted age interval of the to-be-recognized object; and determining whether the to-be-recognized object is a child based on the predicted age and the predicted age interval. The method overcomes the limitation of relying on single feature information for recognition, reduces the adverse effects of face or body occlusion on image recognition, and helps to improve the generalization, fault tolerance and accuracy of child recognition of image recognition.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a method and apparatus for image recognition, a storage medium and an electronic device. BACKGROUND

[0002] Currently, for the caring services for children in the car cabin, such as child left reminding, child dangerous behavior alarming, and safety seat correct wearing prompting, it is necessary to identify children from images collected by cameras arranged in the cabin.

[0003] In the related art, the method for identifying children in the cabin usually includes the following two kinds: the first kind is to estimate the age based on the facial features in the image to predict whether the object is a child; the second kind is to estimate the height of the object based on the skeleton key points in the image by pixel-level operation to estimate the length of the skeleton, and then compare the height of the object with a preset height threshold to predict the age of the object. Limited by the type, installation position of the camera in the cabin, and the seat shielding and many other factors, the performance of the two recognition methods in actual application is poor, for example, when the face is shielded, it may lead to the inability to identify by the first method; and when the body part of the object is shielded, it may lead to low recognition accuracy of the second method. SUMMARY

[0004] To solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide a method and apparatus for image recognition, a storage medium and an electronic device.

[0005] According to an aspect of an embodiment of the present disclosure, a method for image recognition is provided, comprising: determining a face image, body region information, and key point information of a human body skeleton of a to-be-identified object from a to-be-identified image; predicting an age of the to-be-identified object based on the face image by using a first pre-trained prediction model to obtain a predicted age of the to-be-identified object; predicting an age interval in which the to-be-identified object is located based on the body region information and the key point information of the human body skeleton by using a second pre-trained prediction model to obtain a predicted age interval of the to-be-identified object; and determining whether the to-be-identified object is a child based on the predicted age and the predicted age interval.

[0006] According to still another aspect of the embodiments of the present disclosure, an apparatus for image recognition is provided, which comprises: an information acquisition unit configured to determine a face image of a to-be-recognized object, body region information and key point information of a body skeleton from a to-be-recognized image; a first prediction unit configured to predict an age of the to-be-recognized object based on the face image by using a pre-trained first prediction model, to obtain a predicted age of the to-be-recognized object; a second prediction unit configured to predict an age interval in which the to-be-recognized object is located based on the body region information and the key point information of the body skeleton by using a pre-trained second prediction model, to obtain a predicted age interval of the to-be-recognized object; and a result determination unit configured to determine whether the to-be-recognized object is a child based on the predicted age and the predicted age interval.

[0007] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores a computer program for implementing the above method.

[0008] According to still another aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises: a processor; a memory for storing executable instructions of the processor; and the processor is configured to read the executable instructions from the memory and execute the instructions to implement the above method.

[0009] The method for image recognition of the present disclosure can predict the age of the to-be-recognized object by using the face information in the to-be-recognized image, and simultaneously use the body region information and the key point information of the body skeleton in the to-be-recognized image. The age interval of the to-be-recognized object is predicted, and the predicted age and the predicted age interval are fused to determine whether the to-be-recognized object is a child. The limitation of relying on single feature information for recognition is overcome, the adverse effects of face or body being blocked on image recognition are reduced, and the generalization, fault tolerance and accuracy of child recognition of image recognition are improved.

[0010] The technical solutions of the present disclosure will be described in further detail below by means of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. The drawings provided herein are for illustrative purposes only and constitute part of the detailed description. They serve the purpose of providing further understanding of the present disclosure and together with the detailed description explain the present disclosure. They do not constitute a limitation on the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 is a scene diagram to which the method for image recognition of the present disclosure is applied;

[0013] Figure 2This is a schematic flowchart of an embodiment of the image recognition method disclosed herein;

[0014] Figure 3 This is a flowchart illustrating the process of determining a face image, human body region information, and key point information of the human skeleton in one embodiment of the image recognition method disclosed herein.

[0015] Figure 4 This is a schematic diagram of a process for predicting age ranges in one embodiment of the image recognition method disclosed herein;

[0016] Figure 5 This is a schematic diagram of the structure of one embodiment of the image recognition device disclosed herein;

[0017] Figure 6 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0018] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0019] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0020] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0021] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0022] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0023] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0024] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0025] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0028] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0029] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0030] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0031] SUMMARY

[0032] In the process of developing this disclosure, the inventors discovered that when using facial or body features to identify whether an object in an image is a child, related technologies are easily affected by environmental factors, leading to poor image recognition performance. For example, when relying on facial features for recognition, if the object's facial area is occluded, facial recognition is prone to failure or inaccuracy; when relying on body features for recognition, if the object's body is occluded, pixel-level calculation errors in bone length are likely to be large, also making it difficult to guarantee accuracy. Furthermore, this method has poor generalization ability and cannot adapt to complex scenarios.

[0033] It is evident that the above methods are easily limited by environmental factors, resulting in poor accuracy and generalization of image recognition.

[0034] Exemplary system

[0035] Figure 1 This is a schematic diagram of a scenario in which the image recognition method of this disclosure is applicable, such as... Figure 1 As shown, a camera 100 installed in the car cabin can capture images 110 inside the cabin. Images 110 can typically include one or more occupants. In order to identify whether an occupant is a child so as to provide corresponding care services, the camera 100 can send the captured images 110 to an electronic device 120 for image recognition, such as an in-vehicle computer or a cloud server.

[0036] Electronic device 120 can determine from image 110 the face image 111, face region information 112, and key point information 113 of the human skeleton of one or more objects to be identified (i.e., passengers in the cabin) from image 110. Then, the face image 111 is processed using a first prediction model 130 to predict the predicted age of the object to be identified; at the same time, the human region information 112 and the key point information 113 of the human skeleton are processed using a second prediction model 140 to predict the predicted age range of the object to be identified; then, based on the predicted age and the predicted range, it is determined whether the object to be identified is a child.

[0037] Exemplary method

[0038] Figure 2 This is a schematic flowchart of an embodiment of the image recognition method disclosed herein. This embodiment can be applied to... Figure 1 On the electronic device 120 shown, such as Figure 2 As shown, the process includes the following steps:

[0039] Step 210: Determine the face image, human body region information, and key point information of the human skeleton of the object to be identified from the image to be identified.

[0040] In this embodiment, the image to be identified usually refers to an image containing an object to be identified. For example, it may be an image of the interior of a car cabin captured by a camera installed inside the cabin, which may contain one or more passengers, and each passenger is an object to be identified.

[0041] Human body region information can characterize the position and region of the human body in the image to be identified, such as the coordinate information of the human body detection box.

[0042] Keypoint information of the human skeleton can characterize the positional information of the skeletal keypoints of the object to be identified in the image to be identified, such as the pixel coordinates of the skeletal keypoints. As an example, the keypoint information of the human skeleton may include the pixel coordinates of the 15 joints of the human body in the image to be identified and their confidence scores. It should be noted that the number of joints can be set according to actual needs or algorithms, and this disclosure does not limit it.

[0043] As an example, the execution entity (e.g.) Figure 1 The electronic device 120 shown can use an object detection algorithm to segment the face image, human body region information and key point information of human skeleton corresponding to each object to be identified from the image to be identified.

[0044] Step 220: Using the pre-trained first prediction model, predict the age of the object to be identified based on the face image, and obtain the predicted age of the object to be identified.

[0045] In this embodiment, the first prediction model represents the correspondence between a face image and the predicted age. For example, a fully connected network, a convolutional neural network, or a recurrent neural network can be used as the first prediction model. By performing feature extraction, feature mapping, and other processing on the face image, the confidence scores of multiple preset age values ​​are output. Then, the age value with the highest confidence score is used as the predicted age of the object to be identified corresponding to the face image.

[0046] Step 230: Using the pre-trained second prediction model, predict the age range of the object to be identified based on human body region information and key point information of human skeleton, and obtain the predicted age range of the object to be identified.

[0047] In this embodiment, the second prediction model represents the correspondence between human body region information, key point information of human skeleton and preset age range.

[0048] As an example, the second prediction model can be a convolutional neural network or a recurrent neural network. By performing feature encoding and logistic regression (e.g., a softmax function) on human body region information and key point information of human skeleton, the confidence of multiple preset age intervals is estimated, and then the age interval with the highest confidence is used as the predicted age interval of the object to be identified.

[0049] Optionally, the second prediction model can employ a lightweight fully connected network to reduce the number of parameters in the network model, which helps to reduce the performance requirements of the execution entity in the prediction process, thereby increasing the frame rate.

[0050] In this embodiment, the age range can correspond to different age stages in a person. For example, different age ranges can be set for children, adolescents, and adults. For instance, 0-6 years old is the age range for children, 7-12 years old is the age range for adolescents, and 13-99 years old is the age range for adults.

[0051] Step 240: Based on the predicted age and predicted age range, determine whether the subject to be identified is a child.

[0052] In this embodiment, the executing entity can fuse the predicted age obtained in step 220 and the predicted age range obtained in step 230 according to a preset fusion strategy to determine whether the object to be identified is a child.

[0053] It is understandable that when the predicted age is within the predicted age range, it means that the two prediction results match, and at this time, it can be determined whether the subject to be identified is a child based on either one.

[0054] When the predicted age is not within the predicted age range, one of the predicted results can be selected as the basis according to the preset fusion strategy, and the object to be identified can be determined as a child.

[0055] As an example, the priority of predicted age and predicted age range can be determined based on the degree of occlusion of the subject's face and body. For instance, when the degree of face occlusion (e.g., the ratio of the occluded area of ​​the face to the total face area) is greater than the degree of body occlusion (e.g., the ratio of the occluded area of ​​the body to the total body area), a higher priority can be given to the predicted age range. When the predicted age and predicted age range conflict, the predicted age range can be used to determine whether the subject is a child. Specifically, assuming the predicted age of the subject is 18 years old and the predicted age range is 0-6 years old, the subject can be identified as a child based on the predicted age range. Conversely, when the degree of face occlusion is less than the degree of body occlusion, a higher priority can be given to the predicted age.

[0056] For example, priority can be set between the first and second prediction models based on their accuracy during the testing phase. When there is a conflict between the predicted age and the predicted age range, the prediction result with higher priority can be selected as the basis for judgment.

[0057] The image recognition method in this embodiment can predict the age of the subject by using facial information in the image to be recognized, and simultaneously predict the age range of the subject by using human body region information and key point information of the human skeleton in the image to be recognized. The predicted age and predicted age range are then fused to determine whether the subject is a child. This overcomes the limitations of relying on a single feature for recognition, reduces the adverse effects of face or body occlusion on image recognition, and helps improve the generalization, fault tolerance, and accuracy of child recognition in image recognition.

[0058] Furthermore, in the process of developing this disclosure, the applicant discovered that in related technologies, when predicting age based on skeletal information, the explicit skeletal length is usually calculated based on the coordinates of key points on the human body to obtain explicit height information, and then the age of the subject is estimated based on a preset correspondence between height and age. In contrast, the image recognition method in this embodiment directly determines whether the subject to be identified is a child in an end-to-end manner, without the need for complex processes such as calculating skeletal length, height information, and setting a correspondence between height and age, thus simplifying the recognition process.

[0059] Next, refer to Figure 3 , Figure 3 This is a flowchart illustrating the process of determining a face image, human body region information, and key point information of the human skeleton of an object to be identified in one embodiment of the image recognition method disclosed herein, as shown below. Figure 3 As shown, the process includes the following steps:

[0060] Step 310: Use the face branch network in the pre-trained multi-task detection network to perform face detection on the image to be recognized, and obtain the face detection box.

[0061] In this embodiment, the face detection box can represent the area covered by the face of the object to be identified in the image to be identified.

[0062] Step 320: Extract face images from the image to be identified based on the face detection bounding box.

[0063] Step 330: Using the human body branch network in the multi-task detection network, perform human body detection on the object to be identified, obtain the human body detection box, and determine the coordinates of the human body detection box as human body region information.

[0064] In this embodiment, the human body detection box can represent the area covered by the body of the object to be identified in the image to be identified.

[0065] Step 340: Using the human body branch network in the multi-task detection network, based on the human body detection box, predict the coordinates and confidence of the key points of the human skeleton of the object to be identified, and obtain the key point information of the human skeleton.

[0066] In a specific example, the executing entity can extract image features from the image to be identified using the backbone network of a multi-task detection model. Then, a face branch network is used to perform face detection processing on these features, estimating the face region in the image and representing it with a face detection bounding box. Simultaneously, a body branch network is used to perform human detection processing on the image features, estimating the body region in the image and representing it with a body detection bounding box. Next, regression processing can be performed on the pixels in the image region marked by the body detection bounding box to predict the confidence (i.e., probability value) of each pixel as a human skeletal keypoint. Specifically, assuming a pre-set number of 15 human skeletal keypoints, each pixel can include the confidence values ​​corresponding to each of the 15 keypoints. Then, for each keypoint, the pixel with the highest confidence is selected as that keypoint, thus obtaining the coordinates and confidence values ​​of the keypoints of the human skeleton of the object to be identified, thereby obtaining the keypoint information of the human skeleton. In this way, by using a multi-task detection model, we can obtain the coordinates of the face detection box, the coordinates of the human body detection box, and the coordinates and confidence scores of the human skeleton key points. Then, based on the coordinates of the face detection box, we can extract the face image from the image to be recognized.

[0067] Figure 3 In the illustrated embodiment, a multi-task detection model is used to process the image to be identified, identifying the face detection box, body detection box, and key point information of the human skeleton of the object to be identified in the image. Then, the face image is extracted from the image to be identified based on the face detection box, which can improve the efficiency of obtaining the feature information required for subsequent prediction steps.

[0068] In some optional embodiments of any of the above embodiments, before performing step 220, the image to be recognized may be preprocessed as follows: the face image is adjusted to a preset size; the pixel values ​​in the resized face image are normalized.

[0069] In this embodiment, the preset size can characterize the input size of the first prediction model.

[0070] As an example, the executing entity can normalize the pixel values ​​in the resized face image using the following formula (1).

[0071]

[0072] Where x represents the pixel value and a represents the normalization parameter. For example, when a is 128, the pixel value can be normalized to the interval [-1, 1]; when a is 255, the pixel can be normalized to the interval [0, 1].

[0073] In this embodiment, preprocessing the face image helps to further improve the accuracy of the first prediction model.

[0074] Furthermore, the preprocessing of the face image may also include: identifying facial key points from the face image and obtaining the coordinate information of the facial key points; and adjusting the face pose in the face image to a preset pose based on the coordinate information of the facial key points.

[0075] In this embodiment, the preset pose typically refers to the pose of the face when it is directly facing the image acquisition device. By adjusting the face pose, the information loss caused by the face orientation can be reduced, which helps to further improve the accuracy of the first prediction model.

[0076] As an example, the executing entity can identify the coordinate information of facial key points from a face image using a key point detection model. This model could be, for example, ResNet (Deep Residual Network), DenseNet (Densely connected convolutional networks), or MobileNetV2. The facial key points can be set according to actual needs; for example, it could include five key points: two eyes, a nose, and two corners of the mouth. Then, the facial pose can be estimated using the relative positions between these key points, and finally, an affine transformation can be used to adjust the estimated facial pose to a preset pose.

[0077] In some optional implementations of any of the above embodiments, step 220 may further include: predicting a face image using a first prediction model and outputting a first probability vector, the first probability vector including a first preset number of confidence values ​​of age values; and determining the predicted age based on the number of confidence values ​​in the probability vector that are greater than a preset probability threshold.

[0078] In this embodiment, the dimension of the first probability vector represents the number of preset age values. For example, if the age range predicted by the first prediction model is 1-85 years old, the dimension of the first probability vector is 85, where each dimension corresponds to an age value.

[0079] The preset probability threshold can be set empirically or determined through statistical analysis. As an example, the first prediction model can be a fully connected network, a convolutional neural network, or a recurrent neural network, with an output probability vector of dimension 85. Assuming that the number of confidence scores greater than 0.5 in the first probability vector is 6, then the predicted age of the object to be identified is 6 years old.

[0080] In one example of this implementation, the predicted age can be obtained using the following formula (2):

[0081]

[0082] Where N represents the number of dimensions of the first probability vector, I represents the face image, and f k (i) represents the probability value of the k-th dimension in the first probability vector corresponding to the face image, 1[f k (I)>0.5] means when f k When (I) > 0.5, it is treated as 1 in the summation operation.

[0083] For example, assuming the first probability vector has a dimension of 85 (i.e., the prediction range of the first prediction model is 1-85 years old), for a face image, the predicted 85-dimensional first probability vector P = [p1, p2, ..., p...] is obtained. 85 ], p i =f i (I) (i = 1, 2, ..., 85) represents the probability that the predicted age corresponding to the face image is greater than or equal to the age value i. Assuming the actual age corresponding to the face image is 20 years old, then p1 is a probability value close to 1, for example, p1 = 0.9. Similarly, p2 - p... 20 Both are probability values ​​close to 1, and p 21 -p 85 These are probability values ​​close to 0, such as 0.1 or 0.2. Based on this, the number of dimensions with probability values ​​greater than a preset probability threshold (e.g., 0.5) can be determined, where p1-p 20 All are probability values ​​greater than 0.5, p 21 -p 85 All values ​​are less than 0.5, resulting in a predicted age of 20.

[0084] As an example, the first prediction model can be trained as follows: Obtain sample face images and their labels. The label can include probability values ​​for each dimension corresponding to the sample face image. These probability values ​​can be set to 0 or 1, specifically determined by the relationship between the actual age of the object represented by the sample face image and the age value corresponding to that dimension. For example, if the actual age of the object represented by the sample face image is greater than or equal to the age value corresponding to that dimension, then the probability value for that dimension is 1; otherwise, it is 0. Using the sample face image as input and the label as the desired output, the initial first prediction model can be trained to obtain the trained first prediction model.

[0085] In this embodiment, the predicted age of the object to be identified is estimated by the confidence level in the first probability vector. This more granular approach helps to improve the accuracy of age prediction.

[0086] Next, refer toFigure 4 , Figure 4 The diagram illustrates a flowchart of predicting an age range in one embodiment of the image recognition method of this disclosure. In some optional implementations of any of the above embodiments, step 230 may also employ... Figure 4 The process is shown below. Figure 4 As shown, the process includes the following steps:

[0087] Step 410: Determine the height and area of ​​the human body detection box based on the human body region information.

[0088] Step 420: Determine the upper body height of the object to be identified based on the key point information of the human skeleton.

[0089] As an example, the height of the upper body of the object to be identified can be determined based on the coordinates of the key points corresponding to the shoulder and hip joints.

[0090] Step 430: Encode the human body region information, the height and area of ​​the human body detection box, the height of the upper body, and the key point information of the human skeleton into a feature vector.

[0091] In this embodiment, the feature vector can represent the human feature information of the object to be identified contained in the image to be identified. The human body region information, the height and area of ​​the human body detection box, the height of the upper body, and the key point information of the human skeleton are all in numerical form, and multiple numerical values ​​can be directly encoded into the form of feature vector.

[0092] In a specific example, the human body region information can include the pixel coordinates of two diagonal points, corresponding to 4 values; the height and area of ​​the human body detection box, and the upper body height, each correspond to 1 value; the key point information of the human skeleton can include the pixel coordinates of key points corresponding to 15 joints and their confidence scores, with each pixel coordinate having 1 value for its horizontal and vertical coordinates, and 1 value for its confidence score. In this way, the key point information of the human skeleton can correspond to 45 values. Combining the 4 values ​​corresponding to the human body region information and the 3 values ​​corresponding to the height and area of ​​the human body detection box and the upper body height, we can obtain 52 values. The feature vector formed by this can include 52 dimensions.

[0093] In some optional implementations of this embodiment, the feature vector can be generated as follows: based on the resolution of the image to be identified, the coordinates of the human body detection box and the coordinates of the key points of the human skeleton are normalized; the processed coordinates of the human body detection box, the coordinates of the key points of the human skeleton, the confidence of the key points of the human skeleton, the upper body height, and the height and area of ​​the human body detection box are encoded into a feature vector.

[0094] As an example, if the resolution of the image to be recognized is m×n, then m can be used to normalize the horizontal coordinates of the human detection box and the horizontal coordinates of the key points of the human skeleton, and n can be used to normalize the vertical coordinates of the human detection box and the vertical coordinates of the key points of the human skeleton.

[0095] In this embodiment, the coordinates of the human detection box and the coordinates of the key points of the human skeleton are normalized and then encoded to generate feature vectors, which can further improve the prediction accuracy of the second prediction model.

[0096] Step 440: Input the feature vector into the second prediction model to obtain the second probability vector, which includes the confidence scores of a second preset number of age intervals.

[0097] As an example, the second prediction model could be a multilayer perceptron. By classifying and predicting feature vectors, it can output a second probability vector, where each value represents the confidence level for a preset age range. For instance, if there are three preset age ranges, the second probability vector has a dimension of 3, corresponding to the confidence levels for each of the three age ranges.

[0098] Step 450: Determine the predicted age range based on the second probability vector.

[0099] For example, the age range corresponding to the highest confidence level in the second probability vector can be determined as the predicted age range.

[0100] exist Figure 4 In the illustrated embodiment, the height and area of ​​the human detection box and the upper body height of the object to be identified can be determined based on human region information and key point information of the human skeleton. The human region information, the height and area of ​​the human detection box, the upper body height, and the key point information of the human skeleton are encoded to generate a high-dimensional feature vector. Based on this feature vector, the predicted age range of the object to be identified is then predicted. On the one hand, age range prediction can be achieved without calculating the length information of the human skeleton to obtain explicit height information; on the other hand, the dimensionality of the feature information required for the prediction process of the second prediction model can be increased, thereby improving the prediction accuracy of the second prediction model.

[0101] In some optional implementations of any of the above embodiments, step 240 may further include: when a first preset condition is met, determining whether the object to be identified is a child based on the predicted age range, wherein the first preset condition includes one of the following: the face occlusion area in the face image is greater than a preset area threshold and the number of visible key points in the key points of the human skeleton is greater than a preset number; or, the face occlusion area is not greater than the preset area threshold and the difference between the age range in which the predicted age is located and the predicted age range is greater than a preset difference threshold; when the first preset condition is not met, determining whether the object to be identified is a child based on the age range in which the predicted age is located.

[0102] In this embodiment, different strategies are used to fuse the predicted age and the predicted age range for different scenarios to determine the recognition result. This makes the image recognition method in this embodiment applicable to different scenarios, which helps to improve the applicability of the image recognition method and ensures the accuracy in different scenarios.

[0103] Any of the image recognition methods provided in this disclosure can be executed by any suitable device with image recognition capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the image recognition methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the image recognition methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0104] Exemplary apparatus

[0105] Next, refer to Figure 5 , Figure 5 A schematic diagram of the structure of an embodiment of the image recognition apparatus of this disclosure is shown, such as... Figure 5 As shown, the device includes: an information acquisition unit 510 configured to determine the face image, human body region information, and key point information of the human skeleton of the object to be identified from the image to be identified; a first prediction unit 520 configured to predict the age of the object to be identified based on the face image using a pre-trained first prediction model, thereby obtaining the predicted age of the object to be identified; a second prediction unit 530 configured to predict the age range of the object to be identified based on the human body region information and key point information of the human skeleton using a pre-trained second prediction model, thereby obtaining the predicted age range of the object to be identified; and a result determination unit 540 configured to determine whether the object to be identified is a child based on the predicted age and the predicted age range.

[0106] In one embodiment, the information acquisition unit 510 includes: a face detection module configured to perform face detection on the image to be identified using a face branch network in a pre-trained multi-task detection network to obtain a face detection box; a face image extraction module configured to extract a face image from the image to be identified based on the face detection box; a human body detection module configured to perform human body detection on the object to be identified using a human body branch network in a multi-task detection network to obtain a human body detection box, and determine the coordinates of the human body detection box as human body region information; and a key point prediction module configured to predict the coordinates and confidence of the human skeleton key points of the object to be identified based on the human body branch network and the human body detection box to obtain the key point information of the human skeleton.

[0107] In one embodiment, the first prediction unit 520 further includes: a vector output module configured to predict a face image using a first prediction model and output a first probability vector, the first probability vector including a first preset number of confidence values ​​of age values; and an age prediction module configured to determine the predicted age based on the number of confidence values ​​in the probability vector that are greater than a preset probability threshold.

[0108] In one embodiment, the device further includes a preprocessing unit configured to: adjust the face image to a preset size; and normalize the pixel values ​​in the resized face image.

[0109] In one embodiment, the preprocessing unit is further configured to: identify facial key points from the face image and obtain the coordinate information of the facial key points; and adjust the face pose in the face image to a preset pose based on the coordinate information of the facial key points.

[0110] In one embodiment, the second prediction unit 530 further includes: a detection box calculation module configured to determine the height and area of ​​a human detection box based on human region information; a height calculation module configured to determine the upper body height of the object to be identified based on key point information of the human skeleton; a vector encoding module configured to encode the human region information, the height and area of ​​the human detection box, the upper body height, and the key point information of the human skeleton into a feature vector; a prediction module configured to input the feature vector into a second prediction model to obtain a second probability vector, the second probability vector including the confidence of a second preset number of age intervals; and an interval determination module configured to determine the predicted age interval based on the second probability vector.

[0111] In one embodiment, the vector encoding module is further configured to: normalize the coordinates of the human detection box and the coordinates of the key points of the human skeleton based on the resolution of the image to be recognized; and encode the processed coordinates of the human detection box and the coordinates of the key points of the human skeleton, the confidence of the key points of the human skeleton, the upper body height, and the height and area of ​​the human detection box into feature vectors.

[0112] In one embodiment, the result determination unit 540 is further configured to: when a first preset condition is met, determine whether the object to be identified is a child based on the predicted age range. The first preset condition includes one of the following: the face occlusion area in the face image is greater than a preset area threshold and the number of visible key points in the human skeleton is greater than a preset number; or, the face occlusion area is not greater than the preset area threshold and the difference between the predicted age range and the predicted age range is greater than a preset difference threshold. When the first preset condition is not met, determine whether the object to be identified is a child based on the predicted age range.

[0113] Exemplary electronic device

[0114] Below, for reference Figure 6 To describe an electronic device according to embodiments of the present disclosure. Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 6 As shown, the electronic device 600 includes one or more processors 610 and memory 620.

[0115] The processor 610 may be a central processing unit (CPU) or other form of processing unit with image recognition and / or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.

[0116] The memory 620 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the image recognition methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0117] In one example, the electronic device 600 may also include an input device 630 and an output device 640, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0118] For example, the input device 630 can be the microphone or microphone array described above, used to capture the input signal from the sound source, or it can be a communication network connector used to receive the acquired input signal.

[0119] In addition, the input device 630 may also include, for example, a keyboard, a mouse, etc.

[0120] The output device 640 can output various information to the outside, including determined distance information, direction information, etc. The output device 640 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0121] Of course, for the sake of simplicity, Figure 6 Only some of the components of the electronic device 600 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 600 may include any other suitable components depending on the specific application.

[0122] Exemplary computer program product and computer readable storage medium

[0123] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the image recognition methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0124] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0125] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the image recognition methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0126] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0129] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0130] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0131] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0132] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0133] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An image recognition method, comprising: The image to be identified is used to determine the face image, human body region information, and key point information of the human skeleton of the object to be identified. Using a pre-trained first prediction model, the age of the object to be identified is predicted based on the face image, thus obtaining the predicted age of the object to be identified; Using a pre-trained second prediction model, based on the human body region information and the key point information of the human skeleton, the age range of the object to be identified is predicted, thus obtaining the predicted age range of the object to be identified. The second prediction model predicts the predicted age range of the object to be identified based on feature vectors. These feature vectors are obtained by encoding the human body region information, the height and area of ​​the human body detection box, the upper body height of the object to be identified, and the key point information of the human skeleton. The height and area of ​​the human body detection box are determined based on the human body region information, and the upper body height is determined based on the key point information of the human skeleton. Based on the predicted age and the predicted age range, it is determined whether the object to be identified is a child.

2. The method according to claim 1, wherein, The process of determining the face image, human body region information, and key point information of the human skeleton of the object to be identified from the image to be identified includes: The face detection bounding box is obtained by using the face branch network in the pre-trained multi-task detection network to perform face detection on the image to be identified. Based on the face detection bounding box, the face image is extracted from the image to be identified; Using the human body branch network in the multi-task detection network, human body detection is performed on the object to be identified to obtain a human body detection box, and the coordinates of the human body detection box are determined as the human body region information; Using the human branch network and based on the human detection box, the coordinates and confidence scores of the key points of the human skeleton of the object to be identified are predicted to obtain the key point information of the human skeleton.

3. The method according to claim 1, wherein, The step of using a pre-trained first prediction model to predict the age of the object to be identified based on the face image, and obtaining the predicted age of the object to be identified, includes: The first prediction model is used to predict the face image and output a first probability vector, which includes the confidence level of a first preset number of age values. The predicted age is determined based on the number of confidence levels in the probability vector that are greater than a preset probability threshold.

4. The method according to claim 1, wherein, Before using a pre-trained first prediction model to predict the age of the object to be identified based on the face image, the method further includes performing the following preprocessing on the face image: Adjust the face image to a preset size; The pixel values ​​in the resized face image are normalized.

5. The method according to claim 4, wherein, After normalizing the pixel values ​​in the resized face image, the preprocessing further includes: Facial key points are identified from the face image, and the coordinate information of the facial key points is obtained; Based on the coordinate information of the facial key points, the facial pose in the face image is adjusted to a preset pose.

6. The method according to any one of claims 2 to 5, wherein, The step of using a pre-trained second prediction model to predict the age range of the object to be identified based on the human body region information and the key point information of the human skeleton, and obtaining the predicted age range of the object to be identified, includes: Based on the human body region information, the height and area of ​​the human body detection frame are determined; Based on the key point information of the human skeleton, the upper body height of the object to be identified is determined; The human body region information, the height and area of ​​the human body detection box, the upper body height, and the key point information of the human skeleton are encoded into feature vectors. The feature vector is input into the second prediction model to obtain a second probability vector, which includes the confidence level of a second preset number of age intervals. The predicted age range is determined based on the second probability vector.

7. The method according to claim 6, wherein, The step of encoding the human body region information, the height and area of ​​the human body detection box, the upper body height, and the key point information of the human skeleton into a feature vector includes: Based on the resolution of the image to be identified, the coordinates of the human body detection box and the coordinates of the key points of the human skeleton are normalized. The coordinates of the processed human body detection box, the coordinates of the key points of the human skeleton, the confidence level of the key points of the human skeleton, the height of the upper body, and the height and area of ​​the human body detection box are encoded into the feature vector.

8. The method according to any one of claims 1 to 5, wherein, The step of determining whether the subject to be identified is a child based on the predicted age and the predicted age range includes: When the first preset condition is met, it is determined whether the object to be identified is a child based on the predicted age range. The first preset condition includes one of the following: the face occlusion area in the face image is greater than a preset area threshold and the number of visible key points in the key points of the human skeleton is greater than a preset number; or, the face occlusion area is not greater than the preset area threshold and the difference between the age range in which the predicted age is located and the predicted age range is greater than a preset difference threshold. When the first preset condition is not met, the object to be identified is determined to be a child based on the age range in which the predicted age is located.

9. An image recognition apparatus, comprising: The information acquisition unit is configured to determine the face image, human body region information, and key point information of the human skeleton of the object to be identified from the image to be identified; The first prediction unit is configured to use a pre-trained first prediction model to predict the age of the object to be identified based on the face image, thereby obtaining the predicted age of the object to be identified. The second prediction unit is configured to use a pre-trained second prediction model to predict the age range of the object to be identified based on the human body region information and the key point information of the human skeleton, thereby obtaining the predicted age range of the object to be identified. The second prediction model predicts the predicted age range of the object to be identified based on feature vectors. The feature vectors are obtained by encoding the human body region information, the height and area of ​​the human body detection box, the upper body height of the object to be identified, and the key point information of the human skeleton. The height and area of ​​the human body detection box are determined based on the human body region information, and the upper body height is determined based on the key point information of the human skeleton. The result determination unit is configured to determine whether the object to be identified is a child based on the predicted age and the predicted age range.

10. A computer-readable storage medium storing a computer program for performing the method according to any one of claims 1-8.

11. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-8.

Images