Object recognition method, device, computer-readable storage medium, and electronic device

Abstract

This application discloses an object recognition method, apparatus, computer-readable storage medium, and electronic device, relating to the field of artificial intelligence technology. The method includes: acquiring at least two images of a target object acquired from different acquisition angles; performing feature extraction processing on the at least two images to obtain facial image features in each image; performing feature co-occurrence analysis on the facial image features in each image to determine the target facial image features matching the target object; aggregating the target facial image features to obtain facial features to be recognized; and performing object recognition on the target object based on the facial features to be recognized to obtain a recognition result. This application effectively improves the accuracy and efficiency of object recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to an object recognition method, apparatus, computer-readable storage medium, and electronic device. Background Technology

[0002] Facial recognition is widely used in various industries for object identification. Typically, a camera is installed on the terminal to perform facial recognition on the target object. For example, a merchant's payment terminal can perform facial recognition on the target object.

[0003] Currently, when performing object recognition, the terminal device is usually fixed and only has a camera installed at one angle. This is easily limited by factors such as the height of the object, making it inconvenient to collect images for object recognition. Moreover, the quality of the collected images is often poor, resulting in low accuracy and efficiency in object recognition. Summary of the Invention

[0004] This application provides an object recognition method and related apparatus, which can effectively improve the accuracy and efficiency of object recognition.

[0005] To address the aforementioned technical problems, this application provides the following technical solutions:

[0006] According to one embodiment of this application, an object recognition method is applied to a terminal. The method includes: acquiring at least two images of a target object to be recognized, acquired from different acquisition angles; performing feature extraction processing based on the at least two images to be recognized to obtain facial image features in each image to be recognized; performing feature co-occurrence analysis on the facial image features in each image to be recognized to determine the target facial image features matching the target object; aggregating the target facial image features to obtain facial features to be recognized; and performing object recognition on the target object based on the facial features to be recognized to obtain a recognition result.

[0007] According to one embodiment of this application, an object recognition device is applied to a terminal, comprising: a acquisition module for acquiring at least two images of a target object to be recognized from different acquisition angles; an extraction module for performing feature extraction processing based on the at least two images to be recognized to obtain facial image features in each image to be recognized; an analysis module for performing feature co-occurrence analysis on the facial image features in each image to be recognized to determine the target facial image features matching the target object; an aggregation module for aggregating the target facial image features to obtain facial features to be recognized; and a recognition module for performing object recognition on the target object based on the facial features to be recognized to obtain a recognition result.

[0008] In some embodiments of this application, the analysis module includes: a similarity calculation unit, used to calculate the similarity between facial image features in every two images to be identified; a candidate feature acquisition unit, used to determine facial image features corresponding to similarities higher than a predetermined feature co-occurrence similarity threshold, and obtain candidate facial image features; and a target feature acquisition unit, used to determine target facial image features matching the target object based on the candidate facial image features.

[0009] In some embodiments of this application, the target feature acquisition unit includes: a feature quantity calculation subunit, used to calculate the feature quantity corresponding to each face image feature in the candidate face image features; and a quantity comparison subunit, used to determine the face image features whose feature quantity is higher than a predetermined quantity threshold as the target face image features matching the target object.

[0010] In some embodiments of this application, the aggregation module includes: a normalization unit, used to normalize each of the target face image features to obtain a normalized feature corresponding to each of the target face image features; and an aggregation unit, used to aggregate the normalized features corresponding to all the target face image features to obtain the face feature to be identified.

[0011] In some embodiments of this application, the aggregation unit includes: a spatial transformation subunit, used to perform feature space transformation processing on the normalized features corresponding to each target face image feature to obtain spatial transformation features corresponding to each target face image feature; and a feature aggregation subunit, used to aggregate the spatial transformation features corresponding to each target face image feature to obtain the face features to be identified.

[0012] In some embodiments of this application, the feature aggregation subunit includes: a weighted processing subunit, used to perform weighted processing on the spatial transformation features corresponding to each target face image feature to obtain a weighted feature corresponding to each target face image feature; and a combination processing subunit, used to perform spatial combination processing on the weighted features corresponding to each target face image feature to generate the face feature to be identified.

[0013] In some embodiments of this application, the combined processing subunit is used to: perform splicing processing on the weighted features corresponding to each target facial image feature to obtain spliced ​​features; and use the spliced ​​features as the facial features to be identified.

[0014] In some embodiments of this application, the combined processing subunit is used to: perform fusion processing on the weighted features corresponding to each target facial image feature to obtain fused features; and use the fused features as the facial features to be identified.

[0015] In some embodiments of this application, the quantity comparison subunit is configured to include: a recognition degree calculation subunit, configured to take each facial image feature whose number of features is higher than a predetermined number threshold as a feature to be matched, and calculate the facial recognition degree corresponding to each feature to be matched; and a recognition degree comparison subunit, configured to determine the features to be matched whose facial recognition degree is higher than the predetermined recognition degree as the target facial image features to be matched with the target object.

[0016] In some embodiments of this application, the recognition rate calculation subunit is used to: detect the face area and face symmetry based on each of the features to be matched; and perform a weighted score based on the face area and face symmetry corresponding to each of the features to be matched, so as to obtain the face recognition rate corresponding to each of the features to be matched.

[0017] In some embodiments of this application, the terminal includes at least two cameras located at different acquisition heights; the acquisition module includes a triggering unit for triggering the at least two cameras to acquire images of the target object from different acquisition heights, thereby obtaining at least two acquired images to be identified.

[0018] In some embodiments of this application, the recognition module includes: a remote recognition unit, used to send the facial features to be recognized to a server, so that the server can perform object recognition on the target object based on the facial features to be recognized, and obtain a recognition result.

[0019] According to another embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a computer's processor, causes the computer to perform the methods described in the embodiments of this application.

[0020] According to another embodiment of this application, an electronic device includes: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the methods described in the embodiments of this application.

[0021] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0022] In this embodiment, at least two images of the target object to be identified are acquired from different acquisition angles; feature extraction processing is performed on the at least two images to be identified to obtain facial image features in each image to be identified; feature co-occurrence analysis is performed on the facial image features in each image to be identified to determine the target facial image features that match the target object; the target facial image features are aggregated to obtain the facial features to be identified; and object recognition is performed on the target object based on the facial features to be identified to obtain the recognition result.

[0023] In this way, the terminal can acquire at least two images of the target object from different acquisition angles. By extracting facial image features from each image and performing feature co-occurrence analysis, the target facial image features matching the target object are determined. Then, the target facial image features are aggregated and processed to obtain the facial features to be identified for object recognition. Thus, the target object does not need to be limited by factors such as height and other factors to spend time cooperating with image acquisition by changing postures such as standing on tiptoe. It can efficiently acquire images of the target object from different acquisition angles, and at the same time, it does not have high requirements for image acquisition quality, and can also accurately obtain recognition results, effectively improving the accuracy and efficiency of object recognition and enhancing the user experience. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of a system to which embodiments of this application can be applied is shown.

[0026] Figure 2 A schematic diagram of another system to which embodiments of this application can be applied is shown.

[0027] Figure 3 A flowchart of an object recognition method according to an embodiment of this application is shown.

[0028] Figure 4 A flowchart of feature co-occurrence analysis according to an embodiment of this application is shown.

[0029] Figure 5 A flowchart illustrating the filtering of target facial image features according to an embodiment of this application is shown.

[0030] Figure 6 A schematic diagram of a scenario for filtering target facial image features according to one embodiment is shown.

[0031] Figure 7 A schematic diagram of a scenario for filtering target facial image features according to one embodiment is shown.

[0032] Figure 8 A flowchart of a feature aggregation method according to an embodiment of this application is shown.

[0033] Figure 9 This diagram illustrates a scenario where object recognition is performed.

[0034] Figure 10 A schematic diagram of a scenario for object recognition using related technologies is shown.

[0035] Figure 11 A block diagram of an object recognition device according to an embodiment of this application is shown.

[0036] Figure 12 A block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Figure 1 A schematic diagram of a system 100 to which embodiments of this application can be applied is shown. For example... Figure 1 As shown, system 100 may include server 101 and terminal 102. Server 101 and terminal 102 can be connected directly or indirectly via wireless communication, and this application does not impose any special restrictions on this connection.

[0039] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0040] In one implementation of this example, server 101 is a cloud server that can provide artificial intelligence cloud services, such as large-scale multi-user online object recognition AI cloud services. AI cloud services are generally also referred to as AIaaS (AI as a Service). This is a mainstream service model for AI platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.

[0041] Terminal 102 can be any device, including but not limited to mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, VR / AR devices, smartwatches, and computers, etc.

[0042] In one embodiment of this example, terminal 102 can acquire at least two images of a target object to be identified, captured from different acquisition angles; perform feature extraction processing based on the at least two images to be identified to obtain facial image features in each image; perform feature co-occurrence analysis on the facial image features in each image to determine the target facial image features matching the target object; aggregate the target facial image features to obtain facial features to be identified; and perform object recognition on the target object based on the facial features to be identified to obtain a recognition result. Specifically, terminal 102 can be equipped with at least two cameras 1021 with different acquisition angles (e.g., different acquisition heights) to acquire at least two images of the target object to be identified from different acquisition angles. Terminal 102 can upload the facial features to be identified to server 101, enabling server 101 to perform object recognition on the target object based on the facial features to obtain a recognition result.

[0043] Figure 2 A schematic diagram of another system 200 to which embodiments of this application can be applied is shown. For example... Figure 2 As shown, system 200 can be a distributed system formed by connecting client 201 and multiple nodes 202 through network communication.

[0044] Taking a distributed system as an example, see blockchain system. Figure 2 , Figure 2This is an optional structural diagram of the distributed system 200 provided in this application embodiment applied to a blockchain system. It consists of multiple nodes 202 and clients 201, forming a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In the distributed system, any machine, such as a server, can join and become a node 202 (each node 202 can be, for example, a server). Figure 1 The server 101 and node 202 can provide object recognition services. The nodes include a hardware layer, a middleware layer, an operating system layer, and an application layer.

[0045] See Figure 2 The functions of each node in the blockchain system shown include:

[0046] 1) Routing: A basic function of nodes used to support communication between nodes.

[0047] In addition to routing capabilities, nodes can also have the following functions:

[0048] 2) Applications are deployed in the blockchain to implement specific business needs. They record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system. When other nodes successfully verify the source and integrity of the record data, they add the record data to a temporary block.

[0049] For example, the business logic implemented by the application includes:

[0050] 2.1) A wallet is used to provide the function of conducting electronic currency transactions, including initiating transactions (i.e., sending the transaction record of the current transaction to other nodes in the blockchain system; after other nodes successfully verify the transaction, they store the transaction record data in the temporary block of the blockchain as a response to acknowledge the validity of the transaction; of course, the wallet also supports querying the remaining electronic currency in the electronic currency address.

[0051] 2.2) Shared ledger, used to provide functions such as storage, query and modification of ledger data. It sends the record data of the operation on the ledger data to other nodes in the blockchain system. After the other nodes verify the validity, as a response to acknowledge the validity of the ledger data, they store the record data in a temporary block. They can also send confirmation to the node that initiated the operation.

[0052] 2.3) Smart contracts are computerized protocols that can execute the terms of a contract. They are implemented through code deployed on a shared ledger that executes when certain conditions are met. Based on actual business needs, the code is used to complete automated transactions, such as querying the logistics status of goods purchased by a buyer and transferring the buyer's electronic money to the merchant's address after the buyer signs for the goods. Of course, smart contracts are not limited to executing contracts for transactions; they can also execute contracts for processing received information.

[0053] 3) A blockchain consists of a series of blocks that are sequentially generated. Once a new block is added to the blockchain, it will not be removed. The blocks contain the data submitted by the nodes in the blockchain system.

[0054] In one embodiment of this example, the terminal can acquire at least two images of the target object from different acquisition angles via client 201; perform feature extraction processing based on the at least two images to obtain facial image features in each image; perform feature co-occurrence analysis on the facial image features in each image to determine the target facial image features matching the target object; aggregate the target facial image features to obtain the facial features to be identified; and perform object recognition on the target object based on the facial features to be identified to obtain the recognition result. Client 201 can upload the facial features to be identified to node 202, enabling node 202 to perform object recognition on the target object based on the facial features to obtain the recognition result.

[0055] Figure 3 A flowchart illustrating an embodiment of an object recognition method according to this application is shown. The execution subject of this object recognition method can be any terminal, such as... Figure 1 The terminal 102 shown or as Figure 2 The terminal corresponding to client 201 shown.

[0056] like Figure 3 As shown, the object recognition method may include steps S310 to S350.

[0057] Step S310: Obtain at least two images of the target object to be identified from different acquisition angles;

[0058] Step S320: Perform feature extraction processing based on at least two images to be identified to obtain facial image features in each image to be identified;

[0059] Step S330: Perform feature co-occurrence analysis on the facial image features in each image to be identified in order to determine the target facial image features that match the target object;

[0060] Step S340: Aggregate the features of the target face image to obtain the face features to be identified;

[0061] Step S350: Based on the facial features to be identified, the target object is identified to obtain the identification result.

[0062] The following describes the specific process of each step in object recognition.

[0063] In step S310, at least two images of the target object to be identified are acquired from different acquisition angles.

[0064] In this example implementation, different acquisition angles may include different acquisition heights and different acquisition directions. By acquiring at least two images of the target object from different acquisition angles, images containing the target object's face can be acquired even when the target object is in a relatively casual image acquisition state. Furthermore, acquiring at least two images of the target object from different acquisition angles ensures that at least two images containing the target object's face can be acquired even when the target object is in a relatively casual image acquisition state, avoiding poor image quality caused by the target object's dynamic movements, differences in ambient light sources, and differences in the fixed installation angle of a single camera.

[0065] In one embodiment, the terminal includes at least two cameras located at different acquisition heights; step S310, acquiring at least two images of the target object captured from different acquisition angles, includes: triggering at least two cameras to capture images of the target object from different acquisition heights, and obtaining at least two captured images of the target object.

[0066] By setting at least two cameras at different acquisition heights on the terminal, images of the target object can be acquired from different acquisition heights, avoiding the inability to acquire the image of the target object due to factors such as height or physical defects, and also better adapting to objects with different height ranges.

[0067] In step S320, feature extraction processing is performed based on at least two images to be identified to obtain facial image features in each image to be identified.

[0068] In this example implementation, feature extraction processing is performed on each of the at least two images to be identified, which can extract facial image features from each image to be identified. Facial image features may include features of the eyebrows, nose, eyes, mouth, and ears.

[0069] The feature extraction process for each image to be recognized can be performed by first detecting the face region in each image, then detecting the corresponding regions of each part within the face region, and finally extracting the coordinate data of the feature points of each part. The set of extracted coordinate data in each image to be recognized can then be used as the face image feature of each image. Alternatively, the set of extracted coordinate data in each image to be recognized can be processed by convolutional neural network to obtain a feature vector, which can then be used as the face image feature of each image to be recognized.

[0070] In step S330, feature co-occurrence analysis is performed on the facial image features in each image to be identified in order to determine the target facial image features that match the target object.

[0071] In this example implementation, since the images to be identified are acquired from different acquisition angles, the images to be identified from different acquisition angles may contain the faces of other objects. Feature co-occurrence analysis is performed on the facial image features in each image to be identified, that is, the target facial image features that appear together in all images to be identified are analyzed. If the target facial image features appear frequently in different images to be identified, it indicates that the target facial image features match the target object. Based on the determined target facial image features, the recognition accuracy of the target object can be improved.

[0072] In one embodiment, see Figure 4 Step S330 involves performing feature co-occurrence analysis on the facial image features in each image to be identified to determine the target facial image features that match the target object, including:

[0073] Step S410: Calculate the similarity between facial image features in every two images to be identified; Step S420: Determine the facial image features corresponding to similarities higher than a predetermined feature co-occurrence similarity threshold to obtain candidate facial image features; Step S430: Determine the target facial image features to match the target object based on the candidate facial image features.

[0074] The similarity between facial image features in two images to be identified can be measured and calculated using methods such as Euclidean distance or histogram. Then, facial image features with similarity values ​​higher than a predetermined feature co-occurrence similarity threshold (which can be set according to actual conditions) are taken as candidate facial image features. Since the candidate facial image features appear frequently in different images to be identified, the target facial image features that match the target object can be accurately determined based on the candidate facial image features.

[0075] Facial image features with similarity scores exceeding a predetermined feature co-occurrence similarity threshold are selected as candidate facial image features. For example, if the predetermined feature co-occurrence similarity threshold is 45%, and the images to be identified include four images: A, B, C, and D, the similarity of facial image features between A and B is 90%, between A and C is 50%, between A and D is 30%, between B and C is 48%, between B and D is 40%, and between C and D is 25%, then the candidate facial image features include facial image features from three images to be identified: A, B, and C.

[0076] In one embodiment, see Figure 5 Step S430, determining the target face image features matching the target object based on the candidate face image features, includes:

[0077] Step S510: Calculate the number of features corresponding to each face image feature in the candidate face image features; Step S520: Determine the face image features with a feature number higher than a predetermined threshold as the target face image features that match the target object.

[0078] The candidate facial image features may include facial image features from at least one image to be identified, with each image to be identified corresponding to one facial image feature. Each facial image feature may include at least one part feature. The number of features corresponding to each facial image feature may be the number of part features included in each facial image feature or the number of feature points.

[0079] Facial image features with a feature count exceeding a predetermined threshold are identified as target facial image features for matching the target object. For example, candidate facial image features include facial image features from three images to be identified: A, B, and C. If the facial image feature corresponding to A is A1, the facial image feature corresponding to B is B1, and the facial image feature corresponding to C is C1, then if the predetermined threshold is 45, the feature count corresponding to A1 is 60, the feature count corresponding to B1 is 50, and the feature count corresponding to C1 is 35. Therefore, facial image features with a feature count exceeding the predetermined threshold, namely A1 and B1, are identified as target facial image features for matching the target object. This can further improve the accuracy of target object recognition.

[0080] In one embodiment, step S520, determining facial image features with a feature count exceeding a predetermined threshold as target facial image features matching the target object, includes:

[0081] Facial image features with a number of features exceeding a predetermined threshold are used as features to be matched, and the facial recognition score corresponding to each feature to be matched is calculated; features to be matched with a facial recognition score exceeding a predetermined recognition score are determined as target facial image features to match the target object.

[0082] Face recognition score is the degree to which a target object can be identified based on facial image features. In one example, the face recognition score can be determined based on the name of the feature part in each feature to be matched. For example, the face recognition score corresponding to the feature to be matched including the eyes, nose and mouth is 5, and the face recognition score corresponding to the feature to be matched including the eyes and mouth is 3, etc. In another example, the face recognition score can be calculated based on the detection of the face area and face symmetry for each feature to be matched.

[0083] By identifying facial features with a recognition score higher than a predetermined score as target facial image features for matching the target object, the accuracy of target object recognition can be further improved.

[0084] In one embodiment, calculating the facial recognition score corresponding to each feature to be matched includes:

[0085] The facial area and facial symmetry are detected based on each feature to be matched; a weighted score is then calculated based on the facial area and facial symmetry corresponding to each feature to obtain the facial recognition score for each feature to be matched.

[0086] The face area refers to the area of ​​the face contained in the face image to be identified, and the face symmetry refers to the symmetry of the face on both sides of the face's midline. Based on each feature to be matched, the face area and face symmetry can be detected. Then, the face area and face symmetry can be weighted and summed according to a predetermined weighting coefficient to obtain the weighted sum as the face recognition score corresponding to the feature to be matched. The larger the face area and face symmetry, the higher the weight of the frontal face.

[0087] See one example. Figure 6 and Figure 7 The method of obtaining facial recognition accuracy through weighted scoring sets a predetermined recognition score as an appropriate threshold. Features to be matched with facial recognition scores higher than the predetermined recognition score accurately represent frontal facial images (e.g., ...). Figure 6 and Figure 7 The features in the image (marked with a checkmark) can be used to accurately identify objects, both standing and sitting.

[0088] This approach, which identifies features that exceed the predetermined recognition threshold as target facial image features for matching the target object, can improve the reliability of facial recognition calculation.

[0089] In step S340, the features of the target face image are aggregated to obtain the face features to be identified.

[0090] In this example implementation, the aggregation process aggregates facial image features corresponding to the same object but different images to be identified into a feature space. The target facial image features may include multiple features. By aggregating multiple target facial image features into a feature space, the facial features to be identified are obtained. Based on the facial features to be identified, the recognition accuracy of the target object can be effectively improved.

[0091] In one embodiment, see Figure 8 Step S340: Aggregate the features of the target face image to obtain the face features to be identified, including:

[0092] Step S610: Normalize the features of each target face image to obtain the normalized features corresponding to each target face image feature; Step S620: Aggregate the normalized features corresponding to all target face image features to obtain the face features to be identified.

[0093] Normalizing the features of the target face image can involve extracting the maximum value, average value, and variance of the target face image features. Normalizing the target face image features into normalized features such as maximum value, average value, or variance, and then performing aggregation processing based on the normalized features can, to a certain extent, eliminate the differences between the target face image features corresponding to the images to be identified captured from different acquisition angles, thereby improving the recognition accuracy of the face features to be identified.

[0094] In one embodiment, step S620 involves aggregating the normalized features corresponding to all target facial image features to obtain the facial features to be identified, including:

[0095] For each target face image feature, the normalized features are subjected to feature space transformation processing to obtain the spatial transformation features corresponding to each target face image feature; the spatial transformation features corresponding to each target face image feature are aggregated to obtain the face features to be identified.

[0096] Feature space transformation processing involves reducing the dimensionality of normalized features through spatial transformation. For example, normalized features can be multiplied by a predetermined feature matrix to perform spatial transformation. After obtaining the spatially transformed features, they are aggregated to obtain the facial features to be identified, which can further improve the recognition accuracy of the facial features to be identified.

[0097] In one embodiment, the spatial transformation features corresponding to each target facial image feature are aggregated to obtain the facial features to be identified, including:

[0098] The spatial transformation features corresponding to each target face image feature are weighted to obtain the weighted features corresponding to each target face image feature; the weighted features corresponding to each target face image feature are then spatially combined to generate the face features to be identified.

[0099] The spatial transformation features corresponding to each target facial image feature are weighted by multiplying them by a weighting coefficient to obtain weighted features. The weighting coefficient is determined based on the facial recognition accuracy of each target facial image feature; higher facial recognition accuracy results in a larger weighting coefficient. The sum of the weighting coefficients for all target facial image features can be one. Spatial combination processing is then performed on the weighted features corresponding to each target facial image feature to generate the facial features to be recognized, further improving the accuracy of facial recognition.

[0100] In one embodiment, the weighted features corresponding to each target face image feature are spatially combined to generate a face feature to be identified, including: splicing the weighted features corresponding to each target face image feature to obtain spliced ​​features; and using the spliced ​​features as the face feature to be identified.

[0101] The stitching process can be either concatenating the weighted features corresponding to all target face image features into a single feature string as the stitching feature, or stitching the weighted features corresponding to all target face image features into the same feature matrix as the stitching feature.

[0102] In one embodiment, the weighted features corresponding to each target face image feature are spatially combined to generate a face feature to be identified, including: fusing the weighted features corresponding to each target face image feature to obtain a fused feature; and using the fused feature as the face feature to be identified.

[0103] The fusion process involves combining the weighted features corresponding to all target facial image features together. For example, the sum of the weighted features corresponding to all target facial image features can be used as the fusion feature.

[0104] In step S350, object recognition is performed on the target object based on the facial features to be recognized, and the recognition result is obtained.

[0105] In this example implementation, the similarity between the facial features to be identified and the preset facial features in the pre-collected facial feature library is compared. The preset facial feature with the highest similarity to the facial features to be identified in the facial feature library is used as the object matching feature. Then, the target object information is determined based on the object information (such as name or ID number) associated with the object matching feature.

[0106] In one embodiment, object recognition of a target object based on facial features to be recognized includes: sending the facial features to be recognized to a server so that the server can perform object recognition on the target object based on the facial features to be recognized and obtain a recognition result.

[0107] In this embodiment, after the terminal collects the facial features to be identified, it uploads them to the server. The server can efficiently compare the similarity between the facial features to be identified and the preset facial features in the pre-collected facial feature library. The preset facial feature with the highest similarity to the facial features to be identified in the facial feature library is used as the object matching feature. Then, the target object information is determined based on the object information (such as name or ID number) associated with the object matching feature, thereby improving the object recognition efficiency.

[0108] In this way, based on steps S310 to S350, the terminal can acquire at least two images to be recognized from different acquisition angles for the target object. By extracting facial image features from each image to be recognized and performing feature co-occurrence analysis, the target facial image features matching the target object are determined. Then, the target facial image features are aggregated and processed to obtain the facial features to be recognized for object recognition. Thus, the target object does not need to be limited by factors such as height and spend time changing postures to cooperate with image acquisition. It can efficiently acquire images to be recognized from different acquisition angles, and at the same time, the requirements for image acquisition quality are not high, and the recognition results can be accurately obtained. This effectively improves the accuracy and efficiency of object recognition and enhances the user experience.

[0109] See Figure 9 By applying the embodiments of this application, images of the target object are captured from different angles by multiple cameras, enabling normal identification of objects in any pose. Objects of different heights do not need to bend over or stand on tiptoe to match the camera's shooting height, thus improving recognition accuracy and efficiency. In contrast, in the prior art, see [reference needed] Figure 10 If a seated object is captured with a low-quality image, it will fail to be recognized, resulting in low recognition accuracy. A seated object may require assistance from other people to lift it up for recognition, which leads to low recognition efficiency.

[0110] To facilitate better implementation of the object recognition method provided in this application, this application also provides an object recognition device based on the above-described object recognition method. The meanings of the terms used are the same as in the object recognition method described above, and specific implementation details can be found in the descriptions within the method embodiments. Figure 11 A block diagram of an object recognition device according to an embodiment of this application is shown.

[0111] like Figure 11As shown, the object recognition device 700 may include a collection module 710, an extraction module 720, an analysis module 730, an aggregation module 740, and a recognition module 750. The object recognition device 700 can be applied to a terminal.

[0112] The acquisition module 710 can be used to acquire at least two images of the target object to be identified from different acquisition angles; the extraction module 720 can be used to perform feature extraction processing based on the at least two images to be identified to obtain facial image features in each image to be identified; the analysis module 730 can be used to perform feature co-occurrence analysis on the facial image features in each image to be identified to determine the target facial image features matched by the target object; the aggregation module 740 can be used to aggregate the target facial image features to obtain facial features to be identified; and the recognition module 750 can be used to perform object recognition on the target object based on the facial features to be identified to obtain the recognition result.

[0113] In some embodiments of this application, the analysis module includes: a similarity calculation unit, used to calculate the similarity between facial image features in every two images to be identified; a candidate feature acquisition unit, used to determine facial image features corresponding to similarities higher than a predetermined feature co-occurrence similarity threshold, and obtain candidate facial image features; and a target feature acquisition unit, used to determine target facial image features matching the target object based on the candidate facial image features.

[0114] In some embodiments of this application, the target feature acquisition unit includes: a feature quantity calculation subunit, used to calculate the feature quantity corresponding to each face image feature in the candidate face image features; and a quantity comparison subunit, used to determine the face image features whose feature quantity is higher than a predetermined quantity threshold as the target face image features matching the target object.

[0115] In some embodiments of this application, the aggregation module includes: a normalization unit, used to normalize each of the target face image features to obtain a normalized feature corresponding to each of the target face image features; and an aggregation unit, used to aggregate the normalized features corresponding to all the target face image features to obtain the face feature to be identified.

[0116] In some embodiments of this application, the aggregation unit includes: a spatial transformation subunit, used to perform feature space transformation processing on the normalized features corresponding to each target face image feature to obtain spatial transformation features corresponding to each target face image feature; and a feature aggregation subunit, used to aggregate the spatial transformation features corresponding to each target face image feature to obtain the face features to be identified.

[0117] In some embodiments of this application, the feature aggregation subunit includes: a weighted processing subunit, used to perform weighted processing on the spatial transformation features corresponding to each target face image feature to obtain a weighted feature corresponding to each target face image feature; and a combination processing subunit, used to perform spatial combination processing on the weighted features corresponding to each target face image feature to generate the face feature to be identified.

[0118] In some embodiments of this application, the combined processing subunit is used to: perform splicing processing on the weighted features corresponding to each target facial image feature to obtain spliced ​​features; and use the spliced ​​features as the facial features to be identified.

[0119] In some embodiments of this application, the combined processing subunit is used to: perform fusion processing on the weighted features corresponding to each target facial image feature to obtain fused features; and use the fused features as the facial features to be identified.

[0120] In some embodiments of this application, the quantity comparison subunit is configured to include: a recognition degree calculation subunit, configured to take each facial image feature whose number of features is higher than a predetermined number threshold as a feature to be matched, and calculate the facial recognition degree corresponding to each feature to be matched; and a recognition degree comparison subunit, configured to determine the features to be matched whose facial recognition degree is higher than the predetermined recognition degree as the target facial image features to be matched with the target object.

[0121] In some embodiments of this application, the recognition rate calculation subunit is used to: detect the face area and face symmetry based on each of the features to be matched; and perform a weighted score based on the face area and face symmetry corresponding to each of the features to be matched, so as to obtain the face recognition rate corresponding to each of the features to be matched.

[0122] In some embodiments of this application, the terminal includes at least two cameras located at different acquisition heights; the acquisition module includes a triggering unit for triggering the at least two cameras to acquire images of the target object from different acquisition heights, thereby obtaining at least two acquired images to be identified.

[0123] In some embodiments of this application, the recognition module includes: a remote recognition unit, used to send the facial features to be recognized to a server, so that the server can perform object recognition on the target object based on the facial features to be recognized, and obtain a recognition result.

[0124] In this way, based on the object recognition device 700, the terminal can acquire at least two images of the target object from different shooting angles. By extracting facial image features from each image and performing feature co-occurrence analysis, the target facial image features matching the target object are determined. Then, the target facial image features are aggregated and processed to obtain the facial features to be recognized for object recognition. Thus, the target object does not need to be limited by factors such as height and other factors, and it does not need to spend time changing postures such as standing on tiptoe to cooperate with image acquisition. It can efficiently acquire images of the target object from different shooting angles, and at the same time, it does not have high requirements for image acquisition quality, and can also obtain accurate recognition results, effectively improving the accuracy and efficiency of object recognition and enhancing the user experience. Multiple images of the target object taken by the camera, whether the target object is looking up or down, can be recognized normally. Objects of different heights do not need to stand on tiptoe or bend over to match the shooting height of the camera.

[0125] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0126] Furthermore, embodiments of this application also provide an electronic device, which can be a terminal or a server, such as... Figure 12 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0127] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 12 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0128] The processor 801 is the control center of the electronic device. It connects to various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions of the computer device and processes data, thereby performing overall detection of the electronic device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0129] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0130] The electronic device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0131] The electronic device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0132] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 802 according to the following instructions, and the processor 801 runs the computer programs stored in the memory 802 to realize various functions, such as the processor 801 executing:

[0133] At least two images of the target object to be identified are acquired from different acquisition angles; feature extraction processing is performed on the at least two images to be identified to obtain facial image features in each image; feature co-occurrence analysis is performed on the facial image features in each image to determine the target facial image features that match the target object; the target facial image features are aggregated to obtain facial features to be identified; object recognition is performed on the target object based on the facial features to be identified to obtain the recognition result.

[0134] In one embodiment, when performing feature co-occurrence analysis on facial image features in each image to be identified to determine the target facial image features matching the target object, the processor 801 may execute: calculating the similarity between facial image features in every two images to be identified; determining the facial image features corresponding to the similarity that is higher than a predetermined feature co-occurrence similarity threshold to obtain candidate facial image features; and determining the target facial image features matching the target object based on the candidate facial image features.

[0135] In one embodiment, when determining the target face image feature matching the target object based on the candidate face image features, the processor 801 may perform the following: calculate the number of features corresponding to each face image feature in the candidate face image features; and determine the face image features whose number of features is higher than a predetermined threshold as the target face image features matching the target object.

[0136] In one embodiment, when the target face image features are aggregated to obtain the face features to be identified, the processor 801 may perform the following: normalize each target face image feature to obtain a normalized feature corresponding to each target face image feature; aggregate the normalized features corresponding to all target face image features to obtain the face features to be identified.

[0137] In one embodiment, when the normalized features corresponding to all the target face image features are aggregated to obtain the face features to be identified, the processor 801 may perform the following: perform feature space transformation processing on the normalized features corresponding to each target face image feature to obtain the spatial transformation features corresponding to each target face image feature; and aggregate the spatial transformation features corresponding to each target face image feature to obtain the face features to be identified.

[0138] In one embodiment, when the spatial transformation features corresponding to each target face image feature are aggregated to obtain the face feature to be identified, the processor 801 may perform the following: weighting the spatial transformation features corresponding to each target face image feature to obtain weighted features corresponding to each target face image feature; and spatially combining the weighted features corresponding to each target face image feature to generate the face feature to be identified.

[0139] In one embodiment, when the weighted features corresponding to each target facial image feature are spatially combined to generate the facial feature to be identified, the processor 801 may perform the following: concatenating the weighted features corresponding to each target facial image feature to obtain a concatenated feature; and using the concatenated feature as the facial feature to be identified.

[0140] In one embodiment, when the weighted features corresponding to each target facial image feature are spatially combined to generate the facial feature to be identified, the processor 801 may perform the following: perform fusion processing on the weighted features corresponding to each target facial image feature to obtain fused features; and use the fused features as the facial feature to be identified.

[0141] In one embodiment, when determining facial image features with a feature quantity exceeding a predetermined threshold as target facial image features matching the target object, the processor 801 may execute: taking each facial image feature with a feature quantity exceeding the predetermined threshold as a feature to be matched, calculating the facial recognition degree corresponding to each feature to be matched; and determining the features to be matched with facial recognition degrees exceeding the predetermined recognition degree as target facial image features matching the target object.

[0142] In one embodiment, when calculating the facial recognition score corresponding to each of the features to be matched, the processor 801 may perform the following: detect the facial area and facial symmetry based on each of the features to be matched; perform a weighted score based on the facial area and facial symmetry corresponding to each of the features to be matched, so as to obtain the facial recognition score corresponding to each of the features to be matched.

[0143] In one embodiment, the terminal includes at least two cameras located at different acquisition heights; when acquiring at least two images of the target object to be identified from different acquisition angles, the processor 801 may execute: triggering the at least two cameras to acquire images of the target object from different acquisition heights, thereby obtaining at least two acquired images to be identified.

[0144] In one embodiment, when performing object recognition on the target object based on the facial features to be recognized, the processor 801 may execute: sending the facial features to be recognized to the server, so that the server performs object recognition on the target object based on the facial features to be recognized, and obtains a recognition result.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0146] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program that can be loaded by a processor to perform the steps in any of the methods provided in embodiments of this application.

[0147] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0148] Since the computer program stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that the methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0149] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments of this application.

[0150] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0151] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. An object recognition method, characterized in that, Applied to a terminal, the method includes: Acquire at least two images of the target object to be identified, captured from different acquisition angles; Based on the at least two images to be identified, feature extraction processing is performed to obtain facial image features in each image to be identified; Feature co-occurrence analysis is performed on the facial image features in each image to be identified to determine the target facial image features that match the target object, including: Calculate the similarity between facial image features in every two images to be identified; The facial image features corresponding to the similarity scores that are higher than a predetermined feature co-occurrence similarity threshold are determined to obtain candidate facial image features; Calculate the number of features corresponding to each face image feature in the candidate face image features, and take each face image feature whose number of features is higher than a predetermined threshold as the feature to be matched; The face area and face symmetry are detected based on each of the features to be matched, and a weighted score is performed based on the face area and face symmetry corresponding to each of the features to be matched, so as to obtain the face recognition degree corresponding to each feature to be matched. The facial recognition score that is higher than the predetermined recognition score is identified as the target facial image feature to be matched with the target object; The features of the target face image are aggregated to obtain the face features to be identified; Based on the facial features to be identified, object recognition is performed on the target object to obtain the recognition result.

2. The method according to claim 1, characterized in that, The step of aggregating the features of the target facial image to obtain the facial features to be identified includes: Each of the target face image features is normalized to obtain the normalized feature corresponding to each of the target face image features; The normalized features corresponding to all the target facial image features are aggregated to obtain the facial features to be identified.

3. The method according to claim 2, characterized in that, The step of aggregating the normalized features corresponding to all the target facial image features to obtain the facial features to be identified includes: The normalized features corresponding to each target face image feature are subjected to feature space transformation processing to obtain the spatial transformation features corresponding to each target face image feature; The spatial transformation features corresponding to each target facial image feature are aggregated to obtain the facial features to be identified.

4. The method according to claim 3, characterized in that, The step of aggregating the spatial transformation features corresponding to each target facial image feature to obtain the facial features to be identified includes: The spatial transformation features corresponding to each of the target face image features are weighted to obtain the weighted features corresponding to each of the target face image features; The weighted features corresponding to each target facial image feature are spatially combined to generate the facial features to be identified.

5. The method according to claim 4, characterized in that, The step of spatially combining the weighted features corresponding to each target facial image feature to generate the facial features to be identified includes: The weighted features corresponding to each of the target facial image features are concatenated to obtain the concatenated features; The spliced ​​features are used as the facial features to be identified.

6. The method according to claim 4, characterized in that, The step of spatially combining the weighted features corresponding to each target facial image feature to generate the facial features to be identified includes: The weighted features corresponding to each of the target facial image features are fused to obtain the fused features; The fused features are used as the facial features to be identified.

7. The method according to claim 1, characterized in that, The terminal includes at least two cameras, which are located at different acquisition heights; The acquisition of at least two images of the target object acquired from different acquisition angles includes: The at least two cameras are triggered to capture images of the target object from different acquisition heights, resulting in at least two images to be identified.

8. The method according to claim 1, characterized in that, The object recognition based on the facial features to be identified includes: The facial features to be identified are sent to the server so that the server can perform object recognition on the target object based on the facial features to be identified and obtain the recognition result.

9. An object recognition device, characterized in that, Applied to a terminal, the device includes: The acquisition module is used to acquire at least two images of the target object to be identified from different acquisition angles; The extraction module is used to perform feature extraction processing based on the at least two images to be identified, so as to obtain facial image features in each image to be identified; The analysis module is used to perform feature co-occurrence analysis on the facial image features in each image to be identified, so as to determine the target facial image features that match the target object; The analysis module includes: A similarity calculation unit is used to calculate the similarity between facial image features in every two images to be identified; The candidate feature acquisition unit is used to determine the facial image features corresponding to the similarity that is higher than a predetermined feature co-occurrence similarity threshold, and to obtain candidate facial image features. The feature quantity calculation subunit is used to calculate the number of features corresponding to each face image feature in the candidate face image features; The recognition rate calculation subunit is used to take facial image features whose number of each feature exceeds a predetermined threshold as features to be matched; it is also used to detect facial area and facial symmetry based on each feature to be matched, and to perform a weighted score based on the facial area and facial symmetry corresponding to each feature to be matched, so as to obtain the facial recognition rate corresponding to each feature to be matched. The recognition rate comparison subunit is used to determine the facial recognition rate of the feature to be matched as the target facial image feature that matches the target object if the facial recognition rate is higher than the predetermined recognition rate. An aggregation module is used to aggregate the features of the target facial image to obtain the facial features to be identified. The recognition module is used to perform object recognition on the target object based on the facial features to be recognized, and obtain the recognition result.

10. The object recognition device according to claim 9, characterized in that, The aggregation module includes: The normalization unit is used to normalize each of the target face image features to obtain the normalized features corresponding to each of the target face image features. The aggregation unit is used to aggregate the normalized features corresponding to all the target facial image features to obtain the facial features to be identified.

11. The object recognition device according to claim 10, characterized in that, The aggregation unit includes: The spatial transformation subunit is used to perform feature space transformation processing on the normalized features corresponding to each target face image feature to obtain the spatial transformation features corresponding to each target face image feature. The feature aggregation subunit is used to aggregate the spatial transformation features corresponding to each target face image feature to obtain the face feature to be identified.

12. The object recognition device according to claim 11, characterized in that, The feature aggregation subunit includes: The weighted processing subunit is used to perform weighted processing on the spatial transformation features corresponding to each target face image feature to obtain the weighted features corresponding to each target face image feature; The combined processing subunit is used to perform spatial combination processing on the weighted features corresponding to each target facial image feature to generate the facial features to be identified.

13. The object recognition device according to claim 12, characterized in that, The combined processing subunit is used for: The weighted features corresponding to each of the target facial image features are concatenated to obtain the concatenated features; The spliced ​​features are used as the facial features to be identified.

14. The object recognition device according to claim 12, characterized in that, The combined processing subunit is used for: The weighted features corresponding to each of the target facial image features are fused to obtain the fused features; The fused features are used as the facial features to be identified.

15. The object recognition device according to claim 9, characterized in that, The terminal includes at least two cameras, which are located at different acquisition heights; The acquisition module includes: The triggering unit is used to trigger the at least two cameras to capture images of the target object from different acquisition heights, thereby obtaining at least two images to be identified.

16. The object recognition device according to claim 9, characterized in that, The identification module includes: A remote recognition unit is used to send the facial features to be recognized to a server, so that the server can perform object recognition on the target object based on the facial features to be recognized and obtain a recognition result.

17. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the method described in any one of claims 1 to 8.

18. An electronic device, characterized in that, include: Memory, which stores computer programs; A processor reads a computer program stored in memory to perform the method described in any one of claims 1 to 8.

19. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the object recognition method as described in any one of claims 1 to 8.

Images