Model training method, device, equipment and medium executed at terminal device

Abstract

Provided are a model training method, apparatus, device and medium executed at a terminal device. The method includes: receiving a first instance of a face recognition model from a cloud server; acquiring multiple face images; annotating each face image in the multiple face images based on the first instance of the face recognition model; verifying the identity annotation result corresponding to each face image in the multiple face images based on a preset verification strategy; generating a sample data set; training the first instance of the face recognition model based on the sample data set to update the parameters of the first instance of the face recognition model; and transmitting update data representing the updated parameters to the cloud server so that the cloud server can update a second instance of the face recognition model deployed at the cloud server.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a model training method, apparatus, computer equipment, vehicle, and computer-readable storage medium executed at a terminal device. Background Art

[0002] With the rapid development of face recognition technology, face recognition technology is increasingly used in people's daily lives, such as face recognition for entering stations, face recognition for paying at supermarkets, and face recognition for logging into mobile apps. With the development of smart vehicle-related technologies, face recognition technology is also increasingly used in vehicles to identify and authenticate users. Summary of the invention

[0003] According to one aspect of the present disclosure, a model training method performed at a terminal device is provided, comprising: receiving a first instance of a face recognition model from a cloud server; acquiring multiple face images captured by a camera arranged at the terminal device; annotating each face image in the multiple face images based on the first instance of the face recognition model to obtain an identity annotation result corresponding to each face image in the multiple face images; verifying the identity annotation result corresponding to each face image in the multiple face images based on a preset verification strategy to obtain multiple identity tags; generating a sample data set based on the multiple face images and the multiple identity tags respectively corresponding to the multiple face images; training the first instance of the face recognition model based on the sample data set to update the parameters of the first instance of the face recognition model; and transmitting update data representing the updated parameters to the cloud server so that the cloud server can update a second instance of the face recognition model deployed at the cloud server.

[0004] According to another aspect of the present disclosure, a face recognition method is provided, including: obtaining a face image of a user; and inputting the face image into a face recognition model to obtain an identity labeling result of the face image, wherein the face recognition model is trained according to the model training method performed at the terminal device as described above.

[0005] According to another aspect of the present disclosure, a model training device executed at a terminal device is provided, comprising: a receiving unit configured to receive a first instance of a face recognition model from a cloud server; an acquiring unit configured to acquire a plurality of face images captured by a camera arranged at the terminal device; a labeling unit configured to label each face image in the plurality of face images based on the first instance of the face recognition model to obtain an identity labeling result corresponding to each face image in the plurality of face images; a verifying unit configured to verify the identity labeling result corresponding to each face image in the plurality of face images based on a preset verification strategy to obtain the plurality of identity tags; a generating unit configured to generate a sample data set based on the plurality of face images and the plurality of identity tags respectively corresponding to the plurality of face images; a training unit configured to train the first instance of the face recognition model based on the sample data set to update the parameters of the first instance of the face recognition model; and a transmitting unit configured to transmit update data representing the updated parameters to the cloud server so that the cloud server can update a second instance of the face recognition model deployed at the cloud server.

[0006] According to another aspect of the present disclosure, a face recognition device is provided, including: an acquisition unit, configured to acquire a face image of a user; and an input unit, configured to input the face image into a face recognition model to obtain an identity labeling result of the face image, wherein the face recognition model is trained according to the model training method executed at the terminal device as described above.

[0007] According to another aspect of the present disclosure, a computer device is provided, comprising: at least one processor; and at least one memory on which a computer program is stored, wherein when the computer program is executed by the at least one processor, the at least one processor executes the model training method performed at the terminal device as described above.

[0008] According to another aspect of the present disclosure, a vehicle is provided, comprising the model training apparatus executed at a terminal device as described above or the computer device as described above.

[0009] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the model training method performed at the terminal device as described above.

[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, causes the processor to execute the model training method performed at a terminal device as described above.

[0011] These and other aspects of the disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1 It is a flowchart for iteratively updating a model in the related art;

[0014] Figure 2 is a schematic diagram illustrating an example system in which the various methods described herein may be implemented according to an exemplary embodiment;

[0015] Figure 3 is a flow chart illustrating a model training method 300 performed at a terminal device according to an exemplary embodiment;

[0016] Figure 4 is a schematic diagram illustrating a process of iteratively updating a face recognition model based on a vehicle according to an exemplary embodiment;

[0017] Figure 5 is a flow chart illustrating a method for verifying an identity tagging result of a facial image corresponding to a user according to an exemplary embodiment;

[0018] Figure 6 is a flow chart illustrating a method for verifying an identity labeling result corresponding to each of a plurality of facial images according to an exemplary embodiment;

[0019] Figure 7 is a flow chart illustrating a face recognition method 700 according to an exemplary embodiment;

[0020] Figure 8 is a schematic block diagram illustrating a model training apparatus 800 executed at a terminal device according to an exemplary embodiment;

[0021] Fig. 9 is a schematic block diagram illustrating a face recognition apparatus 900 according to an exemplary embodiment;

[0022] Fig.10 is a block diagram illustrating an exemplary computer device that can be applied to the exemplary embodiments. DETAILED DESCRIPTION

[0023] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0024] The terms used in the description of various examples described in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. As used herein, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". In addition, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations.

[0025] In related technologies, after a face recognition model is put online, it is necessary to iteratively train and update the model based on the results of model verification or user feedback, so as to solve some problems existing in the actual application of the model and optimize the performance of the model.

[0026] Figure 1 A schematic diagram of a process for iteratively updating a model in related technologies is shown.

[0027] In the related art, for a machine learning model deployed in a terminal device, such as a face recognition model deployed on an onboard computer of a vehicle, Figure 1 The iterative process may include step S110, collecting image data, step S120, compressing and uploading, step S130, data collection, step S140, data cleaning and step S150, model iteration.

[0028] In some related technologies, the vehicle can first collect images of the interior of the vehicle through the on-board camera device and temporarily store them in the vehicle's memory; when the vehicle is charging or the on-board computer is idle, the collected image data can be compressed and uploaded to the cloud database; the cloud database receives image data from each vehicle in real time, and performs data cleaning on each image data, such as labeling the data to generate sample data, etc.; and based on the sample data generated after data cleaning, the model deployed on the cloud server is iteratively trained to optimize the performance of the model.

[0029] However, iterative model updates using the above-mentioned technologies require uploading user image data to the cloud database, which inevitably brings data security risks.

[0030] According to an embodiment of the present disclosure, a model training method executed at a terminal device is provided, which can ensure the effectiveness of the model training without uploading the impact data collected by the terminal device to the cloud.

[0031] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0032] Exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0033] Figure 2 is a schematic diagram illustrating an example system 200 in which the various methods described herein may be implemented, according to an example embodiment.

[0034] refer to Figure 2 The system 200 includes an in-vehicle system 210, a server 220, and a network 230 that communicatively couples the in-vehicle system 210 and the server 220.

[0035] The vehicle-mounted system 210 includes a display 214 and an application (APP) 212 that can be displayed via the display 214. The application 212 can be an application installed by default in the vehicle-mounted system 210 or downloaded and installed by the user 202, or a small program as a lightweight application. In the case where the application 212 is a small program, the user 202 can directly run the application 212 on the vehicle-mounted system 210 by searching for the application 212 in the host application (for example, by the name of the application 212, etc.) or scanning the graphic code of the application 212 (for example, a bar code, a QR code, etc.), without installing the application 212. In some embodiments, the vehicle-mounted system 210 may include one or more processors and one or more memories (not shown), and the vehicle-mounted system 210 is implemented as a vehicle-mounted computer. In some embodiments, the vehicle-mounted system 210 may include more or less display screens 214 (for example, excluding the display screen 214), and / or one or more speakers or other human-computer interaction devices. In some embodiments, the vehicle-mounted system 210 may not communicate with the server 220.

[0036] Server 220 may represent a single server, a cluster of multiple servers, a distributed system, or a cloud server that provides basic cloud services (such as cloud database, cloud computing, cloud storage, cloud communication). It will be understood that although Figure 2 The server 220 is shown to communicate with only one vehicle system 210 , but the server 220 can provide background services for multiple vehicle systems at the same time.

[0037] The network 230 allows wireless communication and information exchange between vehicles and X ("X" means vehicle, road, pedestrian or Internet, etc.) according to the agreed communication protocol and data exchange standard. Examples of the network 230 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks such as the Internet. The network 230 can be a wired or wireless network. In one example, the network 230 can be an in-vehicle network, an inter-vehicle network and / or an in-vehicle mobile Internet.

[0038] For the purpose of the embodiments of this disclosure, Figure 2 In the example, the application 212 may be a face recognition application, which may provide functions related to face recognition and verification for the user. Correspondingly, the server 220 may be a server used together with the face recognition application. The server 220 may provide the application 212 running in the vehicle system 210 with services for matching facial features and identity information for the user based on a pre-established cloud user information database. Alternatively, the server 220 may also provide the pre-established cloud user information database to the vehicle system 210, and the application 212 running in the vehicle system 210 may provide the user with services related to face recognition and verification based on the cloud user information database.

[0039] Figure 3 is a flowchart illustrating a model training method 300 performed at a terminal device according to an exemplary embodiment. The method 300 may be performed in a vehicle-mounted system (e.g., Figure 2 The execution of each step of the method 300 may be performed at the vehicle-mounted system 210 shown in FIG. Figure 3 In some embodiments, the method 200 may be performed by a vehicle-mounted system (e.g., the vehicle-mounted system 210) and a server (e.g., the server 220) in combination. Hereinafter, the steps of the method 300 are described in detail by taking the vehicle-mounted system 210 as an example.

[0040] refer to Figure 3 , method 300 includes steps S310 to S370.

[0041] Step S310, receiving a first instance of a face recognition model from a cloud server;

[0042] Step S320: acquiring a plurality of face images captured by a camera arranged at a terminal device;

[0043] Step S330: Based on the first instance of the face recognition model, label each face image in the multiple face images to obtain an identity labeling result corresponding to each face image in the multiple face images;

[0044] Step S340: based on a preset verification strategy, verify the identity labeling result corresponding to each of the multiple face images to obtain multiple identity labels;

[0045] Step S350: generating a sample data set based on a plurality of face images and a plurality of identity tags respectively corresponding to the plurality of face images;

[0046] Step S360: training the first instance of the face recognition model based on the sample data set to update the parameters of the first instance of the face recognition model; and

[0047] Step S370: Transmit the updated data representing the updated parameters to the cloud server so that the cloud server can update the second instance of the face recognition model deployed at the cloud server.

[0048] According to the above embodiment, facial images are collected through the camera at the terminal device, and the facial images are annotated through the first instance of the facial recognition model deployed on the terminal device, and the identity annotation results are verified accordingly, so as to generate sample data based on the verified identity annotation results and train the model. Through the above method, the validity of the generated sample data can be guaranteed while ensuring the security of the user's privacy data, thereby ensuring effective iterative training of the first instance of the model and effective updating of the parameters of the second instance of the model deployed on the cloud server.

[0049] The face recognition model deployed on the cloud server can be a face recognition model obtained through preliminary training on the cloud server based on existing sample data, and it already has a certain ability to perform face recognition on users.

[0050] In some embodiments, the face recognition model deployed on the cloud server may also be a face recognition model that has undergone one or more update iterations.

[0051] The cloud server can simultaneously send a first instance of the face recognition model to at least one terminal device. Sending the first instance can be achieved by sending the parameters of the current face recognition model to each terminal device. After receiving the above parameters, the terminal device can deploy the face recognition model on the terminal device based on these parameters. As used herein, the term "instance" refers to a copy of the face recognition model.

[0052] The terminal device may be, for example, a computer, which may control a camera to collect facial images in a wired or wireless manner and perform corresponding processing on each facial image.

[0053] In some embodiments, a face recognition model deployed on a terminal device (i.e., a first instance of a face recognition model) can first perform feature extraction on a face image to obtain a feature vector of the face image; then, a feature vector with the highest similarity to the feature vector is recalled from a preset cloud user information database, and user identity information corresponding to the feature vector (e.g., a user's identity ID) is obtained, thereby annotating the face image based on the user identity information.

[0054] The cloud user information database can be deployed to communicate with the cloud server and store a full amount of facial images and identity information entered by users themselves.

[0055] In some embodiments, the cloud user information database may also store a mapping relationship between each user's facial image features and their identity ID, wherein the facial image features are feature codes obtained through a currently unupdated facial recognition model deployed on the cloud.

[0056] In some embodiments, the data in the cloud user information database can also be sent by the cloud server to each terminal device, thereby saving the time of information transmission during the face recognition process. The data transmitted to the terminal device for storage may only include the mapping relationship between the full amount of user face image features and their identity ID.

[0057] The preset verification strategy can be used to determine whether the identity annotation results of the face image are accurate, and delete or correct inaccurate results, thereby improving the accuracy of the identity annotation results. The accuracy of the sample data generated based on the verified identity annotation results can be higher than the annotation results of the existing model, so training the model with this sample data can effectively improve the accuracy of the model.

[0058] In some embodiments, after a predetermined number of face images are collected, the face recognition model deployed on the terminal device may be trained once, or the model may be trained once every predetermined time. After the model training is completed, the updated data of the model (e.g., updated model parameters, or the amount of change before and after each model parameter is updated, etc.) may be uploaded to the cloud server for the cloud server to update the face recognition model deployed at the cloud server (i.e., the second instance of the face recognition model).

[0059] In some embodiments, the terminal device may be a vehicle, and a facial recognition model deployed at the vehicle may be used to perform facial recognition and identity authentication on a user, thereby logging into the user's account based on the authentication result.

[0060] Figure 4A schematic diagram of a process 400 for iteratively updating a face recognition model based on a vehicle according to an exemplary embodiment is shown.

[0061] In some embodiments, reference Figure 4 The method for iteratively updating the face recognition model based on the vehicle may include steps S410 to S470. The specific steps are as follows:

[0062] Step S410: During driving, an image of the interior of the vehicle is collected by a camera arranged at the vehicle;

[0063] Step S420: receiving a first instance of a face recognition model sent by a cloud server, and deploying the first instance of the face recognition model at a vehicle, for example, in an onboard computer of the vehicle;

[0064] Step S430: When the vehicle is idle, for example, when the vehicle is charging, the collected images may be cleaned, for example, a face image in the image may be recognized and cropped out, or identity information may be annotated and identity information may be verified for the face image, and a sample data set may be constructed based on the face image after identity information is verified;

[0065] Step S440: When the vehicle is idle, the model may be iteratively trained based on the sample data set;

[0066] Step S450: When the vehicle is idle, uploading updated data to the cloud server based on the updated parameters of the model;

[0067] Step S460: The cloud server may collect update data, and the collected update data may come from at least one vehicle;

[0068] Step S470: After collecting sufficient update data, the cloud server updates the parameters of the second instance of the face recognition model deployed on the cloud server.

[0069] In some embodiments, the above-mentioned steps of data cleaning, model training, and data update uploading can be terminated at any time, and their execution progress can be saved accordingly; the next time the vehicle is idle, the corresponding operations can be continued based on the execution progress.

[0070] In some embodiments, obtaining multiple face images captured by a camera arranged at a terminal device includes: obtaining multiple images captured by the camera; detecting face regions from the multiple images; and storing the detected multiple face regions as multiple face images.

[0071] In some embodiments, the terminal device may be a vehicle, and the process of acquiring multiple face images may be: first, a camera mounted in the vehicle is used to take a picture of the scene inside the vehicle, thereby acquiring multiple images, at which time, each image may include one or more face images, or may not include a face image; face detection is performed on each image (for example, a face detection model may be applied), thereby acquiring the region of each face in the image, and the image is cropped to save the region as a face image for storage. Thus, a large amount of storage space can be saved.

[0072] In some embodiments, based on a preset verification strategy, verifying the identity labeling result corresponding to each facial image in multiple facial images includes: verifying the identity labeling result of the facial image corresponding to the user based on the preset identity information of the user stored in a preset database.

[0073] Therefore, the corresponding identity labeling results can be verified through the obtainable user identity information, thereby improving the accuracy of the identity labeling results. The sample data generated based on the verified identity labeling results can have higher label accuracy than the labeling results of the existing model. Training the model based on such sample data can effectively improve the accuracy of the model.

[0074] The preset database may be a database storing some user identity information. The user's preset identity information may include, for example, the user's mobile phone number, email address, name, and other information.

[0075] In some embodiments, when the terminal device is an operating vehicle, the preset database may be an operating platform database associated with the operating vehicle. Based on the preset identity information of the user stored in the preset database, verifying the identity labeling result of the facial image corresponding to the user may include: obtaining the preset identity information from the operating platform database; and correcting the identity labeling result of the facial image corresponding to the user in the vehicle based on the preset identity information.

[0076] The preset identity information stored in the operation platform database may include, for example, the user's mobile phone number, email address, name, and other information. It will be understood that, in this article, the acquisition and use of user identity information does not violate public order and good morals. Relevant technical personnel can obtain the user identity information stored in the corresponding operation platform database based on the commercial cooperation between the operation platform and the user's authorization. In some cases, when a user takes a taxi through the operation platform, or the user is the driver of the operating vehicle, the user identity information of the vehicle at that time can be matched through information such as the time corresponding to the facial image.

[0077] At the same time, the user's identity information may also be stored in a preset cloud user information database (the user information database is used to provide facial feature vectors for matching during face recognition, which stores the correspondence between the user's facial feature vector and his / her identity ID, as well as other identity information of the user, such as mobile phone number, etc.). Through the user's identity information, the identity ID corresponding to the user can be found in the preset cloud user information database, so as to verify the identity labeling result of the face image based on the identity ID.

[0078] When the identity labeling result is the same as the identity ID, the identity labeling result is judged to be verified and the identity labeling result and its corresponding facial image are stored; when the identity labeling result is different from the identity ID, the identity labeling result is judged to be verified and failed, and the identity labeling result is corrected based on the identity ID, and the corrected identity labeling result and its corresponding facial image are stored.

[0079] Therefore, when the vehicle is an operating vehicle, it is possible to obtain user information from the operating platform and search in the user information database based on the user information to obtain the information previously entered into the database by the user; correcting the face annotation results based on the information can improve the accuracy of the identity annotation results, and the sample data generated based on the verified identity annotation results can have a higher label accuracy than the annotation results of the existing model. Training the model based on such sample data can effectively improve the accuracy of the model.

[0080] In some embodiments, the terminal device may be a vehicle equipped with an onboard computer, and the preset database may be a cloud user information database associated with the onboard computer, the cloud user information database including user identity information pre-entered by the user. Figure 5 As shown, based on the preset identity information of the user stored in the preset database, the identity labeling result of the face image corresponding to the verified user may include:

[0081] Step S510: in response to determining that the user's personal device is connected to the vehicle-mounted computer, sending a request for obtaining device information of the personal device to the personal device;

[0082] Step S520, in response to determining that the acquisition request is approved, acquiring device information;

[0083] Step S530: searching the cloud user information database for user identity information associated with the device information; and

[0084] Step S540: In response to finding the user identity information associated with the device information, correct the identity labeling result of the facial image corresponding to the user based on the user identity information.

[0085] In some cases, when a user is riding in a vehicle, he or she may connect his or her personal device (such as a mobile phone) to the vehicle computer or vehicle system through, for example, Bluetooth or wireless network. At this time, the device information of the personal device (such as a mobile phone number, login information of related applications such as music software, etc.) may be requested from the user, and the device information of the personal device may be obtained after the user agrees.

[0086] At the same time, the user's identity information may also be stored in a preset cloud user information database (the cloud user information database is used to provide facial feature vectors for matching during face recognition, which stores the corresponding relationship between the user's facial feature vector and his / her identity ID, as well as other identity information of the user, such as mobile phone number, etc.). Through the device information of the user's personal device, the identity ID corresponding to the user can be found in the preset cloud user information database, so as to verify the identity labeling result of the face image based on the identity ID.

[0087] When the identity labeling result is the same as the identity ID, the identity labeling result is judged to be verified and the identity labeling result and its corresponding facial image are stored; when the identity labeling result is different from the identity ID, the identity labeling result is judged to be verified and failed, and the identity labeling result is corrected based on the identity ID, and the corrected identity labeling result and its corresponding facial image are stored.

[0088] Therefore, when a user connects to the onboard computer through a personal device, the user can request the device information of the personal device and search the cloud user information database based on the device information to obtain the identity information previously entered into the database by the user. Correcting the face annotation results based on this information can improve the accuracy of the identity annotation results; and the sample data generated based on the verified identity annotation results can have a higher label accuracy than the annotation results of the existing model. Training the model based on such sample data can effectively improve the accuracy of the model.

[0089] In some embodiments, the terminal device may be a vehicle equipped with an onboard computer, the preset database may be a cloud user information database associated with the onboard computer, and the cloud user information database may include user identity information pre-entered by the user and the user's corresponding family member relationship information. Figure 6 As shown, based on the preset verification strategy, verifying the identity labeling result corresponding to each face image in the multiple face images may include:

[0090] Step S610: Based on the cloud user information database, obtain the user identity information of each logged-in user of the vehicle-mounted computer and the corresponding family member relationship information;

[0091] Step S620: Analyze the public social data of each logged-in user and each family member of each logged-in user to obtain a candidate set of drivers and passengers corresponding to the vehicle; and

[0092] Step S630: verify the identity labeling result by determining whether the identity labeling result corresponding to each face image in the multiple face images exists in the driver and passenger candidate set.

[0093] In some cases, when a user enters his or her identity information into the cloud user information database, the identity information of his or her family members and the relationship information between each family member will also be entered at the same time. For each logged-in user of the vehicle, based on the above relationship information and the social data disclosed by each family member on the social platform, the social relationship graph can be mined to obtain the information of all possible persons who may be associated with the vehicle, thereby forming a candidate set of drivers and passengers corresponding to the vehicle.

[0094] In some embodiments, the identity labeling result is verified by determining whether the identity labeling result corresponding to each of the multiple face images exists in the driver and passenger candidate set. In the example, the specific operations may include:

[0095] Determine whether the identity labeling result of a certain user's face image on the vehicle is within the range of the candidate set of drivers and passengers corresponding to the vehicle. The identity labeling result corresponding to the face image is obtained by inputting the face image into the face recognition model to obtain its image feature code, and matching the image feature code with the highest feature similarity in the cloud user information database, and obtaining the identity labeling result corresponding to the face image based on the mapping relationship between the feature code and the identity ID.

[0096] When it is determined that the identity annotation result is within the candidate set range, the feature similarity can be multiplied by the first preset weight to obtain the weighted feature similarity. Subsequently, it is determined whether the weighted feature similarity is greater than or equal to the preset threshold. When the weighted feature similarity is greater than or equal to the preset threshold, it is determined that the identity annotation result is verified, and the identity annotation result and its corresponding face image are stored; when the weighted feature similarity is less than the preset threshold, it is determined that the identity annotation result is not verified, and the identity annotation result and its corresponding face recognition image are deleted. In some embodiments, the personnel in the above-mentioned driver and passenger candidate set can be further divided into a core driver and passenger candidate set and a marginal driver and passenger candidate set, wherein the core driver and passenger candidate set can include the owner and his family members, and the marginal driver and passenger candidate set can include people with certain social relations with the above-mentioned core driver and passenger. In addition, different weights are set for the core driver and passenger and the marginal driver and passenger, respectively, wherein the weight corresponding to the core driver and passenger is greater than the weight corresponding to the marginal driver and passenger.

[0097] When it is determined that the identity labeling result is not within the candidate set, the feature similarity can be multiplied by the second preset weight to obtain a weighted feature similarity, wherein the second preset weight is less than the first preset weight. Subsequently, it is determined whether the weighted feature similarity is greater than or equal to the preset threshold. When the weighted feature similarity is greater than or equal to the preset threshold, it is determined that the identity labeling result has been verified, and the identity labeling result and its corresponding face image are stored; when the weighted feature similarity is less than the preset threshold, it is determined that the identity labeling result has not been verified, and the identity labeling result and its corresponding face recognition image are deleted.

[0098] It is understandable that relevant technical personnel can set the above-mentioned first preset weight, second preset weight and preset threshold according to actual needs, and no limitation is made here.

[0099] Thus, the candidate set of drivers and passengers who may take the vehicle is obtained through the above method. Correcting the face annotation results based on this information can improve the accuracy of the identity annotation results; and the sample data generated based on the verified identity annotation results can have a higher label accuracy than the annotation results of the existing model. Training the model based on such sample data can effectively improve the accuracy of the model.

[0100] In some embodiments, the multiple facial images include multiple first facial images at the same spatial position acquired within a preset time period, and based on a preset verification strategy, verifying the identity labeling results corresponding to each facial image in the multiple facial images includes: in response to determining that there are multiple different identity labeling results among the multiple first identity labeling results corresponding to the multiple first facial images, obtaining the identity labeling result with the largest number among the multiple identity labeling results; and using the identity labeling result with the largest number to correct the multiple first identity labeling results.

[0101] In some cases, such as during a period of continuous driving, the spatial position of each user in the car usually does not change. During this process (i.e., within a preset period of time), the multiple facial images obtained at each position in the car should correspond to the same user. However, due to factors such as changes in the light in the car and changes in the sitting posture of the driver and passengers, the identity labeling results of multiple facial images of the same user may be different within the preset period of time.

[0102] In some embodiments, it is possible to determine whether the multiple facial images collected at the same location are all within a preset time period by determining the time information corresponding to each facial image. If they are all within a preset time period, it is considered that the multiple facial images at the same location within the preset time period should correspond to the same identity labeling result. When the multiple facial images at the same location within the preset time period correspond to multiple identity labeling results, the identity labeling result with the largest number is considered to be the correct labeling result, and other different identity labeling results are corrected based on this labeling result.

[0103] Therefore, by unifying the labeling results of multiple facial images of the same user within the same preset time period through the above method, the accuracy of the identity labeling results can be improved; and the sample data generated based on the verified identity labeling results can have higher label accuracy than the labeling results of the existing model. Training the model based on such sample data can effectively improve the accuracy of the model.

[0104] In some embodiments, when preparing a sample data set, the annotation information of the facial image can be verified by one or more of the above methods, and the sample data set can be generated based on the verified annotation information and the corresponding facial image.

[0105] In some embodiments, the model training method performed at the terminal device may further include: receiving updated model parameters of the face recognition model from the cloud server to update the parameters of the first instance of the face recognition model again. The updated model parameters are obtained by the cloud server updating the parameters of the second instance of the face recognition model in response to the number of update data received from at least one terminal device being greater than a preset number threshold.

[0106] When the cloud server receives enough update data from one or more terminal devices, the parameters of the second instance of the face recognition model are updated based on the update data.

[0107] In some embodiments, the update data uploaded by each terminal device may be the change in each parameter of the first instance of the face recognition model before and after training.

[0108] In some embodiments, the parameters of the second instance of the face recognition model can be updated by calculating the average value of multiple parameter changes corresponding to each parameter from different terminal devices or different training processes, and applying the average value to update the model parameters.

[0109] After the second instance of the face recognition model is updated, the updated parameters can be sent to each terminal device, thereby updating the first instance of the face recognition model deployed in each terminal device, thereby achieving iterative optimization of the face recognition model deployed in each terminal device and improving the face recognition accuracy of the model.

[0110] Although various operations are depicted in the drawings as being in a particular order, this should not be understood as requiring that the operations must be performed in the particular order shown or in sequential order, nor should it be understood as requiring that all illustrated operations must be performed to achieve desirable results.

[0111] Figure 7 is a flow chart illustrating a face recognition method 700 according to an exemplary embodiment.

[0112] like Figure 7 As shown, the face recognition method 700 may include:

[0113] Step S710: obtaining a user's facial image; and

[0114] Step S720: input the facial image into the facial recognition model to obtain the identity labeling result of the facial image, wherein the facial recognition model is trained according to the model training method executed at the terminal device as described above.

[0115] Figure 8 is a schematic block diagram illustrating a model training apparatus 800 executed at a terminal device according to an exemplary embodiment.

[0116] like Figure 8 As shown, the model training device 800 executed at the terminal device may include:

[0117] A receiving unit 810 is configured to receive a first instance of a face recognition model from a cloud server;

[0118] An acquisition unit 820 is configured to acquire a plurality of face images captured by a camera arranged at the terminal device;

[0119] A labeling unit 830 is configured to label each of the multiple face images based on the first instance of the face recognition model to obtain an identity labeling result corresponding to each of the multiple face images;

[0120] The verification unit 840 is configured to verify the identity labeling result corresponding to each face image in the plurality of face images based on a preset verification strategy to obtain the plurality of identity labels;

[0121] A generating unit 850 is configured to generate a sample data set based on the multiple face images and multiple identity tags respectively corresponding to the multiple face images;

[0122] A training unit 860 is configured to train the first instance of the face recognition model based on the sample data set to update the parameters of the first instance of the face recognition model; and

[0123] The transmitting unit 870 is configured to transmit update data representing updated parameters to the cloud server so that the cloud server updates the second instance of the face recognition model deployed at the cloud server.

[0124] in, Figure 8 The various units of the device 800 shown in FIG. 8 can be used in conjunction with the reference Figure 3 The steps in the method 300 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 300 are also applicable to the apparatus 800 and the units included therein. For the sake of brevity, some operations, features and advantages are not described in detail here.

[0125] Fig. 9 is a schematic block diagram illustrating a face recognition apparatus 900 according to an exemplary embodiment.

[0126] like Fig. 9 As shown, the face recognition device 900 may include:

[0127] An acquisition unit 910 is configured to acquire a facial image of a user; and

[0128] The input unit 920 is configured to input the facial image into a facial recognition model to obtain user identity information corresponding to the facial image, wherein the facial recognition model is trained according to the model training method performed at the terminal device as described above.

[0129] in, Fig. 9 The various units of the device 900 shown in FIG. 9 can be used in conjunction with the reference Figure 7 The steps in the described method 700 correspond to each other. For the sake of brevity, certain operations, features and advantages are not described in detail here.

[0130] Although specific functions are discussed above with reference to specific modules, it should be noted that the functions of the various modules discussed herein may be divided into multiple modules, and / or at least some functions of multiple modules may be combined into a single module. The specific module discussed herein performing an action includes the specific module itself performing the action, or alternatively the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, the specific module that performs an action may include the specific module itself that performs the action and / or another module that the specific module calls or otherwise accesses to perform the action. As used herein, the phrase "entity A initiates action B" may mean that entity A issues an instruction to perform action B, but entity A itself does not necessarily perform the action B.

[0131] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 8 and 9 The various units described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuits. SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.

[0132] According to one aspect of the present disclosure, a computer device is provided, comprising at least one memory, at least one processor, and a computer program stored in the at least one memory. The at least one processor is configured to execute the computer program to implement the steps of any method embodiment described above.

[0133] According to one aspect of the present disclosure, a vehicle is provided, which includes the apparatus or computer device as described above.

[0134] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0135] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0136] In the following, combined Fig.10 Illustrative examples of such a computer device, non-transitory computer-readable storage medium, and computer program product are described.

[0137] Fig.10 An example configuration of a computer device 1000 that can be used to implement the methods described herein is shown. For example, Figure 2 The server 220 and / or the vehicle system 210 shown in FIG. 1 may include an architecture similar to the computer device 1000. The above-mentioned apparatus 800 or apparatus 900 may also be implemented in whole or at least in part by the computer device 1000 or a similar device or system.

[0138] The computer device 1000 may include at least one processor 1002, memory 1004, communication interface(s) 1006, a display device 1008, other input / output (I / O) devices 1010, and one or more mass storage devices 1012, which can communicate with each other, such as via a system bus 1014 or other appropriate connection.

[0139] The processor 1002 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. The processor 1002 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, the processor 1002 may be configured to obtain and execute computer-readable instructions stored in the memory 1004, mass storage device 1012, or other computer-readable media, such as program code of an operating system 1016, program code of an application program 1018, program code of other programs 1020, and the like.

[0140] The memory 1004 and the mass storage device 1012 are examples of computer-readable storage media for storing instructions that are executed by the processor 1002 to implement the various functions described above. For example, the memory 1004 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, the mass storage device 1012 may generally include a hard drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc. The memory 1004 and the mass storage device 1012 may all be collectively referred to herein as memory or computer-readable storage media, and may be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by the processor 1002 as a specific machine configured to implement the operations and functions described in the examples herein.

[0141] A number of programs may be stored on mass storage device 1012. These programs include operating system 1016, one or more application programs 1018, other programs 1020, and program data 1022, and they may be loaded into memory 1004 for execution. Examples of such applications or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components / functions: method 300 and / or method 700 (including any suitable steps of method 300, 700), and / or other embodiments described herein.

[0142] Although in Fig.10 1000, but modules 1016, 1018, 1020, and 1022, or portions thereof, may be implemented using any form of computer-readable media accessible by computer device 1000. As used herein, "computer-readable media" includes at least two types of computer-readable media, namely, computer-readable storage media and communication media.

[0143] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computer device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism. Computer-readable storage media defined herein do not include communication media.

[0144] One or more communication interfaces 1006 are used to exchange data with other devices, such as through a network, a direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as IEEE 802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a cellular network ... TM The communication interface 1006 may facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interface 1006 may also provide for communication with external storage devices (not shown) such as in a storage array, network attached storage, storage area network, etc.

[0145] In some examples, a display device 1008 such as a monitor may be included for displaying information and images to the user. Other I / O devices 1010 may be devices that receive various inputs from the user and provide various outputs to the user, and may include a touch input device, a gesture input device, a camera, a keyboard, a remote control, a mouse, a printer, an audio input / output device, and the like.

[0146] The technology described herein can be supported by these various configurations of the computer device 1000, and is not limited to the specific examples of the technology described herein. For example, the functionality can also be implemented in whole or in part on the "cloud" by using a distributed system. The cloud includes and / or represents a platform for resources. The platform abstracts the underlying functions of the hardware (e.g., server) and software resources of the cloud. Resources can include applications and / or data that can be used when performing computing processing on a server away from the computer device 1000. Resources can also include services provided over the Internet and / or through a subscriber network such as a cellular or Wi-Fi network. The platform can abstract resources and functions to connect the computer device 1000 to other computer devices. Therefore, the implementation of the functions described herein can be distributed throughout the cloud. For example, the functions can be implemented partially on the computer device 1000 and partially through a platform that abstracts the functions of the cloud.

[0147] Although the present disclosure has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative and schematic, not restrictive; the present disclosure is not limited to the disclosed embodiments. By studying the drawings, the disclosure and the appended claims, those skilled in the art will be able to understand and implement variations to the disclosed embodiments when practicing the claimed subject matter. In the claims, the word "comprising" does not exclude other elements or steps that are not listed, the indefinite article "a" or "an" does not exclude a plurality, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". The mere fact that certain measures are recorded in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A model training method performed at a terminal device, the method comprising: receiving a first instance of a face recognition model from a cloud server; Acquire a plurality of face images captured by a camera arranged at the terminal device; Based on the first instance of the face recognition model, each face image in the multiple face images is labeled to obtain an identity labeling result corresponding to each face image in the multiple face images, including performing the following operations for each face image in the multiple face images: Based on the first example, performing feature extraction on the face image to obtain a feature vector of the face image; Recalling a preset feature vector having the highest similarity to the feature vector from a preset cloud user information database; and Obtaining a user ID corresponding to the preset feature vector in the cloud user information database, so as to use the user ID as an identity labeling result of the face image; Based on a preset verification strategy, the identity labeling result corresponding to each of the multiple face images is verified to obtain multiple identity labels, including: Based on the preset identity information of the user stored in the preset database, verify the identity labeling result of the face image corresponding to the user, wherein the cloud user information database also includes identity information corresponding to the user identity ID, and the identity information includes at least one of a mobile phone number, an email address, and a name. In response to the terminal device being an operating vehicle, the preset database is an operating platform database associated with the operating vehicle, and wherein, based on the preset identity information of the user stored in the preset database, verifying the identity labeling result of the face image corresponding to the user includes: Acquiring the preset identity information from the operation platform database includes: determining the preset identity information of the user who rides the operation vehicle at the time based on the time corresponding to the face image, the preset identity information including at least one of a mobile phone number, an email address, and a name; and Correcting the identity labeling result of the face image corresponding to the user based on the preset identity information includes: In response to the preset identity information being identical to the identity information corresponding to the identity labeling result of the face image, determining that the identity labeling result of the face image has been verified; and In response to the preset identity information being different from the identity information corresponding to the identity tagging result of the face image, determining that the identity tagging result of the face image has failed verification, and correcting the identity tagging result to the user identity ID corresponding to the preset identity information of the user; Generate a sample data set based on the multiple facial images and multiple identity tags corresponding to the multiple facial images respectively; Based on the sample data set, training the first instance of the face recognition model to update the parameters of the first instance of the face recognition model; and The updated data representing the updated parameters is transmitted to the cloud server so that the cloud server can update the second instance of the face recognition model deployed at the cloud server.

2. The method according to claim 1, wherein: The terminal device is a vehicle equipped with an on-board computer, the preset database is a cloud user information database associated with the on-board computer, the cloud user information database includes user identity information pre-entered by the user, and wherein the identity labeling result of verifying the face image corresponding to the user based on the preset identity information of the user stored in the preset database includes: In response to determining that the personal device of the user is connected to the vehicle-mounted computer, sending a request for obtaining device information of the personal device to the personal device; In response to determining that the acquisition request is approved, acquiring the device information; Searching the cloud user information database for user identity information associated with the device information; and In response to finding the user identity information associated with the device information, the identity labeling result of the facial image corresponding to the user is corrected based on the user identity information.

3. The method according to claim 1, wherein: The terminal device is a vehicle equipped with an on-board computer, the preset database is a cloud user information database associated with the on-board computer, the cloud user information database includes user identity information pre-entered by the user and the user's corresponding family member relationship information, and wherein, based on the preset verification strategy, verifying the identity labeling result corresponding to each of the multiple face images includes: Based on the cloud user information database, obtaining user identity information of each logged-in user of the vehicle-mounted computer and corresponding family member relationship information; Acquire a candidate set of drivers and passengers corresponding to the vehicle by analyzing the public social data of each logged-in user and each family member corresponding to each logged-in user; and The identity labeling result is verified by determining whether the identity labeling result corresponding to each face image in the multiple face images exists in the driver and passenger candidate set.

4. The method according to claim 1, wherein: The multiple facial images include multiple first facial images at the same spatial position acquired within a preset time period, and wherein, based on a preset verification strategy, verifying the identity labeling result corresponding to each facial image in the multiple facial images includes: In response to determining that there are a plurality of different identity labeling results among the plurality of first identity labeling results corresponding to the plurality of first facial images, obtaining the identity labeling result with the largest number among the plurality of identity labeling results; The plurality of first identity labeling results are corrected using the identity labeling results with the largest number.

5. The method according to claim 1, wherein: The acquiring of a plurality of face images collected by a camera arranged at the terminal device comprises: Acquire a plurality of images captured by the camera; Detecting a human face region from the plurality of images; and The detected multiple face regions are stored as the multiple face images.

6. The method according to any one of claims 1 to 5, further comprising: Receive updated model parameters of the face recognition model from the cloud server to update the parameters of the first instance of the face recognition model again, wherein the updated model parameters are obtained by the cloud server updating the parameters of the second instance of the face recognition model in response to the number of update data received from at least one terminal device being greater than a preset number threshold.

7. A face recognition method, the method comprising: Get the user's face image; as well as The facial image is input into a facial recognition model to obtain an identity labeling result of the facial image, wherein the facial recognition model is trained according to the method according to any one of claims 1-6.

8. A model training device executed at a terminal device, the device comprising: A receiving unit, configured to receive a first instance of a face recognition model from a cloud server; an acquisition unit, configured to acquire a plurality of face images captured by a camera arranged at the terminal device; The labeling unit is configured to label each face image in the plurality of face images based on the first instance of the face recognition model to obtain an identity labeling result corresponding to each face image in the plurality of face images, including performing the following operations for each face image in the plurality of face images: Based on the first example, performing feature extraction on the face image to obtain a feature vector of the face image; Recalling a preset feature vector having the highest similarity to the feature vector from a preset cloud user information database; and Obtaining a user ID corresponding to the preset feature vector in the cloud user information database, so as to use the user ID as an identity labeling result of the face image; The verification unit is configured to verify the identity labeling result corresponding to each face image in the plurality of face images based on a preset verification strategy to obtain a plurality of identity labels, including: Based on the preset identity information of the user stored in the preset database, verify the identity labeling result of the face image corresponding to the user, wherein, the cloud user information database also includes identity information corresponding to the user identity ID, and the identity information includes at least one of a mobile phone number, an email address, and a name. In response to the terminal device being an operating vehicle, the preset database is an operating platform database associated with the operating vehicle, and wherein, based on the preset identity information of the user stored in the preset database, verifying the identity labeling result of the face image corresponding to the user includes: Acquiring the preset identity information from the operation platform database includes: determining the preset identity information of the user who rides the operation vehicle at the time based on the time corresponding to the face image, the preset identity information including at least one of a mobile phone number, an email address, and a name; and Correcting the identity labeling result of the face image corresponding to the user based on the preset identity information includes: In response to the preset identity information being identical to the identity information corresponding to the identity labeling result of the face image, determining that the identity labeling result of the face image has been verified; and In response to the preset identity information being different from the identity information corresponding to the identity tagging result of the face image, determining that the identity tagging result of the face image has failed verification, and correcting the identity tagging result to the user identity ID corresponding to the preset identity information of the user; A generating unit configured to generate a sample data set based on the multiple face images and multiple identity tags respectively corresponding to the multiple face images; a training unit configured to train the first instance of the face recognition model based on the sample data set to update parameters of the first instance of the face recognition model; and The transmitting unit is configured to transmit update data representing updated parameters to the cloud server so that the cloud server updates the second instance of the face recognition model deployed at the cloud server.

9. A face recognition device, comprising: An acquisition unit, configured to acquire a facial image of a user; as well as An input unit is configured to input the facial image into a facial recognition model to obtain user identity information corresponding to the facial image, wherein the facial recognition model is trained according to the method according to any one of claims 1-6.

10. A computer device comprising: at least one processor; as well as at least one memory having a computer program stored thereon, Wherein, when the computer program is executed by the at least one processor, the at least one processor is caused to perform the method according to any one of claims 1 to 7.

11. A vehicle comprising the apparatus of claim 9 or the computer device of claim 10.

12. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

13. A computer program product, comprising a computer program, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 7.

Images