A method and device for identifying a person based on AR glasses, AR glasses and medium

Abstract

The present application relates to a method, device, AR glasses and medium for character recognition based on AR glasses, and belongs to the technical field of face recognition, wherein the method comprises: obtaining target multimedia information, wherein the target multimedia information comprises a target object; obtaining facial features of the target object based on the target multimedia information; determining the target object in the current scene based on the facial features; and controlling the display of the target object in a preset manner. The present application has the effect of improving the accuracy of identifying the target person.

Description

Technical Field

[0001] The present application relates to the technical field of face recognition, and in particular to a method and device for character recognition based on AR glasses, AR glasses, and a medium. Background Art

[0002] AR glasses are also called augmented reality glasses. AR glasses cover virtual objects directly in the user's direct field of view through transparent glass. Users of AR glasses can observe the real world and the virtual world at the same time. With the rapid development of artificial intelligence, AR glasses are combined with face recognition technology and applied in many occasions.

[0003] In the related art, the method of combining AR glasses with face recognition to identify the target person may include: capturing the face in real time through the camera of the AR glasses, and comparing the facial features with the pre-stored face database to determine the target person, and displaying the target person through AR glasses, etc.

[0004] However, if AR glasses are used to identify target persons at various large-scale events such as ball games and concerts, due to the large number of performers, the facial information of all the performers may not be pre-stored in the facial database, which may make it difficult to accurately identify the target person and may not be able to accurately display the target person. Summary of the invention

[0005] In order to improve the accuracy of identifying a target person, the present application provides a person recognition method, device, AR glasses and medium based on AR glasses.

[0006] In a first aspect, the present application provides a method for character recognition based on AR glasses, comprising:

[0007] Acquiring target multimedia information, wherein the target multimedia information includes a target object;

[0008] Acquire facial features of the target object based on the target multimedia information;

[0009] Determine the target object in the current scene based on the facial features;

[0010] The target object is displayed by controlling in a preset manner.

[0011] By adopting the above technical solution, when the user wears AR glasses in the current scene, the AR glasses determine the target object to be identified through the target multimedia information, and then obtain the facial features of the target object, and save the facial features of the target object to the face database, so as to determine the target object from the current scene according to the facial features of the target object and display the target object. Therefore, AR glasses are suitable for various occasions, and can track and locate the target person according to the multimedia information at the event site to obtain the person's features. The application scenarios are more flexible, and the recognition speed and accuracy of the target person are improved.

[0012] In another possible implementation manner, the target multimedia information includes at least two objects, and acquiring facial features of the target objects based on the target multimedia information includes:

[0013] Acquire a character list based on the target multimedia information, and display the character list in combination with the current scene;

[0014] The target object selected by the user based on the character list is obtained, and the facial features of the target object are obtained.

[0015] By adopting the above technical solution, after all objects in the target multimedia information are obtained, all objects are displayed in the form of a character list, which is convenient for users to select target characters, improves the user experience of AR glasses, and facilitates users to quickly select target characters.

[0016] In another possible implementation, the acquiring a character list based on the target multimedia information includes:

[0017] Obtaining facial features corresponding to each object in the target multimedia information;

[0018] Determine object attribute information corresponding to each object based on the facial features corresponding to each object, wherein the object attribute information includes: name, gender, age and occupation;

[0019] The character list is generated according to the object attribute information respectively corresponding to each of the objects.

[0020] By adopting the above technical solution, the object attribute characteristics corresponding to each object in the character list are obtained through the facial features in the target multimedia information, so that all object attribute information can be obtained more accurately, making it easier for users to select target objects from the character list more intuitively.

[0021] In another possible implementation, the acquiring a character list based on the target multimedia information includes:

[0022] Identifying text information in the target multimedia information;

[0023] Filtering out keywords from the text information, wherein the keywords include character name information and / or activity name information;

[0024] Obtaining object attribute information corresponding to each of the keywords to obtain object attribute information corresponding to each object;

[0025] Obtain a character list based on the object attribute information corresponding to each object.

[0026] By adopting the above technical solution, the corresponding object attribute characteristics of each object in the character list are obtained through the text information in the target multimedia information. The object attribute information corresponding to each object is searched through keywords, and all object attribute information can be obtained more quickly, which is convenient for users to select the target object from the character list more intuitively.

[0027] In another possible implementation, the method further includes:

[0028] Acquire the current character features of the target object, the character features including the facial features, clothing features, hairstyle features and body features, and establish a character model based on the character features;

[0029] The target object is tracked and selected in the multimedia information of the current scene based on the character model.

[0030] By adopting the above technical solution, the character features of the target object are obtained and a character model is established. The target object is tracked from the multimedia information of the current scene through the character model, thereby reducing the possibility of AR glasses losing the selection when the facial features of the target object cannot be recognized, and increasing the comparison library for character recognition, so as to track the selected target object more quickly.

[0031] In another possible implementation, the tracking and selecting the target object in the multimedia information of the current scene based on the character model includes:

[0032] Extracting a first character feature, where the first character feature is a character feature of all objects in a current scene captured by a camera of the AR glasses;

[0033] Based on the comparison between the clothing features, hairstyle features and body features in the first character features and the character features of the target object, a second character feature that is consistent with the comparison is screened out, and the second character feature is the character feature of the candidate target object;

[0034] Determine whether the number of the second character features is greater than one, and if so, determine a target object with consistent comparison based on the comparison of the candidate facial features of the candidate target object and the facial features of the target object;

[0035] The target object is selected by tracking.

[0036] By adopting the above technical scheme, the clothing features, hairstyle features and body features of the first character are first compared with the character features of the target object. Then, when the number of candidate target objects initially screened is greater than one, the target object is further determined by screening through facial features. The preliminary screening is performed through features such as clothing, hairstyle and body that are easier to identify and screen, thereby improving the speed of determining the target object.

[0037] In another possible implementation manner, the tracking and selecting the target object includes:

[0038] Establishing a coordinate system based on the multimedia information of the current scene;

[0039] Obtaining historical coordinates of a target object in multimedia information of the current scene at a previous moment;

[0040] Acquire coordinates of all objects from multimedia information of the current scene at the current moment;

[0041] Objects whose distances from the historical coordinates of the target object are less than a preset deviation value are screened according to the coordinates of all objects, and the screened objects are determined to be the target objects.

[0042] By adopting the above technical solution, according to the rule that the moving positions of target objects in consecutive moments are similar, the coordinates of all objects are compared with the historical coordinates of the target object, the target object is screened out, and the target object is determined more accurately and quickly.

[0043] In another possible implementation, the method further includes:

[0044] Acquiring the target object's foot motion information by using an image recognition model, wherein the foot motion information includes the orientation information of the two toes and the projection distance information between the two toes, and the orientation information of the two toes includes the angle between each toe and a vertical plane parallel to the camera;

[0045] Determining predicted moving direction information of the target object according to the orientation information of the two toes, wherein the predicted moving direction information of the target object includes an angle between the predicted moving direction of the target object and the vertical plane;

[0046] Based on the same predicted moving direction of the target object, determining the predicted moving speed information of the target object according to the projection distance information between the two toes;

[0047] Display identification information, the identification information is used to identify the predicted moving direction of the target object and the predicted moving speed of the target object

[0048] By adopting the above technical solution, the predicted moving direction of the target object is determined by judging the orientation information of the two toes of the target object, and the moving speed of the target object is determined by judging the projection distance information between the two toes. In addition, identification information is used to indicate the predicted moving direction and moving speed of the target object, so that the user can move the AR glasses according to the identification information and track the target object more intuitively.

[0049] In another possible implementation, the method further includes:

[0050] Acquire multimedia information of the same scene located near the geographical location of the AR glasses through the Internet, where the multimedia information of the same scene is multimedia information related to the current scene uploaded to the Internet by nearby users;

[0051] The character features in the multimedia information of the same scene are obtained, the character features having the highest similarity with the target character features are screened out as newly added character features, and the newly added character features of the target object are saved, and the target character features are the character features of the target object.

[0052] By adopting the above technical solution, multimedia information of the same location and scene is obtained through the Internet, and new character features with the highest similarity to the target character features are obtained therefrom. The new character features are used to enrich the comparison library of the target object, thereby improving the accuracy of AR glasses in determining the target object.

[0053] In a second aspect, the present application provides a person recognition device based on AR glasses, comprising:

[0054] A first acquisition module, used to acquire target multimedia information, wherein the target multimedia information includes a target object;

[0055] A second acquisition module, used for acquiring facial features of the target object based on the target multimedia information;

[0056] A third acquisition module, used to determine the target object in the current scene based on the facial features;

[0057] The display module is used to control the display of the target object in a preset manner.

[0058] By adopting the above technical solution, when the user wears AR glasses in the current scene, the first acquisition module and the second acquisition module determine the target object to be identified through the target multimedia information, and then obtain the facial features of the target object, and save the facial features of the target object to the face database, so that the third acquisition module determines the target object from the current scene according to the facial features of the target object, and the display module controls the display of the target object. Therefore, AR glasses are suitable for various occasions, and the target person can be tracked and located by obtaining the character features according to the multimedia information at the event site. The application scenarios are more flexible, and the recognition speed and accuracy of the target person are improved.

[0059] In another possible implementation manner, the target multimedia information includes at least two objects, wherein the second acquisition module includes:

[0060] A character list generating unit, used for acquiring a character list based on target multimedia information, and displaying the character list in combination with the current scene;

[0061] The target object acquisition unit is used to acquire the target object selected by the user based on the character list and acquire the facial features of the target object.

[0062] In another possible implementation, when the character list generating unit generates the character list according to the face information of the object, it is specifically used to:

[0063] Obtaining facial features corresponding to each object in the target multimedia information;

[0064] Based on the facial features corresponding to each object, determine the object attribute information corresponding to each object, the object attribute information includes: name, gender, age and occupation;

[0065] Generate a character list according to the object attribute information corresponding to each object.

[0066] In another possible implementation, when the character list generating unit generates the character list according to the text information in the target multimedia information, it is specifically used to:

[0067] Recognize text information in target multimedia information;

[0068] Filter out keywords from the text information, where the keywords include person name information and / or activity name information;

[0069] Obtaining object attribute information corresponding to each keyword to obtain object attribute information corresponding to each object;

[0070] Obtain a character list based on the object attribute information corresponding to each object.

[0071] Furthermore, the device also includes:

[0072] The character feature acquisition module is used to acquire the current character features of the target object, including facial features, clothing features, hairstyle features and body features, and to establish a character model based on the character features;

[0073] The tracking and selection module is used to track and select the target object in the multimedia information of the current scene based on the character model.

[0074] In another possible implementation, the selected module is tracked, specifically for:

[0075] Extracting a first character feature, where the first character feature is a character feature of all objects in the current scene captured by a camera of the AR glasses;

[0076] Based on the comparison between the clothing features, hairstyle features and body features in the first character features and the character features of the target object, a second character feature that is consistent with the comparison is screened out, and the second character feature is the character feature of the candidate target object;

[0077] Determine whether the number of the second character features is greater than one, and if so, determine the target object with consistent comparison based on the comparison of the candidate facial features of the candidate target object and the facial features of the target object;

[0078] Track the selected target object.

[0079] In another possible implementation, the tracking selected module is specifically used in the step of tracking the selected target object:

[0080] Establishing a coordinate system based on multimedia information of the current scene;

[0081] Obtain the historical coordinates of the target object in the multimedia information of the current scene at the previous moment;

[0082] Obtaining coordinates of all objects from multimedia information of the current scene at the current moment;

[0083] Objects whose distances from historical coordinates of the target object are less than a preset deviation value are screened according to the coordinates of all objects, and the screened objects are determined to be the target objects.

[0084] In another possible implementation, the device further includes:

[0085] A two-foot motion information acquisition module is used to acquire the two-foot motion information of the target object by using an image recognition model, wherein the two-foot motion information includes the orientation information of the two toes and the projection distance information between the two toes, and the orientation information of the two toes includes the angle between each toe and the vertical plane parallel to the camera;

[0086] A predicted moving direction information determination module, which determines the predicted moving direction information of the target object according to the orientation information of the two toes, wherein the predicted moving direction information of the target object includes an angle between the predicted moving direction of the target object and a vertical plane;

[0087] A predicted moving speed determination module, used to determine the predicted moving speed information of the target object according to the projection distance information between the two toes based on the predicted moving direction of the same target object;

[0088] The identification information display module is used to display identification information, where the identification information is used to identify the predicted moving direction and the predicted moving speed of the target object.

[0089] In another possible implementation, the device further includes:

[0090] The multimedia information acquisition module of the same scene is used to acquire multimedia information of the same scene located near the geographical location of the AR glasses through the Internet. The multimedia information of the same scene is multimedia information related to the current scene uploaded to the Internet by nearby users;

[0091] The newly added character feature acquisition module is used to obtain character features in multimedia information of the same scene, select character features with the highest similarity to target character features as newly added character features, and save the newly added character features of the target object. The target character features are the character features of the target object.

[0092] In a third aspect, the present application provides an AR glasses comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute any method in the first aspect.

[0093] By adopting the above technical solution, when the user wears AR glasses in the current scene, the processor loads and executes the computer program on the memory, so that the AR glasses determine the target object to be identified through the target multimedia information, and then obtain the facial features of the target object, and save the facial features of the target object to the face database, so that the AR glasses determine the target object from the current scene according to the facial features of the target object and display the target object. Therefore, the AR glasses are suitable for various occasions, and the target person can be tracked and located by obtaining the character features according to the multimedia information at the event site. The application scenarios are more flexible, and the recognition speed and accuracy of the target person are improved.

[0094] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the computer program according to any one of the methods in the first aspect.

[0095] By adopting the above technical solution, when the user wears AR glasses in the current scene, the processor loads and executes the computer program on the readable storage medium, so that the AR glasses determine the target object to be identified through the target multimedia information, and then obtain the facial features of the target object, and save the facial features of the target object to the face database, so that the AR glasses determine the target object from the current scene according to the facial features of the target object and display the target object. Therefore, the AR glasses are suitable for various occasions, and the target person can be tracked and located by obtaining the character features according to the multimedia information at the event site. The application scenarios are more flexible, and the recognition speed and accuracy of the target person are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a schematic diagram of the overall process of the person recognition method based on AR glasses in an embodiment of the present application.

[0097] Figure 2 It is a flowchart of the steps of determining the target object through character features in an embodiment of the present application.

[0098] Figure 3 It is a structural schematic diagram of a person recognition device based on AR glasses in an embodiment of the present application.

[0099] Figure 4 It is a structural block diagram of the AR glasses in an embodiment of the present application. DETAILED DESCRIPTION

[0100] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0101] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0102] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0103] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0104] The present application embodiment provides a method for character recognition based on AR glasses, referring to Figure 1 , the main process of the method is described as follows (step S101 to step S104):

[0105] Step S101: Acquire target multimedia information, where the target multimedia information includes a target object.

[0106] For the embodiments of the present application, the target multimedia information may be information such as images or videos that can be recognized by AR glasses. Specifically, the target multimedia information may be posters or promotional videos of an event site, and the posters or promotional videos of the event site may include characters appearing at the event site.

[0107] The target object is the person that the user wants to see through the AR glasses.

[0108] Step S102: acquiring facial features of the target object based on the target multimedia information.

[0109] Specifically, the AR glasses first recognize all objects in the multimedia information, and then the user selects a target object from all objects.

[0110] In an embodiment of the present application, the user can select the target object through the buttons or touch screen on the AR glasses; or, select the target object using gestures, and the camera of the AR glasses recognizes the user's gestures to determine the target object.

[0111] Furthermore, after the AR glasses determine the target object selected by the user, they obtain the facial features of the target object, including facial features, face shape features, etc. At the same time, the AR glasses save the facial features in a face database.

[0112] Step S103, determining a target object in the current scene based on facial features.

[0113] Specifically, when the user uses AR glasses in the current scene, the AR glasses recognize the facial features of multiple objects in the current scene, and compare the facial features of the objects in the current scene with the facial features of the target objects in the face database. However, due to the makeup of the target object or the light of the current scene, the facial features of the target object in the target multimedia information and the current scene may not be completely consistent. Therefore, after comparing the facial features of the objects in the current scene with the facial features of the target object, the AR glasses determine the object with the highest similarity as the target object.

[0114] Step S104, controlling the display of the target object in a preset manner.

[0115] Specifically, after the AR glasses determine the target object, the target object is highlighted. In the embodiment of the present application, the target object can be highlighted by displaying an enlarged image of the target object, displaying an identification box to select the target object, displaying an arrow to track and indicate the target object, etc., which is conducive to the user directly observing the target object through the AR glasses.

[0116] An embodiment of the present application provides a method for displaying a person based on AR glasses. In different environments, the AR glasses first obtain the facial features of the target object based on the target multimedia information related to the current scene, and can store the temporarily obtained facial features in a face database. Then, the target object is determined in the current scene based on the facial features and the target object is displayed, thereby improving the accuracy of identifying the target person and being suitable for accurately tracking the target object in various scenes.

[0117] Furthermore, if the target multimedia information contains at least two objects, it is necessary to determine the target object from the at least two objects, wherein the facial features of the target object are obtained based on the target multimedia information, including: obtaining a character list based on the target multimedia information, and displaying the character list in combination with the current scene; obtaining the target object selected by the user based on the character list, and obtaining the facial features of the target object.

[0118] Among them, when the AR glasses obtain all the objects, a character list is generated according to the information of all the objects, and the character list is displayed on the AR glasses, so that the user can view the object information through the character list more intuitively and conveniently, and then select the target object.

[0119] The user can scroll up and down the character list through the buttons or touch screen on the AR glasses, and then select the target object through the buttons, or double-click the touch screen to determine the target object; in addition, the user also makes a gesture of scrolling the character list in front of the AR glasses. The AR glasses recognize the gesture and make the character list slide up and down according to the gesture, and when the user's hand clicks the target object, the AR glasses determine the target object selected by the user.

[0120] In another possible implementation, the AR glasses may generate a character list based on the facial information of all objects, wherein the character list is obtained based on the target multimedia information, including (steps S11 to S13):

[0121] Step S11 (not shown in the figure), obtaining facial features corresponding to each object in the target multimedia information.

[0122] Step S12 (not shown in the figure) determines object attribute information corresponding to each object based on the facial features corresponding to each object. The object attribute information includes: name, gender, age and occupation.

[0123] Specifically, AR glasses can determine the object attribute information corresponding to each object through the Internet. First, the facial features corresponding to each object are compared with the facial features on the Internet to determine the objects with consistent comparisons, and then obtain the object attribute information corresponding to the object, where the object attribute information includes but is not limited to name, gender, age and occupation, and can be supplemented according to user settings.

[0124] Step S13 (not shown in the figure), generating a character list according to the object attribute information corresponding to each object.

[0125] Specifically, the AR glasses can list the object attribute information corresponding to each object in sequence to generate a character list, and display it on the AR glasses. The user can slide the character list and select the target object from it.

[0126] For example, at a concert, the following character list can be generated:

[0127] Object A Male, 25 years old, singer Object B Female, 21 years old, singer Object C Male, 27 years old, actor, singer

[0128] In another possible implementation of the embodiment of the present application, a character list may be generated through text information in the target multimedia information, and obtaining the character list based on the target multimedia information includes (steps S21 to S24):

[0129] Step S21 (not shown in the figure), identifying text information in the target multimedia information.

[0130] The text information in the target multimedia information includes the event name, event date, event address, names of people participating in the event, etc. For example, AR glasses can obtain text information such as the concert name, concert start date and time, concert address, singer name, etc. from the poster or promotional video of the concert.

[0131] Step S22 (not shown in the figure), filter out keywords from the text information, the keywords include character name information and / or activity name information.

[0132] Step S23 (not shown in the figure), obtaining object attribute information corresponding to each keyword to obtain object attribute information corresponding to each object.

[0133] Among them, when keywords containing character name information are filtered out, the object attribute information corresponding to the character name information can be directly searched through the Internet. When keywords containing activity name information are filtered out, the character name of the activity can be participated in through the Internet to obtain the object attribute information corresponding to the character name.

[0134] Step S24 (not shown in the figure), obtaining a character list according to the object attribute information corresponding to each object.

[0135] The method of obtaining the character list is the same as that of the above step S13, which will not be repeated here.

[0136] Specifically, after the AR glasses obtain keywords through text recognition, they can search the object attribute information corresponding to each keyword through the Internet, and then obtain the object attribute information corresponding to each object, so as to more accurately determine the object and improve the speed of obtaining the character list.

[0137] Furthermore, when the target object is facing away from the camera of the AR glasses, the AR glasses cannot recognize the facial features of the target object, which may easily lead to the phenomenon of losing the target object during tracking. In order to further improve the accuracy of determining the target object, the embodiment of the present application also includes the step of determining the target object by character features, referring to Figure 2 , including (step S31 to step S32):

[0138] Step S31, obtaining the current character features of the target object, the character features including facial features, clothing features, hairstyle features and body features, and establishing a character model based on the character features.

[0139] Specifically, since the makeup of the target object in the current scene may be different from the makeup in the target multimedia information, after the AR glasses determine the target object, the facial features of the target object are acquired again, so that the AR glasses can accurately determine the target object in the current scene.

[0140] Furthermore, AR glasses can determine the target object through the acquired clothing features, hairstyle features and body features, wherein clothing features include clothing color and style, hairstyle features include hair color, length, hairstyle, body features include height, body contour, body proportion, etc. The target object is determined according to the character features, the comparison library is increased, and if there are many characters in the current scene, the embodiment of the present application is not limited to determining the target object by face feature recognition, so as to improve the accuracy of determining the target object.

[0141] Step S32: Tracking a selected target object in the multimedia information of the current scene based on the character model.

[0142] AR glasses combine facial features, clothing features, hairstyle features and body features to build a character model, track and select the target object based on the character model, and improve the accuracy of identifying the target object.

[0143] Furthermore, in order to accurately track the selected target object, tracking the selected target object in the multimedia information of the current scene based on the character model may specifically include (steps S311 to S314):

[0144] Step S311 (not shown in the figure), extracting a first character feature, where the first character feature is a character feature of all objects in the current scene captured by the camera of the AR glasses;

[0145] Step S312 (not shown in the figure), based on the comparison of clothing features, hairstyle features and body features in the first character features with the character features of the target object, a second character feature that is consistent with the comparison is screened out, and the second character feature is used as the character feature of the candidate target object;

[0146] Among them, the clothing features, hairstyle features and body features of the target object may be more different from those of other objects, and are more eye-catching in the current scene, making it easier for AR glasses to identify and distinguish them. Therefore, AR glasses first perform preliminary identification to determine the target object through the object's clothing, hairstyle and body, thereby increasing the speed of determining the target object.

[0147] Step S313 (not shown in the figure) determines whether the number of the second character features is greater than one. If so, based on the comparison between the candidate facial features of the candidate target object and the facial features of the target object, a target object with consistent comparison is determined.

[0148] Step S314 (not shown in the figure), tracking the selected target object.

[0149] Specifically, if the number of the second character features is equal to one, the AR glasses determine the target object and track the selected target object without further determining the target object through facial features; when the number of the second character features is greater than one, there is an object that is very similar to the target object in clothing, hairstyle and body shape, that is, the AR glasses need to perform further comparison and screening through facial features to determine the target object and then track the selected target object.

[0150] Furthermore, in order to reduce the amount of calculation for determining the target person and improve the accuracy of tracking the selected target object, tracking the selected target object includes (steps Sa to Sd):

[0151] Step Sa (not shown in the figure) is to establish a coordinate system based on the multimedia information of the current scene.

[0152] Specifically, the AR glasses can set the origin of the coordinate system at the center of the multimedia information of the current scene, and divide the multimedia information of the current scene into four quadrants. When an object appears in the multimedia information of the current scene, the AR glasses can obtain the coordinates of each object.

[0153] To simplify calculations, AR glasses can obtain coordinate points representing the position of the object based on a certain part of the object's body. Furthermore, since the object's head shakes less than the body, AR glasses obtain the coordinates of the center point of the object's head.

[0154] Step Sb (not shown in the figure) is to obtain the historical coordinates of the target object in the multimedia information of the current scene at the last moment.

[0155] Specifically, the AR glasses obtain the coordinates of the center point of the head of the target object in the multimedia information of the current scene at the previous moment, which are the historical coordinates of the target object. The historical coordinates can represent the position of the target object at the previous moment.

[0156] Step Sc (not shown in the figure) is to obtain the coordinates of all objects based on the multimedia information of the current scene at the current moment.

[0157] Step Sd (not shown in the figure) is to screen objects whose distances from the historical coordinates of the target object are less than a preset deviation value according to the coordinates of all objects, and determine the screened objects as the target objects.

[0158] Among them, in the multimedia information of the current scene, the position of the target object moves very little in the continuous moments, so the position of the target object at the current moment is less different from the position of the target object at the previous moment. When the AR glasses determine the target object, they compare the historical coordinates of the target object with the coordinates of all objects at the current moment to screen out the target object, reduce the frequency of character recognition, and achieve rapid tracking of the target object, thereby improving the user experience when using the AR glasses.

[0159] In another possible implementation of the present application, in order to facilitate the user to track the moving target object when using the AR glasses, it also includes (steps S41 to S44):

[0160] Step S41 (not shown in the figure) is to obtain the double-foot motion information of the target object by using an image recognition model. The double-foot motion information includes the orientation information of the two toes and the projection distance information between the two toes. The orientation information of the two toes includes the angle between each toe and the vertical plane parallel to the camera.

[0161] Generally speaking, when a target object moves, the moving direction is basically consistent with the direction of its toes, so AR glasses can predict the moving direction of the target object by the direction of its toes.

[0162] First, the AR glasses obtain the position of the toes and heels through the image recognition model, and form a line between the toes and heels of each foot. The AR glasses determine the angle between each toe and the vertical plane parallel to the camera through the inclination of the line of each foot. In addition, the AR glasses form a line between the toes of the two feet, and the measured length of the line between the toes of the two feet is the projection distance information between the two toes.

[0163] Step S42 (not shown in the figure), determining the predicted moving direction information of the target object according to the orientation information of the two toes, the predicted moving direction information of the target object includes the angle between the predicted moving direction of the target object and the vertical plane.

[0164] The AR glasses take the average of the orientation information of the two toes to obtain the orientation information as the predicted moving direction information of the target object.

[0165] Step S43 (not shown in the figure), based on the same predicted moving direction of the target object, the predicted moving speed information of the target object is determined according to the projection distance information between the two toes.

[0166] For example, when the target object moves in a certain direction, the greater the actual distance between the two toes, the faster the target object moves. Then, the greater the corresponding projection distance between the two toes in the multimedia information of the current scene. It can be deduced that: under the same predicted moving direction, the greater the projection distance information between the two toes, the faster the target object moves. Therefore, AR glasses determine the speed of the predicted moving speed information of the target object based on the size of the projection distance information.

[0167] Step S44 (not shown in the figure) displays identification information, where the identification information is used to identify the predicted moving direction and the predicted moving speed of the target object.

[0168] Among them, the identification information can be an arrow, and the direction of the arrow indicates the predicted moving direction of the target object. The AR glasses can use the arrow to indicate the predicted moving speed of the target object through methods such as the thickness of the arrow, the flashing frequency, and the depth of color, so that users can more intuitively know the predicted moving trajectory of the target object and move the field of view of the AR glasses according to the identification information.

[0169] In another possible implementation of the present application, in order to improve the completeness of the target object's personal information and improve the accuracy of determining the target object, the present application also includes the step of obtaining the target object's new information through the Internet, which may specifically include (steps S51 to S52):

[0170] Step S51 (not shown in the figure) is to obtain multimedia information of the same scene located near the geographical location of the AR glasses through the Internet. The multimedia information of the same scene is multimedia information related to the current scene uploaded to the Internet by nearby users.

[0171] Specifically, the AR glasses obtain the geographical location of the AR glasses through their own positioning module, and then obtain the multimedia information uploaded by the user from the current geographical location through the Internet, and then filter out the multimedia information related to the current scene.

[0172] Step S52 (not shown in the figure) obtains character features in the multimedia information of the same scene, selects character features with the highest similarity to the target character features as new character features, and saves the new character features of the target object. The target character features are the character features of the target object.

[0173] Among them, in the same scene, there will be slight differences in the character features obtained from different perspectives for the target object. Therefore, when the AR glasses obtain the newly added character features, the AR glasses can determine the target object more quickly and accurately through the newly added character features and the target character features obtained by the AR glasses.

[0174] In order to better implement the above method, the present application provides a person recognition device based on AR glasses, referring to Figure 3 , the person recognition device 200 based on AR glasses includes:

[0175] A first acquisition module 201 is used to acquire target multimedia information, where the target multimedia information includes a target object;

[0176] A second acquisition module 202, used to acquire facial features of a target object based on target multimedia information;

[0177] A third acquisition module 203, used to determine a target object in the current scene based on facial features;

[0178] The display module 204 is used to control the display of the target object in a preset manner.

[0179] In another possible implementation, the target multimedia information includes at least two objects, wherein the second acquisition module 202 includes:

[0180] A character list generating unit, used for acquiring a character list based on target multimedia information, and displaying the character list in combination with the current scene;

[0181] The target object acquisition unit is used to acquire the target object selected by the user based on the character list and acquire the facial features of the target object.

[0182] In another possible implementation, when the character list generating unit generates the character list according to the face information of the object, it is specifically used to:

[0183] Obtaining facial features corresponding to each object in the target multimedia information;

[0184] Based on the facial features corresponding to each object, determine the object attribute information corresponding to each object, the object attribute information includes: name, gender, age and occupation;

[0185] Generate a character list according to the object attribute information corresponding to each object.

[0186] In another possible implementation, when the character list generating unit generates the character list according to the text information in the target multimedia information, it is specifically used to:

[0187] Identify text information in target multimedia information;

[0188] Filter out keywords from the text information, where the keywords include person name information and / or activity name information;

[0189] Obtaining object attribute information corresponding to each keyword to obtain object attribute information corresponding to each object;

[0190] Obtain a character list based on the object attribute information corresponding to each object.

[0191] Furthermore, the device also includes:

[0192] The character feature acquisition module is used to acquire the current character features of the target object, including facial features, clothing features, hairstyle features and body features, and to establish a character model based on the character features;

[0193] The tracking and selection module is used to track and select the target object in the multimedia information of the current scene based on the character model.

[0194] In another possible implementation, the selected module is tracked, specifically for:

[0195] Extracting a first character feature, where the first character feature is a character feature of all objects in the current scene captured by a camera of the AR glasses;

[0196] Based on the comparison between the clothing features, hairstyle features and body features in the first character features and the character features of the target object, a second character feature that is consistent with the comparison is screened out, and the second character feature is the character feature of the candidate target object;

[0197] Determine whether the number of the second character features is greater than one, and if so, determine the target object with consistent comparison based on the comparison of the candidate facial features of the candidate target object and the facial features of the target object;

[0198] Track the selected target object.

[0199] In another possible implementation, the tracking selected module is specifically used in the step of tracking the selected target object:

[0200] Establishing a coordinate system based on multimedia information of the current scene;

[0201] Obtain the historical coordinates of the target object in the multimedia information of the current scene at the previous moment;

[0202] Obtaining coordinates of all objects from multimedia information of the current scene at the current moment;

[0203] Objects whose distances from historical coordinates of the target object are less than a preset deviation value are screened according to the coordinates of all objects, and the screened objects are determined to be the target objects.

[0204] In another possible implementation, the device further includes:

[0205] A two-foot motion information acquisition module is used to acquire the two-foot motion information of the target object by using an image recognition model, wherein the two-foot motion information includes the orientation information of the two toes and the projection distance information between the two toes, and the orientation information of the two toes includes the angle between each toe and the vertical plane parallel to the camera;

[0206] A predicted moving direction information determination module, which determines the predicted moving direction information of the target object according to the orientation information of the two toes, wherein the predicted moving direction information of the target object includes an angle between the predicted moving direction of the target object and a vertical plane;

[0207] A predicted moving speed determination module, used to determine the predicted moving speed information of the target object according to the projection distance information between the two toes based on the predicted moving direction of the same target object;

[0208] The identification information display module is used to display identification information, where the identification information is used to identify the predicted moving direction and the predicted moving speed of the target object.

[0209] In another possible implementation, the device further includes:

[0210] The multimedia information acquisition module of the same scene is used to acquire multimedia information of the same scene located near the geographical location of the AR glasses through the Internet. The multimedia information of the same scene is multimedia information related to the current scene uploaded to the Internet by nearby users;

[0211] The newly added character feature acquisition module is used to obtain character features in multimedia information of the same scene, select character features with the highest similarity to target character features as newly added character features, and save the newly added character features of the target object. The target character features are the character features of the target object.

[0212] The various variations and specific examples of the methods in the aforementioned embodiments are also applicable to the character recognition device based on AR glasses in this embodiment. Through the aforementioned detailed description of the character recognition method based on AR glasses, those skilled in the art can clearly understand the implementation method of the character recognition system based on AR glasses in this embodiment. Therefore, for the sake of brevity of the specification, it will not be described in detail here.

[0213] In order to better implement the above method, the present application embodiment provides an AR glasses, referring to Figure 4 , AR glasses 300 include: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a communication bus 302. Optionally, the AR glasses 300 may also include a transceiver 304 and a display 305, and the display 305 is connected to the processor 301 through the bus 302. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the AR glasses 300 does not constitute a limitation on the embodiments of the present application.

[0214] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0215] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc.

[0216] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0217] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.

[0218] Figure 4 The AR glasses shown are merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0219] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned method for character recognition based on AR glasses is implemented.

[0220] In this embodiment, the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device. The computer-readable storage medium may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. Specifically, the computer-readable storage medium may be a portable computer disk, a hard disk, a USB flash drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a podium random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, an optical disk, a magnetic disk, a mechanical encoding device, and any combination thereof.

[0221] The computer program in this embodiment includes program codes for executing all the aforementioned methods, and the program codes may include instructions corresponding to the steps of the methods provided in the above embodiments. The computer program can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network (such as the Internet, a local area network, a wide area network, and / or a wireless network). The computer program can be executed completely on the user's computer or as an independent software package.

[0222] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

[0223] In addition, it is to be understood that relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprises," "comprising," or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device.

Claims

1. A person recognition method based on AR glasses, characterized in that: include: Acquiring target multimedia information, wherein the target multimedia information includes a target object; Acquire facial features of the target object based on the target multimedia information; Determine the target object in the current scene based on the facial features; Controlling the display of the target object in a preset manner; Also includes: Acquiring the target object's foot motion information by using an image recognition model, wherein the foot motion information includes the orientation information of the two toes and the projection distance information between the two toes, and the orientation information of the two toes includes the angle between each toe and a vertical plane parallel to the camera; Determining predicted moving direction information of the target object according to the orientation information of the two toes, wherein the predicted moving direction information of the target object includes an angle between the predicted moving direction of the target object and the vertical plane; Based on the same predicted moving direction of the target object, determining the predicted moving speed information of the target object according to the projection distance information between the two toes; Display identification information, where the identification information is used to identify the predicted moving direction of the target object and the predicted moving speed of the target object.

2. The method according to claim 1, characterized in that ,If the target multimedia information contains at least two objects, then, Acquiring facial features of the target object based on the target multimedia information includes: Acquire a character list based on the target multimedia information, and display the character list in combination with the current scene; The target object selected by the user based on the character list is obtained, and the facial features of the target object are obtained.

3. The method according to claim 2, characterized in that , the acquiring of a character list based on the target multimedia information comprises: Obtaining facial features corresponding to each object in the target multimedia information; Determine object attribute information corresponding to each object based on the facial features corresponding to each object, wherein the object attribute information includes: name, gender, age and occupation; The character list is generated according to the object attribute information respectively corresponding to each of the objects.

4. The method according to claim 1, characterized in that: Also includes: Acquire the current character features of the target object, the character features including the facial features, clothing features, hairstyle features and body features, and establish a character model based on the character features; The target object is tracked and selected in the multimedia information of the current scene based on the character model.

5. The method according to claim 4, characterized in that The tracking and selecting the target object in the multimedia information of the current scene based on the character model includes: Extracting a first character feature, where the first character feature is a character feature of all objects in a current scene captured by a camera of the AR glasses; Based on the comparison between the clothing features, hairstyle features and body features in the first character features and the character features of the target object, a second character feature that is consistent with the comparison is screened out, and the second character feature is the character feature of the candidate target object; Determine whether the number of the second character features is greater than one, and if so, determine a target object with consistent comparison based on the comparison of the candidate facial features of the candidate target object and the facial features of the target object; The target object is selected by tracking.

6. The method according to claim 5, characterized in that The tracking and selecting the target object includes: Establishing a coordinate system based on the multimedia information of the current scene; Obtaining historical coordinates of a target object in multimedia information of the current scene at a previous moment; Acquire coordinates of all objects from multimedia information of the current scene at the current moment; Objects whose distances from the historical coordinates of the target object are less than a preset deviation value are screened according to the coordinates of all objects, and the screened objects are determined to be the target objects.

7. A person recognition device based on AR glasses, characterized in that: include: A first acquisition module, used to acquire target multimedia information, wherein the target multimedia information includes a target object; A second acquisition module, used for acquiring facial features of the target object based on the target multimedia information; A third acquisition module, used to determine the target object in the current scene based on the facial features; A display module, used for controlling the display of the target object in a preset manner; A two-foot motion information acquisition module is used to acquire the two-foot motion information of the target object by using an image recognition model, wherein the two-foot motion information includes the orientation information of the two toes and the projection distance information between the two toes, and the orientation information of the two toes includes the angle between each toe and the vertical plane parallel to the camera; A predicted moving direction information determination module, which determines the predicted moving direction information of the target object according to the orientation information of the two toes, wherein the predicted moving direction information of the target object includes an angle between the predicted moving direction of the target object and a vertical plane; A predicted moving speed determination module, used to determine the predicted moving speed information of the target object according to the projection distance information between the two toes based on the predicted moving direction of the same target object; The identification information display module is used to display identification information, where the identification information is used to identify the predicted moving direction and the predicted moving speed of the target object.

8. An AR glasses, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for character recognition based on AR glasses as described in any one of claims 1 to 6 is implemented.

Images