A user trajectory information retrieval method, device, equipment and storage medium

Abstract

The application provides a user trajectory information retrieval method, device and equipment and a storage medium. The method comprises the following steps: extracting a user feature from a given user image to obtain a first user feature; determining a user identity of the user in the user image by matching the first user feature with a user feature in a user feature library, taking the user identity as a first user identity, the user feature library comprising user features corresponding to a plurality of user identities, the user feature library comprising a standard feature library and a dynamic feature library, each user feature in the dynamic feature library being extracted from a user image collected from a real environment and matching a user feature in the standard feature library; and retrieving trajectory point information of the user in the user image from a user trajectory library based on the first user identity to obtain a target retrieval result, wherein the user trajectory library comprises trajectory point information corresponding to a plurality of user identities. The user trajectory information retrieval method provided by the application has a high trajectory recall rate.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for retrieving user trajectory information. Background Technology

[0002] User trajectory information retrieval refers to retrieving the trajectory information of a user from a user trajectory database, given a user image.

[0003] Current user trajectory information retrieval schemes involve extracting user features from a given user image and then comparing these features with the user features contained in each trajectory point information in a user trajectory database to obtain the user's trajectory information in the given user image. The user trajectory database includes several trajectory point information sets, each containing a captured user image and user features extracted from that image.

[0004] While the above scheme can retrieve user trajectory information, when the resolution of the collected user images is too low, the feature information extracted from the low-resolution user images is limited, which leads to low feature matching accuracy and consequently low recall rate for user trajectory information. Summary of the Invention

[0005] In view of this, the present invention provides a user trajectory information retrieval method, apparatus, device, and storage medium to solve the problem that existing user trajectory information retrieval methods have low recall rates for user trajectory information when the resolution of the acquired user images is too low. The technical solution is as follows:

[0006] Firstly, a user trajectory information retrieval method is provided, including:

[0007] Extract user features from a given user image to obtain the first user feature;

[0008] By matching the first user feature with user features in the user feature library, the user identifier of the user in the user image is determined as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matched with a user feature in the standard feature library.

[0009] Based on the first user identifier, the trajectory point information of the user in the user image is retrieved from the user trajectory database to obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

[0010] Optionally, the first user feature includes a first facial feature and / or a first body feature;

[0011] The standard feature library includes a standard face feature library and a standard body feature library, and the dynamic feature library includes a dynamic face feature library and a dynamic body feature library;

[0012] The user feature library also includes an extended feature library, which includes an extended face feature library and an extended body feature library;

[0013] The facial features in the extended facial feature library are facial features associated with the body features in the dynamic body feature library. The body features in the extended body feature library are body features associated with the facial features in the dynamic facial feature library. The associated body features and facial features come from the same user image.

[0014] Optionally, the dynamic feature library and the extended feature library are dynamically updated, and the process of dynamically updating the dynamic feature library and the extended feature library includes:

[0015] For each acquired image, user features are extracted from the acquired image to obtain a second user feature;

[0016] In the case where the second user feature includes the second facial feature:

[0017] Determine whether a face feature matching the second face feature exists in the standard face feature library; if so, record the second face feature in the dynamic face feature library; if the second user feature also includes a second body feature, record the second body feature in the extended body feature library, wherein the user identifiers corresponding to the second face feature recorded in the dynamic face feature library and the second body feature recorded in the extended body feature library are both user identifiers corresponding to the face feature matching the second face feature in the standard face feature library;

[0018] In the case where the second user feature includes the second body feature:

[0019] Determine whether a body feature matching the second body feature exists in the standard body feature library; if so, record the second body feature in the dynamic body feature library; if the second user feature also includes a second face feature, record the second face feature in the extended face feature library; wherein, the user identifiers corresponding to the second body feature recorded in the dynamic body feature library and the second face feature recorded in the extended face feature library are the user identifiers corresponding to the body feature matching the second body feature in the standard body feature library.

[0020] Optionally, the process of dynamically updating the dynamic feature library and the extended feature library further includes:

[0021] For any face feature library in the dynamic face feature library and the extended face feature library:

[0022] If the second facial feature is entered into the facial feature database, it is determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature is greater than the preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature is deleted according to the preset deletion rules.

[0023] For any one of the dynamic shape feature libraries and the extended shape feature library:

[0024] If the second shape feature is entered into the shape feature library, it is determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature is deleted according to the preset deletion rules.

[0025] Optionally, deleting the facial feature corresponding to the user identifier corresponding to the second facial feature according to a preset deletion rule includes:

[0026] Obtain the scores of each face feature corresponding to the user identifier corresponding to the second face feature, and delete the face feature with the lowest score. The score of a face feature is determined based on the entry time of the face feature into the database and the similarity between the face feature and the face feature corresponding to the user identifier in the standard face feature database.

[0027] The step of deleting the shape feature corresponding to the user identifier corresponding to the second shape feature according to the preset deletion rules includes:

[0028] Obtain the scores of each shape feature corresponding to the user identifier corresponding to the second shape feature, and delete the shape feature with the lowest score. The score of a shape feature is determined based on the entry time of the shape feature into the database and the similarity between the shape feature and the shape feature corresponding to the user identifier in the standard shape feature database.

[0029] Optionally, the priority of each feature library included in the user feature library is as follows: the standard face feature library, the dynamic face feature library, the standard body feature library, the dynamic body feature library, the extended face feature library, and the extended body feature library;

[0030] The step of determining the user identifier of the user in the user image by matching the first user feature with user features in the user feature library includes:

[0031] According to the priority, user features that match the first user feature are determined from the user feature database;

[0032] The user identifier corresponding to the user feature that matches the first user feature in the user feature library is determined as the user identifier of the user in the user image.

[0033] Optionally, the step of retrieving the user's trajectory point information from the user trajectory database based on the first user identifier to obtain the target retrieval result includes:

[0034] The system retrieves several trajectory point information corresponding to the first user identifier from the user trajectory database to obtain the first search result, which is then used as the target search result.

[0035] Optionally, the trajectory point information corresponding to a user identifier includes a user image collected from the real environment and facial features and / or body features extracted from the user image;

[0036] The step of retrieving user trajectory point information from the user trajectory database based on the first user identifier to obtain the target retrieval result further includes:

[0037] Representative shape features are obtained from several trajectory point information corresponding to the first user identifier, wherein the representative shape features include one or more of the following features: front shape features, side shape features, and back shape features;

[0038] The second search result is obtained by retrieving trajectory point information from the user trajectory database that matches the shape features of the representative shape features;

[0039] The trajectory point information contained in the first search result and the trajectory point information contained in the second search result are merged and deduplicated, and the processed search result is used as the target search result.

[0040] Optionally, obtaining representative shape features from the first search result includes:

[0041] Filter trajectory point information containing shape features from the first search results;

[0042] Based on the deflection angle of the shape in the user image contained in the selected trajectory point information, frontal shape features, and / or side shape features, and / or back shape features are obtained from the shape features contained in the selected trajectory point information to obtain representative shape features.

[0043] Optionally, the step of obtaining frontal shape features, and / or side shape features, and / or back shape features from the shape features contained in the filtered trajectory point information based on the deflection angle of the shape in the user image, includes:

[0044] From the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the front shape, as candidate front shape features. Determine the score of each candidate front shape feature based on the deflection angle of the shape corresponding to each candidate front shape feature. Based on the score of each candidate front shape feature, determine the front shape feature from the obtained candidate front shape features.

[0045] And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the side shape as candidate side shape features, determine the score of each candidate side shape feature according to the deflection angle of the shape corresponding to each candidate side shape feature, and determine the side shape features from the obtained candidate side shape features according to the score of each candidate side shape feature.

[0046] And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the back shape as candidate back shape features, determine the score of each candidate back shape feature based on the deflection angle of the shape corresponding to each candidate back shape feature, and determine the back shape feature from the obtained candidate back shape features based on the score of each candidate back shape feature.

[0047] Optionally, determining the frontal shape feature from the acquired candidate frontal shape features based on the score of each candidate frontal shape feature includes:

[0048] Candidate frontal features whose scores are greater than a preset score threshold among the candidate frontal features containing facial features in the trajectory point information are identified as frontal features.

[0049] The step of determining the side profile features from the acquired candidate side profile features based on the score of each candidate side profile feature includes:

[0050] Candidate side profile features whose scores are greater than a preset score threshold among the candidate side profile features that do not contain facial features in the trajectory point information are identified as side profile features.

[0051] The step of determining the back shape features from the acquired candidate back shape features based on the score of each candidate back shape feature includes:

[0052] Candidate back-face features whose scores are greater than a preset score threshold among the candidate back-face features that do not contain facial features in the trajectory point information are identified as back-face features.

[0053] Optionally, obtaining the second search result by retrieving trajectory point information from the user trajectory database that matches the representative shape features includes:

[0054] For each representative shape feature: calculate the similarity between the representative shape feature and each shape feature contained in the user trajectory database; determine the trajectory point information of the shape feature in the user trajectory database whose similarity to the representative shape feature is greater than a preset similarity threshold as the target trajectory point information;

[0055] All obtained target trajectory point information are identified as the second search result.

[0056] Secondly, a user trajectory information retrieval device is provided, including: a user feature extraction module, a user identifier determination module, and a user trajectory retrieval module;

[0057] The user feature extraction module is used to extract user features from a given user image to obtain the first user feature;

[0058] The user identifier determination module is used to determine the user identifier of the user in the user image by matching the first user feature with the user features in the user feature library, and use the first user identifier as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matching a user feature in the standard feature library.

[0059] The user trajectory retrieval module is used to retrieve the trajectory point information of the user in the user image from the user trajectory database based on the first user identifier, and obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

[0060] Thirdly, a processing device is provided, including: a memory and a processor;

[0061] The memory is used to store programs;

[0062] The processor is used to execute the program to implement each step of the user trajectory information retrieval method described above.

[0063] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the various steps of the user trajectory information retrieval method described in any of the preceding claims.

[0064] The user trajectory information retrieval method provided by this invention first extracts user features from a given user image to obtain a first user feature. Then, it determines the user identifier of the user in the given user image by matching the first user feature with user features in a user feature library (which includes user features corresponding to several user identifiers, comprising a standard feature library and a dynamic feature library, where each user feature in the dynamic feature library is extracted from user images collected from real-world environments and matches a user feature in the standard feature library). This first user identifier is then used as the first user identifier. Finally, based on the first user identifier, the trajectory information of the user in the user image is retrieved from the user trajectory library. This user trajectory information retrieval method does not obtain user trajectory information by matching user features extracted from a given user image with user features extracted from a low-resolution acquired image. Instead, it obtains user trajectory information by matching user features extracted from a given user image with user features in a user feature library. This strategy can improve the recall rate of user trajectory information when the acquired image resolution is low. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of the hardware architecture involved in the present invention;

[0067] Figure 2 This is a flowchart illustrating the user trajectory information retrieval method provided in an embodiment of the present invention;

[0068] Figure 3 A schematic diagram illustrating the process of dynamically updating the dynamic feature library and the extended feature library as provided in the embodiments of the present invention;

[0069] Figure 4 This is a schematic diagram of the process of retrieving user trajectory point information from the user image in the user trajectory database based on the first user identifier to obtain the target retrieval result, provided by an embodiment of the present invention.

[0070] Figure 5 This is a schematic diagram of the user trajectory information retrieval device provided in an embodiment of the present invention;

[0071] Figure 6 This is a schematic diagram of the processing device provided in an embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Given that existing user trajectory information retrieval methods have low recall rates, the inventors of this case conducted research and, through continuous research, finally proposed a user trajectory information retrieval method with better performance.

[0074] Before introducing the solution provided by this invention, the hardware architecture involved in this invention will be described first.

[0075] In one possible implementation, such as Figure 1 As shown, the hardware architecture involved in this invention may include: electronic device 101 and server 102.

[0076] For example, electronic device 101 can be any electronic product that can interact with a user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as mobile phone, laptop, tablet computer, PDA, personal computer, wearable device, smart TV, PAD, etc.

[0077] It should be noted that, Figure 1This is just one example; there can be many types of electronic devices, not limited to... Figure 1 The laptop in the middle.

[0078] For example, server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. Server 102 may include processors, memory, and network interfaces, etc.

[0079] For example, electronic device 101 can establish a connection and communicate with server 102 through a wireless communication network; for example, electronic device 101 can establish a connection and communicate with server 102 through a wired network.

[0080] Electronic device 101 acquires a given user image and sends the given user image to server 102. Server 102 retrieves the user's trajectory information in the given user image and sends the retrieval result to electronic device 101.

[0081] In another possible implementation, the hardware architecture involved in this invention may include an electronic device. The electronic device is one with strong data processing capabilities.

[0082] Electronic devices can acquire a given user image and retrieve the user's trajectory information within that image.

[0083] Those skilled in the art should understand that the above-described electronic devices and servers are merely examples, and other existing or future electronic devices or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0084] The user trajectory information retrieval method provided by the present invention will be described in the following embodiments.

[0085] Please see Figure 2 The diagram illustrates a flowchart of a user trajectory information retrieval method provided in an embodiment of the present invention. The method may include:

[0086] Step S201: Extract user features from the given user image to obtain the first user features.

[0087] The given user image is used to retrieve user trajectory information. The user features extracted from the given user image may include only facial features, only body features, or both facial and body features. It should be noted that the specific user features extracted from the given user image depend on the features that can be extracted from the given user image.

[0088] To facilitate differentiation from the user features mentioned later, this embodiment refers to the user features extracted from a given user image as the first user features, the facial features extracted from a given user image as the first facial features, and the body features extracted from a given user image as the first body features.

[0089] Step S202: By matching the first user feature with the user features in the user feature library, the user identifier of the user in the given user image is determined as the first user identifier.

[0090] The user feature database includes user features corresponding to several user identifiers.

[0091] The process of determining the user identifier of a user in a given user image by matching a first user feature with user features in a user feature library includes: determining the user identifier that matches the first user feature from the user feature library, and determining the user identifier in the user feature library that matches the first user feature as the user identifier of the user in the given user image. It should be noted that the user identifier in the user feature library that matches the first user feature is a user feature in the user feature library whose similarity to the first user feature is greater than a preset similarity threshold.

[0092] In one possible implementation, the user feature library may include a standard feature library, where user features are extracted from user-registered images. Since user-registered images are of high quality (usually manually reviewed), the features extracted from them are highly representative. Considering that user-registered images are typically captured against a static background, while user images in the user trajectory library are captured in real-world environments, to improve retrieval performance, the user feature library may also include a dynamic feature library. The dynamic feature library contains user features extracted from user images captured in real-world environments that match a user feature in the standard feature library. Introducing a dynamic feature library can reduce the impact of the environment on retrieval results.

[0093] The standard feature library may include a standard face feature library and a standard body feature library, while the dynamic feature library may include a dynamic face feature library and a dynamic body feature library. Both the standard face feature library and the dynamic face feature library contain face features corresponding to several user identifiers. Both the standard body feature library and the dynamic body feature library contain body features corresponding to several user identifiers. Each face feature in the dynamic face feature library is a face feature that matches a face feature in the standard face feature library, and each body feature in the dynamic body feature library is a body feature that matches a body feature in the standard body feature library.

[0094] In another possible implementation, the user feature library can include not only a standard feature library and a dynamic feature library, but also an extended feature library. This extended feature library includes an extended face feature library and an extended body feature library. The face features in the extended face feature library are those associated with the body features in the dynamic body feature library, and the body features in the extended body feature library are those associated with the face features in the dynamic face feature library. It should be noted that the associated body features and face features originate from the same user image. Introducing an extended feature library can further improve retrieval performance.

[0095] Optionally, the priority of the above feature libraries is as follows: standard face feature library, dynamic face feature library, standard body feature library, dynamic body feature library, extended face feature library, and extended body feature library. When determining user features that match the first user feature from the user feature library, the user features that match the first user feature can be determined from the user feature library according to the above priority.

[0096] For example, if the user feature library includes a standard feature library, a dynamic feature library, and an extended feature library, then: if the first user feature only contains a first face feature, a face feature matching the first face feature can be determined first from the standard face feature library. If a face feature matching the first face feature exists in the standard face feature library, the user identifier corresponding to the face feature matching the first face feature in the standard face feature library is determined as the user identifier of the user in the given user image. If no face feature matching the first face feature exists in the standard face feature library, a face feature matching the first face feature is determined from the dynamic face feature library. If a face feature matching the first face feature exists in the dynamic face feature library, the user identifier corresponding to the face feature matching the first face feature in the dynamic face feature library is determined as the user identifier of the user in the given user image. If no face feature matching the first face feature exists in the dynamic face feature library, then a face feature matching the first face feature is further determined from the extended face feature library. The system first determines the facial features that match the first facial features. If the first user features only contain the first body features, the system first determines the body features that match the first body features from the standard body feature library. If the standard body feature library contains a body feature that matches the first body feature, the user identifier corresponding to the body feature that matches the first body feature in the standard body feature library is determined as the user identifier of the user in the given user image. If the standard body feature library does not contain a body feature that matches the first body feature, the system determines the body features that match the first body feature from the dynamic body feature library. If the dynamic body feature library contains a body feature that matches the first body feature, the user identifier corresponding to the body feature that matches the first body feature in the dynamic body feature library is determined as the user identifier of the user in the given user image. If the dynamic body feature library does not contain a body feature that matches the first body feature, the system further determines the body features that match the first body feature from the extended body feature library.If the first user feature includes both a first body feature and a first face feature, the system first determines a face feature matching the first face feature from a standard face feature library. If a face feature matching the first face feature exists in the standard face feature library, the user identifier corresponding to that face feature is determined as the user identifier of the user in the given user image. If no face feature matching the first face feature exists in the standard face feature library, a face feature matching the first face feature is determined from a dynamic face feature library. If a face feature matching the first face feature exists in the dynamic face feature library, the user identifier corresponding to that face feature is determined as the user identifier of the user in the given user image. If no face feature matching the first face feature exists in the dynamic face feature library, a body feature matching the first body feature is determined from a standard body feature library. If a body feature matching the first body feature exists in the standard body feature library, the user identifier corresponding to that face feature is determined as the user identifier of the user in the given user image. The user identifier corresponding to the first shape feature matched is determined as the user identifier of the user in the given user image. If no shape feature matching the first shape feature exists in the standard shape feature library, then a shape feature matching the first shape feature is determined from the dynamic shape feature library. If a shape feature matching the first shape feature exists in the dynamic shape feature library, then the user identifier corresponding to the shape feature matching the first shape feature in the dynamic shape feature library is determined as the user identifier of the user in the given user image. If no shape feature matching the first shape feature exists in the dynamic shape feature library, then a face feature matching the first face feature is determined from the extended face feature library. If a face feature matching the first face feature exists in the extended face feature library, then the user identifier corresponding to the face feature matching the first face feature in the extended face feature library is determined as the user identifier of the user in the given user image. If no face feature matching the first face feature exists in the extended face feature library, then a shape feature matching the first shape feature is determined from the extended shape feature library.

[0097] For any face feature library among the standard face feature library, dynamic face feature library, and extended face feature library, the process of determining the face features that match the first face feature library from that face feature library may include: calculating the similarity between each face feature in the face feature library and the first face feature; and determining the face features in the face feature library whose similarity to the first face feature is greater than a preset face similarity threshold as the face features that match the first face feature. It should be noted that if there are multiple face features in the face feature library whose similarity to the first face feature is greater than the preset face similarity threshold, then the face feature with the highest similarity to the first face feature among these multiple face features can be determined as the face feature that matches the first face feature library.

[0098] Similarly, for any one of the standard shape feature library, dynamic shape feature library, or extended shape feature library, the process of determining the shape features that match the first shape feature library from that shape feature library includes: calculating the similarity between each shape feature in the shape feature library and the first shape feature; and determining the shape features in the shape feature library whose similarity to the first shape feature is greater than a preset shape similarity threshold as the shape features that match the first shape feature library. It should be noted that if there are multiple shape features in the shape feature library whose similarity to the first shape feature is greater than the preset shape similarity threshold, then the shape feature with the highest similarity to the first shape feature among these multiple shape features is determined as the shape feature that matches the first shape feature.

[0099] For example, the first user feature only includes the first face feature. If the face feature that matches the first face feature is a face feature in the standard face feature library, then the user identifier corresponding to the face feature that matches the first face feature in the standard face feature library is determined as the user identifier of the user in the given user image, and is used as the first user identifier.

[0100] Step S203: Based on the first user identifier, retrieve the user's trajectory information from the user trajectory database to obtain the target retrieval result.

[0101] The user trajectory database includes trajectory point information corresponding to several user identifiers. The trajectory point information corresponding to a user identifier is the information of one trajectory point in the user trajectory. Each trajectory point information includes a user image collected from the real environment, user features (facial features and / or body features) extracted from the user image, and other key information (such as the time and location of the user's appearance).

[0102] It should be noted that the user identifier corresponding to a trajectory point in the user trajectory database is determined by matching the user features in that trajectory point with the user features in the user feature database. Furthermore, the user images in each trajectory point in the user trajectory database can be user images captured by different cameras; that is, the trajectory points in the user trajectory database can be cross-camera trajectory point information.

[0103] The user trajectory information retrieval method provided in this embodiment of the invention first extracts user features from a given user image to obtain a first user feature. Then, by matching the first user feature with user features in a user feature database, a user identifier for the user in the given user image is determined and used as the first user identifier. Finally, based on the first user identifier, the trajectory information of the user in the user image is retrieved from the user trajectory database. This user trajectory information retrieval method, unlike the method in this embodiment, does not obtain user trajectory information by matching user features extracted from a given user image with user features extracted from a low-resolution acquired image. Instead, it obtains user trajectory information by matching user features extracted from a given user image with user features in a user feature database. This strategy can improve the recall rate of user trajectory information when the acquired image resolution is low.

[0104] As mentioned in the above embodiments, the user feature library may include not only a standard feature library, but also a dynamic feature library and an extended feature library. The user features in the dynamic feature library and the extended feature library are dynamically updated. The dynamic update process of the dynamic feature library and the extended feature library will be described below.

[0105] Please see Figure 3 This illustrates a flowchart of the dynamic feature library and the dynamic update process of the extended feature library, which may include:

[0106] Step S301: For each acquired image, extract user features from the acquired image to obtain the second user features.

[0107] In this embodiment, after acquiring the captured image, key points can be extracted from the target in the captured image. The extracted key points are compared with pre-obtained representative facial key points and representative body key points to determine whether the captured image contains a face and / or a body. After determining that the captured image contains a face and / or a body, user features are then extracted from the captured image. It should be noted that representative facial key points are key points that can represent a face, obtained through training on a large number of face images. Similarly, representative body key points are key points that can represent a body, obtained through training on a large number of body images.

[0108] The user features extracted from the acquired images may contain only facial features, only body features, or both. In this embodiment, the user features extracted from the acquired images are referred to as second user features; correspondingly, the facial features extracted from the acquired images are referred to as second facial features, and the body features extracted from the acquired images are referred to as second body features.

[0109] As mentioned above, there are three cases for the second user feature: it contains only the second facial feature, it contains only the second body feature, and it contains both the second facial feature and the second body feature. The processing strategies for each of these three cases will be introduced below.

[0110] If the second user feature only contains the second facial feature, perform the following steps:

[0111] Step S302-a1: Determine whether there is a face feature in the standard face feature library that matches the second face feature. If so, proceed to step S301-a2.

[0112] Step S302-a2: Enter the second facial feature into the dynamic facial feature database.

[0113] The user identifier corresponding to the face feature that matches the second face feature in the standard face feature library is determined as the user identifier corresponding to the second face feature.

[0114] Optionally, after the second facial feature is entered into the dynamic facial feature database, it can be determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature in the dynamic facial feature database is greater than the preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature in the dynamic facial feature database is deleted according to the preset deletion rules.

[0115] It should be noted that each facial feature in the dynamic facial feature database corresponds to a unique user identifier, but one user identifier can correspond to multiple facial features.

[0116] For example, the preset quantity is 5, and the user identifier corresponding to the second facial feature is ID3. After the second facial feature is entered into the dynamic facial feature database, the number of facial features corresponding to user identifier ID3 is 6. If it exceeds 5, then 1 facial feature corresponding to user identifier ID3 will be deleted according to the preset deletion rules.

[0117] Optionally, the process of deleting a face feature corresponding to the user identifier corresponding to the second face feature in the dynamic face feature library according to preset deletion rules may include: obtaining the score of each face feature corresponding to the user identifier corresponding to the second face feature in the dynamic face feature library, and deleting the face feature with the lowest score.

[0118] The score of a facial feature in the dynamic facial feature database is determined based on the time the facial feature was added to the database and the similarity between the facial feature and the corresponding user identifier in the standard facial feature database. Optionally, the score W1 of a facial feature in the dynamic facial feature database can be determined as shown in the following formula:

[0119]

[0120] Where T1 represents the time when the facial feature was entered into the dynamic facial feature database. S1 represents the normalized time, and S1 represents the similarity between the face feature and the corresponding user identifier in the standard face feature library. For example, if the user identifier corresponding to the face feature in the dynamic face feature library is ID3, then S1 is the similarity between the face feature and the face feature corresponding to ID3 in the standard face feature library. If there are multiple face features corresponding to ID3 in the standard face feature library, then S1 can be the maximum similarity among the similarities between the face feature and each face feature corresponding to ID3 in the standard face feature library. It should be noted that 0.8 in the above formula (1) is the weight corresponding to the normalized time, and 0.2 is the weight corresponding to the face feature similarity. This embodiment does not limit the values ​​of the two weights to 0.8 and 0.2, and they can also be other values. For example, the weight corresponding to the normalized time is 0.75, and the weight corresponding to the face feature similarity is 0.25.

[0121] It should be noted that the facial features retained in the dynamic facial feature database are those that were added to the database relatively recently and have a relatively high similarity to the facial features corresponding to the user identifier in the standard facial feature database.

[0122] If the second user feature only contains the second shape feature, perform the following steps:

[0123] Step S302-b1: Determine whether there is a shape feature in the standard shape feature library that matches the second shape feature. If so, proceed to step S302-b2.

[0124] Step S302-b2: Enter the second shape feature into the dynamic shape feature library.

[0125] The user identifier corresponding to the shape feature that matches the second shape feature in the standard shape feature library is determined as the user identifier corresponding to the second shape feature.

[0126] Optionally, after the second shape feature is entered into the dynamic shape feature library, it can be determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature in the dynamic shape feature library is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature in the dynamic shape feature library is deleted according to the preset deletion rules.

[0127] It should be noted that each shape feature in the dynamic shape feature library corresponds to a unique user identifier, but one user identifier can correspond to multiple shape features.

[0128] Optionally, the process of deleting a shape feature corresponding to a user identifier corresponding to a second shape feature in the dynamic shape feature library according to preset deletion rules may include: obtaining the scores of each shape feature corresponding to the user identifier corresponding to the second shape feature in the dynamic shape feature library, and deleting the shape feature with the lowest score.

[0129] The score of a shape feature in the dynamic shape feature library is determined based on the entry time of the shape feature and the similarity between the shape feature and the corresponding shape feature of the user identifier in the standard shape feature library. Optionally, the score of a shape feature in the dynamic shape feature library can be determined as shown in the following formula (2):

[0130]

[0131] Where T2 represents the time when the shape feature was entered into the dynamic shape feature database. S1 represents the normalized time, and S2 represents the similarity between the shape feature and the corresponding user identifier in the standard face feature database. It should be noted that 0.8 in the above formula (2) is the weight corresponding to the normalized time, and 0.2 is the weight corresponding to the shape feature similarity S2. This embodiment does not limit the values ​​of the two weights to 0.8 and 0.2, and they can also be other values. For example, the weight corresponding to the normalized time is 0.75, and the weight corresponding to the shape feature similarity is 0.25.

[0132] It should be noted that the shape features retained in the dynamic shape feature library are those that were added to the library relatively recently and have a relatively high similarity to the shape features corresponding to the user identifier in the standard shape feature library.

[0133] If the second user feature includes both the second facial feature and the second body feature, perform the following steps:

[0134] Step S302-c1-a: Determine whether there is a face feature in the standard face feature library that matches the second face feature. If so, proceed to step S302-c2-a.

[0135] Step S302-c1-b: Determine whether there is a shape feature in the standard shape feature library that matches the second shape feature. If so, proceed to step S302-c2-b.

[0136] Step S302-c2-a: Enter the second facial feature into the dynamic facial feature library and enter the second body feature into the extended body feature library.

[0137] If a face feature matching the second face feature exists in the standard face feature library, the second face feature is entered into the dynamic face feature library. At the same time, the second body feature is entered into the extended body feature library. The user identifier corresponding to the face feature matching the second face feature in the standard face feature library is determined as the user identifier corresponding to the second face feature entered into the dynamic face feature library and the user identifier corresponding to the second body feature entered into the extended body feature library.

[0138] Optionally, after the second facial feature is entered into the dynamic facial feature database, it can be determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature in the dynamic facial feature database is greater than the preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature in the dynamic facial feature database is deleted according to the preset deletion rules.

[0139] Optionally, after the second shape feature is entered into the extended shape feature library, it can be determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature in the extended shape feature library is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature in the extended shape feature library is deleted according to the preset deletion rules.

[0140] Optionally, the process of deleting a shape feature corresponding to a user identifier corresponding to a second shape feature in the extended shape feature library according to preset deletion rules may include: obtaining the scores of each shape feature corresponding to the user identifier corresponding to the second shape feature in the extended shape feature library, and deleting the shape feature with the lowest score.

[0141] The score of a shape feature in the extended shape feature library is determined based on the entry time of the shape feature and the similarity between the shape feature and the corresponding shape feature of the user identifier in the standard shape feature library. Optionally, the score W3 of a shape feature in the extended shape feature library can be determined as shown in formula (3):

[0142]

[0143] Where T3 represents the time when the shape feature was entered into the expanded shape feature library. S1 represents the normalized time, and S2 represents the similarity between the shape feature and the corresponding user identifier in the standard shape feature library. It should be noted that the first 0.5 in the above formula (3) is the weight corresponding to the normalized time, and the second 0.5 is the weight corresponding to 1-S3. This embodiment does not limit the value of both weights to 0.5, and they can also be other values. For example, the weight corresponding to the normalized time is 0.55, and the weight corresponding to 1-S3 is 0.45.

[0144] It should be noted that the shape features retained in the extended shape feature library are those that were added to the library relatively recently and have a relatively low similarity to the shape features corresponding to the user identifier in the standard shape feature library.

[0145] Step S302-c2-b: Enter the second body feature into the dynamic body feature library and enter the second face feature into the extended face feature library.

[0146] If it is determined that there is a body feature in the standard body feature library that matches the second body feature, the second body feature can be entered into the dynamic body feature library, and the second face feature can be entered into the extended face feature library. The user identifier corresponding to the body feature in the standard body feature library that matches the second body feature is determined as the user identifier corresponding to the second body feature entered into the dynamic body feature library and the user identifier corresponding to the second face feature entered into the extended face feature library.

[0147] Optionally, after the second shape feature is entered into the dynamic shape feature library, it can be determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature in the dynamic shape feature library is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature in the dynamic shape feature library is deleted according to the preset deletion rules.

[0148] Optionally, after the second facial feature is entered into the extended facial feature library, it can be determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature in the extended facial feature library is greater than a preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature in the extended facial feature library is deleted according to the preset deletion rules.

[0149] Optionally, the process of deleting a face feature corresponding to the user identifier corresponding to the second face feature in the extended face feature library according to preset deletion rules may include: obtaining the score of each face feature corresponding to the user identifier corresponding to the second face feature in the extended face feature library, and deleting the face feature with the lowest score.

[0150] The score of a face feature in the extended face feature database is determined based on the entry time of the face feature and the similarity between the face feature and the corresponding face feature of the user identifier in the standard face feature database. Optionally, the score W4 of a face feature in the extended face feature database can be determined as shown in formula (4):

[0151]

[0152] Where T4 represents the time it took for this facial feature to be entered into the expanded facial feature database. S4 represents the normalized time, and S4 represents the similarity between the face feature and the corresponding face feature of the user identifier in the standard face feature database. It should be noted that the first 0.5 in the above formula (4) is the weight corresponding to the normalized time, and the second 0.5 is the weight corresponding to 1-S4. This embodiment does not limit the value of both weights to 0.5, and they can also be other values. For example, the weight corresponding to the normalized time is 0.55, and the weight corresponding to 1-S4 is 0.45.

[0153] It should be noted that the facial features retained in the extended facial feature database are those that were added to the database relatively recently and have a relatively low similarity to the facial features corresponding to the user identifier in the standard facial feature database.

[0154] In another embodiment of the present invention, the specific implementation process of "step S203: based on the first user identifier, retrieve the user's trajectory information in the user image from the user trajectory database to obtain the target retrieval result" in the above embodiment will be described.

[0155] There are multiple ways to implement step S203. In one possible implementation, the trajectory point information corresponding to the first user identifier can be obtained from the user trajectory database as the target retrieval result.

[0156] As mentioned in the above embodiments, the user trajectory database includes trajectory point information corresponding to several user identifiers. Assuming that the first user identifier is ID1, several trajectory point information corresponding to ID1 are obtained from the user trajectory database as the target retrieval result.

[0157] To improve trajectory recall, this embodiment provides another implementation of step S203. Please refer to [link to relevant documentation]. Figure 3 The flowchart illustrating this implementation method is shown and may include:

[0158] Step S401: Obtain information on several trajectory points corresponding to the first user identifier from the user trajectory database to obtain the first search result.

[0159] Step S402: Obtain representative shape features from the first search results.

[0160] Representative physical features may include one or more of the following features: frontal physical features, side physical features, and back physical features. It should be noted that frontal physical features refer to features of a frontal shape, side physical features refer to features of a side shape, and back physical features refer to features of a back shape.

[0161] Specifically, the implementation process of step S402 may include:

[0162] Step S4021: Filter the trajectory point information containing shape features from the several trajectory point information corresponding to the first user identifier.

[0163] Each trajectory point information corresponding to the first user identifier includes a user image and user features extracted from the user image (which may only include facial features, or only include body features, or may include both facial features and body features). The purpose of step S4021 is to filter trajectory point information containing body features from the several trajectory point information corresponding to the first user identifier.

[0164] Step S4022: Based on the deflection angle of the shape in the user image contained in the selected trajectory point information, obtain the frontal shape features, and / or side shape features, and / or back shape features from the shape features contained in the selected trajectory point information to obtain representative shape features.

[0165] It should be noted that the deflection angle of a shape refers to the deflection angle of the shape relative to a reference point (such as the front view).

[0166] The process of obtaining frontal body features from the body features contained in the selected trajectory point information based on the deflection angle of the body in the user image can include:

[0167] Step S4022-a1: Obtain the shape features corresponding to the shapes whose deflection angles are within the deflection angle range corresponding to the frontal shape from the selected trajectory point information, and use them as candidate frontal shape features.

[0168] For each selected trajectory point, it can be determined whether the deflection angle of the shape in the user image contained in the trajectory point is within the deflection angle range corresponding to the frontal shape. If so, the shape feature contained in the trajectory point is used as a candidate frontal shape feature.

[0169] For example, the deflection angle range corresponding to the frontal shape is [0, 45°] and [0, -45°]. The deflection angle of the shape in the user image contained in the selected trajectory point information is 30°. Since it is located within [0, 45°], the shape feature contained in the trajectory point information is used as the candidate frontal shape feature.

[0170] Step S4022-a2: Determine the score of each candidate frontal shape feature based on the deflection angle of the shape corresponding to each candidate frontal shape feature.

[0171] Optionally, the score Score1 of a candidate frontal shape feature can be determined according to the following formula:

[0172] Score1=(90°-|α|) / 90° (5)

[0173] Where α represents the deflection angle of the shape corresponding to the candidate frontal shape feature.

[0174] Step S4022-a3: Determine the frontal shape feature from each candidate frontal shape feature based on the score of each candidate frontal shape feature.

[0175] In one possible implementation, candidate frontal features with scores greater than a preset score threshold can be identified as frontal features. It should be noted that if there are multiple candidate frontal features with scores greater than the preset score threshold, the candidate frontal feature with the highest score among these multiple candidate frontal features will be identified as the frontal feature.

[0176] In another possible implementation, candidate frontal features whose scores are greater than a preset score threshold among the candidate frontal features containing facial features in the trajectory point information can be identified as frontal features. For example, candidate frontal features include F... 1a F 2a F 3a and F 4a , of which F 1a and F 2a The trajectory point information contains facial features, F 3a and F 4a The trajectory point information does not contain facial features, F 1a If the score is greater than a preset score threshold (e.g., 0.8), F 2a If the score is less than the preset score threshold, then F will be... 1a It is determined to be a frontal facial feature. It should be noted that if there are multiple candidate frontal facial features with scores greater than a preset score threshold among the candidate frontal facial features containing facial features in the trajectory point information, then the candidate frontal facial feature with the highest score among these multiple candidate frontal facial features will be determined as the frontal facial feature.

[0177] The process of obtaining side profile features from the shape features contained in the selected trajectory point information based on the deflection angle of the shape in the user image can include:

[0178] Step S4022-b1: Obtain the shape features corresponding to the shapes whose deflection angles are within the deflection angle range corresponding to the side shape from the selected trajectory point information, and use them as candidate side shape features.

[0179] For each selected trajectory point, it can be determined whether the deflection angle of the shape in the user image contained in the trajectory point is within the deflection angle range corresponding to the side shape. If so, the shape features contained in the trajectory point are used as candidate side shape features.

[0180] For example, the deflection angle range corresponding to the side shape is [45, 90°] and [-45, -90°]. The deflection angle of the shape in the user image contained in a trajectory point information is 60°. Since it is located within [45, 90°], the shape feature contained in the trajectory point information is used as a candidate side shape feature.

[0181] Step S4022-b2: Determine the score of each candidate side profile feature based on the deflection angle of the corresponding shape.

[0182] Optionally, the score Score2 of a candidate side profile feature can be determined according to the following formula:

[0183] Score2=|β| / 90° (6)

[0184] Where β represents the deflection angle of the shape corresponding to the candidate side profile feature.

[0185] Step S4022-b3: Determine the side profile features from each candidate side profile feature based on the score of each candidate side profile feature.

[0186] In one possible implementation, candidate side profile features with scores greater than a preset score threshold can be identified as side profile features. It should be noted that if there are multiple candidate side profile features with scores greater than the preset score threshold, the candidate side profile feature with the highest score among these multiple candidate side profile features will be identified as the side profile feature.

[0187] In another possible implementation, candidate side profile features whose scores are greater than a preset score threshold among candidate side profile features that do not contain facial features in the trajectory point information can be identified as side profile features. For example, candidate side profile features include F... 1b F 2b F 3b and F 4b , of which F 1b and F 3b The trajectory point information contains facial features, F 2b and F 4b The trajectory point information does not contain facial features, F 2b If the score is less than a preset score threshold (e.g., 0.8), F 4b If the score is greater than the preset score threshold, then F will be... 4bIt is determined to be a side profile feature. It should be noted that if there are multiple candidate side profile features with scores greater than a preset score threshold among the candidate side profile features that do not contain facial features in the trajectory point information, then the candidate side profile feature with the highest score among these multiple candidate side profile features will be determined as the side profile feature.

[0188] The process of obtaining the rear-face shape features from the shape features contained in the selected trajectory point information based on the deflection angle of the shape in the user image can include:

[0189] Step S4022-c1: Obtain the shape features corresponding to the shapes whose deflection angles are within the deflection angle range corresponding to the back shape from the selected trajectory point information, and use them as candidate back shape features.

[0190] For each selected trajectory point, it can be determined whether the deflection angle of the shape in the user image contained in the trajectory point is within the deflection angle range corresponding to the back shape. If so, the shape feature contained in the trajectory point is used as a candidate back shape feature.

[0191] For example, the deflection angle range corresponding to the back shape is [90, 180°] and [-90, -180°]. The deflection angle of the shape in the user image contained in a trajectory point information is -100°. Since it is located within [-90, -180°], the shape feature contained in the trajectory point information is used as a candidate back shape feature.

[0192] Step S4022-c2: Determine the score of each candidate back-side shape feature based on the deflection angle of the shape corresponding to each candidate back-side shape feature.

[0193] Optionally, the score Score3 of a candidate rear-face shape feature can be determined according to the following formula:

[0194] Score3 = |γ| / 180° (7)

[0195] Wherein, γ represents the deflection angle of the shape corresponding to the candidate back-side shape feature.

[0196] Step S4022-c3: Determine the back shape features from each candidate back shape feature based on the score of each candidate back shape feature.

[0197] In one possible implementation, candidate back-face features with scores greater than a preset score threshold can be identified as back-face features. It should be noted that if there are multiple candidate back-face features with scores greater than the preset score threshold, the candidate back-face feature with the highest score among these multiple candidate back-face features will be identified as the back-face feature.

[0198] In another possible implementation, candidate back-facing features whose scores are greater than a preset score threshold among candidate back-facing features that do not contain facial features in the trajectory point information can be identified as back-facing features. For example, candidate back-facing features include F... 1c F 2c F 3c and F 4c , of which F 1c and F 4c The trajectory point information contains facial features, F 2c and F 3c The trajectory point information does not contain facial features, F 2c If the score is greater than a preset score threshold (e.g., 0.8), F 3c If the score is less than the preset score threshold, then F will be... 2c It is determined to be a back view feature. It should be noted that if there are multiple candidate back view features with scores greater than a preset score threshold among the candidate back view features that do not contain facial features in the trajectory point information, then the candidate back view feature with the highest score among these multiple candidate back view features can be determined as the back view feature.

[0199] Step S403: Obtain trajectory point information that matches the shape features and representative shape features contained in the user trajectory database to obtain the second search result.

[0200] Specifically, for each representative shape feature, the similarity between the representative shape feature and each shape feature contained in the user trajectory database is calculated. The trajectory point information of the shape feature in the user trajectory database whose similarity with the representative shape feature is greater than a preset similarity threshold is determined as the target trajectory point information. All the obtained target trajectory point information is determined as the second search result.

[0201] This invention performs a secondary search in the user trajectory database based on representative shape features. The secondary search based on representative shape features can match trajectory point information with abnormal conditions such as occlusion, angle deviation, and color deviation.

[0202] Step S404: Merge and deduplicate the trajectory point information contained in the first search result and the trajectory point information contained in the second search result, and use the processed search result as the target search result.

[0203] It should be noted that when performing trajectory retrieval, in addition to providing the user image, some search conditions can also be provided, such as time and location. After obtaining the target search results, trajectory point information that meets these search conditions can be further filtered from the obtained target search results, and then sorted, paginated, and other processes can be performed on the trajectory point information that meets these search conditions.

[0204] This invention provides a user trajectory information retrieval device. The user trajectory information retrieval device provided in this invention is described below. The user trajectory information retrieval device described below can be referred to in correspondence with the user trajectory information retrieval method described above.

[0205] Please see Figure 5 The diagram shows a schematic of the structure of a user trajectory information retrieval device provided in an embodiment of the present invention. The user trajectory information retrieval device may include: a user feature extraction module 501, a user identifier determination module 502, and a user trajectory retrieval module 503.

[0206] User feature extraction module 501 is used to extract user features from a given user image to obtain the first user feature.

[0207] The user identifier determination module 502 is used to determine the user identifier of the user in the user image by matching the first user feature with the user feature in the user feature library, and use the result as the first user identifier.

[0208] The user feature library includes user features corresponding to several user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is extracted from a user image collected from a real environment and matches a user feature in the standard feature library.

[0209] The user trajectory retrieval module 503 is used to retrieve the trajectory point information of the user in the user image from the user trajectory database based on the first user identifier, and obtain the target retrieval result.

[0210] The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

[0211] Optionally, the first user feature includes a first facial feature and / or a first body feature; the standard feature library includes a standard facial feature library and a standard body feature library, and the dynamic feature library includes a dynamic facial feature library and a dynamic body feature library; the user feature library further includes an extended feature library, which includes an extended facial feature library and an extended body feature library; the facial features in the extended facial feature library are facial features associated with the body features in the dynamic body feature library, and the body features in the extended body feature library are body features associated with the facial features in the dynamic facial feature library, wherein the associated body features and facial features come from the same user image.

[0212] Optionally, the user trajectory information retrieval device provided in this embodiment of the invention further includes a feature database update module. The feature database update module is used to dynamically update the dynamic feature database and the extended feature database. Specifically, the feature database update module is used for:

[0213] For each acquired image, user features are extracted from the acquired image to obtain a second user feature;

[0214] In the case where the second user feature includes the second facial feature:

[0215] Determine whether a face feature matching the second face feature exists in the standard face feature library; if so, record the second face feature in the dynamic face feature library; if the second user feature also includes a second body feature, record the second body feature in the extended body feature library, wherein the user identifiers corresponding to the second face feature recorded in the dynamic face feature library and the second body feature recorded in the extended body feature library are both user identifiers corresponding to the face feature matching the second face feature in the standard face feature library;

[0216] In the case where the second user feature includes the second body feature:

[0217] Determine whether a body feature matching the second body feature exists in the standard body feature library; if so, record the second body feature in the dynamic body feature library; if the second user feature also includes a second face feature, record the second face feature in the extended face feature library; wherein, the user identifiers corresponding to the second body feature recorded in the dynamic body feature library and the second face feature recorded in the extended face feature library are the user identifiers corresponding to the body feature matching the second body feature in the standard body feature library.

[0218] Optionally, the feature library update module is also used for:

[0219] For any face feature library in the dynamic face feature library and the extended face feature library:

[0220] If the second facial feature is entered into the facial feature database, it is determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature is greater than the preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature is deleted according to the preset deletion rules.

[0221] For any one of the dynamic shape feature libraries and the extended shape feature library:

[0222] If the second shape feature is entered into the shape feature library, it is determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature is deleted according to the preset deletion rules.

[0223] Optionally, when the feature library update module deletes a facial feature corresponding to the user identifier corresponding to the second facial feature according to a preset deletion rule, it is specifically used for:

[0224] Obtain the scores of each face feature corresponding to the user identifier corresponding to the second face feature, and delete the face feature with the lowest score. The score of a face feature is determined based on the entry time of the face feature into the database and the similarity between the face feature and the face feature corresponding to the user identifier in the standard face feature database.

[0225] The step of deleting the shape feature corresponding to the user identifier corresponding to the second shape feature according to the preset deletion rules includes:

[0226] Obtain the scores of each shape feature corresponding to the user identifier corresponding to the second shape feature, and delete the shape feature with the lowest score. The score of a shape feature is determined based on the entry time of the shape feature into the database and the similarity between the shape feature and the shape feature corresponding to the user identifier in the standard shape feature database.

[0227] Optionally, the priority of each feature library included in the user feature library is as follows: the standard face feature library, the dynamic face feature library, the standard body feature library, the dynamic body feature library, the extended face feature library, and the extended body feature library;

[0228] When the user identifier determination module 502 determines the user identifier of the user in the user image by matching the first user feature with user features in the user feature database, it is specifically used for:

[0229] According to the priority, user features that match the first user feature are determined from the user feature database;

[0230] The user identifier corresponding to the user feature that matches the first user feature in the user feature library is determined as the user identifier of the user in the user image.

[0231] Optionally, the user trajectory retrieval module 503 may include: a first retrieval module.

[0232] The first retrieval module is used to obtain information on several trajectory points corresponding to the first user identifier from the user trajectory database, and obtain the first retrieval result as the target retrieval result.

[0233] Optionally, the trajectory point information corresponding to a user identifier includes a user image collected from the real environment and facial features and / or body features extracted from the user image;

[0234] The user trajectory retrieval module 503 may also include: a representative body feature acquisition module, a second retrieval module, and a retrieval result determination module.

[0235] The representative shape feature acquisition module is used to acquire representative shape features from the first search results.

[0236] The representative shape features include one or more of the following features: front shape features, side shape features, and back shape features.

[0237] The second retrieval module is used to obtain trajectory point information from the user trajectory database that matches the shape features with the representative shape features, and to obtain the second retrieval result;

[0238] The retrieval result determination module is used to merge and deduplicate the trajectory point information contained in the first retrieval result and the trajectory point information contained in the second retrieval result, and the processed retrieval result is used as the target retrieval result.

[0239] Optionally, when the representative shape feature acquisition module acquires representative shape features from the first search result, it is specifically used for:

[0240] Filter trajectory point information containing shape features from the first search results;

[0241] Based on the deflection angle of the shape in the user image contained in the selected trajectory point information, frontal shape features, and / or side shape features, and / or back shape features are obtained from the shape features contained in the selected trajectory point information to obtain representative shape features.

[0242] Optionally, when the representative shape feature acquisition module acquires frontal shape features and / or side shape features and / or back shape features from the shape features contained in the selected trajectory point information based on the deflection angle of the shape in the user image, it is specifically used for:

[0243] From the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the front shape, as candidate front shape features. Determine the score of each candidate front shape feature based on the deflection angle of the shape corresponding to each candidate front shape feature. Based on the score of each candidate front shape feature, determine the front shape feature from the obtained candidate front shape features.

[0244] And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the side shape as candidate side shape features, determine the score of each candidate side shape feature according to the deflection angle of the shape corresponding to each candidate side shape feature, and determine the side shape features from the obtained candidate side shape features according to the score of each candidate side shape feature.

[0245] And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the back shape as candidate back shape features, determine the score of each candidate back shape feature based on the deflection angle of the shape corresponding to each candidate back shape feature, and determine the back shape feature from the obtained candidate back shape features based on the score of each candidate back shape feature.

[0246] Optionally, the representative shape feature acquisition module, when determining the frontal shape feature from the acquired candidate frontal shape features based on the score of each candidate frontal shape feature, is specifically used for:

[0247] Candidate frontal features whose scores are greater than a preset score threshold among the candidate frontal features containing facial features in the trajectory point information are identified as frontal features.

[0248] Optionally, the representative shape feature acquisition module, when determining the side shape features from the acquired candidate side shape features based on the score of each candidate side shape feature, is specifically used for:

[0249] Candidate side profile features whose scores are greater than a preset score threshold among the candidate side profile features that do not contain facial features in the trajectory point information are identified as side profile features.

[0250] Optionally, the representative shape feature acquisition module, when determining the back shape feature from the acquired candidate back shape features based on the score of each candidate back shape feature, is specifically used for:

[0251] Candidate back-face features whose scores are greater than a preset score threshold among the candidate back-face features that do not contain facial features in the trajectory point information are identified as back-face features.

[0252] Optionally, when the second retrieval module obtains trajectory point information from the user trajectory database that matches the representative shape features, and obtains the second retrieval result, it is specifically used for:

[0253] For each representative shape feature: calculate the similarity between the representative shape feature and each shape feature contained in the user trajectory database; determine the trajectory point information of the shape feature in the user trajectory database whose similarity to the representative shape feature is greater than a preset similarity threshold as the target trajectory point information;

[0254] All obtained target trajectory point information are identified as the second search result.

[0255] The user trajectory information retrieval device provided in this embodiment of the invention has a high trajectory recall rate.

[0256] This invention provides a processing device; please refer to [link / reference]. Figure 6 The diagram shows the structure of the processing device, which may include: a processor 601, a communication interface 602, a memory 603, and a communication bus 604.

[0257] In this embodiment of the invention, the number of processor 601, communication interface 602, memory 603, and communication bus 604 is at least one, and processor 601, communication interface 602, and memory 603 communicate with each other through communication bus 604.

[0258] The processor 601 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0259] The memory 603 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0260] The memory stores a program, which the processor can call. The program is used for:

[0261] Extract user features from a given user image to obtain the first user feature;

[0262] By matching the first user feature with user features in the user feature library, the user identifier of the user in the user image is determined as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matched with a user feature in the standard feature library.

[0263] Based on the first user identifier, the trajectory point information of the user in the user image is retrieved from the user trajectory database to obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

[0264] Optionally, the refined and extended functions of the program can be found in the description above.

[0265] This invention also provides a computer-readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0266] Extract user features from a given user image to obtain the first user feature;

[0267] By matching the first user feature with user features in the user feature library, the user identifier of the user in the user image is determined as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matched with a user feature in the standard feature library.

[0268] Based on the first user identifier, the trajectory point information of the user in the user image is retrieved from the user trajectory database to obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

[0269] Optionally, the refined and extended functions of the program can be found in the description above.

[0270] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0271] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0272] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for retrieving user trajectory information, characterized in that, include: Extract user features from a given user image to obtain the first user feature; By matching the first user feature with user features in the user feature library, the user identifier of the user in the user image is determined as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matched with a user feature in the standard feature library. Based on the first user identifier, the trajectory point information of the user in the user image is retrieved from the user trajectory database to obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

2. The user trajectory information retrieval method according to claim 1, characterized in that, The first user feature includes a first facial feature and / or a first body shape feature; The standard feature library includes a standard face feature library and a standard body feature library, and the dynamic feature library includes a dynamic face feature library and a dynamic body feature library; The user feature library also includes an extended feature library, which includes an extended face feature library and an extended body feature library; The facial features in the extended facial feature library are facial features associated with the body features in the dynamic body feature library. The body features in the extended body feature library are body features associated with the facial features in the dynamic facial feature library. The associated body features and facial features come from the same user image.

3. The user trajectory information retrieval method according to claim 2, characterized in that, The dynamic feature library and the extended feature library are dynamically updated, and the process of dynamically updating the dynamic feature library and the extended feature library includes: For each acquired image, user features are extracted from the acquired image to obtain a second user feature; In the case where the second user feature includes the second facial feature: Determine whether a face feature matching the second face feature exists in the standard face feature library; if so, record the second face feature in the dynamic face feature library; if the second user feature also includes a second body feature, record the second body feature in the extended body feature library, wherein the user identifiers corresponding to the second face feature recorded in the dynamic face feature library and the second body feature recorded in the extended body feature library are both user identifiers corresponding to the face feature matching the second face feature in the standard face feature library; In the case where the second user feature includes the second body feature: Determine whether a body feature matching the second body feature exists in the standard body feature library; if so, record the second body feature in the dynamic body feature library; if the second user feature also includes a second face feature, record the second face feature in the extended face feature library; wherein, the user identifiers corresponding to the second body feature recorded in the dynamic body feature library and the second face feature recorded in the extended face feature library are the user identifiers corresponding to the body feature matching the second body feature in the standard body feature library.

4. The user trajectory information retrieval method according to claim 3, characterized in that, The process of dynamically updating the dynamic feature library and the extended feature library further includes: For any face feature library in the dynamic face feature library and the extended face feature library: If the second facial feature is entered into the facial feature database, it is determined whether the number of facial features corresponding to the user identifier corresponding to the second facial feature is greater than the preset number. If so, one facial feature corresponding to the user identifier corresponding to the second facial feature is deleted according to the preset deletion rules. For any one of the dynamic shape feature libraries and the extended shape feature library: If the second shape feature is entered into the shape feature library, it is determined whether the number of shape features corresponding to the user identifier corresponding to the second shape feature is greater than the preset number. If so, the shape feature corresponding to the user identifier corresponding to the second shape feature is deleted according to the preset deletion rules.

5. The user trajectory information retrieval method according to claim 4, characterized in that, The step of deleting the facial feature corresponding to the user identifier corresponding to the second facial feature according to the preset deletion rules includes: Obtain the scores of each face feature corresponding to the user identifier corresponding to the second face feature, and delete the face feature with the lowest score. The score of a face feature is determined based on the entry time of the face feature into the database and the similarity between the face feature and the face feature corresponding to the user identifier in the standard face feature database. The step of deleting the shape feature corresponding to the user identifier corresponding to the second shape feature according to the preset deletion rules includes: Obtain the scores of each shape feature corresponding to the user identifier corresponding to the second shape feature, and delete the shape feature with the lowest score. The score of a shape feature is determined based on the entry time of the shape feature into the database and the similarity between the shape feature and the shape feature corresponding to the user identifier in the standard shape feature database.

6. The user trajectory information retrieval method according to claim 2, characterized in that, The user feature library includes the following feature libraries in the following order of priority: standard face feature library, dynamic face feature library, standard body feature library, dynamic body feature library, extended face feature library, and extended body feature library. The step of determining the user identifier of the user in the user image by matching the first user feature with user features in the user feature library includes: According to the priority, user features that match the first user feature are determined from the user feature database; The user identifier corresponding to the user feature that matches the first user feature in the user feature library is determined as the user identifier of the user in the user image.

7. The user trajectory information retrieval method according to claim 1, characterized in that, The step of retrieving user trajectory point information from the user trajectory database based on the first user identifier to obtain the target retrieval result includes: The system retrieves several trajectory point information corresponding to the first user identifier from the user trajectory database to obtain the first search result, which is then used as the target search result.

8. The user trajectory information retrieval method according to claim 7, characterized in that, The trajectory point information corresponding to a user identifier includes a user image collected from the real environment and facial features and / or body features extracted from the user image; The step of retrieving user trajectory point information from the user trajectory database based on the first user identifier to obtain the target retrieval result further includes: Representative shape features are obtained from the first search results, wherein the representative shape features include one or more of the following features: front shape features, side shape features, and back shape features; The second search result is obtained by retrieving trajectory point information from the user trajectory database that matches the shape features of the representative shape features; The trajectory point information contained in the first search result and the trajectory point information contained in the second search result are merged and deduplicated, and the processed search result is used as the target search result.

9. The user trajectory information retrieval method according to claim 8, characterized in that, The step of obtaining representative shape features from the first search result includes: Filter trajectory point information containing shape features from the first search results; Based on the deflection angle of the shape in the user image contained in the selected trajectory point information, frontal shape features, and / or side shape features, and / or back shape features are obtained from the shape features contained in the selected trajectory point information to obtain representative shape features.

10. The user trajectory information retrieval method according to claim 9, characterized in that, The step of obtaining frontal shape features, and / or side shape features, and / or back shape features from the shape features contained in the selected trajectory point information based on the deflection angle of the shape in the user image includes: From the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the front shape, as candidate front shape features. Determine the score of each candidate front shape feature based on the deflection angle of the shape corresponding to each candidate front shape feature. Based on the score of each candidate front shape feature, determine the front shape feature from the obtained candidate front shape features. And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the side shape as candidate side shape features, determine the score of each candidate side shape feature according to the deflection angle of the shape corresponding to each candidate side shape feature, and determine the side shape features from the obtained candidate side shape features according to the score of each candidate side shape feature. And / or, from the selected trajectory point information, obtain the shape features corresponding to the shape whose deflection angle is within the deflection angle range corresponding to the back shape as candidate back shape features, determine the score of each candidate back shape feature based on the deflection angle of the shape corresponding to each candidate back shape feature, and determine the back shape feature from the obtained candidate back shape features based on the score of each candidate back shape feature.

11. The user trajectory information retrieval method according to claim 10, characterized in that, The step of determining the frontal shape features from the acquired candidate frontal shape features based on the score of each candidate frontal shape feature includes: Candidate frontal features whose scores are greater than a preset score threshold among the candidate frontal features containing facial features in the trajectory point information are identified as frontal features. The step of determining the side profile features from the acquired candidate side profile features based on the score of each candidate side profile feature includes: Candidate side profile features whose scores are greater than a preset score threshold among the candidate side profile features that do not contain facial features in the trajectory point information are identified as side profile features. The step of determining the back shape features from the acquired candidate back shape features based on the score of each candidate back shape feature includes: Candidate back-face features whose scores are greater than a preset score threshold among the candidate back-face features that do not contain facial features in the trajectory point information are identified as back-face features.

12. The user trajectory information retrieval method according to claim 8, characterized in that, The step of obtaining trajectory point information from the user trajectory database that matches the shape features with the representative shape features to obtain the second search result includes: For each representative shape feature: calculate the similarity between the representative shape feature and each shape feature contained in the user trajectory database; determine the trajectory point information of the shape feature in the user trajectory database whose similarity to the representative shape feature is greater than a preset similarity threshold as the target trajectory point information; All obtained target trajectory point information are identified as the second search result.

13. A user trajectory information retrieval device, characterized in that, include: The module includes a user feature extraction module, a user identifier determination module, and a user trajectory retrieval module. The user feature extraction module is used to extract user features from a given user image to obtain the first user feature; The user identifier determination module is used to determine the user identifier of the user in the user image by matching the first user feature with the user features in the user feature library, and use the first user identifier as the first user identifier. The user feature library includes several user features corresponding to user identifiers. The user feature library includes a standard feature library and a dynamic feature library. Each user feature in the dynamic feature library is a user feature extracted from a user image collected from a real environment and matching a user feature in the standard feature library. The user trajectory retrieval module is used to retrieve the trajectory point information of the user in the user image from the user trajectory database based on the first user identifier, and obtain the target retrieval result. The user trajectory database includes trajectory point information corresponding to several user identifiers. The user identifier corresponding to a trajectory point is determined by matching the user features in the trajectory point information with the user features in the user feature database.

14. A processing apparatus, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the user trajectory information retrieval method as described in any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the user trajectory information retrieval method as described in any one of claims 1 to 12.

Images