Track information determination method and device, equipment and storage medium

Abstract

The invention discloses a trajectory information determination method and device, equipment and a storage medium, and belongs to the technical field of computers. The method comprises the steps that a first image containing a target object is acquired through first image acquisition equipment, first feature information of the target object is determined based on the first image, and the first image acquisition equipment is located at a first position of a target area; determining identity identification information of the target object based on the first feature information; a second image including any object is acquired by using a second image acquisition device, second feature information of any object is determined based on the second image, and the second image acquisition device is located at a second position of the target area; and determining an association relationship between the second feature information and the identity identification information, and determining trajectory information of the target object according to the association relationship and related information of the first image acquisition device and the second image acquisition device. According to the method, the quantity of the identity identification information generated in the target area is reduced, and the accuracy and efficiency of determining the track information are improved.

Description

Methods, devices, equipment and storage media for determining trajectory information

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for determining trajectory information.

[0002] With the development of computer technology, there is an increasing demand for object management, security monitoring, and personalized services based on object trajectory information within target areas. These target areas can include public areas such as parks, shopping malls, and schools.

[0003] In related technologies, facial images captured by image acquisition devices installed in the target area are analyzed to obtain the facial identity document (Face ID) corresponding to the facial image. Image acquisition devices associated with the same Face ID are then linked to obtain the trajectory information of the object.

[0004] During the Face ID determination process, factors such as the installation location and angle of the image acquisition device, lighting conditions, and obstructions can lead to multiple Face IDs for the same object. This reduces the accuracy and efficiency of determining the object's trajectory information using the image acquisition device and Face ID.

[0005]

[0006] This application provides a method, apparatus, device, and storage medium for determining trajectory information, which improves the accuracy and efficiency of determining trajectory information. The technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a method for determining trajectory information, the method comprising:

[0008] A first image containing a target object is acquired using a first image acquisition device. Based on the first image, first feature information of the target object is determined. The first image acquisition device is located at a first position in the target area. The target object is an object that enters the target area.

[0009] The identity information of the target object is determined based on the first feature information;

[0010] A second image containing any object is acquired using a second image acquisition device, and second feature information of the object is determined based on the second image. The second image acquisition device is located at a second position in the target area.

[0011] The association between the second feature information and the identity information is determined, and the trajectory information of the target object is determined based on the association, the relevant information of the first image acquisition device and the second image acquisition device.

[0012] Secondly, embodiments of this application provide a trajectory information determination device, the device comprising:

[0013] The acquisition module is used to acquire a first image containing a target object using a first image acquisition device, and to determine first feature information of the target object based on the first image. The first image acquisition device is located at a first position in the target area, and the target object is an object that enters the target area.

[0014] The determining module is used to determine the identity information of the target object based on the first feature information;

[0015] The acquisition module is further configured to acquire a second image containing any object using a second image acquisition device, determine second feature information of the any object based on the second image, wherein the second image acquisition device is located at a second position in the target area;

[0016] The determining module is further configured to determine the association between the second feature information and the identity information, and determine the trajectory information of the target object based on the association, the relevant information of the first image acquisition device and the second image acquisition device.

[0017] In one possible implementation, the acquisition module is further configured to acquire registration feature information, wherein the registration feature information is feature information of a registered object, and the registered object is an object that has appeared in the target area and has the identity information;

[0018] The determining module is used to determine a first similarity between the first feature information and the registration feature information; and to determine the identity information of the target object based on the first similarity.

[0019] In one possible implementation, the determining module is configured to determine the identity information associated with the registration feature information corresponding to the first similarity as the identity information of the target object when the first similarity is greater than or equal to the first threshold; or, when the first similarity is less than the first threshold, generate the identity information of the target object based on the first feature information.

[0020] In one possible implementation, the determining module is further configured to perform a matching between the first feature information and the job feature information, wherein the job feature information is the feature information of the employees working in the target area; and if the first feature information does not match the job feature information, to perform a step of determining a first similarity between the first feature information and the registration feature information.

[0021] In one possible implementation, the first feature information includes the biometric information and auxiliary feature information of the target object. The auxiliary feature information is feature information that can help determine the identity information of the target object. The determining module is used to determine a first weight of the biometric information in the first feature information and a second weight of the auxiliary feature information in the first feature information; and to determine a first similarity between the first feature information and the registration feature information based on the first weight and the second weight.

[0022] In one possible implementation, the biometric information includes facial feature information, the auxiliary feature information includes head feature information of the target object, and the determining module is further configured to acquire target images acquired by the first image acquisition device and the second image acquisition device, wherein the target image is an image including the target object; determine the facial region and head region of the target object in the target image; and, if the overlap between the facial region and the head region is greater than an overlap threshold, determine the facial feature information and head feature information of the target object based on the facial region and head region in the target image.

[0023] In one possible implementation, the biometric information includes facial feature information, the auxiliary feature information includes clothing feature information of the target object, and the determining module is further configured to acquire target images acquired by the first image acquisition device and the second image acquisition device, wherein the target image is an image including the target object; determine head key points and torso key points of the target image; perform clustering on the head key points and torso key points to obtain the correspondence between the head key points and torso key points belonging to the same target object; and determine the facial feature information and clothing feature information of the target object in the target image according to the correspondence.

[0024] In one possible implementation, the first feature information includes the biometric information of the target object and the feature information of peer objects, wherein the peer objects are objects that have a companion relationship with the target object in the target region. The determining module is used to determine a third weight of the biometric information in the first feature information and a fourth weight of the peer object feature information in the first feature information; and to determine a first similarity between the first feature information and the registration feature information based on the third weight and the fourth weight.

[0025] In one possible implementation, the determining module is further configured to acquire multiple target images acquired by the first image acquisition device and the second image acquisition device, wherein the multiple target images are images including the target object; if the number of target objects and the first object located in the same target image is greater than or equal to a number threshold, and the distance between the first object and the target object in the same target image is less than or equal to a distance threshold, the first object is determined as a peer object of the target object; and the biometric information of the target object and the feature information of the peer object are determined based on the peer object and the target image.

[0026] In one possible implementation, the device further includes a storage module for storing registration feature information of the target object, the registration feature information including first feature information and identity information of the target object;

[0027] The determining module is used to determine the second similarity between the second feature information and the registration feature information; and to determine the association between the second feature information and the identity information based on the second similarity.

[0028] In one possible implementation, the determining module is configured to determine that the identity information associated with the registration feature information corresponding to the second similarity is associated with the second feature information if the second similarity is greater than or equal to the second threshold; or, if the second similarity is less than the second threshold, determine that the second feature information is not associated with the identity information.

[0029] In one possible implementation, the relevant information of the first image acquisition device and the second image acquisition device includes the location and acquisition time of the image acquisition devices. The determining module is used to determine the location of the target object in the target area based on the correlation and the location of the image acquisition devices; and to determine the trajectory information of the target object based on the location of the target object in the target area and the acquisition time.

[0030] In one possible implementation, the target area includes multiple sub-regions, and the acquisition module is further configured to acquire selection information, which is information generated for the selection operation of the identity information or any sub-region;

[0031] The device further includes a display module, which is configured to display the trajectory of the target object based on the identity information and the trajectory information when the selection information is generated by a selection operation for the identity information; or, when the selection information is generated by a selection operation for any sub-region, display the trajectory of the target object in any sub-region based on the any sub-region and the trajectory information of the target object in the target region.

[0032] Thirdly, embodiments of this application provide a computer device, the computer device including a processor and a memory, the memory storing at least one piece of program code, the at least one piece of program code being loaded and executed by the processor, so that the computer device implements any of the above-described methods for determining trajectory information.

[0033] Fourthly, a computer-readable storage medium is also provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the at least one piece of program code being loaded and executed by a processor to enable a computer to implement the method for determining trajectory information as described above.

[0034] Fifthly, a computer program or computer program product is also provided, wherein the computer program or computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement any of the above-described methods for determining trajectory information.

[0035] The technical solution provided in this application has at least the following beneficial effects:

[0036] The technical solution provided in this application provides a first image acquisition device and a second image acquisition device at a first and a second location in a target area. The first image acquisition device acquires a first image of the target object to generate first feature information. After determining the identity information of the target object using the first feature information, the second image acquisition device, which acquires a second image containing any object, associates the second feature information with the identity information, thereby achieving identity recognition of the target object in the target area. Furthermore, the second feature information does not generate new identity information, which can reduce the number of identity information generated in the target area to a certain extent, avoiding redundancy. This can also increase the number of second image acquisition devices that can acquire the same identity information, thereby improving the accuracy and efficiency of determining the trajectory information of the target object using the first and second image acquisition devices.

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 is a structural block diagram of a computer system for a method of determining execution trajectory information provided in an embodiment of this application;

[0039] Figure 2 is a flowchart of a method for determining trajectory information provided in an embodiment of this application;

[0040] Figure 3 is a schematic diagram of a target area provided in an embodiment of this application;

[0041] Figure 4 is a schematic diagram of a feature database provided in an embodiment of this application;

[0042] Figure 5 is a schematic diagram of trajectory information of a target object in a target area provided in an embodiment of this application;

[0043] Figure 6 is a flowchart of a method for determining the trajectory information of a park according to an embodiment of this application;

[0044] Figure 7 is a schematic diagram of a Face ID interaction process for determining people entering a park based on image acquisition, according to an embodiment of this application.

[0045] Figure 8 is an architecture diagram of a trajectory information determination process provided in an embodiment of this application;

[0046] Figure 9 is a schematic diagram of a trajectory information determination device provided in an embodiment of this application;

[0047] Figure 10 is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0048] Figure 11 is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0049] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments.

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0051] First, let's introduce the terms used in the embodiments of this application:

[0052] Feature vector: A feature vector is a form of representation of feature information. It is the information extracted from an image that represents the key features of the image, and the key feature information is represented in the form of a vector to obtain the corresponding feature vector of the image. The key feature information of the image may include, but is not limited to, the image's color feature information, texture feature information, and shape feature information.

[0053] Milvus is an open-source distributed vector database characterized by high availability, high performance, and easy scalability. Milvus can be built upon vector search libraries such as Facebook AI Similarity Search (Faiss), Approximate Nearest Neighbors Oh Yeah (Annoy), and Hierarchical Navigable Small World (HNSW). The core of the Milvus distributed vector database is solving the problem of dense vector similarity retrieval. Based on this vector database, Milvus supports data partitioning and sharding, data persistence, incremental data ingestion, scalar vector hybrid queries, and time travel, and optimizes vector retrieval performance to meet the application requirements of vector retrieval scenarios.

[0054] Image acquisition devices are hardware devices specifically designed to capture images of the real world and convert them into a computer-readable digital format for further processing, storage, display, or analysis. Image acquisition devices typically include cameras, webcams, and scanners.

[0055] Figure 1 is a structural block diagram of a computer system for determining execution trajectory information according to an embodiment of this application. As shown in Figure 1, the computer system includes: multiple image acquisition devices 101 and a server 102. The multiple image acquisition devices 101 may include a first image acquisition device and a second image acquisition device installed at different locations in the target area. Optionally, the computer system may also include a display device 103.

[0056] The image acquisition device 101 can be a fixed-position image acquisition device or a movable-position image acquisition device. The image acquisition device 101 is used to acquire information from a target area and generate an acquired image. The image acquisition device 101 may include a camera, webcam, scanner, and terminal devices equipped with the image acquisition device. The terminal device can be any electronic device product with an image acquisition device. For example, the terminal device can be a smartphone, tablet computer, laptop computer, PC (Personal Computer), PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), smart vehicle system, etc.

[0057] For example, the image acquisition device 101 can be installed at a fixed location in the target area to acquire objects (people) in the target area. The acquired images generated by the image acquisition device 101 may include objects in the target area.

[0058] Image acquisition device 101 can refer to multiple image acquisition devices in general. This embodiment only uses image acquisition device 101 as an example. Those skilled in the art will know that the number of image acquisition devices 101 can be more or less. For example, there can be dozens or hundreds of image acquisition devices 101, or even more. This application embodiment does not limit the number or type of image acquisition devices 101.

[0059] Server 102 provides backend services for image acquisition device 101. Server 102 can be a single server, a server cluster consisting of multiple servers, or any of the following: a cloud computing platform or a virtualization center; this embodiment does not limit the specific type of server 102. Server 102 communicates directly or indirectly with image acquisition device 101 via wired or wireless communication. Server 102 has data receiving, data processing, and data sending functions. Of course, server 102 may also have other functions; this embodiment does not limit the specific functions of server 102.

[0060] For example, server 102 can process the captured images generated by image acquisition device 101 to obtain trajectory information of each object in the target area. After obtaining the trajectory information of each object in the target area, the trajectory information of the target area or the trajectory information of any object can be displayed using display device 103. Display device 103 is a device with display function, and may include electronic devices with displays and projectors, etc.

[0061] Those skilled in the art should understand that the image acquisition device 101 and server 102 described above are merely illustrative examples. Other existing or future image acquisition devices or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0062] The trajectory information determination method provided in this application embodiment can be applied to the computer system shown in FIG1 above. For example, the method can be executed by the image acquisition device and server in FIG1. ​​As shown in FIG2, the method includes the following steps 201 to 204.

[0063] In step 201, a first image containing the target object is acquired using a first image acquisition device, and the first feature information of the target object is determined based on the first image. The first image acquisition device is located at a first position in the target area, and the target object is an object that enters the target area.

[0064] In an exemplary embodiment of this application, the target area is an area where image acquisition devices are installed. For example, the target area may include public areas such as parks, shopping malls, and schools. Multiple locations within the target area are equipped with image acquisition devices, including a first location, which may be the entrance / exit location or a turnstile location of the target area. The image acquisition device installed at the first location is a first image acquisition device, used to capture images of objects entering or exiting the target area to obtain a first image. The image acquisition device may include a camera, webcam, scanner, or a terminal device equipped with an image acquisition device. The number of first image acquisition devices installed at the first location can be one or more. It should be noted that this application provides an exemplary description of the target area; the target area can also be determined based on actual needs, and this application does not impose any limitations on this.

[0065] Figure 3 is a schematic diagram of a target area provided in an embodiment of this application. As shown in Figure 3, the target area may include three areas, namely area A, area B, and area C. Areas A and C have entrances to the target area, and first image acquisition devices (A1, A2, C1, and C2) are installed at the entrances. The first image acquisition devices can acquire images of target objects entering the target area.

[0066] The target object can be any object entering the target area, and the number of target objects can be one or more. The first image acquisition device can continuously acquire images of the acquisition area within the target area to generate a video stream. The acquisition area is the region that the acquisition device can capture, and the acquisition area can be determined based on the parameters of the image acquisition device. The video stream includes multiple frames, and the frame containing the target object is defined as the first image; that is, the first image is the image containing the target object.

[0067] Optionally, the first image acquisition device is a device whose acquisition angle or acquisition position is adjustable. By adjusting at least one of the acquisition position or acquisition angle of the first image acquisition device, a first image containing more information about the target object can be obtained. When there are multiple first image acquisition devices, multiple first image acquisition devices can acquire the same target object, obtaining multiple first images of the same target object at different acquisition positions or different acquisition angles. When there are multiple first images containing the target object, one or more images containing more information about the target object can be selected from the multiple first images as images for determining the first feature information of the target object in subsequent processes.

[0068] Taking a park as the target area, visitors entering the park as the target objects, and surveillance cameras as the primary image acquisition device, this example illustrates the process. At least one surveillance camera is installed at the park entrance, monitoring the entrance and generating video footage. As visitors enter or exit the park through the entrance, the surveillance camera captures images of them. For example, the video footage may include multiple frames; the frame containing the visitor is designated as the first image.

[0069] After determining the first image, the first feature information of the target object can be determined based on the first image. The first feature information can be represented as a multi-dimensional feature vector. The first feature information can include the following three cases:

[0070] Scenario 1: The primary feature information includes the target object's biometric information.

[0071] For example, biometric information includes facial feature information. The following explanation uses facial feature information as an example. A facial region detection algorithm generates multiple candidate boxes in the first image. These candidate boxes may contain facial regions. A classifier (such as a support vector machine or neural network) is used to classify the information in the candidate boxes to determine whether they contain the face of the target object. Optionally, a regressor is used to fine-tune the candidate boxes containing the target object's face, making the size of the candidate boxes more consistent with the size of the facial region. Based on these multiple candidate boxes, the facial region of the target object is determined.

[0072] Feature extraction is performed on the facial region of the target object to obtain the first feature information. For example, a Convolutional Neural Network (CNN) algorithm in deep learning can be used to extract geometric, texture, and color information of the facial region of the target object, and the extracted features are converted into vector representations to obtain the first feature vector of the target object, i.e., the first feature information. Geometric information may include, but is not limited to, the shape, position, and size of the target object's eyes, nose, mouth, and eyebrows, as well as the contour information of the face; texture information may include, but is not limited to, the smoothness of the target object's skin, blemishes, and wrinkles; color features may include the target object's skin color.

[0073] Scenario 2: The first feature information includes the target object's biometric information and auxiliary feature information. The auxiliary feature information is feature information that helps determine the target object's identity; the biometric information includes facial feature information, and the auxiliary feature information may include at least one of head feature information or clothing feature information.

[0074] In one embodiment of this application, facial feature information and head feature information are used as examples of biometric information. The process of determining facial feature information and head feature information may include: acquiring a target image captured by a first image acquisition device, wherein the target image is an image including a target object; determining the facial region and head region of the target object in the target image; and determining the facial feature information and head feature information of the target object based on the facial region and head region in the target image when the overlap between the facial region and head region is greater than an overlap threshold.

[0075] The following explanation uses the example of an object entering the target region as the case study. The target image may include multiple target objects. After determining the facial and head regions of each target object, the overlap between any two facial and head regions can be determined. This overlap is compared to an overlap threshold. When the overlap between the facial and head regions is greater than or equal to the threshold, the facial and head regions are identified as the face and head of the same target object. Feature extraction is then performed on the head region in the target image to obtain the head feature information of the target object. This head feature information may include, but is not limited to, head shape, head size, hair color, and hair length.

[0076] For example, the degree of overlap can be determined by the area of ​​the face region and the head region. For instance, after determining the face region and the head region of the target object in the target image, the intersection area and the union area of ​​the face region and the head region can be determined, and the ratio of the intersection area and the union area can be used as the area of ​​the face region and the head region to determine the degree of overlap.

[0077] The overlap threshold can be set based on at least one of the following: the number of target objects in the target image, the quality of the target image, or the user's actual needs. For example, when the number of target objects in the target image is small, a higher overlap threshold can be set to ensure a high degree of matching between the detected facial and head regions of the target objects, improving the accuracy and reliability of the matching results. When the number of target objects in the target image is large, the overlap threshold can be appropriately lowered to capture as many target objects as possible, increasing the number of detected target objects. As another example, when the quality of the target image is high, the detection algorithm can more accurately identify the facial and head regions of the target objects, allowing for a higher overlap threshold to improve the accuracy of the matching results. When the quality of the target image is low, the overlap threshold can be appropriately lowered to increase the number of detected target objects. Furthermore, different application scenarios or different users have different requirements for the accuracy of the matching results, allowing the overlap threshold to be set based on the user's actual needs.

[0078] It should be noted that the process of acquiring the target image is similar to that of acquiring the first image, and will not be elaborated further here. Furthermore, the process of determining the facial and head regions in the target image is similar to the process of determining the facial region in Case 1, and can be found in the relevant description in Case 1.

[0079] In another embodiment of this application, facial feature information and clothing feature information are used as examples of biometric information. The process of determining facial feature information and clothing feature information may include: acquiring a target image captured by a first image acquisition device, wherein the target image is an image including a target object; determining the head key points and torso key points of the target image; performing clustering on the head key points and torso key points to obtain the correspondence between head key points and torso key points belonging to the same target object; and determining the facial feature information and clothing feature information of the target object in the target image based on the correspondence.

[0080] Taking the target object as an example of an object entering the target region, the following explanation is provided. The target image may include multiple target objects. The target image is processed using a pose estimation model to obtain the head keypoints and torso keypoints of any target object. The pose estimation model may include, but is not limited to, OpenPose, PoseNet, and DensePose models. Head keypoints may include, but are not limited to, keypoints corresponding to the top of the head, eyes, ears, mouth, nose, and chin of any target object. Torso keypoints may include, but are not limited to, keypoints corresponding to the left and right sides of the shoulders, the center point and chest contour keypoints, the center point and left and right sides of the waist, the center point and leg contour keypoints, and the center point of both feet of any target object.

[0081] After identifying the key points of the head and torso, the coordinates of each key point can be obtained. Optionally, the positional coordinates of each key point can be evaluated using human anatomy to determine the confidence level of each key point. A confidence threshold can then be used to filter the key points, retaining those with a confidence level not lower than the threshold and removing those with a confidence level lower than the threshold. The confidence threshold can be set based on actual conditions or experimental results.

[0082] The process involves acquiring the positional features, orientation features, and local image features of each keypoint. Positional features can include the coordinates of the keypoint; orientation features can include the orientation vectors between different keypoints; and local image features can include the texture and color features of the area surrounding the keypoint. A clustering algorithm is then used to cluster the keypoints, resulting in clusters of head and torso keypoints belonging to the same target object, establishing a correspondence. Each target object can correspond to one cluster. Clustering algorithms can include, but are not limited to, K-means clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), and hierarchical clustering algorithms.

[0083] Facial and clothing features of the target object are obtained by extracting features from the regions surrounding key points of the torso that belong to the same cluster in the target image. The process of feature extraction from the regions surrounding the facial and torso key points is similar to that of feature extraction from the facial region, and will not be elaborated further here.

[0084] Scenario 3: The first feature information includes the target object's biometric information and the peer object's feature information.

[0085] In the exemplary embodiments of this application, facial feature information is used as an example of biometric information. In the target area, a peer object moves together with the target object. The process of determining the biometric information of the target object and the feature information of the peer object may include: acquiring multiple target images acquired by a first image acquisition device and a second image acquisition device, wherein the multiple target images are images including the target object; if the number of times the target object and the first object are located in the same target image is greater than or equal to a number threshold, and the distance between the first object and the target object in the same target image is less than or equal to a distance threshold, then the first object is determined as a peer object of the target object; the biometric information of the target object and the feature information of the peer object are determined based on the peer object and the target image.

[0086] The following explanation uses any object entering the target area as an example. The target image may include multiple objects, including both the target object and a first object. That is, the first object is any object in the target image other than the target object. For either the first image acquisition device or the second image acquisition device, a video containing the target object can be generated from the time the image acquisition device begins acquiring the target object until it can no longer acquire it. The second image acquisition device is located at the second position within the target area; it will be described in detail below and will not be elaborated upon here.

[0087] The captured video can include multiple captured images. The number of times the target object and any first object are located in the same target image in the captured video, and the number of images containing the target object, are determined. A quantity threshold is determined by at least one of the following: the total number of images containing the target object or the capture time of the image capture device. Since the target object appears for a different duration in the capture area corresponding to each image capture device, and the video duration of the target object captured by the image capture device varies, meaning the total number of captured images in the video varies, the quantity threshold can differ for different video durations. For example, 90% of the total number of captured images in the video can be used as the quantity threshold; or, if the video can capture 15 frames per second, the total number of captured images can be determined based on the video duration, thus allowing the quantity threshold to be determined using the video duration.

[0088] If the number of times the target object and the first object are located in the same target image is greater than or equal to a certain threshold, the distance between the target object and the first object is determined. For example, by normalizing the image containing the target object and the first object, and constructing a coordinate system based on the normalized image, the coordinates of the target object and the first object in the image are determined, thus obtaining the distance between the target object and the first object. When the distance between the first object and the target object is less than or equal to the distance threshold, the first object can be identified as a peer object of the target object. The distance threshold can be set based on actual conditions. For example, the distance threshold can be set to 3 meters.

[0089] It should be noted that the duration of target object capture varies for each image acquisition device, and different quantity thresholds can be applied to each device. Alternatively, the number of target objects and first objects in the same image can be used as a percentage of the total number of images containing the target object in the captured video to determine the objects traveling alongside the target object. For example, the total number of images in the captured video and the number of images where the target object and first object are in the same image are obtained. The ratio of the number of images where the target object and first object are in the same image to the total number of images is calculated. When the ratio is greater than or equal to a ratio threshold, the first object can be identified as an object that may be traveling alongside the target object. Then, the distance between the target object and the first object is used for further determination of the first object. The ratio threshold can be set based on actual conditions. The process of further determining the first object using the distance between the target object and the first object can be found in the description above and will not be repeated here.

[0090] After identifying the objects in the same row as the target object, feature extraction can be performed on the target object and the objects in the same row in the target image to obtain the facial feature information of the target object and the feature information of the objects in the same row. The process of extracting features from the face of the target object and the objects in the same row is similar to the process of determining facial feature information, and will not be elaborated on here.

[0091] Furthermore, this application provides an illustrative example of the process for determining the first feature information of the target object and the fact that the biometric information of the target object is facial feature information. Other methods can also be used to determine the first feature information of the target object based on the actual situation of the target image and the application scenario. The biometric information may also include other feature information of the target object, and this application does not limit this.

[0092] In step 202, the identity information of the target object is determined based on the first feature information.

[0093] In an exemplary embodiment of this application, the identity information of a target object can be determined through first feature information and a feature database. The feature database may include an open-source distributed vector database and a service database. The distributed vector database may store feature vectors of objects that have visited the target area; each feature vector may include the vector itself and a vector ID. The service database may contain information related to determining the trajectory information of the target object, such as storing the object's ID, vector ID, the association between the object's ID and the vector ID, storage time, and information related to the image acquisition device. Exemplarily, the distributed vector database may include Milvus, and the service database may include at least one of MySQL, PostgreSQL, or MongoDB. This application uses the example of a feature database comprising both a vector database and a service database for illustration; however, the feature database may also be a single database, and this application is not limited to this. Furthermore, this application uses the information of objects that have visited the target area stored in the feature database to determine the identity information of the target object as an example for illustration; however, the information of objects that have visited the target area may also be stored in the form of text documents or tables, and this application is not limited to this either.

[0094] Let's take determining the identity information of a target object through a feature database as an example. A feature database can include multiple feature sets, each containing feature vectors of similar types, i.e., feature information. For example, the feature database can include a registration feature set. The registration feature set can include multiple registration feature information, which can be all registration feature information or registration feature information within a certain time period. For example, the registration feature set can contain all historical registration feature information, or the registration feature information within the registration feature set can contain registration feature information within the last 7 days.

[0095] In one possible implementation, the identity information of the target object is determined by the similarity between the first feature information and the registration feature information. The process of determining the similarity between the first feature information and the registration feature information may include: obtaining the registration feature information, which is the feature information of a registered object, and the registered object is an object that has appeared in the target area and has identity identification information; determining the first similarity between the first feature information and the registration feature information; and determining the identity identification information of the target object based on the first similarity.

[0096] As mentioned above, the feature database can record the feature information of objects that have previously entered the target area and register an identity document (ID) for each object. That is, an object can have at least one feature, and the feature information of the same object is associated with an identity document. The identity document and the object have a unique mapping relationship. That is, each registered feature information in the feature database can include an identity document and at least one feature associated with the identity document.

[0097] For example, object feature information may include facial feature information, and the corresponding identity information may be facial identity information (Face ID). Registration feature information may include Face ID and facial feature information associated with Face ID. Optionally, the identity information included in the registration feature information may be a non-work identity, that is, the object associated with the identity information is not an object working in the target area.

[0098] After obtaining the registered feature information from the feature database, the first similarity between the first feature information and any registered feature information can be determined. The first feature information and the registered feature information can be represented in vector form. For example, both the first feature information and the registered feature information can be 256-dimensional feature vectors. The first similarity between the first feature vector and the registered feature vector is calculated using a similarity measurement method. This similarity measurement method can include, but is not limited to, calculating the cosine similarity and Euclidean distance between the first feature vector and the registered feature vector. The calculation of the cosine similarity between the first feature vector and the registered feature vector will be used as an example for explanation.

[0099] For example, the cosine similarity cosθ between the first feature vector A and the registered feature vector B can be calculated using formula (1):

[0100] In formula (1) above, ||A|| and ||B|| represent the modulus of the first feature vector A and the registered feature vector B, respectively. The value of cosine similarity ranges from -1 to 1. The closer the value is to 1, the more similar the first feature vector A and the registered feature vector B are; the closer the value is to -1, the less similar the first feature vector A and the registered feature vector B are. By calculating the similarity between the first feature vector A and the registered feature vector B, the first similarity between the first feature information and any registered feature information can be obtained. It should be noted that the use of cosine similarity to calculate the first similarity in this application is illustrative only. Other methods can also be used to calculate the first similarity, and this application does not limit this.

[0101] After determining the first similarity, the first similarity and the first feature information can be used to determine the identity information of the target object. For example, if the first similarity is greater than or equal to a first threshold, the identity information associated with the registration feature information corresponding to the first similarity is determined as the identity information of the target object; or, if the first similarity is less than the first threshold, the identity information of the target object is generated based on the first feature information. The first threshold can be set based on actual circumstances. For example, the first threshold can be set to 0.8.

[0102] For example, the registration feature set in the feature database may include m identity feature information, namely ID1 to IDm. When the first similarity between the registration feature information corresponding to IDk (1≤k≤m) and the first feature information is greater than or equal to the first threshold, the identity information of the target object is determined as IDk. When the first similarity between the registration feature information corresponding to ID1 to IDm and the first feature information is less than the first threshold, the identity information IDm+1 corresponding to the target object is generated. The first feature information and the identity information IDm+1 are associated and then updated to the registration feature set as registration feature information.

[0103] The technical solution provided in this application processes images captured by a first image acquisition device positioned at the entrance and exit points of a target area to obtain first feature information of the target object. It then determines a first similarity between the first feature information and registered feature information. If the first similarity is greater than or equal to a first threshold, the identity information associated with the registered feature information is identified as the identity information of the target object. This reduces the number of generated identity information entries, thereby reducing data redundancy in the feature database and saving storage resources. In subsequent processes, when using data from the feature database to determine the trajectory information of the target object, the amount of data processed can be reduced, thus improving the efficiency of determining the trajectory information of the target object.

[0104] Furthermore, since the feature database stores the feature information of the target object, the amount of data in the feature information is less than that in the acquired images, thus requiring less storage space and improving the utilization of storage resources. Moreover, in determining the trajectory information of the target object, the feature information stored in the feature database can be directly used for matching to obtain the target object's identification information, which can improve the efficiency of determining the target object's trajectory information to a certain extent.

[0105] In one embodiment of this application, the first feature information may include biometric information. A first similarity is determined using the biometric information and registration feature information. If the number of registered feature information corresponding to the first similarity greater than or equal to a first threshold in the feature database is one, the ID corresponding to the registered feature information can be associated with the first feature information. If the number of registered feature information corresponding to the first similarity greater than or equal to the first threshold in the feature database is multiple, the ID corresponding to the registered feature information with the highest first similarity can be associated with the first feature information. If the first similarity is less than the first threshold, an ID corresponding to the target object can be generated, and this ID can be associated with the first feature information to obtain the registration feature information corresponding to the target object. The registration feature information is then added to the registration feature set. Determining the identity information of the target object by using the first similarity and first threshold between facial feature information and registration feature information can improve the efficiency of determining the identity information of the target object compared to using multiple feature information.

[0106] In another embodiment of this application, if the first feature information includes other feature information besides biometric information, the first similarity can be determined based on the biometric information and other feature information. The other feature information includes at least one of auxiliary feature information or feature information of a fellow traveler. The auxiliary feature information may include at least one of head feature information or clothing feature information. Alternatively, the first similarity can be determined by biometric information and auxiliary feature information, or by biometric information and feature information of a fellow traveler.

[0107] For example, the process of determining the first similarity based on biometric information and auxiliary feature information includes: determining a first weight of biometric information in the first feature information and a second weight of auxiliary feature information in the first feature information; and determining the first similarity between the first feature information and the registration feature information based on the first weight and the second weight.

[0108] Taking facial features as an example of biometric information, the first weight can be determined by the importance and reliability of facial features in determining the target object's ID; the second weight can be determined by the completeness and uniqueness of auxiliary feature information. For example, if the target image includes a clear and complete facial region of the target object, a higher first weight can be assigned; if the target image shows an incomplete facial region or an unclear face, a lower first weight can be assigned. Similarly, if the target object's head size, hair color, hair length, or clothing is relatively complete or unique in the target image, a higher second weight can be assigned; otherwise, a lower second weight is assigned.

[0109] Calculate the facial feature similarity between facial feature information and registration feature information, as well as the auxiliary feature similarity between auxiliary feature information and registration feature information. The process of calculating facial feature similarity and auxiliary feature similarity is similar to the process of calculating the first similarity mentioned above, and will not be elaborated further here. Calculate the first similarity using facial feature similarity, auxiliary feature similarity, a first weight, and a second weight. First Similarity = First Weight × Facial Feature Similarity + Second Weight × Auxiliary Feature Similarity.

[0110] The technical solution provided in this application determines auxiliary feature information of a target object based on a target image. The auxiliary feature information may include at least one of head feature information or clothing feature information. By utilizing the weights corresponding to facial biometric information and auxiliary feature information, a first similarity between the first feature information and the registered feature information is determined, which can improve the accuracy of determining the first similarity. Especially when the biometric features of the target object are occluded or there is limited biometric information, the auxiliary feature information can be used as supplementary information to the biometric information, thereby improving the flexibility and robustness of determining the first similarity.

[0111] The process of determining the first similarity based on biometric information and peer information includes: determining the third weight of biometric information in the first feature information and the fourth weight of peer information in the first feature information; and determining the first similarity between the first feature information and the registration feature information based on the third and fourth weights.

[0112] For example, the confidence level of a peer object can be determined by at least one of the following: the number of times the target object and peer objects are located in the same image, or the distance between the peer object and the target object in the same image. For instance, the higher the proportion of the number of target objects and peer objects located in the same image in the total number of images, the higher the confidence level. A fourth weight is set by combining the confidence level with the completeness and uniqueness of the feature information of peer objects. The process of determining the third weight is similar to the process of determining the first weight, and the process of calculating the first similarity using the third and fourth weights is similar to the process of calculating the first similarity using the first and second weights, as described above, and will not be repeated here.

[0113] The technical solution provided in this application provides for determining peer objects of a target object based on a target image and a target object. This can increase the dimension of determining identity information. By utilizing the weights corresponding to the biometric information and the feature information of peer objects, the first similarity between the first feature information and the registration feature information can be determined, thereby improving the accuracy of determining the first similarity.

[0114] In an exemplary embodiment of this application, the feature database may further include a set of job features, which may include feature data corresponding to objects working in the target area. Optionally, before determining the first similarity between the first feature information and the registration feature information, the first feature information and the job feature information may be matched, where the job feature information is the feature information of employees working in the target area; if the first feature information and the job feature information do not match, the step of determining the first similarity between the first feature information and the registration feature information is performed.

[0115] For example, the job feature information is feature information generated from the onboarding information of employees within the target area. For instance, the onboarding information may include a facial photo of the employee. By extracting features from the employee's facial area, the job feature information is obtained, and this job feature information can be associated with the employee's identity information (Face ID).

[0116] Because staff move frequently within the target area, the multiple objects captured by the first or second image acquisition device may include a large number of staff. In determining the identity information of the target objects, the first characteristic information and work characteristic information of the target object can be matched firstly. If the similarity between the first characteristic information and the work characteristic information is greater than or equal to a first threshold, the Face ID associated with the work characteristic information can be used as the identity information of the target object. If the similarity between the first characteristic information and the work characteristic information is less than the first threshold, the first similarity between the first characteristic information and the registration characteristic information can be calculated.

[0117] The technical solution provided in this application prioritizes matching the first feature information of the target object with the work feature information. Since the number of staff is relatively fixed and usually much smaller than the number of registered feature information, prioritizing matching with work feature information can significantly reduce the amount of data that needs to be matched, thereby quickly narrowing the search range of the first feature vector in the feature database and improving the efficiency of determining identity information.

[0118] In step 203, a second image containing any object is acquired using a second image acquisition device, and second feature information of any object is determined based on the second image. The second image acquisition device is located at a second position in the target area.

[0119] In an exemplary embodiment of this application, the second location can be an internal location within the target area. The second location can be equipped with multiple second image acquisition devices, which are used to acquire a second image of the target object within the target area. Referring to Figure 3, the internal area of ​​the target area is the area excluding the entrance; that is, the second image acquisition devices are image acquisition devices other than the first image acquisition device among the target area acquisition devices. For example, the second image acquisition devices may include A3 to A6 in area A, B1 to B7 in area B, and C3 to C7 in area C.

[0120] The second image acquisition device can continuously acquire images of all objects within the acquisition area, which is the acquisition area of ​​the second image acquisition device. The acquisition areas of the second image acquisition devices can be the same or different in size. Each second image acquisition device can generate a video stream. Processing each video stream yields a second image containing any object. Then, feature extraction is performed on each object in the second image to obtain the second feature information corresponding to each object. The process of obtaining the second image is similar to that of obtaining the first image. Determining the second feature information of any object using the second image is similar to determining the first feature information of the target object based on the first image; see the relevant description in step 201, which will not be elaborated upon here.

[0121] In step 204, the association between the second feature information and the identity information is determined, and the trajectory information of the target object is determined based on the association, the relevant information of the first image acquisition device and the second image acquisition device.

[0122] Before determining the association between the second feature information and the identity information, the registration feature information of the target object can also be stored. For example, if the registration feature information in the registration feature set of the feature database does not match the first feature information of the target object, that is, if the feature database does not contain the registration information of the target object, registration feature information can be generated using the first feature information of the target object, and the registration feature information of the target object can be stored in the registration feature set. The registration feature information may include, but is not limited to, the first feature vector and the vector ID corresponding to the first feature vector.

[0123] For example, the second feature information may include biometric information, auxiliary feature information, and at least one of the feature information of peers. After determining the second feature information, the identity information of any object can be determined based on the second feature information. The process of determining the identity information of any object based on the second feature information may include: determining a second similarity between the second feature information and the registration feature information; and determining the association between the second feature information and the identity information based on the second similarity.

[0124] Taking the second feature information, which includes biometric information and peer feature information, as an example, the process of determining the second similarity using the second feature information is explained below. The process of determining the second similarity based on biometric information and peer feature information includes: determining the fifth weight of biometric information in the second feature information and the sixth weight of peer feature information in the second feature information; and determining the second similarity between the second feature information and the registration feature information based on the fifth and sixth weights. The process of determining the second similarity between the second feature information and the registration feature information is similar to the process of determining the first similarity between the first feature information and the registration feature information, as described above.

[0125] The technical solution provided in this application determines the second similarity between the second feature information and the registered feature information based on facial feature information and the feature information of objects in the same row, which can improve the accuracy of determining the second similarity. Especially when the target object is occluded, the feature information of objects in the same row can be used to help determine the position of the target object in subsequent processes, thereby increasing the number of images of the target object captured, and thus improving the accuracy of determining the trajectory information of the target object in subsequent processes.

[0126] After determining the second similarity, the association between the second feature information and the identity information can also be determined based on the second similarity. If the second similarity is greater than or equal to the second threshold, it is determined that the second feature information of the identity information associated with the registration feature information corresponding to the second similarity is associated; or, if the second similarity is less than the second threshold, it is determined that the second feature information and the identity information are not associated.

[0127] The second threshold can be the same as or different from the first threshold. For example, the second threshold can be less than the first threshold. Since the second feature vector corresponding to any object does not have the function of generating corresponding identity information, appropriately reducing the second threshold can improve the efficiency of determining the identity information corresponding to any object, thereby facilitating the determination of the trajectory information of the target object.

[0128] For example, the registration feature set in the feature database may include m identity feature information, namely ID1 to IDm. When the second similarity between the registration feature information corresponding to IDp (1≤p≤m) and the second feature information of object P is greater than or equal to the second threshold, then the second feature information is associated with the identity information, that is, the identity information of object P is associated with IDp. When the second similarity between the registration feature information corresponding to ID1 to IDm and the second feature information is less than the second threshold, then it is confirmed that object P is not associated with the registration feature information in the registration feature set. At this time, no new identity information is generated.

[0129] Optionally, the feature database may also include an abnormal feature set, which can be used to store abnormal feature vectors that are not associated with the registered feature information vectors. These abnormal feature vectors may have an abnormal vector ID. When the second feature information and the identity information are not associated, the second feature information can be matched with the abnormal feature vectors. If the match is successful, the abnormal vector ID corresponding to the abnormal feature vector is associated with the second feature information; if the match fails, the second feature information can be inserted as a new abnormal feature vector into the abnormal feature set. By also setting an abnormal feature set in the feature database, the abnormal vector set can be used to determine the feature information of objects that entered the target area through abnormal means, for subsequent viewing and processing by staff.

[0130] In an exemplary embodiment of this application, the various feature sets in the feature database can also be updated according to update information. The update information may include, but is not limited to, first feature information, second feature information, identity information, the association between the first feature information and the identity information, the association between the second feature information and the identity information, and the generation time of each feature information.

[0131] For example, the feature database may include, but is not limited to, a historical feature set, a registration feature set, an operational feature set, and an abnormal feature set. The historical feature set may include registration feature information, abnormal feature information, and operational feature set from the feature database that meet the time requirements.

[0132] Generally, the duration for which feature information is retained in the feature set can be set by the duration of the target object's presence in the target area. Let's take a park as the target area and tourists as the target objects, illustrating how to determine tourist identification information using a registration feature set. If the duration for which the registration feature set retains feature information is short, a large number of abnormal feature vectors will be generated; if the duration for which the registration feature set retains feature information is long, a large number of registration feature information entries need to be retrieved during the process of determining tourist identification information, reducing retrieval efficiency.

[0133] For example, when the retention period for the registered feature set is one day, if a visitor who stays overnight in the park appears directly inside the park the next day, the park's cameras will collect the visitor's feature information. However, if the visitor's identity information cannot be obtained by searching the registered feature set, the visitor's feature information will be judged as abnormal. Therefore, a large number of abnormal feature information may be generated. When the retention period for the registered feature set is 30 days, the number of visitors entering the park within 30 days is relatively large. In other words, the registered feature set contains a large amount of registered feature information, and visitors rarely stay in the park for a long period of time. Therefore, the efficiency of determining the visitor's identity information through the registered feature set is low.

[0134] The following example illustrates a feature database provided in this application, where the registration feature set contains newly added registration feature information within the past 7 days, and the abnormal feature set contains newly added abnormal feature information within the past 7 days. Figure 4 is a schematic diagram of a feature database provided in this application embodiment. As shown in Figure 4, the generation time of newly added registration feature information in the registration feature set is determined. If the time elapsed from the generation time of the registration information to the current time exceeds 7 days, the registration feature set and the historical feature set can be updated. That is, newly added registration feature information in the registration feature set that is more than 7 days old (i.e., newly added registration feature information before 7 days ago) is updated to the historical feature set, while newly added registration feature information within the past 7 days is retained in the registration feature set. Similarly, the abnormal feature set and the historical feature set can be updated.

[0135] It should be noted that the duration of feature information retained in the feature set in this application is an illustrative example. The duration of feature information retained in each feature set can be set according to the actual situation, and this application does not impose any restrictions on it.

[0136] By updating the various feature sets contained in the feature database, the feature information in each feature set of the feature database can be searched during the process of determining identity information. This can reduce the number of feature information that needs to be searched and improve the efficiency of determining identity information to a certain extent.

[0137] In an exemplary embodiment of this application, after determining the association between the second feature information and the identity information of any object, the trajectory information of the target object can be determined based on the association, relevant information of the first image acquisition device and the second image acquisition device. The process of determining the trajectory information of the target object may include: determining the position of the target object in the target area based on the association and the position of the image acquisition device; and determining the trajectory information of the target object based on the position of the target object in the target area and the acquisition time.

[0138] For example, the relevant information of the image acquisition devices may include information about a first image acquisition device and a second image acquisition device. This information may include, but is not limited to, the location and acquisition time of the image acquisition devices. The location of the image acquisition devices can be represented by labels, each label associated with the location coordinates of the image acquisition device. Based on the association between the object acquired by the image acquisition devices and the identification information, the target image acquisition device corresponding to the identification information is determined. The location of the target object within the target area is then determined using the location of the target image acquisition device.

[0139] In one embodiment, the location of the target image acquisition device can be used as the location of the target object. In another embodiment, the relative position information of the target object in the target image is analyzed using a target image containing the target object acquired by the target image acquisition device. This relative position information can be the offset of the target object relative to the center of the target image or the pixel coordinates of the target object in the target image. The position of the target object in the target region is calculated using the relative position information combined with relevant parameters of the image acquisition device. These relevant parameters may include, but are not limited to, the position coordinates, viewing angle, and focal length of the target image acquisition device.

[0140] The following explanation uses a target object as an example. Figure 5 is a schematic diagram of the trajectory information of a target object in a target area provided in an embodiment of this application. As shown in Figure 5, the target image acquisition device that captures the target object is determined by the target object's identification information, and the positions of the target object in the target area are determined to be from position a to position g (represented by gray circles in the figure) based on the relevant information of the target image acquisition device.

[0141] After determining the location of the target object in the target area, the locations are concatenated by combining the acquisition time of the target object acquired by the target image acquisition device to obtain the trajectory information of the target object in the target area. Referring to Figure 5, the target object enters the target area from the entrance of area A in the target area. According to the time (acquisition time) of the target object at each location, the target object moves in the target area based on the trajectory (1) to (9) in sequence, thus obtaining the trajectory information of the target object (dashed arrow).

[0142] Optionally, after determining the location of the target object, trajectory optimization algorithms (such as Kalman filtering algorithm, particle filtering algorithm, etc.) can be used to smooth and optimize the trajectory data, improve the accuracy and continuity of the trajectory, and make the trajectory information of the target object more consistent with the actual movement trajectory of the target object.

[0143] In an exemplary embodiment of this application, after determining the trajectory information of the target object, the trajectory information may also be displayed. The process of displaying the trajectory information may include: obtaining selection information, wherein the selection information is information generated by a selection operation on identity information or any sub-region; if the selection information is information generated by a selection operation on identity information, displaying the trajectory of the target object based on the identity information and the trajectory information; or, if the target region includes multiple sub-regions, and the selection information is information generated by a selection operation on any sub-region, displaying the trajectory of the target object in any sub-region based on any sub-region and the trajectory information of the target object within the target region.

[0144] For example, trajectory information can be displayed in a trajectory information display interface. The trajectory information display interface may include selection controls, allowing users to select the trajectory information to be displayed. For instance, the selection controls may include a first selection control and a second selection control. The first selection control displays the trajectory of one or more target objects; the second selection control displays the trajectory of one or more sub-regions within the target area. After detecting a selection operation on any selection control, selection information can be obtained based on the selection state of the selection control.

[0145] Taking the selection of the first selection control as an example, after the first selection control is selected, selection information related to identity identification information can be obtained. For example, a user can select a region in the target image through the first selection control. The selected region contains at least one target object. The system can analyze the feature information of the target object in the target region to obtain the target object's ID; or, the user can directly select the target object's ID through the first selection control. After the target object's ID is selected, the target object's ID and trajectory information generate the target object's trajectory, which is then displayed in the trajectory information display interface. During the display of the target object, all trajectories of the target object in the target region can be displayed at once, or the target object's trajectory can be dynamically displayed based on the target object's movement process in the target region. This application does not limit the trajectory display method. The process of displaying the trajectories of individual target objects in the target sub-region is similar to the process of displaying the target object's trajectory, and will not be elaborated here.

[0146] The technical solution provided in this application embodiment displays the trajectory of the target object or the trajectory of the target object in any sub-region in the trajectory information display interface by using user selection information and trajectory information. This allows users to intuitively understand the trajectory of the target object. Furthermore, users can flexibly select the trajectory displayed in the trajectory information display interface according to their own needs, which improves the personalization, flexibility and scalability of the trajectory display process.

[0147] The technical solution provided in this application provides a first image acquisition device and a second image acquisition device at a first and a second location in a target area. The first image acquisition device acquires a first image of the target object to generate first feature information. After determining the identity information of the target object using the first feature information, the second image acquisition device, which acquires a second image containing any object, associates the second feature information with the identity information, thereby achieving identity recognition of the target object in the target area. Furthermore, the second feature information does not generate new identity information, which can reduce the number of identity information generated in the target area to a certain extent, avoiding redundancy. This can also increase the number of second image acquisition devices that can acquire the same identity information, thereby improving the accuracy and efficiency of determining the trajectory information of the target object using the first and second image acquisition devices.

[0148] This application also provides an implementation process for a method to determine trajectory information. Taking a target area as a park, an image acquisition device as a camera, a first position as the park entrance / exit position, a second position as the park interior position, first characteristic information and second feature information as facial feature vectors, a feature database including multiple feature sets, each feature set containing multiple feature information, and Face ID as the identity identification information, this will be used as an example for illustration. Figure 6 is a flowchart of a method for determining trajectory information in a park provided by an embodiment of this application. As shown in Figure 6, the method includes the following steps 601 to 620.

[0149] In step 601, the camera captures a facial image.

[0150] In step 602, features are extracted from the facial image to obtain a facial feature vector.

[0151] In one possible implementation, steps 601 to 602 have been described in steps 201 and 203 above, and will not be repeated here.

[0152] In step 603, the facial feature vector is searched in the working feature set.

[0153] In step 604, it is determined whether there are feature vectors in the working feature set that meet the conditions. If yes, proceed to step 610; otherwise, proceed to step 605.

[0154] In step 605, it is determined whether the camera is a park entrance / exit camera or an internal park camera. If it is a park entrance / exit camera, proceed to step 606; if it is an internal park camera, proceed to step 612.

[0155] In step 606, a search is performed in the set of registered features.

[0156] In step 607, the first similarity between the registered feature vector in the registered feature set and the facial feature vector is determined.

[0157] In step 608, it is determined whether the first similarity is greater than the first threshold. If yes, proceed to step 610; otherwise, proceed to step 609.

[0158] In step 609, a new registration feature vector is generated using the facial feature vector and added to the registration feature set.

[0159] In step 610, the vector ID is determined using the registered feature vector.

[0160] In step 611, the Face ID is determined using the vector ID.

[0161] In one possible implementation, steps 603 to 611 have been described in step 202 above and will not be repeated here.

[0162] In step 612, a search is performed in the set of registered features.

[0163] In step 613, the second similarity between the registered feature vector in the registered feature set and the facial feature vector is determined.

[0164] In step 614, it is determined whether the second similarity is greater than the second threshold. If yes, proceed to step 610; otherwise, proceed to step 615.

[0165] In step 615, a search is performed in the set of abnormal features.

[0166] In step 616, it is determined whether there are any feature vectors in the abnormal feature set that meet the conditions. If yes, proceed to step 618; otherwise, proceed to step 617.

[0167] In step 617, a new abnormal feature vector is generated using the facial feature vector and added to the abnormal feature set.

[0168] In step 618, the vector ID is determined using the abnormal feature vector.

[0169] In step 619, the location of the camera that captured Face ID and the time when the camera acquired the facial image are obtained.

[0170] In step 620, the location of Face ID is determined, and the trajectory of Face ID in the park is recorded.

[0171] In one possible implementation, steps 612 to 620 have been described in step 204 above and will not be repeated here.

[0172] The technical solution provided in this application embodiment uses cameras installed at the entrances and exits of the park and inside the park to acquire facial images of people entering and inside the park. Feature extraction is performed on these facial images to obtain facial feature vectors. After determining a person's Face ID using the facial feature vectors corresponding to the facial images captured by the cameras entering the park, the identity of people in the park is achieved by associating the facial feature vectors corresponding to the facial images captured by the cameras inside the park with the already determined Face IDs. Furthermore, the facial feature vectors corresponding to the facial images captured by the cameras inside the park do not generate new Face IDs, which can reduce the number of Face IDs generated inside the park to a certain extent, avoiding Face ID redundancy. This can also increase the number of second image acquisition devices that can acquire the same Face ID, thereby improving the accuracy and efficiency of determining the trajectory information of people using the park's cameras.

[0173] This application also provides an interactive process for determining identity information. The interactive process can be jointly executed by an image acquisition device and a server, wherein the server may include a logic processing unit, a feature vector library service unit, a feature vector library, and a business library. Taking a target area as a park, the target object as a person entering the park, the image acquisition device as a first image acquisition device, and the identity information of the target object as Face ID as an example, the following explanation is provided. Figure 7 is a schematic diagram of a Face ID interactive process for determining a person entering a park based on image acquisition, according to an embodiment of this application. As shown in Figure 7, the interaction includes the following steps 701 to 723.

[0174] In step 701, the image acquisition device acquires facial images of people entering the park.

[0175] In step 702, the image acquisition device sends the facial features corresponding to the facial image to the logic processing unit.

[0176] In step 703, the logic processing unit processes the facial features to obtain a multidimensional facial feature vector.

[0177] In step 704, the logic processing unit calls the vector search interface of the feature vector library service unit.

[0178] In step 705, the feature vector library service unit sends a feature vector search request to the feature vector library.

[0179] In step 706, the feature vector library performs a vector search based on the vector search request and generates search results.

[0180] In step 707, the feature vector library sends the search results to the feature vector library service unit.

[0181] In step 708, the feature vector library service unit sends the search results to the logic processing unit.

[0182] In an exemplary embodiment of this application, the facial feature vector can be a vector V, which can be an n-dimensional feature vector. When the feature vector library searches for vector V, it can calculate the similarity between a feature vector (any face vector) in the feature vector library and vector V, and compare this similarity with a similarity threshold to obtain the search results. Where the search results include feature vectors, the similarity between the feature vector and vector V is greater than or equal to the similarity threshold. For example, the similarity threshold can be set to 0.8. If the search results contain k feature vectors, the two feature vectors with the highest similarity can be retained. The corresponding procedure can be:

[0183] In step 709, the logic processing unit determines whether the search result is empty. If it is empty, steps 710 to 719 are executed; if it is not empty, steps 720 to 723 are executed.

[0184] For example, the logic processing unit determines the status of the search results. When the search results are empty, it means that there are no feature vectors in the feature vector library that meet the conditions, so the result of empty search results ("code":400) is returned, and steps 710 to 719 are executed; when the search results are not empty, it means that the person has entered the park, so the result of empty search results ("code":200) is returned, and the feature vector ID and the similarity (distance) between the feature vector and the vector V are displayed.

[0185] When the search results are empty, the corresponding program can be:

[0186] When the search results are not empty, the corresponding program can be:

[0187] In step 710, the logic processing unit sends a feature vector registration request to the feature vector library service unit.

[0188] In step 711, the vector library service unit generates a registration feature vector based on the vector registration request.

[0189] In step 712, the vector library service unit sends the registered feature vector to the feature vector library.

[0190] In step 713, the feature vector library generates a registration vector ID corresponding to the registered feature vector.

[0191] In step 714, the feature vector library sends the registration vector ID to the vector library service unit.

[0192] In step 715, the vector library service unit generates the registration result.

[0193] In step 716, the vector library service unit sends the registration result to the logic processing unit, and the registration result includes the registered vector ID.

[0194] In step 717, the logic processing unit sends the registration vector ID to the business database.

[0195] In step 718, the business library generates the corresponding Face ID based on the registration vector ID.

[0196] In step 719, the business library sends the Face ID corresponding to the registration vector ID to the logic processing unit.

[0197] In an exemplary embodiment of this application, if the search results are empty, a request can be made to insert vector V into the feature vector library. For example, a feature vector registration request can be generated based on vector V. The feature vector registration request can be a POST request, and vector V can be inserted into the feature vector library based on the POST request.

[0198] When the search results are empty, the program that generates the POST request can be:

[0199] After inserting vector V into the feature vector library, the system can return the registration success status and vector ID. The corresponding program can be:

[0200] Here, 1703639911311478000 is the vector ID of vector V. After determining the vector ID, it can be inserted into the business database, and a unique identifier is generated as the Face ID of the person corresponding to the vector ID. The business database can store business information such as Face ID, the ID of the image acquisition device that captured the person's facial image, the vector ID, the acquisition time, and the storage address of the facial image.

[0201] In step 720, the logic processing unit generates a query request based on the vector ID contained in the search results.

[0202] In step 721, the logic processing unit sends a query request to the business database.

[0203] In step 722, the business database obtains the Face ID corresponding to the vector ID based on the query request.

[0204] In step 723, the business library sends the Face ID corresponding to the vector ID to the logic processing unit.

[0205] In an exemplary embodiment of this application, if the search result is not empty, it means that vector V has been stored in the feature vector library. The vector ID corresponding to vector V can be directly determined through the feature vector library. Then, the vector ID is used to query the business database to obtain the Face ID corresponding to the vector ID.

[0206] It should be noted that this application takes the image acquisition device at the entrance and exit of the park as an example. The image acquisition device located inside the park can determine the Face ID of the person included in the image acquired by the image acquisition device inside the park by executing steps 701 to 709 and steps 720 to 723. That is to say, the image acquired by the image acquisition device inside the park does not generate a new Face ID, but is only associated with the already generated Face ID.

[0207] The technical solution provided in this application acquires facial images of people entering the park using image acquisition devices at the park's entrances and exits. The facial images are processed to obtain facial feature vectors, which are then searched in a feature vector library. Based on the search results, the person's Face ID is determined, thus achieving identification of individuals entering the park. Furthermore, the image acquisition devices within the park do not generate new Face IDs, reducing the number of Face IDs generated within the park and avoiding redundancy. This increases the number of image acquisition devices within the park that can acquire the same Face ID, thereby improving the accuracy and efficiency of determining the trajectory information of individuals based on Face IDs using image acquisition devices at the park's entrances and exits and within the park.

[0208] This application also provides an architecture for determining trajectory information. Taking the target area as a park, the target object as tourists entering the park, and the trajectory information as the trajectory of tourists within the park as an example, Figure 8 is an architecture diagram of a trajectory information determination process provided by an embodiment of this application. As shown in Figure 8, the business layer is directly related to the user's business needs and can be used to determine the user's query request based on the user's selection operation on the data dashboard, and to transform the query request into a query instruction that the trajectory information determination system can understand. For example, the business layer can be a visitor flow data dashboard, through which users can select to view the trajectory of any tourist, the trajectory of a portion of the park, or the trajectory of the entire park.

[0209] Let's take the user's need to view the flow of visitors throughout the park as an example. Relevant data can be retrieved through query commands and analyzed to obtain statistical results on visitor flow. For example, images captured by park cameras can be obtained, which may include visitors. The query command determines the association rule configuration, and the relevant information in the association rule configuration is used to filter the captured images to determine those that need to be processed. The IDs of each visitor in the captured images are obtained, and the associated cameras are determined based on the visitor IDs, i.e., the cameras corresponding to the visitors with those IDs are captured. The data from the associated cameras is processed to obtain the locations of each visitor at different times. Based on the time sequence and IDs, the visitor locations are concatenated to obtain the trajectory of each visitor, thus yielding the statistical results on visitor flow.

[0210] The tourist's ID can be determined through Extract-Transform-Load (ETL). For example, real-time ETL data generated by the distributed streaming computing platform Kafka and offline ETL data stored in the business database PostgreSQL are input into the data analysis database Clickhouse. Data processing is then performed on the data in Clickhouse to obtain the tourist's ID. The real-time and offline ETL data can be used to extract features from the captured images, obtaining the tourist's corresponding feature vector. The feature vector can include facial feature vectors, and optionally, it can also include at least one of auxiliary feature vectors and feature vectors of the tourist's companions. It should be noted that this application uses a database containing the business database PostgreSQL and the data analysis database Clickhouse as an example; the database can also include a feature vector library and a business database MongoDB.

[0211] Data processing can be implemented through algorithm services, which may include operation services for a feature vector library. This library stores feature vectors corresponding to images captured by cameras and can categorize feature vectors into different sets based on common characteristics. The operation services for the feature vector library may include connection testing, set creation, set deletion, vector count query, vector insertion, vector search, and vector deletion. For example, features can be extracted from images of visitors entering a park. The extracted vectors can then be searched in the feature vector library. If a vector in the database has low similarity to the extracted vector, it can be inserted into the feature vector library.

[0212] This application also provides a trajectory information determination device. Figure 9 is a schematic diagram of the structure of a trajectory information determination device provided in an embodiment of this application. As shown in Figure 9, the device includes:

[0213] The acquisition module 901 is used to acquire a first image containing a target object using a first image acquisition device, and determine the first feature information of the target object based on the first image. The first image acquisition device is located at a first position in the target area, and the target object is an object that enters the target area.

[0214] The determination module 902 is used to determine the identity information of the target object based on the first feature information;

[0215] The acquisition module 901 is also used to acquire a second image containing any object using the second image acquisition device, determine the second feature information of any object based on the second image, and the second image acquisition device is located at a second position in the target area.

[0216] The determination module 902 is also used to determine the association between the second feature information and the identity information, and to determine the trajectory information of the target object based on the association, the relevant information of the first image acquisition device and the second image acquisition device.

[0217] In one possible implementation, the acquisition module 901 is also used to acquire registration feature information, which is the feature information of the registered object, and the registered object is an object that has appeared in the target area and has identity information;

[0218] The determination module 902 is used to determine the first similarity between the first feature information and the registered feature information; and to determine the identity information of the target object based on the first similarity.

[0219] In one possible implementation, the determining module 902 is used to determine the identity information associated with the registration feature information corresponding to the first similarity as the identity information of the target object when the first similarity is greater than or equal to the first threshold; or, when the first similarity is less than the first threshold, to generate the identity information of the target object based on the first feature information.

[0220] In one possible implementation, the determining module 902 is further configured to perform a matching of the first feature information and the job feature information, wherein the job feature information is the feature information of the employees in the target area; if the first feature information and the job feature information do not match, a step of determining the first similarity between the first feature information and the registration feature information is performed.

[0221] In one possible implementation, the first feature information includes the biometric information and auxiliary feature information of the target object. The auxiliary feature information is feature information that can help determine the identity information of the target object. The determining module 902 is used to determine the first weight of the biometric information in the first feature information and the second weight of the auxiliary feature information in the first feature information; and to determine the first similarity between the first feature information and the registration feature information based on the first weight and the second weight.

[0222] In one possible implementation, the biometric information includes facial feature information, and the auxiliary feature information includes head feature information of the target object. The determination module 902 is further configured to acquire a target image acquired by a first image acquisition device and a second image acquisition device, wherein the target image is an image including the target object; determine the facial region and head region of the target object in the target image; and determine the facial feature information and head feature information of the target object based on the facial region and head region in the target image if the overlap between the facial region and head region is greater than an overlap threshold.

[0223] In one possible implementation, the biometric information includes facial feature information, and the auxiliary feature information includes clothing feature information of the target object. The determining module 902 is further configured to acquire target images acquired by the first image acquisition device and the second image acquisition device, wherein the target image is an image including the target object; determine the head key points and torso key points of the target image; perform clustering on the head key points and torso key points to obtain the correspondence between head key points and torso key points belonging to the same target object; and determine the facial feature information and clothing feature information of the target object in the target image based on the correspondence.

[0224] In one possible implementation, the first feature information includes the biometric information of the target object and the feature information of peer objects. Peer objects are objects that have a companion relationship with the target object in the target area. The determining module 902 is used to determine the third weight of the biometric information in the first feature information and the fourth weight of the peer object feature information in the first feature information; and to determine the first similarity between the first feature information and the registration feature information based on the third weight and the fourth weight.

[0225] In one possible implementation, the determining module 902 is further configured to acquire multiple target images acquired by the first image acquisition device and the second image acquisition device, wherein the multiple target images are images including target objects; if the number of target objects and the first object located in the same target image is greater than or equal to a number threshold, and the distance between the first object and the target object in the same target image is less than or equal to a distance threshold, the first object is determined as a peer object of the target object; and the biometric information of the target object and the feature information of the peer object are determined based on the peer object and the target image.

[0226] In one possible implementation, the trajectory information determination device further includes a storage module (not shown in the figure), which is used to store the registration feature information of the target object, including the first feature information and identity information of the target object;

[0227] The determining module 902 is used to determine the second similarity between the second feature information and the registration feature information; and to determine the association between the second feature information and the identity information based on the second similarity.

[0228] In one possible implementation, the determining module 902 is used to determine, when the second similarity is greater than or equal to the second threshold, that the identity information associated with the registration feature information corresponding to the second similarity is associated with the second feature information; or, when the second similarity is less than the second threshold, to determine that the second feature information and the identity information are not associated.

[0229] In one possible implementation, the relevant information of the first image acquisition device and the second image acquisition device includes the location and acquisition time of the image acquisition device. The determination module 902 is used to determine the position of the target object in the target area based on the correlation and the location of the image acquisition device; and to determine the trajectory information of the target object based on the position of the target object in the target area and the acquisition time.

[0230] In one possible implementation, the target area includes multiple sub-regions, and the acquisition module 901 is also used to acquire selection information, which is information generated for the selection operation of the identity information or any sub-region.

[0231] The trajectory information determination device also includes a display module (not shown in the figure). The display module is used to display the trajectory of the target object based on the identity information and trajectory information when the selected information is generated by a selection operation for identity information; or, when the selected information is generated by a selection operation for any sub-region, to display the trajectory of the target object in any sub-region based on any sub-region and the trajectory information of the target object in the target region.

[0232] The technical solution provided in this application is based on an acquisition module and a determination module. By setting a first image acquisition device and a second image acquisition device at a first and a second location in the target area, the first image acquisition device acquires a first image of the target object to generate first feature information. After determining the identity information of the target object using the first feature information, the second image acquisition device, which acquires a second image containing any object, associates the second feature information with the identity information, thereby achieving identity recognition of the target object in the target area. Furthermore, the second feature information does not generate new identity information, which can reduce the number of identity information generated in the target area to a certain extent, avoiding redundancy. This can also increase the number of second image acquisition devices that can acquire the same identity information, thereby improving the accuracy and efficiency of determining the trajectory information of the target object using the first and second image acquisition devices.

[0233] It should be understood that the above-described apparatus is only illustrated by the division of the functional modules described above when implementing its functions. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0234] Figure 10 is a schematic diagram of a terminal device provided in an embodiment of this application. The terminal device 1000 can be any electronic device product capable of human-computer interaction with a user through one or more methods such as a keyboard, touchpad, remote control, voice interaction, or handwriting device. Examples include PCs (Personal Computers), mobile phones, smartphones, PDAs (Personal Digital Assistants), wearable devices, PPCs (Pocket PCs), tablet computers, smart car systems, smart TVs, smart speakers, and smartwatches.

[0235] Typically, terminal device 1000 includes a processor 1001 and a memory 1002.

[0236] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0237] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the method for determining trajectory information provided in the method embodiments of this application.

[0238] In some embodiments, the terminal device 1000 may also optionally include: a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, and a power supply 1008.

[0239] Peripheral device interface 1003 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002 and peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1001, memory 1002 and peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0240] The radio frequency (RF) circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1004 can communicate with other terminal devices through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0241] Display screen 1005 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1005 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1001 for processing. In this case, display screen 1005 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1005 may be a single screen, disposed on the front panel of terminal device 1000; in other embodiments, display screen 1005 may be at least two, disposed on different surfaces of terminal device 1000 or in a folded design; in still other embodiments, display screen 1005 may be a flexible display screen, disposed on a curved or folded surface of terminal device 1000. Furthermore, display screen 1005 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0242] The camera assembly 1006 is used to acquire images or videos. Optionally, the camera assembly 1006 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal device 1000, and the rear-facing camera is located on the back of the terminal device 1000. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0243] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1001 for processing, or input to the radio frequency circuit 1004 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal device 1000. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0244] The power supply 1008 is used to power the various components in the terminal device 1000. The power supply 1008 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When the power supply 1008 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0245] In some embodiments, the terminal device 1000 further includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: an acceleration sensor 1011, a gyroscope sensor 1012, a pressure sensor 1013, an optical sensor 1014, and a proximity sensor 1015.

[0246] Accelerometer 1011 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal device 1000. For example, accelerometer 1011 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1001 can control display screen 1005 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1011. Accelerometer 1011 can also be used for games or for acquiring user motion data.

[0247] The gyroscope sensor 1012 can detect the orientation and rotation angle of the terminal device 1000. The gyroscope sensor 1012 can work in conjunction with the accelerometer sensor 1011 to collect the user's 3D movements on the terminal device 1000. Based on the data collected by the gyroscope sensor 1012, the processor 1001 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0248] The pressure sensor 1013 can be disposed on the side bezel of the terminal device 1000 and / or on the lower layer of the display screen 1005. When the pressure sensor 1013 is disposed on the side bezel of the terminal device 1000, it can detect the user's grip signal on the terminal device 1000, and the processor 1001 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0249] An optical sensor 1014 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 based on the ambient light intensity collected by the optical sensor 1014. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 based on the ambient light intensity collected by the optical sensor 1014.

[0250] The proximity sensor 1015, also known as a distance sensor, is typically installed on the front panel of the terminal device 1000. The proximity sensor 1015 is used to detect the distance between the user and the front of the terminal device 1000. In one embodiment, when the proximity sensor 1015 detects that the distance between the user and the front of the terminal device 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from a screen-on state to a screen-off state; when the proximity sensor 1015 detects that the distance between the user and the front of the terminal device 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from a screen-off state to a screen-on state.

[0251] Those skilled in the art will understand that the structure shown in FIG10 does not constitute a limitation on the terminal device 1000, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0252] Figure 11 is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more processors 1101 and one or more memories 1102. Each memory 1102 stores at least one line of program code, which is loaded and executed by the processors 1101 to implement the trajectory information determination method provided in the various method embodiments described above. Of course, the server 1100 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 1100 may also include other components for implementing device functions, which will not be elaborated upon here.

[0253] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one piece of program code, which is loaded and executed by a processor to enable a computer to implement any of the above-described methods for determining trajectory information.

[0254] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0255] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement any of the above-described methods for determining trajectory information.

[0256] In this application, the term "at least one of A and B" merely describes the relationship between related objects, indicating that three relationships can exist. For example, "at least one of A and B" can represent: A existing alone, A and B existing simultaneously, and B existing alone. Similarly, "at least one of A, B, and C" indicates that seven relationships can exist, representing: A existing alone, B existing alone, C existing alone, A and B existing simultaneously, A and C existing simultaneously, C and B existing simultaneously, and A, B, and C existing simultaneously. Likewise, "at least one of A, B, C, and D" indicates that fifteen relationships can exist, representing: A existing alone, B existing alone, C existing alone, D existing alone, A and B existing simultaneously, A and C existing simultaneously, A and D existing simultaneously, C and B existing simultaneously, D and B existing simultaneously, C and D existing simultaneously, A, B, and C existing simultaneously, A, B, and D existing simultaneously, A, C, and D existing simultaneously, and A, B, C, and D existing simultaneously.

[0257] In this application, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0258] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the first image, second image, first feature information, second feature information, and identity information involved in this application were all obtained with full authorization.

[0259] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

A method for determining trajectory information, characterized in that, The method includes: acquiring a first image containing a target object using a first image acquisition device; determining first feature information of the target object based on the first image; wherein the first image acquisition device is located at a first position in a target area, and the target object is an object entering the target area; determining identity information of the target object based on the first feature information; acquiring a second image containing any object using a second image acquisition device; determining second feature information of any object based on the second image; wherein the second image acquisition device is located at a second position in the target area; determining the association between the second feature information and the identity information; and determining trajectory information of the target object based on the association, relevant information of the first image acquisition device, and the second image acquisition device. The method according to claim 1, characterized in that, Before determining the identity information of the target object based on the first feature information, the method further includes: obtaining registration feature information, wherein the registration feature information is feature information of a registered object, and the registered object is an object that has appeared in the target area and has the identity information; determining the identity information of the target object based on the first feature information includes: determining a first similarity between the first feature information and the registration feature information; and determining the identity information of the target object based on the first similarity. The method according to claim 2, characterized in that, Determining the identity information of the target object based on the first similarity includes: if the first similarity is greater than or equal to a first threshold, determining the identity information associated with the registration feature information corresponding to the first similarity as the identity information of the target object; or, if the first similarity is less than the first threshold, generating the identity information of the target object based on the first feature information. The method according to claim 2, characterized in that, Before determining the first similarity between the first feature information and the registration feature information, the method further includes: performing a match between the first feature information and the work feature information, wherein the work feature information is the feature information of the employees in the target area; and performing a step of determining the first similarity between the first feature information and the registration feature information if the first feature information and the work feature information do not match. The method according to claim 2, characterized in that, The first feature information includes the biometric information and auxiliary feature information of the target object. The auxiliary feature information is feature information that can help determine the identity information of the target object. Determining the first similarity between the first feature information and the registration feature information includes: determining the first weight of the biometric information in the first feature information and the second weight of the auxiliary feature information in the first feature information; and determining the first similarity between the first feature information and the registration feature information based on the first weight and the second weight. The method according to claim 5, characterized in that, The biometric information includes facial feature information, and the auxiliary feature information includes head feature information of the target object. Before determining the first weight of the biometric information in the first feature information and the second weight of the auxiliary feature information in the first feature information, the method further includes: acquiring a target image acquired by the first image acquisition device and the second image acquisition device, wherein the target image is an image including the target object; determining the facial region and head region of the target object in the target image; and determining the facial feature information and head feature information of the target object based on the facial region and head region in the target image if the overlap between the facial region and the head region is greater than an overlap threshold. The method according to claim 5, characterized in that, The biometric information includes facial feature information, and the auxiliary feature information includes clothing feature information of the target object. Before determining the first weight of the biometric information in the first feature information and the second weight of the auxiliary feature information in the first feature information, the method further includes: acquiring a target image acquired by the first image acquisition device and the second image acquisition device, wherein the target image is an image including the target object; determining the head key points and torso key points of the target image; performing clustering on the head key points and torso key points to obtain the correspondence between the head key points and torso key points belonging to the same target object; and determining the facial feature information and clothing feature information of the target object in the target image according to the correspondence. The method according to claim 2, characterized in that, The first feature information includes the biometric information of the target object and the feature information of peer objects. The peer objects are objects that have a companion relationship with the target object in the target area. Determining the first similarity between the first feature information and the registration feature information includes: determining the third weight of the biometric information in the first feature information and the fourth weight of the peer object's feature information in the first feature information; and determining the first similarity between the first feature information and the registration feature information based on the third weight and the fourth weight. The method according to claim 8, characterized in that, Before determining the third weight of the biometric information in the first feature information and the fourth weight of the feature information of the peer object in the first feature information, the method further includes: acquiring multiple target images acquired by the first image acquisition device and the second image acquisition device, wherein the multiple target images are images including the target object; when the number of target objects and the first object located in the same target image is greater than or equal to a number threshold, and the distance between the first object and the target object in the same target image is less than or equal to a distance threshold, the first object is determined as a peer object of the target object; and the biometric information of the target object and the feature information of the peer object are determined based on the peer object and the target image. The method according to any one of claims 1-9, characterized in that, The method further includes: storing registration feature information of the target object, the registration feature information including first feature information and identity information of the target object; determining the association between the second feature information and the identity information includes: determining a second similarity between the second feature information and the registration feature information; and determining the association between the second feature information and the identity information based on the second similarity. The method according to claim 10, characterized in that, The step of determining the association between the second feature information and the identity information based on the second similarity includes: if the second similarity is greater than or equal to the second threshold, determining that the identity information associated with the registration feature information corresponding to the second similarity is associated with the second feature information; or, if the second similarity is less than the second threshold, determining that the second feature information and the identity information are not associated. The method according to any one of claims 1-9, characterized in that, The relevant information of the first image acquisition device and the second image acquisition device includes the location and acquisition time of the image acquisition devices. The step of determining the trajectory information of the target object based on the correlation and the relevant information of the first image acquisition device and the second image acquisition device includes: determining the location of the target object in the target area based on the correlation and the location of the image acquisition device; and determining the trajectory information of the target object based on the location of the target object in the target area and the acquisition time. The method according to claim 12, characterized in that, The target area includes multiple sub-regions. After determining the trajectory information of the target object based on its position in the target area and the acquisition time, the method further includes: acquiring selection information, which is information generated by a selection operation on the identity information or any sub-region; if the selection information is information generated by a selection operation on the identity information, displaying the trajectory of the target object based on the identity information and the trajectory information; or, if the selection information is information generated by a selection operation on any sub-region, displaying the trajectory of the target object in any sub-region based on the any sub-region and the trajectory information of the target object in the target area. A device for determining trajectory information, characterized in that, The apparatus includes: an acquisition module, configured to acquire a first image containing a target object using a first image acquisition device, and determine first feature information of the target object based on the first image, wherein the first image acquisition device is located at a first position in a target area, and the target object is an object entering the target area; a determination module, configured to determine the identity information of the target object based on the first feature information; the acquisition module is further configured to acquire a second image containing any object using a second image acquisition device, and determine second feature information of the any object based on the second image, wherein the second image acquisition device is located at a second position in the target area; the determination module is further configured to determine the association relationship between the second feature information and the identity information, and determine the trajectory information of the target object based on the association relationship and relevant information of the first image acquisition device and the second image acquisition device. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one piece of program code, the at least one piece of program code being loaded and executed by the processor to enable the computer device to implement the method for determining trajectory information as described in any one of claims 1 to 13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to enable the computer to implement the method for determining trajectory information as described in any one of claims 1 to 13. A computer program product, characterized in that, The computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement the method for determining trajectory information as described in any one of claims 1 to 13.

Images