A cross-camera head de-duplication method based on position prediction and related device

Abstract

The application discloses a cross-camera head de-duplication method based on position prediction and related devices, and the method comprises the following steps: when starting a plurality of cameras to collect images of people in a target area, performing head attribute extraction processing on the people in the main camera image collected by the main camera to obtain the head frame attribute corresponding to the main camera image; using the head frame attribute to predict the position of the people in the field of view of the auxiliary camera based on a multi-layer perception mechanism model to obtain a prediction result; matching the prediction result with the actual position of the people in the corresponding auxiliary camera, and forming the association relationship of the people between the plurality of cameras according to the matching result; and performing head de-duplication processing based on the association relationship of the people between the plurality of cameras. In the embodiment of the application, the cross-camera head matching is realized through spatial position prediction, the repeated identification is reduced, the additional calculation overhead is avoided, and the application has high accuracy and stability.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for deduplicating heads across cameras based on location prediction. Background Technology

[0002] In multi-camera scenarios, cross-camera deduplication of people (REID) typically uses the embedding feature for similarity matching to achieve deduplication across cameras; CNN or Transformer is usually used to extract the image appearance and generate a feature vector; and the cosine similarity between targets represents the similarity between different people.

[0003] In existing multi-camera conferencing systems, the commonly used cross-camera head deduplication method is mainly face or head re-identification (Re-ID) based on appearance features (embedding features). The typical approach is to extract feature vectors from the heads detected by each camera and use similarity metrics (such as cosine distance and Euclidean distance) to determine whether they belong to the same person. This method is relatively stable in frontal face images, but the feature discrimination decreases when there are side faces, back faces, insufficient lighting, or low resolution, which can easily lead to mismatches or duplicate recognition. In addition, embedding extraction and feature comparison require a lot of computing power, which puts a certain pressure on the real-time operation of edge devices. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and related device for cross-camera head deduplication based on location prediction. It achieves cross-camera head matching through spatial location prediction, reduces repeated recognition, avoids additional computational overhead, and has high accuracy and stability.

[0005] To address at least one of the aforementioned technical problems, embodiments of the present invention provide a cross-camera head deduplication method based on location prediction, applicable to head deduplication across multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera, and the method includes: When multiple cameras are activated to capture images of people in the target area, the head attributes of the people in the main camera image are extracted to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate, and center pixel coordinates. Based on the multi-layer perception mechanism model, the head frame attribute is used to predict the position of the person in the field of view of the auxiliary camera, and the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute is obtained. The predicted results are matched with the actual positions of the people in the corresponding auxiliary cameras, and the association between the people and the cameras is formed based on the matching results. Deduplication of heads is performed based on the correlation between people across multiple cameras.

[0006] Optionally, the step of extracting head attributes from the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image includes: Based on the object detection algorithm, the head bounding box of the person in the main camera image is detected and processed to obtain the head bounding box in the main camera image; The head frame in the main camera image is subjected to head attribute extraction processing to obtain the head frame attribute corresponding to the main camera image, wherein the head pixel width is the pixel width of the head frame in the main camera image.

[0007] Optionally, the input end of the multilayer perception mechanism model consists of several input branches, which include one or more of the following: the center pixel coordinates in the head box attribute for receiving input, the depth estimate in the head box attribute for receiving input, and the head pixel width in the head box attribute for receiving input. Each of the input branches consists of a fully connected layer and a nonlinear activation layer.

[0008] Optionally, the multi-layer perception mechanism model is trained using a training set constructed from synchronized data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

[0009] Optionally, the multilayer perception mechanism model uses the head frame attribute to predict the position of the person in the field of view of the auxiliary camera, and obtains the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute, including: After the multilayer perception mechanism model receives the head box attributes, it preprocesses the input head box attributes through several input branches to form the head box attribute preprocessing result output by each input branch. The preprocessed results of the head bounding box attributes output from each input branch are concatenated along the feature dimension to form concatenated feature data. The stitched feature data is processed in several fusion layers of the multilayer perception mechanism model to predict the position of the person in the field of view of the auxiliary camera, so as to obtain the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute. The prediction result is the predicted value of the center coordinates of the person in at least one auxiliary camera.

[0010] Optionally, the step of matching the predicted result with the actual position of the person in the corresponding auxiliary camera, and forming a correlation between the person and multiple cameras based on the matching result, includes: The actual position of the person in the auxiliary camera is extracted by performing actual position extraction processing; Based on a preset error threshold, the prediction result is matched with the actual position of the person in the corresponding auxiliary camera to obtain a matching result; Based on the matching results, the association between people and multiple cameras is formed according to the overlap of heads.

[0011] Optionally, the deduplication of heads based on the correlation between people across multiple cameras includes: When the person is determined to be the same person through the correlation between multiple cameras, an overlap removal strategy is initiated or the same person identifier is assigned to maintain a consistent person trajectory.

[0012] In addition, embodiments of the present invention also provide a cross-camera head deduplication device based on location prediction, applicable to head deduplication across multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera, and the device includes: Extraction module: When multiple cameras are activated to capture images of people in a target area, the module performs head attribute extraction processing on the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate, and center pixel coordinates. Prediction module: used to predict the position of a person in the field of view of the auxiliary camera based on the head frame attribute of the multilayer perception mechanism model, and to obtain the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute. Matching module: used to match the prediction results with the actual positions of the person in the corresponding auxiliary camera, and to form the association relationship between the person and multiple cameras based on the matching results; Deduplication module: Used to deduplicat heads based on the correlation between people across multiple cameras.

[0013] In addition, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement the cross-camera head deduplication method as described in any of the above.

[0014] In addition, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program or code, which, when executed by a processor, implements the cross-camera head deduplication method as described above.

[0015] In this embodiment of the invention, the head attributes of a person in the main camera image are extracted to obtain the head bounding box attributes. The position of the person in the field of view of the auxiliary camera is predicted through a multi-layer perception mechanism model. Finally, the association relationship between the person and multiple cameras is determined by matching to achieve deduplication. The cross-camera ID deduplication accuracy is over 95%, which is about 10-15% higher than traditional embedding methods. The average error of the predicted position across cameras is less than the width of a head bounding box (about 20-30 pixels), which effectively ensures the stability of matching. Since there is no need to calculate the embedding feature, the computational load is reduced by about 50-70% compared to the traditional Re-ID algorithm. In the cases of side face, back head, and low resolution, the discrimination effect of position mapping is better than appearance-based Re-ID, which effectively reduces the repeated detection caused by feature instability. Due to the lightweight nature of the model, when deployed on edge devices, the overall system frame rate is maintained at about 3 FPS and the CPU utilization is reduced to below 60%. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the cross-camera head deduplication method based on location prediction in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structural composition of the cross-camera head deduplication device based on location prediction in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1, please refer to Figure 1 , Figure 1 This is a flowchart illustrating the cross-camera head deduplication method based on location prediction in an embodiment of the present invention.

[0020] like Figure 1 As shown, a cross-camera head deduplication method based on location prediction is applied to deduplication of heads from multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera. The method includes: S101: When multiple cameras are activated to capture images of people in the target area, the head attributes of the people in the main camera image captured by the main camera are extracted to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate and center pixel coordinates. In a specific implementation of this invention, the step of extracting head attributes from the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image includes: detecting the head frames of the people in the main camera image captured by the main camera based on a target detection algorithm to obtain the head frames in the main camera image; and extracting head attributes from the head frames in the main camera image to obtain the head frame attributes corresponding to the main camera image, wherein the head pixel width is the pixel width of the head frame in the main camera image.

[0021] Specifically, multiple cameras (one main camera and at least one auxiliary camera) are first set up outside the meeting area (target area). When multiple cameras are activated, they can cover the target area and capture images of people within the target area, thus obtaining images from the main camera and auxiliary cameras. At this point, an object detection algorithm is needed to check for people in the main camera image captured by the main camera, thereby extracting the head bounding boxes of the people in the main camera image, thus obtaining the head bounding boxes in the main camera image. The object detection algorithm can be a conventional object detection algorithm, such as using an edge extraction algorithm combined with similarity matching, or using a lightweight object detection model to perform object detection.

[0022] After extracting the head bounding box from the main camera image, the head attributes of the head bounding box are extracted to obtain the corresponding head bounding box attributes. These attributes include the head pixel width, depth estimate, and center pixel coordinates. The head pixel width is the pixel width of the head bounding box in the main camera image. The head pixel width is used to represent the relative scale of the person in the main camera's field of view and serves as one of the key features for subsequent multi-sensory mechanism model input. By using the head bounding box attributes, spatial geometric relationships can be expressed using the joint features of pixel width and depth values ​​without relying on the actual physical width estimation. This allows for the acquisition of relatively stable input features even in low-resolution or compressed videos.

[0023] S102: Based on the multilayer perception mechanism model, the head frame attribute is used to predict the position of the person in the field of view of the auxiliary camera, and the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute is obtained. In the specific implementation of this invention, the input end of the multilayer perception mechanism model consists of several input branches, which include one or more of the following: the center pixel coordinates in the head frame attribute for receiving input, the depth estimate in the head frame attribute for receiving input, and the head pixel width in the head frame attribute for receiving input; each of the several input branches consists of a fully connected layer and a nonlinear activation layer.

[0024] Furthermore, the multi-layer perception mechanism model is trained using a training set constructed from synchronous data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

[0025] Furthermore, the multilayer perception mechanism model uses the head frame attribute to predict the position of a person in the field of view of the auxiliary camera, obtaining the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute. This includes: after the multilayer perception mechanism model receives the head frame attribute, it preprocesses the input head frame attribute through several input branches to form a head frame attribute preprocessing result output by each input branch; it concatenates the head frame attribute preprocessing results output by each input branch along the feature dimension to form concatenated feature data; and it uses the concatenated feature data to predict the position of a person in the field of view of the auxiliary camera in several fusion layers of the multilayer perception mechanism model to obtain the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute. The prediction result is the predicted center coordinate value of the person in at least one auxiliary camera.

[0026] Specifically, the multilayer perception mechanism model is a multi-branch lightweight multilayer perceptron (MLP) model, used to predict the corresponding position of a person in the auxiliary camera based on the head attributes extracted from the main camera. The input of the multilayer perception mechanism model consists of several input branches, which include one or more of the following: the center pixel coordinates of the head bounding box attribute, the depth estimate of the head bounding box attribute, and the head pixel width of the head bounding box attribute. The center pixel coordinates represent the relative position of the person in the main camera image plane; the depth estimate represents the relative distance between the person and the main camera; and the head pixel width represents the relative scale of the person in the field of view of the main camera. Input branch one: The center pixel coordinates of the head detection box in the main camera. This is used to characterize the relative position of a person in the main camera image plane; the input has two branches: the depth estimate from the main camera. This is used to reflect the relative distance between the person and the camera; Input branch three: Input the pixel width of the head detection box. The input branches are used to represent the relative scale of the figures; all of the above input branches have been preprocessed to eliminate the influence of different resolutions and scene conditions.

[0027] Each input branch can consist of several fully connected layers and non-linear activation layers (such as LeakyReLU) to extract its own features; the outputs of each branch are concatenated along the feature dimension and further processed through several fusion layers to learn the spatial mapping relationship between the main and side cameras; the final output is the predicted coordinates of the person's center in the side camera. .

[0028] The multilayer perceptron model possesses the following technical characteristics: 1. Lightweight: The total number of network parameters is less than 1M, enabling real-time operation at edge devices (such as RKNN and NPU devices); 2. Branch decoupling structure: Different input dimensions are modeled through independent branches, improving the stability of feature representation; 3. Nonlinear cross-fusion: Feature product terms (e.g., ...) are used in the feature concatenation stage. To enhance the correlation between depth, head width, and position, thereby improving prediction accuracy; the multi-layer perception mechanism model is trained on a multi-camera synchronous dataset to learn the parallax relationship and geometric mapping between the main and side cameras, achieving accurate prediction of the position of people at different distances and angles.

[0029] Among them, the multilayer perception mechanism model is trained using a data training set constructed from synchronous data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

[0030] The head bounding box attributes are input into the multilayer perceptron model. After receiving the head bounding box attributes, the model preprocesses them through several input branches, forming the preprocessed head bounding box attribute output of each input branch. The preprocessed head bounding box attribute outputs of each input branch are then concatenated along the feature dimension to form concatenated feature data. Finally, the concatenated feature data is used in several fusion layers of the multilayer perceptron model to predict the position of the person in the field of view of the auxiliary camera, obtaining the predicted position of the person in the field of view of the auxiliary camera corresponding to the head bounding box attributes. The predicted result is the predicted center coordinate value of the person in at least one auxiliary camera.

[0031] S103: Match the prediction result with the actual position of the person in the corresponding auxiliary camera, and form the association relationship between the person and multiple cameras based on the matching result; In a specific implementation of this invention, the step of matching the prediction result with the actual position of the person in the corresponding auxiliary camera and forming a relationship between the person and multiple cameras based on the matching result includes: extracting the actual position of the person in the auxiliary camera to obtain the actual position of the person in the auxiliary camera; matching the prediction result with the actual position of the person in the corresponding auxiliary camera based on a preset error threshold to obtain a matching result; and forming a relationship between the person and multiple cameras based on the matching result according to the head overlap relationship.

[0032] Specifically, a target extraction algorithm is used to extract people from the images captured by the auxiliary camera, thereby extracting the actual position of the people in the auxiliary camera image. Then, the prediction results are matched with the actual positions of the people in the corresponding auxiliary cameras by using a preset error threshold to obtain the matching results. Finally, the association relationship between people in multiple cameras is formed according to the head overlap relationship based on the matching results.

[0033] The predicted results are matched with the actual positions of the auxiliary cameras to establish the correlation between people in different cameras. The prediction error range (usually within the size of a head frame) is used as a threshold during matching to ensure high-confidence matching. For cases of overlapping heads, the following algorithm is used: 1. Front and back overlap: When two people are in front and back positions with a large depth difference, the pixel width of the head frame of the person in front is significantly larger than that of the person behind. The main camera detection module only detects the head frame of the person in front, so the corresponding position of the person in front will naturally be matched in the auxiliary camera. 2. Left and right overlap: When two people are in left and right positions, the MLP model has learned the left and right correspondence rules through large-scale training with main and auxiliary cameras. It can accurately predict the position of the corresponding auxiliary camera based on the relationship between the head coordinates and width in the main camera, thereby achieving stable matching.

[0034] Since the algorithm only considers unobstructed heads in the main camera view, the larger the field of view of the main camera, the better the overall algorithm effect.

[0035] S104: Deduplicat head processing based on the correlation between people in multiple cameras.

[0036] In the specific implementation of this invention, the deduplication of heads based on the association relationship between multiple cameras includes: when the person is determined to be the same person through the association relationship between multiple cameras, an overlap removal strategy is initiated or the same person identifier is assigned to retain a unified person trajectory.

[0037] Specifically, the deduplication strategy determines that when a person is identified as the same person by multiple cameras, the system will automatically remove duplicate IDs or assign the same person identifier, thus retaining only the unified person trajectory and achieving cross-camera deduplication. Since the multi-layer perception mechanism model is a lightweight model (small parameter data and low computational overhead), it can run in real time on edge devices such as RKNN without affecting the real-time performance and synchronization performance of the overall system.

[0038] The difference between the implementation method in this embodiment and the existing technical solution is that: 1. Using a panoramic camera (main camera) as a reference benchmark, instead of simply projecting and mapping between the main and auxiliary cameras, the panoramic main camera in the conference room (providing a more comprehensive view of the participants' positions) serves as a unified benchmark for cross-camera deduplication. The head detection bounding box, depth information, and coordinate information from the panoramic lens are used as input features to predict the corresponding positions in the auxiliary camera. This approach, unlike existing embedding-based Re-ID or traditional geometric calibration methods, reduces mismatches and unifies IDs across cameras.

[0039] 2. A lightweight Multilayer Perceptron (MLP) model is introduced. Instead of using traditional geometric projection or complex camera calibration matrices to calculate cross-camera positions, this embodiment uses a lightweight MLP model to learn the nonlinear mapping relationship between the main camera and the auxiliary camera. This "nonlinear mapping relationship" refers to the complex spatial correspondence between the main camera and the auxiliary camera in a real-world installation environment due to factors such as parallax, angular shift, perspective distortion, lens distortion, and changes in subject depth. This relationship is difficult to accurately express using linear matrices or analytical models. This embodiment uses an MLP model composed of multiple linear layers and nonlinear activation functions (e.g., ReLU, GELU, or SiLU). The introduction of nonlinear activation functions gives the model a general function approximation capability, enabling efficient fitting of the complex nonlinear mapping between the main camera and the auxiliary camera, thus maintaining high prediction accuracy under different camera arrangements and subject poses. Using the head attributes (including pixel coordinates, pixel width, and depth estimate) from the main camera as input, the model undergoes multiple nonlinear transformations to output the predicted center position of the corresponding auxiliary camera. The MLP model is trained through supervised learning, with the optimization objective of minimizing the spatial error between the predicted position and the actual detection result, thereby automatically learning the implicit geometric relationship between the two cameras. This embodiment uses a neural network structure for automatic modeling, without relying on precise calibration matrices or geometric assumptions, and possesses high robustness, strong adaptability, and low computational complexity. It is particularly suitable for edge deployment environments with large installation errors of multiple cameras or dynamically changing scenes. The input features mainly include: head width, depth value, pixel coordinates, etc. The output is the predicted center position of the head of the auxiliary camera. Moreover, the MLP model has a small number of parameters and high computational efficiency, and can run in real time on edge devices such as RKNN.

[0040] Compared with existing embedding-based cross-camera deduplication methods, this embodiment achieves the following improvements: 1. Improved deduplication accuracy: In multi-camera scenario tests with 5-10 people in a conference room, the cross-camera ID deduplication accuracy reached over 95%, which is about 10-15% higher than the traditional embedding method.

[0041] 2. High prediction accuracy: the average error of the cross-camera prediction position is less than the width of a human head frame (about 20-30 pixels), which effectively ensures the stability of the matching.

[0042] 3. Improved computational efficiency: Since there is no need to calculate embedding features, this solution reduces the amount of computation by about 50-70% compared to the traditional Re-ID algorithm; when running on RKNN edge devices, the overall system frame rate is maintained at around 3 FPS, and the CPU utilization rate is reduced to below 60%.

[0043] 4. Enhanced robustness: In cases of side profiles, back views, and low resolution, the discrimination performance of position mapping is superior to appearance-based Re-ID, effectively reducing duplicate detections caused by feature instability.

[0044] Example 2, please refer to Figure 2 , Figure 2 This is a schematic diagram of the structural composition of the cross-camera head deduplication device based on location prediction in an embodiment of the present invention.

[0045] like Figure 2 As shown, a cross-camera head deduplication device based on location prediction is applied to deduplication of heads from multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera, and the device includes: Extraction module 201: When multiple cameras are activated to capture images of people in a target area, the module performs head attribute extraction processing on the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate, and center pixel coordinates. In a specific implementation of this invention, the step of extracting head attributes from the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image includes: detecting the head frames of the people in the main camera image captured by the main camera based on a target detection algorithm to obtain the head frames in the main camera image; and extracting head attributes from the head frames in the main camera image to obtain the head frame attributes corresponding to the main camera image, wherein the head pixel width is the pixel width of the head frame in the main camera image.

[0046] Specifically, multiple cameras (one main camera and at least one auxiliary camera) are first set up outside the meeting area (target area). When multiple cameras are activated, they can cover the target area and capture images of people within the target area, thus obtaining images from the main camera and auxiliary cameras. At this point, an object detection algorithm is needed to check for people in the main camera image captured by the main camera, thereby extracting the head bounding boxes of the people in the main camera image, thus obtaining the head bounding boxes in the main camera image. The object detection algorithm can be a conventional object detection algorithm, such as using an edge extraction algorithm combined with similarity matching, or using a lightweight object detection model to perform object detection.

[0047] After extracting the head bounding box from the main camera image, the head attributes of the head bounding box are extracted to obtain the corresponding head bounding box attributes. These attributes include the head pixel width, depth estimate, and center pixel coordinates. The head pixel width is the pixel width of the head bounding box in the main camera image. The head pixel width is used to represent the relative scale of the person in the main camera's field of view and serves as one of the key features for subsequent multi-sensory mechanism model input. By using the head bounding box attributes, spatial geometric relationships can be expressed using the joint features of pixel width and depth values ​​without relying on the actual physical width estimation. This allows for the acquisition of relatively stable input features even in low-resolution or compressed videos.

[0048] Prediction module 202: is used to predict the position of a person in the field of view of the auxiliary camera based on the head frame attribute of the multilayer perception mechanism model, and obtain the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute. In the specific implementation of this invention, the input end of the multilayer perception mechanism model consists of several input branches, wherein the several input branches include one or more of the following: the center pixel coordinates in the head box attribute for receiving input, the depth estimate in the head box attribute for receiving input, and the head pixel width in the head box attribute for receiving input; each of the several input branches consists of a fully connected layer and a nonlinear activation layer.

[0049] Furthermore, the multi-layer perception mechanism model is trained using a training set constructed from synchronous data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

[0050] Furthermore, the multilayer perception mechanism model uses the head frame attribute to predict the position of a person in the field of view of the auxiliary camera, obtaining the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute. This includes: after the multilayer perception mechanism model receives the head frame attribute, it preprocesses the input head frame attribute through several input branches to form a head frame attribute preprocessing result output by each input branch; it concatenates the head frame attribute preprocessing results output by each input branch along the feature dimension to form concatenated feature data; and it uses the concatenated feature data to predict the position of a person in the field of view of the auxiliary camera in several fusion layers of the multilayer perception mechanism model to obtain the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute. The prediction result is the predicted center coordinate value of the person in at least one auxiliary camera.

[0051] Specifically, the multilayer perception mechanism model is a multi-branch lightweight multilayer perceptron (MLP) model, used to predict the corresponding position of a person in the auxiliary camera based on the head attributes extracted from the main camera. The input of the multilayer perception mechanism model consists of several input branches, which include one or more of the following: the center pixel coordinates of the head bounding box attribute, the depth estimate of the head bounding box attribute, and the head pixel width of the head bounding box attribute. The center pixel coordinates represent the relative position of the person in the main camera image plane; the depth estimate represents the relative distance between the person and the main camera; and the head pixel width represents the relative scale of the person in the field of view of the main camera. Input branch one: The center pixel coordinates of the head detection box in the main camera. This is used to represent the relative position of the person in the main camera image plane; Input branch two: Input the depth estimate of the main camera. This is used to reflect the relative distance between the person and the camera; Input branch three: Input the pixel width of the head detection box. The input branches are used to represent the relative scale of the figures; all of the above input branches have been preprocessed to eliminate the influence of different resolutions and scene conditions.

[0052] Each input branch can consist of several fully connected layers and non-linear activation layers (such as LeakyReLU) to extract its own features; the outputs of each branch are concatenated along the feature dimension and further processed through several fusion layers to learn the spatial mapping relationship between the main and auxiliary cameras; the final output is the predicted coordinates of the person's center in the auxiliary camera. .

[0053] The multilayer perceptron model possesses the following technical characteristics: 1. Lightweight: The total number of network parameters is less than 1M, enabling real-time operation at edge devices (such as RKNN and NPU devices); 2. Branch decoupling structure: Different input dimensions are modeled through independent branches, improving the stability of feature representation; 3. Nonlinear cross-fusion: Feature product terms (e.g., ...) are used in the feature concatenation stage. To enhance the correlation between depth, head width, and position, thereby improving prediction accuracy; the multi-layer perception mechanism model is trained on a multi-camera synchronous dataset to learn the parallax relationship and geometric mapping between the main and auxiliary cameras, achieving accurate prediction of the position of people at different distances and angles.

[0054] Among them, the multilayer perception mechanism model is trained using a data training set constructed from synchronous data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

[0055] The head bounding box attributes are input into the multilayer perceptron model. After receiving the head bounding box attributes, the model preprocesses them through several input branches, forming the preprocessed head bounding box attribute output of each input branch. The preprocessed head bounding box attribute outputs of each input branch are then concatenated along the feature dimension to form concatenated feature data. Finally, the concatenated feature data is used in several fusion layers of the multilayer perceptron model to predict the position of the person in the field of view of the auxiliary camera, obtaining the predicted position of the person in the field of view of the auxiliary camera corresponding to the head bounding box attributes. The predicted result is the predicted center coordinate value of the person in at least one auxiliary camera.

[0056] Matching module 203: is used to match the prediction result with the actual position of the person in the corresponding auxiliary camera, and form the association relationship between the person and multiple cameras based on the matching result; In a specific implementation of this invention, the step of matching the prediction result with the actual position of the person in the corresponding auxiliary camera and forming a relationship between the person and multiple cameras based on the matching result includes: extracting the actual position of the person in the auxiliary camera to obtain the actual position of the person in the auxiliary camera; matching the prediction result with the actual position of the person in the corresponding auxiliary camera based on a preset error threshold to obtain a matching result; and forming a relationship between the person and multiple cameras based on the matching result according to the head overlap relationship.

[0057] Specifically, a target extraction algorithm is used to extract people from the images captured by the auxiliary camera, thereby extracting the actual position of the people in the auxiliary camera image. Then, the prediction results are matched with the actual positions of the people in the corresponding auxiliary cameras by using a preset error threshold to obtain the matching results. Finally, the association relationship between people in multiple cameras is formed according to the head overlap relationship based on the matching results.

[0058] The predicted results are matched with the actual positions of the auxiliary cameras to establish the correlation between people in different cameras. The prediction error range (usually within the size of a head frame) is used as a threshold during matching to ensure high-confidence matching. For cases of overlapping heads, the following algorithm is used: 1. Front and back overlap: When two people are in front and back positions with a large depth difference, the pixel width of the head frame of the person in front is significantly larger than that of the person behind. The main camera detection module only detects the head frame of the person in front, so the corresponding position of the person in front will naturally be matched in the auxiliary camera. 2. Left and right overlap: When two people are in left and right positions, the MLP model has learned the left and right correspondence rules through large-scale training with main and auxiliary cameras. It can accurately predict the position of the corresponding auxiliary camera based on the relationship between the head coordinates and width in the main camera, thereby achieving stable matching.

[0059] Since the algorithm only considers unobstructed heads in the main camera view, the larger the field of view of the main camera, the better the overall algorithm effect.

[0060] Deduplication module 204: Used to perform deduplication of heads based on the correlation between people in multiple cameras.

[0061] In the specific implementation of this invention, the deduplication of heads based on the association relationship between multiple cameras includes: when the person is determined to be the same person through the association relationship between multiple cameras, an overlap removal strategy is initiated or the same person identifier is assigned to retain a unified person trajectory.

[0062] Specifically, the deduplication strategy determines that when a person is identified as the same person by multiple cameras, the system will automatically remove duplicate IDs or assign the same person identifier, thus retaining only the unified person trajectory and achieving cross-camera deduplication. Since the multi-layer perception mechanism model is a lightweight model (small parameter data and low computational overhead), it can run in real time on edge devices such as RKNN without affecting the real-time performance and synchronization performance of the overall system.

[0063] The difference between the implementation method in this embodiment and the existing technical solution is that: 1. Using a panoramic camera (main camera) as a reference benchmark, instead of simply projecting and mapping between the main and auxiliary cameras, the panoramic main camera in the conference room (providing a more comprehensive view of the participants' positions) serves as a unified benchmark for cross-camera deduplication. The head detection bounding box, depth information, and coordinate information from the panoramic lens are used as input features to predict the corresponding positions in the auxiliary camera. This approach, unlike existing embedding-based Re-ID or traditional geometric calibration methods, reduces mismatches and unifies IDs across cameras.

[0064] 2. A lightweight Multilayer Perceptron (MLP) model is introduced. Instead of using traditional geometric projection or complex camera calibration matrices to calculate cross-camera positions, this embodiment uses a lightweight MLP model to learn the nonlinear mapping relationship between the main camera and the auxiliary camera. This "nonlinear mapping relationship" refers to the complex spatial correspondence between the main camera and the auxiliary camera in a real-world installation environment due to factors such as parallax, angular shift, perspective distortion, lens distortion, and changes in subject depth. This relationship is difficult to accurately express using linear matrices or analytical models. This embodiment uses an MLP model composed of multiple linear layers and nonlinear activation functions (e.g., ReLU, GELU, or SiLU). The introduction of nonlinear activation functions gives the model a general function approximation capability, enabling efficient fitting of the complex nonlinear mapping between the main camera and the auxiliary camera, thus maintaining high prediction accuracy under different camera arrangements and subject poses. Using the head attributes (including pixel coordinates, pixel width, and depth estimate) from the main camera as input, the model undergoes multiple nonlinear transformations to output the predicted center position of the corresponding auxiliary camera. The MLP model is trained through supervised learning, with the optimization objective of minimizing the spatial error between the predicted position and the actual detection result, thereby automatically learning the implicit geometric relationship between the two cameras. This embodiment uses a neural network structure for automatic modeling, without relying on precise calibration matrices or geometric assumptions, and possesses high robustness, strong adaptability, and low computational complexity. It is particularly suitable for edge deployment environments with large installation errors of multiple cameras or dynamically changing scenes. The input features mainly include: head width, depth value, pixel coordinates, etc. The output is the predicted center position of the head of the auxiliary camera. Moreover, the MLP model has a small number of parameters and high computational efficiency, and can run in real time on edge devices such as RKNN.

[0065] Compared with existing embedding-based cross-camera deduplication methods, this embodiment achieves the following improvements: 1. Improved deduplication accuracy: In multi-camera scenario tests with 5-10 people in a conference room, the cross-camera ID deduplication accuracy reached over 95%, which is about 10-15% higher than the traditional embedding method.

[0066] 2. High prediction accuracy: the average error of the cross-camera prediction position is less than the width of a human head frame (about 20-30 pixels), which effectively ensures the stability of the matching.

[0067] 3. Improved computational efficiency: Since there is no need to calculate embedding features, this solution reduces the amount of computation by about 50-70% compared to the traditional Re-ID algorithm; when running on RKNN edge devices, the overall system frame rate is maintained at around 3 FPS, and the CPU utilization rate is reduced to below 60%.

[0068] 4. Enhanced robustness: In cases of side profiles, back views, and low resolution, the discrimination performance of position mapping is superior to appearance-based Re-ID, effectively reducing duplicate detections caused by feature instability.

[0069] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the cross-camera head deduplication method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0070] This invention also provides a computer application running on a computer, which is used to perform the cross-camera head deduplication method of any of the above embodiments.

[0071] also, Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0072] This invention also provides an electronic device, such as... Figure 3 As shown. The electronic device includes a processor 302, a memory 303, an input unit 304, and a display unit 305, among other devices. Those skilled in the art will understand that... Figure 3The structural components of the illustrated electronic device do not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 303 can be used to store application program 301 and various functional modules. Processor 302 runs application program 301 stored in memory 303, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. The memory disclosed in this invention includes, but is not limited to, these types of memory. The memory disclosed in this invention is only an example and not a limitation.

[0073] Input unit 304 is used to receive signal input and user-input keywords. Input unit 304 may include a touch panel and other input devices. The touch panel can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel) and drive the corresponding connection device according to a pre-set program; other input devices may include, but are not limited to, one or more of physical keyboards, function keys (such as play control buttons, power buttons, etc.), trackballs, mice, joysticks, etc. Display unit 305 can be used to display user-input information or information provided to the user, as well as various menus of the terminal device. Display unit 305 may be in the form of a liquid crystal display, organic light-emitting diode, etc. Processor 302 is the control center of the terminal device, connecting various parts of the entire device through various interfaces and lines, and performing various functions and processing data by running or executing software programs and / or modules stored in memory 303, and calling data stored in memory.

[0074] As one embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more applications 301, wherein the one or more applications 301 are stored in the memory 303 and configured to be executed by the one or more processors 302, and the one or more applications 301 are configured to perform the cross-camera head deduplication method corresponding to any of the embodiments described above.

[0075] In this embodiment of the invention, the head attributes of a person in the main camera image are extracted to obtain the head bounding box attributes. The position of the person in the field of view of the auxiliary camera is predicted through a multi-layer perception mechanism model. Finally, the association relationship between the person and multiple cameras is determined by matching to achieve deduplication. The cross-camera ID deduplication accuracy is over 95%, which is about 10-15% higher than traditional embedding methods. The average error of the predicted position across cameras is less than the width of a head bounding box (about 20-30 pixels), which effectively ensures the stability of matching. Since there is no need to calculate the embedding feature, the computational load is reduced by about 50-70% compared to the traditional Re-ID algorithm. In the cases of side face, back head, and low resolution, the discrimination effect of position mapping is better than appearance-based Re-ID, which effectively reduces the repeated detection caused by feature instability. Due to the lightweight nature of the model, when deployed on edge devices, the overall system frame rate is maintained at about 3 FPS and the CPU utilization is reduced to below 60%.

[0076] Furthermore, the above provides a detailed description of a cross-camera head deduplication method and related apparatus based on location prediction provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for deduplicating heads across cameras based on location prediction, characterized in that, A method for deduplicating heads using multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera, the method comprising: When multiple cameras are activated to capture images of people in the target area, the head attributes of the people in the main camera image are extracted to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate, and center pixel coordinates. Based on the multi-layer perception mechanism model, the head frame attribute is used to predict the position of the person in the field of view of the auxiliary camera, and the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute is obtained. The predicted results are matched with the actual positions of the people in the corresponding auxiliary cameras, and the association between the people and the cameras is formed based on the matching results. Deduplication of heads is performed based on the correlation between people across multiple cameras.

2. The method for deduplicating heads across cameras according to claim 1, characterized in that, The step of extracting head attributes from the images captured by the main camera to obtain the head bounding box attributes corresponding to the main camera image includes: Based on the object detection algorithm, the head bounding box of the person in the main camera image is detected and processed to obtain the head bounding box in the main camera image; The head frame in the main camera image is subjected to head attribute extraction processing to obtain the head frame attribute corresponding to the main camera image, wherein the head pixel width is the pixel width of the head frame in the main camera image.

3. The method for deduplicating heads across cameras according to claim 1, characterized in that, The input end of the multilayer perception mechanism model consists of several input branches, which include one or more of the following: the center pixel coordinates in the head box attribute for receiving input, the depth estimate in the head box attribute for receiving input, and the head pixel width in the head box attribute for receiving input. Each of the input branches consists of a fully connected layer and a nonlinear activation layer.

4. The method for deduplicating heads across cameras according to claim 3, characterized in that, The multi-layer perception mechanism model is trained using a training set constructed from synchronous data from multiple cameras to achieve a convergent model that learns the disparity relationship and geometric mapping relationship between the main camera and at least one auxiliary camera.

5. The method for deduplicating heads across cameras according to claim 3, characterized in that, The multilayer perception mechanism-based model uses the head frame attribute to predict the position of a person in the field of view of the auxiliary camera, obtaining the prediction result of the person's position in the field of view of the auxiliary camera corresponding to the head frame attribute, including: After the multilayer perception mechanism model receives the head box attributes, it preprocesses the input head box attributes through several input branches to form the head box attribute preprocessing result output by each input branch. The preprocessed results of the head bounding box attributes output from each input branch are concatenated along the feature dimension to form concatenated feature data. The stitched feature data is processed in several fusion layers of the multilayer perception mechanism model to predict the position of the person in the field of view of the auxiliary camera, so as to obtain the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute. The prediction result is the predicted value of the center coordinates of the person in at least one auxiliary camera.

6. The method for deduplicating heads across cameras according to claim 1, characterized in that, The step of matching the predicted results with the actual positions of the person in the corresponding auxiliary cameras, and forming a correlation between the person and multiple cameras based on the matching results, includes: The actual position of the person in the auxiliary camera is extracted by performing actual position extraction processing; Based on a preset error threshold, the prediction result is matched with the actual position of the person in the corresponding auxiliary camera to obtain a matching result; Based on the matching results, the association between people and multiple cameras is formed according to the overlap of heads.

7. The method for deduplicating heads across cameras according to claim 1, characterized in that, The deduplication of heads based on the correlation between people across multiple cameras includes: When the person is determined to be the same person through the correlation between multiple cameras, an overlap removal strategy is initiated or the same person identifier is assigned to maintain a consistent person trajectory.

8. A cross-camera head deduplication device based on location prediction, characterized in that, A device for deduplicating heads using multiple cameras, wherein the multiple cameras include at least one main camera and at least one auxiliary camera, the device comprising: Extraction module: When multiple cameras are activated to capture images of people in a target area, the module performs head attribute extraction processing on the people in the main camera image captured by the main camera to obtain the head frame attributes corresponding to the main camera image. The head frame attributes include the head pixel width, depth estimate, and center pixel coordinates. Prediction module: used to predict the position of a person in the field of view of the auxiliary camera based on the head frame attribute of the multilayer perception mechanism model, and to obtain the prediction result of the position of the person in the field of view of the auxiliary camera corresponding to the head frame attribute. Matching module: used to match the prediction results with the actual positions of the person in the corresponding auxiliary camera, and to form the association relationship between the person and multiple cameras based on the matching results; Deduplication module: Used to deduplicat heads based on the correlation between people across multiple cameras.

9. An electronic device comprising a processor and a memory, characterized in that, The processor runs a computer program or code stored in the memory to implement the cross-camera head deduplication method as described in any one of claims 1 to 7.

10. A computer-readable storage medium for storing computer programs or code, characterized in that, When the computer program or code is executed by a processor, the method for deduplicating heads across cameras as described in any one of claims 1 to 7 is implemented.

Images