Complementary dynamic perspective camera coordination method based on polar coordinate transformation

Abstract

The application discloses a kind of complementary dynamic visual angle camera coordination methods based on polar coordinate transformation, including steps: S1, target detection network, camera positioning network and cross-view target association network training;S2, top view and side view are applied to target detection network to obtain two view target positions;S3, human detection module is generated according to two view target positions Calculation generates two-view heat map;S4, according to the position of the camera positioning network identified by the photographer to the origin of the side view photographer Polar coordinate transformation is carried out on the view heat map;S5, according to the heat map after polar coordinate transformation Application search network of viewing direction determines side view angle;S6, based on the target position identified in step S3 and the side view angle determined in step S5, calculate target similarity matrix and input cross-view target association network to obtain matching matrix output, end, the application is measured with tracking algorithm Similarity of cross-view target, effectively ensure the accuracy of measurement result and the performance of cross-view association.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of complementary view collaborative analysis, and particularly relates to a complementary dynamic view camera collaboration method based on polar coordinate transformation. BACKGROUND

[0002] The background technology involved in the present application is as follows:

[0003] (1) CenterNet (see document [1]): CenterNet is a key point-based target detection network that explores the visual pattern in each crop box at the minimum cost, thereby solving the problem that the key point-based method in target detection cannot obtain correct bounding boxes. CenterNet improves the previous one-stage CornerNet, and regards each detection target as a triplet, instead of a pair of key points, thereby significantly improving the precision and recall rate. In the present method, a human body detection module is applied by using a similar network of CenterNet, so as to detect the positioning of all persons in the top view and the side view respectively.

[0004] (2) Bidirectional recurrent neural network (see document [2]): The bidirectional recurrent neural network (BRNN) extends each training sequence into two recurrent neural networks (RNN) in the forward and backward directions respectively, and each is connected with an output layer. This structure provides the output layer with complete past and future context information of each point in the input sequence, thereby enabling the network to be trained in both time directions. In the present method, the BRNN architecture is used to construct the association subnetwork.

[0005] (3) Hungarian algorithm (see document [8]): The Hungarian algorithm is a combinatorial optimization algorithm that uses an augmented path to solve the task allocation problem in polynomial time. In the present method, for the target matching task, the Hungarian algorithm is applied on the predicted matching matrix M to convert the output into a binary assignment matrix A as the final target association result. SUMMARY

[0006] The purpose of the present application is to realize the collaborative analysis of complementary view videos by associating the contents in the top view and the side view. The realization of the present method is based on three basic tasks, i.e., camera wearer positioning, observation direction search and complementary view multi-person human body detection and association. The present application combines spatial perception position representation and cross-view video collaboration, and completes joint camera recognition and target association by simultaneously processing the following three tasks: i) positioning the head-up camera in the top view; ii) determining the field of view direction of the head-up camera; iii) detecting and associating the same target in the complementary view.

[0007] In view of the above technical problems, the technical scheme adopted by the present application is: a spatial perception framework for complementary mobile camera identification and target association, which is mainly explained from two aspects: one is the effective response strategy adopted for actual problems, and the other is the detection and association implementation process based on the above strategy, specifically.

[0008] A complementary dynamic view camera cooperation method based on polar coordinate transformation, which detects all human targets in the top view and side view respectively by using a human body detection module, uses a spatial perception position representation to embed the spatial position distribution of objects in different views, and applies polar coordinate transformation to bridge the view gap between the top view and the side view; comprising the following steps:

[0009] S1, training the target detection network, camera positioning network by the top view and side view data set of target position annotation, side view camera position, side view angle and matching matrix annotation; at the same time, training the cross-view target association network by the result of target position annotation and the annotation of matching matrix;

[0010] S2, applying the top view and side view to the target detection network to obtain the target position of the two views;

[0011] S3, the human body detection module calculates and generates two-view heat maps according to the target position of the two views;

[0012] S4, identifying the position of the photographer according to the camera positioning network, and performing polar coordinate transformation on the view heat map with the side view photographer as the origin;

[0013] S5, determining the side view angle according to the heat map after polar coordinate transformation;

[0014] S6, based on the target position identified in step S3 and the side view angle determined in step S5, calculating the target similarity matrix and inputting the cross-view target association network to obtain the matching matrix output, and ending.

[0015] Further, the S1 step trains the target detection network and the camera positioning network by the top view and side view data set of human target detection, side view camera position and angle; at the same time, the cross-view target association network is trained by the annotation of the cross-view target matching matrix;

[0016] 101, training the target detection network;

[0017] 102, training the camera positioning network, wherein: the camera positioning network needs to complete the positioning of the side view camera position, the top view, and the estimation of the camera observation direction at the same time; comprising the following steps:

[0018] calculating the correlation score of the compressed top view heat map and the compressed side view heat map

[0019] where * denotes convolution operation, denotes the part of the compressed top-down heat map f t cut by a sliding window with width w, W is the width of f t , s∈[1,W] is the left boundary of the sliding window; when s≥W-w, the left region of f t is used to supplement to the right of the right boundary of f t , obtaining the correlation score corresponding to each observation direction

[0020] 103, the observation direction in the camera positioning network is supervised training by the following formula;

[0021]

[0022] where: r is the score vector of each observation direction predicted in step 102, the length is W, r gt is the score vector of each observation direction determined by the real observation direction, the length is also W, the real observation direction score is 1, and the rest is in turn according to the deviation from the real direction;

[0023] 104, the camera position in the positioning network is supervised learning by the following formula,

[0024]

[0025] where: τ is a preset parameter, f s is the compressed side view heat map, f t (o) is the compressed top-down heat map obtained by polar coordinate transformation and accumulation with the correct position as the origin, f t (o′) is the compressed top-down heat map obtained by polar coordinate transformation and accumulation with the wrong position as the origin;

[0026] 105, the training of cross-view association network; including the following steps:

[0027] According to the detection results of the top view target P i (i∈{1,2,…,M}) and the side view target Q i (i∈{1,2,…,N}), the target distance in x-axis direction is calculated

[0028] where: denotes the normalized x-axis coordinate of target P i ,Q j , D is a distance measurement function to obtain the similarity matrix of all targets between two views

[0029] 106. Calculate the target distance along the y-axis using the method described in step 105, and use the depth of the photographed target to represent the distance between the target and the camera in the side view; Indicates target P i The normalized y-axis coordinate, d(Q) j ) represents the target Q j The similarity matrix S is calculated using the same method as in step 105, based on the depth of the similarity matrix. y ;

[0030] 107. A bidirectional RNN architecture is adopted as the basic structure of the cross-view target association network: the similarity matrix S is arranged in row order. x S y Reshape it into a vector and feed it back into the first BiRNN network, using the similarity matrix S. x S y The vector is reshaped in column order and fed back into a second BiRNN network; three fully connected layers and a sigmoid function are applied to obtain the resulting matching matrix M. x (or M) y );

[0031] 108. Construct a cross-view target association network using supervised training based on the following formula:

[0032]

[0033] Where: M x M y It is the predicted matching matrix, M gt Given the actual matching matrix, the matrix cross-entropy loss function L is used. cro To measure the consistency between two matrices.

[0034] Furthermore, in step S4, the camera positioning network identifies the photographer's position and performs polar coordinate transformation on the viewpoint heatmap with the side-view photographer as the origin.

[0035] 301. Input the top-view image and a side-view image into the pedestrian target detection module to generate a top-view target distribution heatmap F. t Top view image target set Heat map of target distribution from the side view F s ;

[0036] 302. Based on the top-view target distribution heat map F t Select the origin of the coordinate system (c x ,c y Then, a polar coordinate transformation is performed in an arbitrary direction to obtain the transformed top-view heat map. Right now such that where w, h are the width and height of F t , W, H are the width and height of F , here we set H = h, W = kw, hyper-parameter k >= 1;

[0037] 303, according to the principle of polar coordinate expansion, the vertical direction (side view observation field of view) in the side view heat map F s corresponds to the vertical direction (360-degree field of view) in the polar coordinate expansion F , and combined with the side view observation range theta, the side view heat map can be regarded as the part of F ;

[0038] 304, in order to simulate the occlusion phenomenon of the side view angle, we accumulate along the y-axis direction according to F to obtain the compressed top view heat map f t ; at the same time, the side view target distribution heat map F is value-added along the y-axis direction to obtain the compressed side view heat map F

[0039] Advantages

[0040] The present application proposes a spatial perception position representation method to embed the spatial distribution of the target, and independently designs a camera recognition and target matching network to simultaneously perform three basic tasks of collaborative analysis of complementary view video. It has strong practicability and accurate results, and has the following advantages:

[0041] a. Strong practicability. Compared with other methods of top view camera and side view camera collaborative analysis (see documents [3][4][5][6][7]), this method deletes two assumptions that are usually not met in reality: 1) the top view camera has a certain tilt angle to realize the partial visibility of the human body and use appearance matching for lateral correlation; and 2) the direction of the side view camera is the same as the moving direction of the camera wearer. Therefore, it has stronger practical applicability.

[0042] b. High accuracy. This method avoids using exhaustive search strategy to realize cross-view target similarity measurement, and uses human body detection and tracking algorithm results as auxiliary input. Therefore, the accuracy of the measurement can be guaranteed to a certain extent.

[0043] c.The main idea of the method is to explore the spatial location layout of the target object in two views. All human targets in the top view and side view are detected by the human body detection module, respectively. Based on the detection results, the spatial perception position representation is used to embed the spatial position distribution of the objects in different views, and the polar coordinate transformation is applied to bridge the view gap between the top view and the side view. Based on the above spatial perception position representation, the camera recognition module and the target association module are combined to find the position of the side view camera and its viewing angle in the top view at the same time, so as to complete the target matching in the two views. The method does not require the top view camera to have a certain tilt angle, nor does it require the direction of the side view camera to be the same as the moving direction of the camera wearer, so it has strong practical applicability; in addition, the method avoids using human body detection results as input, and assists in measuring the similarity of cross-view targets by using a tracking algorithm, effectively ensuring the accuracy of the measurement results and the performance of cross-view association. The method can effectively complete the basic task of multi-person scene analysis in complementary views, and has the advantages of strong practical applicability, high measurement accuracy and excellent performance. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 : top view (a) and side view (b) illustration;

[0045] Figure 2 : spatial perception depth framework structure diagram. DETAILED DESCRIPTION

[0046] The present application is based on top view and side view, and performs side view camera wearer positioning, observation direction search, and cross-complementary view multi-person human body detection and association.

[0047] As shown in Figure 1 , Figure 2 , the present application provides a complementary dynamic view camera cooperative method based on polar coordinate transformation to overcome the technical problems existing in the prior art:

[0048] (1) Effective coping strategy:

[0049] a) Associate first-person and third-person views: usually, the third-person view has a self-centered view or a certain tilt angle of the monitoring view, and its observation height and angle are similar to those of the first-person camera. The difference is that in this task, the third-person camera is installed on a drone and produces a top view image, so that the field of view of the two views overlaps very limitedly. In the present application, cross-complementary view multi-person human body detection and association is a very challenging re-identification (re-id) problem;

[0050] b) remove the assumption that does not match the actual situation: previous recent work on top view and other camera collaboration analysis is based on two assumptions: 1) the top view camera has a certain tilt angle to achieve partial visibility of the human body and use appearance matching to correlate horizontally; 2) the direction of the side view camera is the same as the moving direction of the camera wearer.

[0051] To solve the above problems, the present application provides an effective solution:

[0052] A complementary dynamic view camera collaboration method based on polar coordinate transformation, which uses a human body detection module to detect all human targets in the top view and side view respectively, uses a spatial perception position representation to embed the spatial position distribution of objects in different views, and applies polar coordinate transformation to bridge the view gap between the top view and the side view; comprising the following steps:

[0053] S1, train the target detection network, camera positioning network through the top view and side view data set of target position annotation, side view camera position, side view angle and matching matrix annotation; at the same time, train the cross-view target association network through the result of target position annotation and the annotation of matching matrix;

[0054] S2, apply the top view and side view to the target detection network to obtain the target position of the two views;

[0055] S3, the human body detection module calculates and generates two-view heat maps according to the target position of the two views;

[0056] S4, according to the camera positioning network to identify the position of the photographer, the polar coordinate transformation is carried out on the view heat map taking the side view photographer as the origin;

[0057] S5, according to the heat map after polar coordinate transformation, the search network of observation direction is applied to determine the side view angle;

[0058] S6, based on the target position identified in step S3 and the side view angle determined in step S5, calculate the target similarity matrix and input the cross-view target association network to obtain the matching matrix output, and end.

[0059] a) Spatial-aware position representation: A given pair of complementary top-view and side-view images are input into the human detection module. A CNN (Convolutional Neural Network) architecture based on a three-branch CenterNet is used, specifically including a heatmap branch, an offset branch, and a target size branch, where the heatmap branch is used to estimate the center position of the target object. To bridge the gap of the spatial position layout of the target in the two views, a polar coordinate transformation is applied to the heatmap branch of the top-view image, while the heatmap branch of the side-view image can be directly used as the spatial-aware position representation. With the field of view angle θ of the side-view camera, the side-view heatmap can be regarded as an extension of the top-view heatmap, so as to match the spatial distribution of the object position in the side-view image and the corresponding sub-region in the top-view image.

[0060] b) Two-stage complementary view collaboration network: including observation direction search, camera positioning, and camera positioning network. First, the heatmap branches of the side-view and top-view images are matched, and the relevant scores corresponding to each observation direction candidate are obtained. Thereafter, the side-view camera position O in the top-view image is taken as the origin of the polar coordinate transformation, which is sampled from the positions of all objects in the top-view image. Based on the above settings, the framework of the camera position and field of view direction camera positioning network is as shown in the description accompanying drawings FIG. 6B. Figure 2

[0061] (2) Spatial-aware method implementation: Spatial awareness refers to the process of identifying and representing the position and direction of the camera corresponding to the horizontal image, and correlating the targets in different images according to the images of the same scene taken from the top-view and side-view angles. The complete process includes five parts: position representation, network training, photographer identification, observation direction determination, and multi-person target correlation.

[0062] a) Position representation: In order to bridge the gap between the top-view and side-view images, a polar coordinate transformation is applied to the top-view heatmap target to render the 360-degree field of view of the side-view camera:

[0063] Step S1: input the top-view angle image and a side-view angle image into the human detection module to obtain the top-view heatmap F t , the top-view image target set , and the side-view heatmap F s ;

[0064] Step S2: according to the top-view heatmap F t , select the coordinate origin (c x , c y ) and determine an arbitrary direction (such as north) to perform polar coordinate transformation to obtain the transformed top-view heatmap has such that where w, h are the width and height of F t , and W, H are ​The width and height are set here as H = h, W = κw, and the hyperparameter κ ≥ 1;

[0065] Step S3: To simulate the occlusion phenomenon of the side view, the side view heatmap is generated. The values ​​are accumulated along the y-axis to obtain the compression side view heatmap. At the same time, according to The radial values ​​are accumulated to obtain the compression top view heat map f. t ;

[0066] Step S4: Based on the principle of polar coordinate expansion, view the heat map F from the side. s The vertical direction (observation field of view from the side view) and the polar coordinate unfolding Corresponding to the vertical direction (360-degree field of view) and combined with the observation range θ of the side-view field of view, the side-view heatmap can be regarded as... of part.

[0067] b) Network training: Training the target detection network, camera localization network, and cross-view target association network:

[0068] Step S1: Train the CenterNet object detection network;

[0069] Step S2: Calculate the correlation score between the top-view compression heatmap and the side-view compression heatmap. Where * denotes convolution operation. This represents a compressed top view heatmap f extracted using a sliding window with a width w equal to the width of the side view heatmap. t In the part, W is f t The width of the sliding window, s∈[1,W], is the left boundary of the sliding window. When s≥Ww, f is used. t The left side region is added to f t To the right of the right boundary. Obtain the correlation score for each observation direction.

[0070] Step S3: Perform supervised training for observation direction search, loss function Where r is the score vector of each observation direction predicted in step 102, and its length is W. gt It is a score vector for each observation direction determined by the actual observation direction, and its length is also W. The actual observation direction has a score of 1, and the others decrease in order of their deviation from the actual direction.

[0071] Step S4: Supervised learning of camera position using a triple loss method, loss function. Where τ is a preset parameter, fs is a compressed side-view heatmap, fto is a compressed top-view heatmap obtained by polar coordinate transformation and accumulation with the correct position as the origin, and f t (o′) is a compressed top view heatmap obtained by polar coordinate transformation and accumulation with the error position as the origin;

[0072] Step S5: Based on the target P in the top view in step S1 i (i∈{1,2,…,M}) and side view target Q i Calculate the target distance along the x-axis (left and right directions in the side view) based on the detection results (i∈{1,2,…,N}). Indicates target P i Q j The normalized x-axis coordinates are given, and D is a distance measurement function. The similarity matrix of all targets between the two viewpoints is obtained.

[0073] Step S6: Calculate the target distance along the y-axis (proximity / distance in the side view) using a method similar to Step S5, and use the depth of the target to represent the distance between the target and the camera in the side view. Indicates target P i The normalized y-axis coordinate, d(Q) j ) represents the target Q j The depth is calculated in step S5, and the similarity matrix S is calculated. y ;

[0074] Step S7: Construct the association subnetwork using a bidirectional RNN architecture. The similarity matrix is ​​reshaped into a vector in row-wise order and fed into the first BiRNN. Then, the similarity matrix is ​​reshaped into a vector in column-wise order and fed into the second BiRNN. Three fully connected layers and a sigmoid function are applied to obtain the resulting matching matrix M. x (or M) y );

[0075] Step S8: Calculate the loss function Where M x M y It is the predicted matching matrix, M gt Given the actual matching matrix, the matrix cross-entropy loss function L is used. cro This is used to measure the consistency between the two matrices. Based on the loss function, supervised training is performed to construct a cross-view target association network.

[0076] c) Photographer identification: Using object detection and camera localization networks, the photographer's position is determined, thereby establishing the origin of the polar coordinate transformation.

[0077] Step S1: Collect the target set detected in the top view image All targets are input into the camera positioning network to obtain the confidence score of each target;

[0078] Step S2: Select the target with the highest confidence level The predicted camera position;

[0079] Step S3: Perform polar coordinate transformation on the top view heatmap, using O as the origin of the polar coordinate transformation.

[0080] d) Determining the observation direction: An observation direction search network is applied to determine the observation direction based on the top-view heatmap and side-view heatmap after polar coordinate transformation.

[0081] Step S1: Calculate the correlation score of the observation direction based on the top view heatmap and the side view heatmap after polar coordinate transformation;

[0082] Step S2: Select the direction with the highest score as the predicted direction of observation.

[0083] e) Multi-target association: Apply a cross-view target association network to associate targets identified from two different viewpoints.

[0084] Step S1: Based on the target detection results of the top view and side view, and the viewing direction, calculate the similarity matrix S of all targets between the two views. x and S y ;

[0085] Step S2: Place S x and S y Input the cross-view target association network and obtain the matching matrix M output by the network, which is used as the prediction of target association.

[0086] References

[0087] [1]K.Duan, S.Bai, L.Xie, H.Qi, Q.Huang and Q.Tian, ​​"CenterNet: KeypointTriplets for Object Detection," 2019IEEE / CVF International Conference on Computer Vision (ICCV), 2019

[0088] [2]M.Schuster and KKPaliwal, "Bidirectional recurrent neural networks," in IEEE Transactions on Signal Processing, 1997.11

[0089] [3] Y. Shi, L. Liu, X. Yu, and H. Li. Spatial-aware feature aggregation for image based cross-view geolocalization. In NeurIPS, 2019.3

[0090] [4] R. Han, J. Zhao, W. Feng, Y. Gan, L. Wan, and S. Wang. Complementary-view co-interest person detection.

[0091] [5] R. Han, W. Feng, Y. Zhang, J. Zhao, and S. Wang. Multiple human association and tracking from egocentric and complementary top views. IEEE TPAMI, 2021.1, 2, 3, 7, 8, 9

[0092] [6] S. Ardeshir and A. Borji. Egocentric meets top-view. IEEE TPAMI, 2018.2, 3

[0093] [7] S. Ardeshir and A. Borji. Integrating egocentric videos in top-view surveillance videos: Joint identification and temporal alignment. In ECCV, 2018.2, 3, 8

[0094] [8] H. W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):83-97, 1955.6, 8, 9.

Claims

1. A polar coordinate transformation based complementary dynamic view camera collaboration method, characterized in that: The method utilizes a human body detection module to detect all human targets in the top view and the side view respectively, uses spatial position representation to embed the spatial position distribution relationship of objects in different views, and applies polar coordinate transformation to unify the view features between the top view and the side view; The method comprises the following steps: S1, training a target detection network and a camera positioning network through a top view and a side view dataset labeled by human target position, side view camera position, side view angle and a matching matrix; meanwhile, training a cross-view target association network through the result of target position labeling and the labeling of the matching matrix; S2, obtaining target positions in the two views through the target detection network applied to the top view and the side view respectively; S3, a human body detection module calculates and generates two-view target distribution heat maps according to the target positions in the two views; S4, identifying a photographer position through the camera positioning network, and performing polar coordinate transformation on the view heat map with the side view photographer as the origin; S5, determining a side view angle through a search network applied to the heat map after the polar coordinate transformation; S6, calculating a target similarity matrix based on the target positions identified in step S3 and the side view angle determined in step S5, inputting the target similarity matrix into the cross-view target association network to obtain a matching matrix output, and ending.

2. The polar coordinate transformation based complementary dynamic view camera coordination method according to claim 1, characterized in that: The S1 step trains a target detection network and a camera positioning network through a top view and a side view dataset labeled by human target detection, side view camera position and angle; meanwhile, trains a cross-view target association network through labeling of a cross-view target matching matrix; 101, training a target detection network; 102, training a camera positioning network, wherein the camera positioning network needs to complete positioning of a side view camera position, a top view and estimation of a camera observation direction at the same time; comprising the following steps: Computing a correlation score for a compressed top-view and a compressed side-view thermogram where * denotes a convolution operation, denotes the compressed top-down thermogram f intercepted using a sliding window of width w of the side view thermogram t , W is the width of f t , s e [1, W] is the left boundary of the sliding window; when s > W-w, the left region of f t is used to complement to the right of the right boundary of f t , resulting in the correlation score corresponding to each viewing direction 103, supervising and training the observation direction in the camera positioning network through the following formula; wherein: r is the score vector of each observation direction predicted in step 102, with length W, r gt is the score vector of each observation direction determined by the real observation direction, also with length W, with the score of the real observation direction being 1 and the rest decreasing in turn according to the deviation from the real direction; 104, supervising and learning the camera position in the positioning network through the following formula, where τ is a preset parameter, f s is the compressed side view heat map, f t (o) is the compressed top view heat map obtained by polar coordinate transformation and accumulation with the correct position as the origin, f t (o′) is the compressed top view heat map obtained by polar coordinate transformation and accumulation with the incorrect position as the origin; 105, training a cross-view association network; comprising the following steps: According to the top view target P i (i∈{1,2,…,M}) and the side view target Q i (i∈{1,2,…,N}) of the detection result, the target distance in the x-axis direction is calculated where: represents the target P i , Q j normalized x-coordinate of the target, D is a distance measurement function that obtains a similarity matrix of all targets between two views 106. Using the method of step 105 to calculate the target distance in the y-axis direction, using the depth of the target object to represent the distance between the target and the camera in the side view; there is The normalized y-axis coordinate of the target P i , d(Q j ) represents the depth of the target Q j , and the method of step 105 calculates the similarity matrix S y ; 107、Adopting bidirectional RNN architecture as the basic structure of cross-view target association network: by respectively reshaping the similarity matrix S x , S y into vectors in row order and feeding them back to the first BiRNN network, and respectively reshaping the results obtained in column order into vectors and feeding them back to the second BiRNN network; applying a three-layer fully connected layer network and a Sigmod function to obtain the result matching matrix M x , M y ; 108, constructing a cross-view target association network through supervised training according to the following formula: Where: M x M y It is the predicted matching matrix, M gt Given the actual matching matrix, the matrix cross-entropy loss function L is used. cro To measure the consistency between two matrices.

3. The polar coordinate transformation based complementary dynamic view camera coordination method according to claim 1, characterized in that: The step S4 is a process of identifying a photographer position through the camera positioning network, and performing polar coordinate transformation on a top view target distribution heat map with the side view photographer as the origin; 301、the top-view perspective image and a side-view perspective image are input into a pedestrian target detection module to generate a top-view target distribution heat map F t , a top-view image target set and a side-view target distribution heat map F s ; 302. According to the top view target distribution heat map F t , select the coordinate origin (c x ,c y ) and determine an arbitrary direction to perform polar coordinate transformation to obtain the transformed top view heat map That is So that where w, h are the width and height of F t W, H are the width and height of F where H = h, W = Kw, for a hyperparameter K >

1.

303. According to the principle of polar coordinate expansion, the vertical direction (side view viewing angle observation field) in the side view heat map F s corresponds to the vertical direction (360-degree field of view) in the polar coordinate expansion , and in combination with the side view observation range θ, the side view heat map can be regarded as the part of the . 304、To simulate the occlusion phenomenon of side view perspective, we accumulate the value of f along the y-axis direction to get the compressed top view heat map f t ; at the same time, accumulate the value of side view target distribution heat map f along the y-axis direction to get the compressed side view heat map f ​​

Images