A target positioning method fusing unmanned aerial vehicle and regional monitoring and a storage medium

Abstract

This invention discloses a target localization method integrating UAV and area surveillance cameras, comprising the following steps: synchronously acquiring observation data of the same target from both the UAV and the area surveillance camera, and verifying target consistency through spatiotemporal alignment and feature matching; constructing a geometric observation equation set containing four nonlinear equations based on the horizontal and pitch observation angles of the target from the surveillance camera and the UAV; rewriting the geometric observation equation set as a residual function, calculating the partial derivatives of the residual function with respect to the target variable, and constructing the Jacobian matrix; using the midpoint of the line connecting the UAV and the camera as the initial value for iteration, solving for the target's three-dimensional coordinates using the Gauss-Newton iterative method, terminating the iteration when the residual norm converges to a preset threshold or reaches the maximum number of iterations, and outputting the target's position coordinates. This invention combines observation data from both the UAV and the surveillance camera, effectively improving the accuracy of target localization and avoiding the problem of accuracy degradation in complex environments.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) positioning technology, specifically to a target positioning method and storage medium that integrates UAV and area monitoring. Background Technology

[0002] In many practical applications in modern society, such as security patrols, wilderness searches, and logistics tracking, drones are widely used due to their flexibility and efficiency. These applications often require drones to quickly and accurately determine the location of targets to ensure the smooth progress of missions. However, traditional drone positioning methods have significant limitations in complex environments.

[0003] Traditional drone positioning primarily relies on onboard GPS and inertial navigation (IMU) devices. In open, unobstructed environments, these devices can provide relatively accurate positioning information. However, in complex urban environments, mountainous valleys, and other areas with severe signal obstruction or interference, GPS signals may be blocked or reflected by obstacles such as buildings and mountains, leading to a sharp decline in positioning accuracy. Meanwhile, while inertial navigation devices can provide relatively stable positioning information for short periods, their accuracy is affected over extended periods due to accumulated errors.

[0004] Furthermore, relying solely on data acquired by the drone's own sensors to calculate target location has limitations in perspective. Drones can typically only observe targets from the air, making it difficult to comprehensively grasp the surrounding environmental information. This limitation in perspective may lead to misjudgment or omission of targets, affecting the accuracy of positioning.

[0005] On the other hand, widely distributed surveillance video resources contain a wealth of ground information, providing a visual view of the target and its surrounding environment. However, currently, these surveillance video resources are not effectively integrated with the positioning capabilities of drones. Surveillance video data is typically used for post-event review and analysis, rather than for real-time target positioning by drones. Summary of the Invention

[0006] The purpose of this invention is to address the problems existing in the prior art by providing a target localization method that integrates unmanned aerial vehicles (UAVs) and area monitoring. By combining observation data from UAVs and surveillance cameras for target localization, the accuracy degradation of traditional localization methods in complex environments is avoided, significantly improving the accuracy of target localization.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A target localization method integrating UAVs and area surveillance includes the following steps:

[0009] S1. Simultaneously acquire observation data of the same target from drones and area surveillance cameras, and verify the consistency of the target through spatiotemporal alignment and feature matching;

[0010] S2, based on the surveillance camera C(x) c y c , z c The horizontal and pitch observation angles (θ) of the target P c , φ c ), and U(x) drones u y u , z u The horizontal and pitch observation angles (θ) of the target P u , φ u Construct a set of geometric observation equations containing four nonlinear equations;

[0011] S3. Rewrite the geometric observation equations into a residual function F(x, y, z); calculate the partial derivatives of the residual function F(x, y, z) with respect to the target variable, and construct the Jacobian matrix J.

[0012] S4. Using the midpoint of the line connecting the drone and the camera as the initial value for iteration, the Gauss-Newton iteration method is used to solve for the three-dimensional coordinates of the target. When the residual norm converges to the preset threshold or the maximum number of iterations is reached, the iteration is terminated, and the three-dimensional position coordinates of the target P are output.

[0013] In step S2,

[0014] The projection of the target onto the horizontal plane and the horizontal distance between the drone and the surveillance camera are calculated as follows:

[0015]

[0016] Calculate the vertical height differences between the target, the drone, and the surveillance camera, respectively:

[0017] zz c =d c tanφ c ;

[0018] zz u =d u tanφ u ;

[0019] Using azimuth and elevation angles, four equations are defined, expressed as follows:

[0020]

[0021] In step S3, the geometric observation equations are rewritten as residual functions, expressed as:

[0022]

[0023] Where f1 is the equation for the horizontal azimuth angle of the surveillance camera; f2 is the equation for the pitch angle of the surveillance camera; f3 is the equation for the horizontal azimuth angle of the UAV; and f4 is the equation for the pitch angle of the UAV.

[0024] Calculate the partial derivatives of the target variables (x, y, z) in each of the four equations above, and then rearrange to obtain the Jacobian matrix J:

[0025]

[0026] The optimal target position is obtained by adjusting the target variable (x, y·z) to minimize the sum of squared residuals.

[0027] In step S4, the Gauss-Newton iterative method is used to solve for the target's three-dimensional coordinates, including:

[0028] The Gauss-Newton iteration method has the following formula:

[0029] x n+1 =x n -(J T J) -1 J T F(x n );

[0030] Among them, X n J represents the target three-dimensional coordinate vector in the nth iteration; T Let J be the transpose matrix.

[0031] Step S1 includes:

[0032] S1.1. Perform timestamp synchronization, format standardization, and keyframe extraction on UAV sensor data and surveillance camera data;

[0033] S1.2. An improved Faster R-CNN model is used to extract multi-dimensional features of the target's color, texture, and shape, and feature vectors are constructed.

[0034] S1.3 Calculate the similarity between the UAV target feature vector and the monitoring feature database using the K-nearest neighbor algorithm, and output the target consistency judgment result.

[0035] In step S1.2, the extraction of multi-dimensional features of the target's color, texture, and shape using the improved Faster R-CNN model includes:

[0036] The feature map output by the region proposal network is dynamically weighted using a learnable channel weight matrix, and an attention mask is generated using an activation function. Finally, the enhanced target feature map is obtained by element-wise multiplication, as shown in the formula:

[0037] Fatt =σ(W c ·F RPN )⊙F RPN ;

[0038] Among them, F RPN W is the feature map output by the RPN. c The weight matrix is ​​a learnable weight matrix, σ is the Sigmoid function, and ⊙ represents element-wise multiplication. Through dynamic weighting, the model autonomously selects key features from color, texture, and shape.

[0039] The feature vector includes:

[0040] The feature vector is constructed by fusing the color histogram (hc), HOG texture (ht), and shape descriptor (hs), using the following formula:

[0041] v = Concat(h) c h t h s )·W f ;

[0042] Among them, W f This is the feature fusion weight matrix.

[0043] Step S1.3 includes:

[0044] S1.3.1, Using cosine similarity to match the drone feature vector v u With monitoring feature library v m The formula is:

[0045]

[0046] S1.3.2 Select the samples with the highest Top-K similarity and determine the target ID through weighted voting, using the following formula:

[0047]

[0048] Among them, w i Here, δ is the similarity weight, δ is the indicator function, and K represents the number of samples.

[0049] S5. Perform environmental information-assisted correction on the output target location coordinates, specifically including:

[0050] S5.1 Construct a 3D model of the target scene, and label the reflectivity ρ, refractive index n, and transparency α of the material of each building's facade, forming a set of buildings S. obj .

[0051] S5.2 Calculate the line-of-sight path equation L(t) from the UAV to the target, and the equation with the set of buildings S. obj Perform spatial collision detection;

[0052] S5.3 When occlusion is detected, perform Snell's law refraction calculation on the glass material and Lambertian reflection calculation on the wall material.

[0053] S5.4. Based on the cumulative angular offset from multiple reflections and / or refraction paths, correct the azimuth angle of the UAV target and output the corrected azimuth angle θ. cor ;

[0054] S5.5, Correct the UAV target azimuth angle θ cor Substitute the values ​​from steps S2-S4 to recalculate the corrected target position coordinates.

[0055] Step S5.2 includes:

[0056] S5.2.1 Establish the virtual line-of-sight path equation L(t) from the UAV to the target:

[0057] S5.2.2 Perform dense sampling on the line-of-sight path equation L(t), calculate the distance between each sampling point and the nearest building, and if there is at least one sampling point whose distance from the building is less than the spatial resolution threshold, it is determined to be an obstruction.

[0058] Step S5.4 includes: cumulatively calculating the angular offset of multiple light interactions, and determining the initial azimuth angle θ of the UAV based on the cumulative angular offset. u After correction, the corrected UAV target azimuth angle θ is obtained. cor .

[0059] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method steps.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] 1. By integrating drone sensor data and surveillance camera data, spatiotemporal alignment and feature matching were achieved, ensuring the consistency and accuracy of the data source;

[0062] 2. By using the azimuth and elevation angles of the UAV and the monitoring camera, a set of geometric observation equations containing four nonlinear equations is constructed, and the Gauss-Newton iterative method is used to solve the three-dimensional coordinates of the target. This avoids the problem of accuracy degradation of traditional positioning methods in complex environments and significantly improves the accuracy of target positioning.

[0063] 3. By calculating the line-of-sight path equation from the UAV to the target, performing spatial collision detection with the building set, and performing corresponding refraction and reflection calculations on the glass and wall materials, the azimuth angle of the UAV target was corrected, effectively solving the positioning problem in complex environments. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a schematic diagram of the overall process of a target positioning method that integrates UAV and regional monitoring according to an embodiment of this application. Detailed Implementation

[0066] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The sequence number of each step in this application does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0068] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. It should also be understood that although the terms "first," "second," etc., are used in the text to describe various elements in some embodiments of this application, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first table may be named a second table, and similarly, a second table may be named a first table, without departing from the scope of the various described embodiments. Both the first table and the second table are tables, but they are not the same table.

[0069] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0070] Traditional drone positioning primarily relies on onboard GPS and inertial navigation (IMU) devices. In open, unobstructed environments, these devices can provide relatively accurate positioning information. However, in complex urban environments, mountainous valleys, and other areas with severe signal obstruction or interference, GPS signals may be blocked or reflected by obstacles such as buildings and mountains, leading to a sharp decline in positioning accuracy. Meanwhile, while inertial navigation devices can provide relatively stable positioning information for short periods, their accuracy is affected over extended periods due to accumulated errors.

[0071] Furthermore, relying solely on data acquired by the drone's own sensors to calculate target location has limitations in perspective. Drones can typically only observe targets from the air, making it difficult to comprehensively grasp the surrounding environmental information. This limitation in perspective may lead to misjudgment or omission of targets, affecting the accuracy of positioning.

[0072] On the other hand, widely distributed surveillance video resources contain a wealth of ground information, providing a visual view of the target and its surrounding environment. However, currently, these surveillance video resources are not effectively integrated with the positioning capabilities of drones. Surveillance video data is typically used for post-event review and analysis, rather than for real-time target positioning by drones.

[0073] To address the above technical problems, embodiments of this application provide a target localization method integrating unmanned aerial vehicles (UAVs) and area monitoring, comprising the following steps S1-S5. For example... Figure 1 As shown.

[0074] Step S1: Simultaneously acquire observation data of the same target from the drone and the area monitoring camera, and verify the consistency of the target through spatiotemporal alignment and feature matching.

[0075] In some implementations, step S1 includes steps S1.1-S1.3.

[0076] Step S1.1: Synchronize the timestamps, standardize the format, and extract keyframes from the drone sensor data and the monitoring camera data.

[0077] In some implementations, a multi-source data fusion platform is constructed to perform timestamp synchronization, format standardization, and keyframe extraction on UAV sensor data and surveillance camera data. Specifically:

[0078] Drone sensor data acquisition includes: acquiring sensor data such as GPS, IMU, and camera through the drone SDK; adding a precise timestamp (UTC time) to each data point; and converting the data into a unified JSON format.

[0079] Surveillance camera data acquisition includes: connecting to the surveillance camera using the RTSP protocol to acquire video streams in real time; parsing the video stream to extract the timestamps and image data of each frame; storing the video data in H.264 format and generating corresponding metadata files.

[0080] Timestamp synchronization includes using the NTP protocol to synchronize the timestamps of drone sensor data and surveillance camera data, ensuring that the timestamps of all data sources are consistent.

[0081] Keyframe extraction includes: using OpenCV to detect moving targets in video frames from surveillance video data, extracting frames containing the targets as keyframes; annotating the keyframes, extracting information such as the target's position, size, and category, and storing them in JSON format.

[0082] Step S1.2: Use the improved Faster R-CNN model to extract multi-dimensional features of the target's color, texture, and shape, and construct feature vectors.

[0083] In some implementations, step S1.2, which involves using an improved Faster R-CNN model to extract multi-dimensional features of the target's color, texture, and shape, includes steps S1.2.1-S1.2.2.

[0084] Step S1.2.1: Dynamically weight the feature map output by the region proposal network using a learnable channel weight matrix, generate an attention mask using an activation function, and finally obtain the enhanced target feature map by element-wise multiplication. The formula is as follows:

[0085] F att =σ(W c ·F RPN )⊙F RPN ;

[0086] Among them, F RPN W is the feature map output by the RPN. cThe learning weight matrix is ​​σ, where σ is the Sigmoid function and ⊙ represents element-wise multiplication. Through dynamic weighting, the model autonomously selects key features from color, texture, and shape.

[0087] Specifically, a learnable channel weight matrix Wc is introduced based on the feature map FRPN output by the Region Proposal Network (RPN). The dimension of this matrix is ​​the same as the number of channels in the feature map, and it is used to dynamically adjust the weights of each channel. The channel weight matrix is ​​processed using the sigmoid activation function σ to generate an attention mask. The generated attention mask is then multiplied element-wise with the feature map output by the RPN to obtain the enhanced target feature map.

[0088] Through the aforementioned dynamic weighting mechanism, the model can autonomously select key features in color, texture, and shape, thereby improving the accuracy of feature extraction.

[0089] Step S1.2.2: Extract the following multi-dimensional features from the enhanced target feature map:

[0090] Color histogram (hc): Calculates the color distribution histogram of the target region to describe the color characteristics of the target.

[0091] HOG texture (ht): The texture features of the target are extracted using the Histogram of Oriented Gradients (HOG) algorithm.

[0092] Shape descriptor (hs): Extracts the shape features of the target using a shape description algorithm.

[0093] The feature vector is constructed by fusing the color histogram (hc), HOG texture (ht), and shape descriptor (hs), using the following formula:

[0094] v = Concat(h) c h t h s )·W f ;

[0095] Among them, W f This is the feature fusion weight matrix, used to adjust the contribution of each feature in the vector.

[0096] Step S1.3: Calculate the similarity between the UAV target feature vector and the monitoring feature database using the K-nearest neighbor algorithm, and output the target consistency judgment result.

[0097] In some implementations, step S1.3 includes steps S1.3.1-S1.3.2.

[0098] Step S1.3.1: Use cosine similarity to match the UAV feature vector v u With monitoring feature library v m The formula is:

[0099]

[0100] Among them, the UAV target feature vector v u The improved Faster R-CNN model extracts multi-dimensional features, including color, texture, and shape. (Monitoring feature library v) m It contains feature vectors of multiple known targets, each vector corresponding to a target ID. s(v u v m ) represents the cosine similarity, with a value range of [-1, 1]. The larger the value, the higher the similarity.

[0101] Step S1.3.2: Select the samples with the highest Top-K similarity and determine the target ID through weighted voting, using the following formula:

[0102]

[0103] Among them, w i Here, δ represents the similarity weight, δ is the indicator function, and K represents the number of samples.

[0104] Specifically, weights w are assigned to each Top-K sample. i The weight value is its relationship with v u The cosine similarity is calculated; the occurrence frequency of each target ID in the Top-K samples is counted, and a weighted calculation is performed based on the weights; the target ID with the highest weighted score is selected as the consistency determination result.

[0105] Step S2: Based on the surveillance camera C(x) c y c , z c The horizontal and pitch observation angles (θ) of the target P c , φ c ), and U(x) drones u y u , z u The horizontal and pitch observation angles (θ) of the target P u , φ u ); Construct a set of geometric observation equations containing four nonlinear equations.

[0106] In some embodiments, step S2 includes steps S2.1-S2.3:

[0107] Step S2.1: Calculate the target's projection on the horizontal plane and the horizontal distance between the drone and the surveillance camera, respectively:

[0108]

[0109] Step S2.2: Calculate the vertical height differences between the target, the drone, and the surveillance camera, respectively:

[0110] zz c =d c tanφ c ;

[0111] zz u =d u tanφ u ;

[0112] Step S2.3: Define four equations using azimuth and elevation angles, expressed as:

[0113]

[0114] Wherein, formula (1) is the horizontal observation equation of the surveillance camera for target P; f2 is the pitch observation equation of the surveillance camera for target P; f3 is the horizontal observation equation of the UAV for target P; and f4 is the pitch observation equation of the UAV for target P.

[0115] Step S3: Rewrite the geometric observation equations into a residual function F(x, y, z); calculate the partial derivatives of the residual function F(x, y, z) with respect to the target variable, and construct the Jacobian matrix J.

[0116] In some embodiments, step S3 includes steps S3.1-S3.2.

[0117] Step S3.1: Rewrite the geometric observation equations as residual functions, expressed as:

[0118]

[0119] Where f1 is the equation for the horizontal azimuth angle of the surveillance camera; f2 is the equation for the pitch angle of the surveillance camera; f3 is the equation for the horizontal azimuth angle of the UAV; f4 is the equation for the pitch angle of the UAV; F(x, y, z) is the residual function vector; and (x, y, z) is the target variable.

[0120] Step S3.2: Calculate the partial derivatives of the target variables (x, y, z) in the four equations above, and simplify to obtain the Jacobian matrix J:

[0121]

[0122] The optimal target position is obtained by adjusting the target variables (x, y, z) to minimize the sum of squared residuals.

[0123] Step S4: Using the midpoint of the line connecting the drone and the camera as the initial value for iteration, the Gauss-Newton iteration method is used to solve for the three-dimensional coordinates of the target. When the residual norm converges to the preset threshold or the maximum number of iterations is reached, the iteration is terminated, and the three-dimensional position coordinates of the target P are output.

[0124] In some implementations, in step S4, the Gauss-Newton iteration method is used to solve for the target's three-dimensional coordinates, as shown in the formula:

[0125] x n+1 =x n -(J T J) -1 J T F(x n );

[0126] Among them, X n J represents the target three-dimensional coordinate vector in the nth iteration; T Let J be the transpose matrix.

[0127] For example, suppose the location coordinates of the surveillance camera and the drone are as follows:

[0128] The coordinates of the surveillance camera are: C(xc, yc, zc) = (0, 0, 0);

[0129] UAV coordinates: U(xu, yu, zu) = (10, 20, 30);

[0130] The true coordinates of target P are (xp, yp, zp) = (5, 10, 15). Through observation, the horizontal azimuth (θc, θu) and pitch (φc, φu) of target P as perceived by the monitoring camera and the drone are obtained.

[0131] Use the midpoint of the line connecting the surveillance camera and the drone as the initial value for iteration:

[0132]

[0133] In the first iteration (n=0), the residual function F0 and the Jacobian matrix J0 are calculated according to the method in step S3.

[0134] Update target coordinates:

[0135]

[0136] The calculation yielded:

[0137]

[0138] The residual norm is calculated to be 0.5, which does not meet the preset threshold ∈ = 0.01, so the iteration continues.

[0139] In the second iteration (n=1), the residual function F1 and the Jacobian matrix J1 are calculated according to the method in step S3.

[0140] Update target coordinates:

[0141]

[0142] The calculation yielded:

[0143]

[0144] The residual norm is calculated to be 0.05, which does not meet the preset threshold ∈ = 0.01, so the iteration continues.

[0145] In the second iteration (n=2), the target's three-dimensional coordinates are obtained as follows:

[0146]

[0147] The residual norm is calculated to be 0.005. When the preset threshold ∈ = 0.01 is reached, the iteration is terminated.

[0148] Step S5: Perform environmental information-assisted correction on the output target location coordinates.

[0149] In some embodiments, step S5 specifically includes steps S5.1-S5.5.

[0150] Step S5.1: Construct a 3D model of the target scene, and annotate the reflectivity ρ, refractive index n, and transparency α of the material of each building facade to form a set of buildings S. obj .

[0151] In some embodiments, geographic information data of the target scene is obtained using various means, including:

[0152] LiDAR mapping: Obtaining precise three-dimensional coordinates and shape information of buildings and terrain features.

[0153] Satellite remote sensing imagery: acquires information on a wide range of topographic features, such as mountains, rivers, and valleys.

[0154] On-site measurement: Supplementary measurements are taken on details such as the building's exterior materials and window distribution.

[0155] Based on the collected data, a high-precision three-dimensional geographic information model is constructed, which records detailed building information, including the building's precise location, shape, height, structure, and physical properties of the facade material such as reflectivity (ρ), refractive index (n), and transparency (α).

[0156] Organize the building information in the scene into a set Sobj. Each building records the following attributes: coordinate range, height and shape, and the reflectivity (ρ), refractive index (n), and transparency (α) of the facade material.

[0157] Step S5.2: Calculate the line-of-sight path equation L(t) from the UAV to the target, and compare it with the set of buildings S. obj Perform space collision detection.

[0158] In some embodiments, step S5.2 includes the following steps S5.2.1-S5.2.2.

[0159] S5.2.1 Establish the virtual line-of-sight path equation L(t) from the UAV to the target, as follows:

[0160] L(t)=P u +t·(P p -P u )

[0161] Where P u =(x u y u , z u P represents the coordinates of the UAV; p =(x p y p , z p ) represents the coordinates of the target P; t is a parameter with a value range of [0, 1], representing the position on the path.

[0162] In S5.2.2, the line-of-sight path equation L(t) is densely sampled, and the distance between each sampling point and the nearest building is calculated. If at least one sampling point is less than the spatial resolution threshold in distance from the building, it is determined to be an obstruction.

[0163] Specifically, the virtual line-of-sight path equation L(t) is densely sampled to generate a series of sampling points {P1, P2, ..., P}. N}, where the coordinates of each sampling point are:

[0164] P i =P u +t i ·(P p -P u )

[0165] in i = 1, 2, ..., N, where N is the total number of sampling points.

[0166] For each sampling point P i Iterate through the set of buildings S obj Calculate sampling point P iDistance to each building. Choose the smallest distance as d. i .

[0167] If there exists at least one sampling point P i Distance d from the nearest building i If the value is less than the spatial resolution threshold, it is considered an occlusion.

[0168] Step S5.3: When occlusion is detected, perform Snell's law refraction calculation on the glass material and Lambertian reflection calculation on the wall material.

[0169] In some embodiments, step S5.3 specifically includes:

[0170] Refraction calculation for glass:

[0171] Calculate the angle θ of incidence between the light ray and the glass surface. i The formula is:

[0172]

[0173] Calculate the angle of refraction θ according to Snell's law. t The formula is:

[0174] n1sin(θ i )=n2sin(θ t );

[0175] Where n1 is the refractive index of air and n2 is the refractive index of glass.

[0176] According to the angle of refraction θ t The formula for calculating the direction vector t of the refracted ray is:

[0177]

[0178] Reflection calculation of wall material:

[0179] Calculate the angle θ of incidence between the light ray and the wall surface. i The formula is:

[0180]

[0181] Where v is the direction vector of the light ray, and n is the normal vector of the wall surface.

[0182] According to the incident angle θ i The formula for calculating the direction vector r of the reflected ray is:

[0183] r = V - 2(V·n)n.

[0184] According to the Lambert reflection model, the intensity Ir of the reflected light is calculated using the following formula:

[0185] I r =I i ·ρ·cos(θ i );

[0186] Among them, I i ρ is the intensity of the incident light, and ρ is the reflectivity of the wall material.

[0187] Based on the calculation results, the light propagation path is corrected.

[0188] Step S5.4: Based on the cumulative angular offset from multiple reflections and / or refraction paths, correct the azimuth angle of the UAV target and output the corrected azimuth angle θ. cor .

[0189] In some embodiments, step S5.4 includes:

[0190] The cumulative calculation of the angular offset from multiple light interactions is as follows:

[0191]

[0192] Where: Δθ j Let be the angular offset of the j-th ray interaction. N is the total number of ray interactions.

[0193] The initial azimuth angle θu of the UAV is corrected based on the cumulative angle offset Δθtotal, using the following formula:

[0194] θ cor =θ u +Δθ total ;

[0195] Where, θ u θcor is the initial azimuth angle of the UAV, and θcor is the corrected target azimuth angle of the UAV.

[0196] Step S5.5: Adjust the corrected UAV target azimuth angle θ cor Substitute the values ​​from steps S2-S4 to recalculate the corrected target position coordinates.

[0197] By correcting the azimuth angle of the UAV target and recalculating the target position coordinates, the accuracy of target positioning can be significantly improved.

[0198] In summary, this method achieves spatiotemporal alignment and feature matching between UAV sensor data and surveillance camera data. Specifically, it employs the NTP protocol for timestamp synchronization to ensure time consistency of the data sources, and uses an improved Faster R-CNN model to extract multi-dimensional features such as target color, texture, and shape to construct feature vectors. The similarity between the UAV target feature vector and the surveillance feature database is calculated using the K-nearest neighbor algorithm, outputting a target consistency determination result to ensure that both images capture the same target.

[0199] This method constructs a set of geometric observation equations containing four nonlinear equations. By using a UAV and a monitoring camera to observe the target's horizontal and vertical angles, the target's position in three-dimensional space is accurately calculated. The geometric observation equations are rewritten as residual functions, and their partial derivatives with respect to the target variables are calculated to construct the Jacobian matrix. The Gauss-Newton iterative method is then used to solve for the target's three-dimensional coordinates. This not only avoids the accuracy degradation problem of traditional positioning methods in complex environments but also significantly improves the accuracy of target positioning through iterative optimization.

[0200] This method constructs a 3D model of the target scene, annotating the physical properties of building facade materials such as reflectivity, refractive index, and transparency to form a building set. By calculating the line-of-sight path equation from the UAV to the target, spatial collision detection is performed with the building set. When occlusion is detected, Snell's law refraction calculation is performed on glass materials, and Lambert reflection calculation is performed on wall materials. The cumulative angular offset from multiple reflections and / or refraction paths is used to correct the UAV target azimuth angle and recalculate the target position coordinates. This effectively solves the problem of decreased positioning accuracy caused by signal obstruction or interference in complex environments, significantly improving the robustness and adaptability of target positioning.

[0201] In a second aspect, this application provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method steps.

[0202] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A target localization method integrating unmanned aerial vehicles (UAVs) and area monitoring, characterized in that, Includes the following steps: S1. Simultaneously acquire observation data of the same target from drones and area surveillance cameras, and verify the consistency of the target through spatiotemporal alignment and feature matching; S2, based on surveillance cameras Horizontal and pitch observation angles of target P and drones Horizontal and pitch observation angles of target P Construct a set of geometric observation equations containing four nonlinear equations; The projection of the target onto the horizontal plane and the horizontal distance between the drone and the surveillance camera are calculated as follows: ； ； Calculate the vertical height differences between the target, the drone, and the surveillance camera, respectively: ； ； Using azimuth and elevation angles, four equations are defined, expressed as follows: ； S3. Rewrite the geometric observation equations as residual functions, expressed as: ； Where f1 is the equation for the horizontal azimuth angle of the surveillance camera; f2 is the equation for the pitch angle of the surveillance camera; f3 is the equation for the horizontal azimuth angle of the UAV; and f4 is the equation for the pitch angle of the UAV. Calculate the target variable in each of the four equations above. The partial derivatives are used to simplify and obtain the Jacobian matrix. : ； By adjusting the target variable To minimize the sum of squared residuals, the optimal target location is obtained; S4. Using the midpoint of the line connecting the drone and the camera as the initial value, the Gauss-Newton iteration method is used to solve for the target's three-dimensional coordinates; the iteration formula for the Gauss-Newton iteration method is: ； in, This represents the target three-dimensional coordinate vector in the nth iteration; express The transpose of the matrix; The iteration terminates when the residual norm converges to a preset threshold or the maximum number of iterations is reached, and the three-dimensional position coordinates of the target P are output.

2. The target positioning method integrating UAV and area monitoring according to claim 1, characterized in that, Step S1 includes: S1.

1. Perform timestamp synchronization, format standardization, and keyframe extraction on UAV sensor data and surveillance camera data; S1.

2. An improved Faster R-CNN model is used to extract multi-dimensional features of the target's color, texture, and shape, and feature vectors are constructed. S1.3 Calculate the similarity between the UAV target feature vector and the monitoring feature database using the K-nearest neighbor algorithm, and output the target consistency judgment result.

3. The target positioning method integrating UAV and regional monitoring according to claim 2, characterized in that, In step S1.2, the extraction of multi-dimensional features of the target's color, texture, and shape using the improved Faster R-CNN model includes: The feature map output by the region proposal network is dynamically weighted using a learnable channel weight matrix, and an attention mask is generated using an activation function. Finally, the enhanced target feature map is obtained by element-wise multiplication, as shown in the formula: ； Among them, F RPN W is the feature map output by the RPN. c The weight matrix is ​​a learnable weight matrix, σ is the Sigmoid function, and ⊙ represents element-wise multiplication. Through dynamic weighting, the model autonomously selects key features from color, texture, and shape. The feature vector includes: The feature vector is constructed by fusing the color histogram (hc), HOG texture (ht), and shape descriptor (hs), as shown in the formula: ； Among them, W f This is the feature fusion weight matrix.

4. The target positioning method integrating UAV and area monitoring according to claim 2, characterized in that, Step S1.3 includes: S1.3.1, Using cosine similarity to match the drone feature vector v u With monitoring feature library v m The formula is: ； S1.3.2 Select the samples with the highest Top-K similarity and determine the target ID through weighted voting, using the following formula: ； Among them, w i Here, δ represents the similarity weight, δ is the indicator function, and K represents the number of samples.

5. The target positioning method integrating UAV and area monitoring according to claim 1, characterized in that, include: S5. Perform environmental information-assisted correction on the output target location coordinates, specifically including: S5.1 Construct a 3D model of the target scene, and label the reflectivity ρ, refractive index n, and transparency α of the material of each building's facade, forming a set of buildings S. obj ; S5.2 Calculate the line-of-sight path equation L(t) from the UAV to the target, and the equation with the set of buildings S. obj Perform spatial collision detection; S5.3 When occlusion is detected, perform Snell's law refraction calculation on the glass material and Lambertian reflection calculation on the wall material; S5.

4. Based on the cumulative angular offset from multiple reflections and / or refraction paths, correct the azimuth angle of the UAV target and output the corrected azimuth angle θ. cor ; S5.5, Correct the UAV target azimuth angle θ cor Substitute the values ​​from steps S2-S4 to recalculate the corrected target position coordinates.

6. The target positioning method integrating UAV and area monitoring according to claim 5, characterized in that, Step S5.2 includes: S5.2.1 Establish the virtual line-of-sight path equation L(t) from the UAV to the target: S5.2.2 Perform dense sampling on the line-of-sight path equation L(t), calculate the distance between each sampling point and the nearest building, and if there is at least one sampling point whose distance from the building is less than the spatial resolution threshold, it is determined to be an obstruction.

7. The target positioning method integrating UAV and area monitoring according to claim 5, characterized in that, Step S5.4 includes: cumulatively calculating the angular offset of multiple light interactions, and based on the cumulative angular offset, initially adjusting the UAV... Azimuth θ u After correction, the corrected UAV target azimuth angle θ is obtained. cor .

8. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed by the processor, it implements the steps of the method described in claims 1 to 7.

Images