A target identification positioning and presenting method, device, equipment and storage medium

Abstract

This application discloses a target recognition, localization, and presentation method. The method includes: acquiring an aerial image; rotating the aerial image according to the heading angle information of the aerial photography device so that its direction is the same as the direction of the geographic map; establishing a target detection model; inputting the rotated aerial image into the target detection model to obtain an aerial image with the target object identified; establishing a first coordinate system in the identified aerial image and obtaining the pixel position of the target object according to camera parameters; establishing a second coordinate system between the aerial photography device and the identified aerial image, and obtaining the geographic coordinates of the target object according to the heading angle information of the aerial photography device, its relative ground height, and geographic coordinates; and visually annotating and presenting the located target on a pre-generated 3D digital terrain of the aerial photography area based on the geographic coordinates of the target object. This invention realizes the conversion from target detection pixel coordinates to physical world latitude and longitude coordinates, improving the accuracy of target localization.

Description

Technical Field

[0001] This application relates to the field of aerial photography identification and positioning technology, and more specifically, to a target identification, positioning and presentation method, apparatus, equipment and storage medium. Background Technology

[0002] In vision-based visible light camera-based UAV target recognition and localization, the current main approach is a coarse estimation method. This method assumes that the physical world coordinates of the detected target are approximately equal to the physical world coordinates of the geometric center of the aerial photograph in which the target is located. Since the geometric center coordinates of each orthophoto are known, the closer the target's position in the photograph is to the geometric center, the more accurate the physical world coordinates provided by this method, but the error is relatively large. This coarse estimation method provides low accuracy in physical world localization of the target, failing to make good use of the target's pixel coordinate information and the orthophoto attitude information of the fixed-wing UAV.

[0003] Existing algorithms that perform well in remote sensing / UAV target detection employ a top-down structure to fuse multi-scale features. This combines high-level semantic information with low-level detail information to enhance target feature representation and provide contextual information for small targets, improving discriminative power. However, while the fused features highlight foreground targets, they also introduce a significant amount of background information. This is because the top-down structure sequentially stacks the deepest features onto the shallowest features. Since the deepest features have a large receptive field, adding them to small, shallow targets introduces an excessively large receptive field, resulting in too much background interference. This large receptive field degrades the performance of small target detection. Especially for vehicles viewed from a UAV perspective, which are mostly small-scale targets, the top-down feature fusion structure is unfavorable for target detection. Summary of the Invention

[0004] To address at least one deficiency or improvement need in the prior art, the present invention provides a target recognition, positioning and presentation method, apparatus, device and storage medium, which realizes the conversion from target detection pixel coordinates to physical world latitude and longitude coordinates, thereby improving the accuracy of target positioning.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a target identification, localization, and presentation method is provided, the method comprising the following steps:

[0006] Acquire aerial images and rotate them according to the heading angle information of the aerial photography equipment so that the direction of the aerial images is the same as that of the geographic map;

[0007] A target detection model is established, and the rotated aerial image is input into the trained target detection model to obtain an aerial image after the target object is identified.

[0008] A first coordinate system is established in the identified aerial image, and the pixel position of the target object is obtained according to the camera parameters;

[0009] A second coordinate system is established using the aerial photography equipment and the labeled aerial images. The geographic coordinates of the target object are obtained based on the heading angle information, relative ground height, and geographic coordinates of the aerial photography equipment.

[0010] Based on the geographic coordinates of the target object, the positioning target is visualized and marked on the pre-generated 3D digital terrain of the aerial photography area.

[0011] Furthermore, in the aforementioned target visual recognition, localization, and presentation method, the step of establishing a first coordinate system in the identified aerial image and obtaining the pixel position of the target object based on camera parameters specifically includes:

[0012] Based on the camera parameters, obtain the pixel size of the entire aerial image, the pixel coordinates of the image center point, and the pixel coordinates of the target object;

[0013] Based on the pixel size of the entire aerial image, the pixel coordinates of the first boundary point and the second boundary point are obtained; wherein, the distance between the first boundary point and the center point of the image in a first direction is 0, and the distance between the second boundary point and the center point of the image in a second direction is 0, wherein the first direction and the second direction are perpendicular to each other and are both horizontal directions.

[0014] Furthermore, in the aforementioned target visual recognition, positioning, and presentation method, the step of establishing a second coordinate system between the aerial photography equipment and the identified aerial image, and obtaining the geographic coordinates of the target object based on the heading angle information, relative ground altitude, and geographic coordinates of the aerial photography equipment, specifically includes:

[0015] Based on the absolute value of the difference between the geographic abscissa of the target object and the center point of the image, the absolute value of the difference between the geographic abscissa of the first boundary point and the center point of the image, the distance between the target object and the center point of the image in the first direction in the aerial image, and the proportional relationship between the distance between the first boundary point and the center point of the image in the first direction in the aerial image, a first equation is established.

[0016] Based on the absolute value of the difference between the pixel abscissa of the target object and the center point of the image, the absolute value of the difference between the pixel abscissa of the first boundary point and the center point of the image, the distance between the target object and the center point of the image in the first direction in the aerial image, and the proportional relationship between the distance between the first boundary point and the center point of the image in the first direction in the aerial image, a second equation is established.

[0017] A third-party program is established based on the relationship between the geographic abscissa of the image center point, the geographic abscissa of the first boundary point, the distance between the first boundary point and the image center point in the first direction in the aerial image, and the first heading angle of the aerial photography equipment in the first direction.

[0018] Based on the trigonometric relationship between the distance from the aerial photography equipment to the center point of the image, the first heading angle, and the distance between the first boundary point in the aerial image and the center point of the image in the first direction, a fourth equation is established;

[0019] Based on the first equation, the second equation, the third equation, and the fourth equation, establish the first set of equations to solve for the geographic x-coordinate of the target object.

[0020] Furthermore, the above-mentioned target visual recognition, positioning, and presentation method, wherein establishing a second coordinate system with the aerial photography equipment and the identified aerial image, and obtaining the geographic coordinates of the target object based on the heading angle information, relative ground height, and geographic coordinates of the aerial photography equipment, further includes:

[0021] Based on the absolute value of the difference between the geographic vertical coordinates of the target object and the center point of the image, the absolute value of the difference between the geographic vertical coordinates of the second boundary point and the center point of the image, the distance between the target object and the center point of the image in the second direction in the aerial image, and the proportional relationship between the distance between the second boundary point and the center point of the image in the second direction in the aerial image, a fifth equation is established.

[0022] Based on the absolute value of the difference between the pixel ordinates of the target object and the center point of the image, the absolute value of the difference between the pixel ordinates of the second boundary point and the center point of the image, the distance between the target object and the center point of the image in the second direction in the aerial image, and the proportional relationship between the distance between the second boundary point and the center point of the image in the second direction in the aerial image, a sixth equation is established.

[0023] Based on the relationship between the geographic ordinate of the image center point, the geographic ordinate of the second boundary point, the distance between the second boundary point and the image center point in the second direction in the aerial image, and the second heading angle of the aerial photography equipment in the second direction, the seventh equation is established.

[0024] Based on the trigonometric relationship between the distance from the aerial photography equipment to the center point of the image, the second heading angle, and the distance between the second boundary point in the aerial image and the center point of the image in the second direction, the eighth equation is established;

[0025] Based on the fifth, sixth, seventh, and eighth equations mentioned above, establish the second set of equations to solve for the geographic coordinates of the target object.

[0026] Furthermore, in the aforementioned target visual recognition, localization, and presentation method, the step of inputting the rotated aerial image into the trained target detection model to obtain an aerial image with the target object identified specifically includes:

[0027] The feature maps of aerial images at different resolutions are input into the channel attention module and the spatial attention module. The attention weight of each feature map is calculated sequentially along the two different dimensions of channel and space, and adaptive feature refinement is performed. The refined feature maps of all features are then output.

[0028] Furthermore, the above-mentioned target visual recognition, localization, and presentation method, in which the rotated aerial image is input into the trained target detection model to obtain the identified target object image, further includes:

[0029] In the refined feature map, adjacent shallow feature maps and deep feature maps are selected for feature fusion. The fused feature maps are then integrated using nonlinear operations and convolution operations to identify the target object.

[0030] Furthermore, in the above-mentioned target visual recognition, localization, and presentation method, the target detection model is trained alternately based on the cross-entropy loss function, which is defined as follows:

[0031] L(p t )=-α(1-p t )γlog(p t )

[0032] Where the adjustable parameter γ≥0, α is the weighting factor, and p t The definition is as follows:

[0033]

[0034] Where p represents the probability that the identified image content is the target object;

[0035] By adjusting the value of the adjustable parameter γ, the weight of difficult samples can be increased to improve detection performance.

[0036] According to a second aspect of the present invention, a target identification, positioning, and presentation device is also provided, comprising:

[0037] The image acquisition module is used to acquire aerial images and rotate them according to the heading angle information of the aerial photography equipment so that the direction of the aerial images is the same as the direction of the geographic map.

[0038] The target recognition module establishes a target detection model and inputs the rotated aerial image into the trained target detection model to obtain an aerial image after the target object is identified.

[0039] The target localization module establishes a first coordinate system in the identified aerial image and obtains the pixel position of the target object based on camera parameters; it establishes a second coordinate system with the aerial photography equipment and the identified aerial image and obtains the geographic coordinates of the target object based on the heading angle information of the aerial photography equipment, relative ground height, and geographic coordinates.

[0040] The error assessment module visualizes and marks the target object on a pre-generated 3D digital terrain of the aerial photography area based on the geographical coordinates of the target object.

[0041] According to a third aspect of the present invention, a target identification, positioning and presentation device is also provided, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of any of the methods described above.

[0042] According to a fourth aspect of the invention, a storage medium is also provided, which stores a computer program executable by a target visual recognition, positioning and presentation device, which, when run on the target visual recognition, positioning and presentation device, causes the target visual recognition, positioning and presentation device to perform the steps of any of the methods described above.

[0043] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0044] (1) The present invention provides a target identification, positioning and presentation method, device, equipment and storage medium. By establishing a coordinate system between the identified aerial image and the aerial photography equipment, the geographical coordinates of the target object are calculated based on the distance of the target object relative to the center point of the aerial image, combined with the geographical coordinates of the center point of the aerial image and the relationship between the image pixel coordinates and the actual geographical coordinates. This calculation method utilizes the pixel information of the target object and converts the target detection pixel coordinates into physical world latitude and longitude coordinates, which is highly efficient and greatly improves the accuracy of target positioning.

[0045] (2) The present invention provides a target recognition, localization and presentation method, device, equipment and storage medium, which fuses the features of adjacent layers into shallow small targets to provide effective contextual information, and adopts an improved cross-entropy loss function to alternately train and optimize the balance strategy to solve the problem of sample imbalance, thereby improving the mAP (mean accuracy) index of the target detection and recognition model.

[0046] (3) The present invention provides a target identification, positioning and presentation method, device, equipment and storage medium that, with the help of the three-dimensional digital terrain of the aerial photography area, realizes the visualization annotation, presentation and coordinate error measurement of the latitude and longitude coordinates of the target positioning result. It can combine key reference objects on the ground to visualize and measure the target positioning error and perform intuitive analysis. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a target identification, localization, and presentation method provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the channel attention module and spatial attention module structure provided in the embodiments of this application;

[0050] Figure 3 This is a schematic diagram of the adjacent layer feature fusion process provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram of the target object coordinates provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0053] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0054] On the one hand, this application provides a method for target identification, localization, and presentation. Figure 1Please refer to the flowchart illustrating a target recognition, localization, and presentation method provided in this application embodiment. Figure 1 The method includes the following steps:

[0055] (1) Acquire aerial images and rotate them according to the heading angle information of the aerial photography equipment so that the direction of the aerial images is the same as the direction of the geographic map;

[0056] Specifically, after the aerial images acquired by the aerial photography equipment are pre-processed, they need to be combined with the heading angle information of the aerial photography equipment at the time of shooting, that is, the angle information between the aerial photography equipment and the geographic north, and the image needs to be rotated clockwise according to the heading angle so that the north of the image coincides with the north direction of the geographic map, so that the second directional boundary of the image is parallel to the north-south direction.

[0057] (2) Establish a target detection model, input the rotated aerial image into the trained target detection model, and obtain an aerial image after the target object is identified;

[0058] Specifically, Figure 2 This is a schematic diagram of the channel attention module and spatial attention module provided in an embodiment of this application. Please refer to... Figure 2 The channel attention module uses a channel attention mechanism to calculate the weights of each channel in the input image and to enhance or suppress the weights of different channels. The spatial attention module uses a spatial attention mechanism to supplement the channel attention and to increase the attention weights based on the direction of the channel.

[0059] First, feature maps of different resolutions from the aerial images are input into the channel attention module. The channel attention mechanism compresses the spatial dimension of the input feature map through a Squeeze operation, and then generates weights for each feature through an Excitation operation. These weights are applied to each of the original feature channels to learn the importance of different channels and enhance or suppress the weights of different channels. Then, the feature map after passing through the channel attention module is input into the spatial attention module. The spatial attention mechanism increases the attention weights based on the channel direction, resulting in a feature map with refined features.

[0060] Furthermore, shallow feature maps are selected from all the feature maps after feature refinement and fused with adjacent deep feature maps containing contextual information. The fused feature maps are then integrated using nonlinear operations and convolution operations to identify the target object.

[0061] Specifically, Figure 3 This is a schematic diagram of the adjacent layer feature fusion process provided in an embodiment of this application. Please refer to... Figure 3 First, the deep feature map F n F' is obtained by feature learning after a 2×2 deconvolution operation. nAt this time, feature map F' n The scale is the deep feature map F n Twice that of the shallow feature map F to be fused n-1 The scales are the same; then the adjacent shallow feature maps F are... n-1 After a 3×3 convolution operation, the feature map F' is obtained by feature integration through a ReLU nonlinear operation layer. n-1 Finally, feature maps F' with the same scale and channel dimension are... n and F' n-1 Feature fusion is achieved by point-to-point addition. The sum is then passed through a ReLU nonlinear operation layer and a 3×3 convolutional layer to integrate the fused features, resulting in the prediction layer P. n-1 Where n≥1 and is a natural number, representing the number of layers in the feature map.

[0062] The Adjacent Connection Module (ACM) selects two adjacent layers of features for fusion. While extracting more abstract semantic features, it integrates shallow and deep features into the same feature domain. This avoids the problem of background interference caused by a large receptive field due to multiple connections, provides effective contextual information for small targets, and enhances the feature representation of targets from the perspective of UAVs. The fused features play a positive role in target detection.

[0063] Typical object detection networks use the cross-entropy loss function to perform classification tasks. This loss function has the same weight for all samples, meaning each sample contributes equally to the overall loss. However, due to the severe imbalance in drone and vehicle datasets, the accumulated loss from a large number of simple samples contributes significantly to the overall loss, while the contribution from a small number of difficult samples is negligible. This makes it difficult for the network to extract discriminative features.

[0064] To address this issue, this application employs an alternating training method based on an improved cross-entropy loss function, wherein the improved cross-entropy loss function sets a modulation factor (1-p). t )γ, so that each sample can be predicted according to the classification probability p t The size of the adjustment factor automatically adjusts its contribution to the overall loss function. In other words, this adjustment factor can automatically reduce the contribution of simple samples to the loss function and increase the weight of difficult samples, so that the network can focus more on training difficult samples during the training process, thereby improving the sample imbalance problem to a certain extent.

[0065] The improved cross-entropy loss function is defined as shown in equation (1).

[0066] L(p t )=-α(1-pt )γlog(p t (1)

[0067] Where the adjustable parameter γ≥0, α is the weighting factor, and p t Defined as in equation (2),

[0068]

[0069] Where p represents the probability that the identified image content is the target object.

[0070] As γ increases, the loss value of easily classified samples (samples with higher probability or simple samples) becomes smaller and smaller, while the loss value of difficult-to-classify samples (samples with lower probability or difficult samples) becomes larger and larger. Therefore, the network can reduce the contribution of simple samples to the loss function and increase the weight of difficult samples, so that the network can focus more on training difficult samples.

[0071] In the first stage, the cross-entropy loss function is used to perform preliminary iterative training on the network to learn the overall sample feature distribution. Simple samples can obtain discriminative feature representations in this stage, which allows difficult samples to improve their discriminative features by using the feature representations of simple sample cards.

[0072] In the second stage, the modulation factor (1-p) is used. t By increasing the weight of difficult samples (large trucks), more attention is given to these samples, thus ensuring they receive sufficient training. This alternating training strategy can improve the performance of UAV vehicle target detection.

[0073] (3) Establish a first coordinate system in the identified aerial image and obtain the pixel position of the target object according to the camera parameters;

[0074] Specifically, Figure 4 A schematic diagram showing the coordinates of the target object provided in an embodiment of this application. Please refer to... Figure 4 Based on the camera parameters, the pixel size (X) of the entire aerial image is obtained. W ,Y H The pixel coordinates of the image center point are (X... C ,Y C The pixel coordinates of the target object are (X... T ,Y T The pixel coordinates of the first boundary point are (X... W ,Y H / 2), the pixel coordinates of the second boundary point are (X W / 2,Y H ); where the distance d between the first boundary point and the image center point in the first direction is d. hThe distance in the second direction is 0, and the distance in the second direction between the second boundary point and the image center point is d. w The distance in the first direction is 0, and the distance between the target object and the center point of the image in the first direction is d. yc The distance d between the target object and the center point of the image in the second direction is xc The first direction is the x-axis of the first coordinate system, which is the north-south direction geographically. The second direction is the y-axis of the second coordinate system, which is the east-west direction geographically.

[0075] (4) Establish a second coordinate system with the aerial photography equipment and the identified aerial image, and obtain the geographic coordinates of the target object based on the heading angle information, relative ground height and geographic coordinates of the aerial photography equipment;

[0076] Based on the absolute value of the difference between the geographic abscissa of the target object and the image center point, the absolute value of the difference between the geographic abscissa of the first boundary point and the image center point, the distance between the target object and the image center point in the first direction in the aerial image, and the proportional relationship between the distance between the first boundary point and the image center point in the first direction in the aerial image, a first equation is established; based on the absolute value of the difference between the pixel abscissa of the target object and the image center point, the absolute value of the difference between the pixel abscissa of the first boundary point and the image center point, the distance between the target object and the image center point in the first direction in the aerial image, and the proportional relationship between the distance between the first boundary point and the image center point in the first direction in the aerial image, a first equation is established. First, establish the second equation; second, establish the third equation based on the relationship between the geographic abscissa of the image center point, the geographic abscissa of the first boundary point, the distance between the first boundary point and the image center point in the aerial image along the first direction, and the first heading angle of the aerial photography equipment in the abscissa direction of the second coordinate system; third, establish the fourth equation based on the trigonometric relationship between the distance from the aerial photography equipment to the image center point in the first direction, the first heading angle, and the distance between the first boundary point and the image center point in the first direction; and fourth, establish the first set of equations based on the first equation, the second equation, the third equation, and the fourth equation to solve for the geographic abscissa of the target object.

[0077] Based on the absolute values ​​of the differences between the geographic ordinates of the target object and the image center point, the absolute values ​​of the differences between the geographic ordinates of the second boundary point and the image center point, the distance between the target object and the image center point in the second direction in the aerial image, and the proportional relationships between the distances between the first boundary point and the image center point in the second direction in the aerial image, a fifth equation is established. Based on the absolute values ​​of the differences between the pixel ordinates of the target object and the image center point, the absolute values ​​of the differences between the pixel ordinates of the second boundary point and the image center point, the distances between the target object and the image center point in the second direction in the aerial image, and the proportional relationships between the distances between the second boundary point and the image center point in the second direction in the aerial image, a fifth equation is established. First, establish the sixth equation. Second, establish the seventh equation based on the relationship between the geographic ordinate of the image center point, the geographic ordinate of the second boundary point, the distance between the second boundary point and the image center point in the aerial image along the second direction, and the second heading angle of the aerial photography equipment in the ordinate direction of the second coordinate system. Third, establish the eighth equation based on the trigonometric relationship between the distance from the aerial photography equipment to the image center point in the first direction, the second heading angle, and the distance between the second boundary point and the image center point in the aerial image along the second direction. Fourth, establish the second set of equations based on the fifth, sixth, seventh, and eighth equations above, and solve for the geographic ordinate of the target object.

[0078] Specifically, the geographic coordinates of the image center point are set to (N). C ,E C The geographic coordinates of the target object are (N). X ,E Y The geographic coordinates of the first boundary point are (N). B ,E B The geographic coordinates of the second boundary point are (N). D ,E D The orthographic projection of the aerial photography equipment falls exactly on the center point of the image, and the distance between the aerial photography equipment and the center point of the image is H. The angle between the orthographic projection line and the line connecting the aerial photography equipment and the first boundary point is the first heading angle α, and the angle between the orthographic projection line and the line connecting the aerial photography equipment and the second boundary point is the second heading angle β.

[0079] Furthermore, in the physical world, every 1 kilometer interval along a meridian (north-south direction), the longitude remains constant while the latitude changes by approximately 1 / 111 degree (111 units are km). Along a parallel (east-west direction), at latitude θ, the latitude remains constant while the longitude changes by approximately 1 / (111*cosθ) degree. Therefore, the problem can be transformed into finding the geographic latitude and longitude coordinates (N) of the target based on all these known conditions. X ,E YThe problem is as follows. Furthermore, the latitude and longitude here refer to the latitude and longitude of the Chinese region, specifically E longitude and N latitude. Below, we will discuss four cases based on the relative position of the target object and the image center point after rotation, and solve for the latitude and longitude coordinates of the target object.

[0080] In the rotated image, when the target object is in the area to the lower right of the image center, the following equations (3)-(6) are obtained based on the proportional relationship between the image pixel coordinates and the geographic coordinates.

[0081]

[0082]

[0083]

[0084] d h =Htanα (6)

[0085] The geographic latitude coordinates N of the target object can be obtained by solving equations (3)-(6). X As shown in equation (7),

[0086]

[0087] Similarly, we can obtain the following equations (8)-(10).

[0088]

[0089]

[0090]

[0091] d w =Htanβ (11)

[0092] The geographic longitude coordinates E of the target object are calculated according to equations (8)-(11). Y As shown in equation (12),

[0093]

[0094] Finally, the geographic latitude and longitude coordinates (N) of the target object are obtained. X ,E Y As shown in equation (13)

[0095]

[0096] Similarly, in the rotated image, when the target object is in the area to the upper right of the image center, the geographic latitude and longitude coordinates of the target object are obtained as shown in equation (14).

[0097]

[0098] In the rotated image, when the target object is located in the area to the lower left of the image center, the geographic latitude and longitude coordinates of the target object are obtained as shown in Equation (15).

[0099]

[0100] In the rotated image, when the target object is located in the area above and to the left of the image center, the geographic coordinates of the target object are obtained as shown in Equation (16).

[0101]

[0102] In summary, the latitude and longitude coordinates of the target object can be expressed as equation (17).

[0103]

[0104] (5) Based on the geographic coordinates of the target object, the positioning target is visualized and marked on the pre-generated three-dimensional digital terrain of the aerial photography area. The actual error of the target positioning is measured by combining the rotated aerial photography image and the key landform references on the three-dimensional digital map.

[0105] In one specific embodiment, vehicle targets with a target detection confidence level of 0.78 are identified and located, and the resulting positioning error is approximately 10 meters, which can meet the target positioning accuracy requirements of fixed-wing UAV aerial photography in most industries, such as fiber optic cable line inspection.

[0106] On the other hand, this application also provides a target recognition, positioning and presentation device, which includes an image acquisition module, a target recognition module, a target positioning module and an error evaluation module.

[0107] The image acquisition module is used to perform image preprocessing on the aerial images acquired orthogonally by the aerial photography equipment, and then rotate the image clockwise according to the heading angle so that the direction of the aerial image is the same as the direction of the geographic map, that is, the north of the image coincides with the north direction of the geographic map, and the vertical boundary of the image is parallel to the north-south direction.

[0108] The target recognition module inputs the rotated aerial image into the target recognition module. After operations such as feature map refinement and feature fusion, the target objects in the image are identified.

[0109] The target recognition module comprises a spatial and channel dual attention module and a feature fusion module. Images captured by aerial photography equipment often contain misleading geographical features and easily confused non-targets due to their large coverage area. The channel and spatial dual attention module extracts areas of high interest, resisting misleading information and focusing on useful targets. This module is simple, effective, and lightweight; integrating it into a convolutional neural network allows for the extraction of key regions from feature maps and enables end-to-end training. The feature fusion module selects adjacent feature layers for fusion, avoiding the problem of excessively large receptive fields introducing background interference caused by multiple connections. This provides effective contextual information for small targets, enhancing the feature representation of weak targets like vehicles from the UAV's perspective.

[0110] The target localization module solves for the geographic coordinates of the target object by establishing a set of multiple equations based on the distance of the target object relative to the center point of the aerial image, the geographic coordinates of the center point of the aerial image, and the relationship between the image pixel coordinates and the actual geographic coordinates.

[0111] The error assessment module visualizes and marks the target object on a pre-generated 3D digital terrain of the aerial photography area based on the target object's geographical coordinates. It then combines the rotated aerial images with key landform references on the 3D digital map to measure the actual error in the target positioning.

[0112] This application also provides a target recognition, localization, and presentation device, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of the above-described target visual recognition, localization, and presentation method.

[0113] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0114] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0118] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0120] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0121] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for target recognition, localization, and presentation, characterized in that, Includes the following steps: Acquire aerial images and rotate them according to the heading angle information of the aerial photography equipment so that the direction of the aerial images is the same as that of the geographic map; A target detection model is established, and the rotated aerial image is input into the trained target detection model to obtain an aerial image after the target object is identified. A first coordinate system is established in the identified aerial image, and the pixel position of the target object is obtained according to the camera parameters; A second coordinate system is established using the aerial photography equipment and the identified aerial image. The geographic coordinates of the target object are obtained based on the heading angle information, relative ground altitude, and geographic coordinates of the aerial photography equipment. Specifically, this includes: establishing a first equation based on the ratio between the absolute value of the difference between the geographic abscissa of the target object and the image center point, the absolute value of the difference between the geographic abscissa of the first boundary point and the image center point, the distance in the first direction between the target object and the image center point in the aerial image, and the proportional relationship between the distance in the first direction between the first boundary point and the image center point in the aerial image; and establishing a first equation based on the absolute value of the difference between the pixel abscissa of the target object and the image center point, the absolute value of the difference between the pixel abscissa of the first boundary point and the image center point, and the distance in the first direction between the target object and the image center point in the aerial image. A second equation is established based on the proportional relationship between the distance in one direction and the distance in the first direction between the first boundary point and the center point of the image in the aerial image. A third equation is established based on the relationship between the geographic abscissa of the center point of the image, the geographic abscissa of the first boundary point, the distance in the first direction between the first boundary point and the center point of the image in the aerial image, and the first heading angle of the aerial equipment in the first direction. A fourth equation is established based on the trigonometric relationship between the distance from the aerial equipment to the center point of the image, the first heading angle, and the distance in the first direction between the first boundary point and the center point of the image in the aerial image. A first set of equations is established based on the first equation, the second equation, the third equation, and the fourth equation to solve for the geographic abscissa of the target object. Based on the geographic coordinates of the target object, the positioning target is visualized and marked on the pre-generated 3D digital terrain of the aerial photography area.

2. The target identification, localization, and presentation method as described in claim 1, wherein, A first coordinate system is established in the identified aerial image, and the pixel position of the target object is obtained according to the camera parameters, specifically including: Based on the camera parameters, obtain the pixel size of the entire aerial image, the pixel coordinates of the image center point, and the pixel coordinates of the target object; Based on the pixel size of the entire aerial image, the pixel coordinates of the first boundary point and the second boundary point are obtained; wherein, the distance between the first boundary point and the center point of the image in a first direction is 0, and the distance between the second boundary point and the center point of the image in a second direction is 0, wherein the first direction and the second direction are perpendicular to each other and are both horizontal directions.

3. The target identification, localization, and presentation method as described in claim 2, wherein, A second coordinate system is established using the aerial photography equipment and the labeled aerial images. The geographic coordinates of the target object are obtained based on the heading angle information, relative ground altitude, and geographic coordinates of the aerial photography equipment. This also includes: Based on the absolute value of the difference between the geographic vertical coordinates of the target object and the center point of the image, the absolute value of the difference between the geographic vertical coordinates of the second boundary point and the center point of the image, the distance between the target object and the center point of the image in the second direction in the aerial image, and the proportional relationship between the distance between the second boundary point and the center point of the image in the second direction in the aerial image, a fifth equation is established. Based on the absolute value of the difference between the pixel ordinates of the target object and the center point of the image, the absolute value of the difference between the pixel ordinates of the second boundary point and the center point of the image, the distance between the target object and the center point of the image in the second direction in the aerial image, and the proportional relationship between the distance between the second boundary point and the center point of the image in the second direction in the aerial image, a sixth equation is established. Based on the relationship between the geographic ordinate of the image center point, the geographic ordinate of the second boundary point, the distance between the second boundary point and the image center point in the second direction in the aerial image, and the second heading angle of the aerial photography equipment in the second direction, the seventh equation is established. Based on the trigonometric relationship between the distance from the aerial photography equipment to the center point of the image, the second heading angle, and the distance between the second boundary point in the aerial image and the center point of the image in the second direction, the eighth equation is established; Based on the fifth, sixth, seventh, and eighth equations mentioned above, establish the second set of equations to solve for the geographic coordinates of the target object.

4. The target recognition, localization, and presentation method as described in claim 1, wherein the rotated aerial image is input into the trained target detection model to obtain an aerial image with the target object identified, specifically including: The feature maps of aerial images at different resolutions are input into the channel attention module and the spatial attention module. The attention weight of each feature map is calculated sequentially along the two different dimensions of channel and space, and adaptive feature refinement is performed. The refined feature maps of all features are then output.

5. The target recognition, localization, and presentation method as described in claim 4, wherein the rotated aerial image is input into the trained target detection model to obtain the identified target object image, further includes: In the refined feature map, adjacent shallow feature maps and deep feature maps are selected for feature fusion. The fused feature maps are then integrated using nonlinear operations and convolution operations to identify the target object.

6. The target recognition, localization, and presentation method as described in claim 4, wherein the target detection model is trained alternately based on a cross-entropy loss function, the cross-entropy loss function being defined as follows: Adjustable parameters ≥ 0, As a weighting factor, The definition is as follows: in, This indicates the probability that the identified image content is the target object; By adjusting the adjustable parameters The size of the sample is increased, and the weight of difficult samples is increased to improve detection performance.

7. A target recognition, positioning, and presentation device, characterized in that, include: The image acquisition module is used to acquire aerial images and rotate them according to the heading angle information of the aerial photography equipment so that the direction of the aerial images is the same as the direction of the geographic map. The target recognition module establishes a target detection model and inputs the rotated aerial image into the trained target detection model to obtain an aerial image after the target object is identified. The target localization module establishes a first coordinate system in the identified aerial image and obtains the pixel position of the target object based on camera parameters. A second coordinate system is established using the aerial photography equipment and the identified aerial image. The geographic coordinates of the target object are obtained based on the aerial photography equipment's heading angle, relative ground altitude, and geographic coordinates. Specifically, this includes: establishing a first equation based on the ratio between the absolute value of the difference between the geographic abscissa of the target object and the image center point, the absolute value of the difference between the geographic abscissa of the first boundary point and the image center point, the distance between the target object and the image center point in the first direction in the aerial image, and the proportional relationship between the distance between the first boundary point and the image center point in the first direction in the aerial image; and establishing a first equation based on the absolute value of the difference between the pixel abscissa of the target object and the image center point, the absolute value of the difference between the pixel abscissa of the first boundary point and the image center point, and the distance between the target object and the image center point in the first direction in the aerial image. A second equation is established based on the proportional relationship between the distance in one direction and the distance in the first direction between the first boundary point and the center point of the image in the aerial image. A third equation is established based on the relationship between the geographic abscissa of the center point of the image, the geographic abscissa of the first boundary point, the distance in the first direction between the first boundary point and the center point of the image in the aerial image, and the first heading angle of the aerial equipment in the first direction. A fourth equation is established based on the trigonometric relationship between the distance from the aerial equipment to the center point of the image, the first heading angle, and the distance in the first direction between the first boundary point and the center point of the image in the aerial image. A first set of equations is established based on the first equation, the second equation, the third equation, and the fourth equation to solve for the geographic abscissa of the target object. The error assessment module visualizes and marks the target object on a pre-generated 3D digital terrain of the aerial photography area based on the geographical coordinates of the target object.

8. A target recognition, positioning, and presentation device, characterized in that, It includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of the method according to any one of claims 1 to 6.

9. A storage medium, characterized in that, It stores a computer program that can be executed by a target visual recognition, positioning and presentation device. When the computer program is run on the target visual recognition, positioning and presentation device, the target visual recognition, positioning and presentation device performs the steps of the method according to any one of claims 1 to 6.

Images