A target detection method based on key point positioning in remote sensing images

Abstract

The present invention provides a method, device, electronic device and readable storage medium for target detection based on key point positioning in remote sensing images. By defining a rotating bounding box based on labels of feature points, compared with existing methods, it avoids the problem of angular periodicity and the problem of length and width interchange, saves training costs, and can represent the orientation information of the target. The model proposed by the present invention adopts an anchor-free frame design to identify central key points, directional boundary points and lateral widths, introduces improvements such as a two-dimensional Taylor expansion method of a heat map and a guided offset module, reduces quantization errors while improving the detection accuracy of the model and reducing the recognition error rate. The algorithm model proposed by the present invention effectively enhances the target detection capability of remote sensing images with a large field of view, accurately and quickly obtains the coordinates and rotation orientation information of the target of interest, and effectively improves the detection efficiency of the model.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method, device, electronic device and readable storage medium for target detection based on key point positioning in remote sensing images. Background Art

[0002] Remote sensing technology typically involves carrying various image sensors aboard aircraft such as airplanes, balloons, and satellites at altitudes ranging from hundreds of meters to thousands of kilometers. The sensors collect information about ground features within the survey area according to technical requirements, enabling applications in production and scientific research. With the rapid development of image sensing and aerospace technology, the acquisition of remote sensing images has become increasingly convenient. Detecting targets of interest is a key area of ​​remote sensing imagery application. Research on how to quickly and accurately extract and utilize effective information from remote sensing images is crucial for civilian applications, such as disaster management and urban development analysis. Because remote sensing images typically contain enormous amounts of data, manual interpretation for target identification results in low data utilization, poor timeliness, and vulnerability to physical, mental, and subjective influences. Therefore, employing computer vision technology to reduce labor costs and achieve efficient automated detection is crucial.

[0003] Compared to images taken from a ground-level perspective, the difficulties of remote sensing images are: (1) Remote sensing images are captured by high-altitude, high-resolution imaging equipment, resulting in large pixel sizes and a wide imaging range. (2) Targets in remote sensing images are unevenly distributed, and localized dense scenes, such as vehicles and ships, are prone to appear. (3) In remote sensing images, the orientation angle of targets is arbitrary, and targets are rotationally invariant.

[0004] Traditional object detection algorithms, primarily based on sliding window regions, suffer from redundant selection strategies and high time complexity. Furthermore, hand-crafted features lack robustness to diverse changes. Their limited use cases make them incapable of meeting the current requirements for object detection in remote sensing imagery. Object detection algorithms based on shallow machine learning are also being phased out due to their limited feature learning capabilities and inability to process large amounts of data.

[0005] With the tremendous development of deep learning, particularly deep convolutional neural networks, in image processing, object detection in optical remote sensing images using deep convolutional neural networks offers significant advantages over traditional manual feature extraction methods. However, existing deep learning models are not well suited for detecting densely packed rotating objects in remote sensing imagery. Current deep learning models for rotating object detection suffer from issues such as length-width swapping and angular periodicity, hindering model training.

[0006] Therefore, based on the characteristics of remote sensing images, designing a deep learning model that avoids the length-width interchange problem and the angle periodicity problem and improving the detection ability of densely rotating targets in remote sensing images is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0007] In view of this, an embodiment of the present invention provides a target detection method based on key point positioning in remote sensing images, which effectively enhances the target detection capability of large-field-of-view remote sensing images, accurately and quickly obtains the coordinates and rotation orientation information of the target of interest, and effectively improves the detection efficiency of the model.

[0008] In a first aspect, the present invention provides a method for target detection based on key point positioning in a remote sensing image, comprising:

[0009] Acquiring an optical remote sensing image, and preprocessing the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images;

[0010] Inputting the sub-remote sensing image into a pre-trained target detection model, obtaining parameter information of the bounding box through the network neck, and obtaining a sub-target recognition result of the sub-remote sensing image;

[0011] The target detection result of the optical remote sensing image is obtained by splicing the multiple sub-target recognition results.

[0012] As an optional solution, before inputting the sub-remote sensing image into a pre-trained target detection model, obtaining parameter information of the bounding box through the network neck, and obtaining the sub-target recognition result of the sub-remote sensing image, the method further includes:

[0013] The training set is used to train the detection model to be trained to obtain the target detection model with optimized model parameters.

[0014] As an optional solution, before using the training set to train the detection model to obtain the target detection model with optimized model parameters, the method further includes:

[0015] Obtain an optical remote sensing training image with a target of interest marked thereon, and crop and segment the optical remote sensing training image according to the target pixel size to obtain a plurality of sub-remote sensing training images;

[0016] Data enhancement processing is performed on the multiple sub-remote sensing training images to obtain a training set.

[0017] As an optional solution, the target detection model adopts a deep feature fusion network, shallow feature information and deep feature information are interconnected and calculated, and a feature pyramid structure is used to detect targets of different pixel sizes.

[0018] As an optional solution, the target detection model is divided into a key point detection module, a lateral width detection module, and a guide offset module. The heat map output by the key point detection module obtains the relative coordinate information of the key point by calculating the response extreme point. The coordinate value is a floating-point number obtained by Taylor expansion.

[0019] The horizontal width detection module is used to predict the horizontal width of the target bounding box, where the horizontal width is the bounding box size information perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points;

[0020] The guide offset module calculates the position deviation of the center point coordinates and the direction boundary point to match the two key points, obtains the key point pair of the target through the guide offset, and combines the horizontal width information to form all the label parameter information of the rotating bounding box.

[0021] As an optional solution, the data enhancement method includes at least one of affine transformation, noise addition, artificial minority class oversampling method SMOTE (Synthetic Minority Over-sampling Technique), and occlusion.

[0022] As an optional solution, the parameter information of the bounding box can be represented as a data with 7 tuples .in, represents the center of the target bounding box, is the x'-axis dimension of the target, It is the direction boundary point, and the vector between the center point and the direction boundary point is the orientation information of the target.

[0023] In a second aspect, the present invention provides a target detection device based on key point positioning in a remote sensing image, comprising:

[0024] an acquisition unit, configured to acquire an optical remote sensing image and preprocess the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images;

[0025] an identification unit, configured to input the sub-remote sensing image into a pre-trained target detection model, obtain parameter information of a bounding box through a network neck, and obtain a sub-target identification result of the sub-remote sensing image;

[0026] The splicing unit is used to splice the multiple sub-target recognition results to obtain the target detection result of the optical remote sensing image.

[0027] In a third aspect, the present invention provides an electronic device, comprising:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned target detection method based on key point positioning in remote sensing images.

[0031] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the above-mentioned target detection method based on key point positioning in remote sensing images.

[0032] The present invention also provides a method, device, electronic device and readable storage medium for target detection based on key point positioning in remote sensing images. By defining a rotating bounding box based on feature point labels, compared with existing methods, it avoids the angle periodicity problem and the length and width interchange problem, saves training costs, and can represent the target's orientation information. The model proposed by the present invention adopts an anchor-free frame design to identify the central key point, directional boundary point and lateral width, and introduces thermal Figure 2 Improvements such as the Vitalik expansion method and the guided offset module reduce quantization errors while improving the model's detection accuracy and lowering the recognition error rate. The proposed algorithm effectively enhances target detection capabilities in large-field-of-view remote sensing imagery, accurately and quickly acquiring the coordinates and rotational orientation information of targets of interest, effectively improving the model's detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a method for target detection based on key point positioning in a remote sensing image is provided in an embodiment of the present invention;

[0034] Figure 2 Schematic diagram of the length-width interchange problem and angle periodicity problem in the prior art;

[0035] Figure 3 A schematic diagram of parameters of a rotating bounding box based on key points in a target detection method based on key point positioning in a remote sensing image is provided in an embodiment of the present invention;

[0036] Figure 4 A structural diagram of a key point-based rotating bounding box detection model in a target detection method based on key point positioning in a remote sensing image is provided in an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of a target detection model guided offset module in a target detection method based on key point positioning in a remote sensing image is provided in an embodiment of the present invention;

[0038] Figure 6A schematic structural diagram of a target detection device based on key point positioning in a remote sensing image is provided in an embodiment of the present invention;

[0039] Figure 7 A structural block diagram of an electronic device is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0041] The terms "first," "second," "third," "fourth," and the like in the specification and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] Combine Figure 1 As shown, the present invention provides a target detection method based on key point positioning in remote sensing images, comprising:

[0043] S101 : Acquire an optical remote sensing image, and pre-process the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images.

[0044] S102: Input the sub-remote sensing image into a pre-trained target detection model, obtain parameter information of the bounding box through the network neck, and obtain a sub-target recognition result of the sub-remote sensing image.

[0045] S103 , performing splicing processing on the multiple sub-target recognition results to obtain a target detection result of the optical remote sensing image.

[0046] It should be noted that the target detection model can be trained in advance, that is, a pre-trained target detection model, and there is no need to train the model in each detection.

[0047] In some embodiments, before inputting the sub-remote sensing image into a pre-trained target detection model, obtaining parameter information of a bounding box through a network neck, and obtaining a sub-target recognition result of the sub-remote sensing image, the method further includes:

[0048] The training set is used to train the detection model to be trained to obtain the target detection model with optimized model parameters.

[0049] In some embodiments, before using the training set to train the detection model to be trained to obtain the target detection model with optimized model parameters, the method further includes:

[0050] Obtain an optical remote sensing training image with a target of interest marked thereon, and crop and segment the optical remote sensing training image according to the target pixel size to obtain a plurality of sub-remote sensing training images;

[0051] Data enhancement processing is performed on the multiple sub-remote sensing training images to obtain a training set.

[0052] In some embodiments, the target detection model uses a deep feature fusion network, shallow feature information and deep feature information are interconnected and calculated, and a feature pyramid structure is used to detect targets of different pixel sizes.

[0053] In some embodiments, the target detection model is divided into a key point detection module, a lateral width detection module, and a guide offset module. The heat map output by the key point detection module obtains the relative coordinate information of the key point by calculating the response extreme point, and the coordinate value is a floating-point number obtained by Taylor expansion calculation;

[0054] The horizontal width detection module is used to predict the horizontal width of the target bounding box, where the horizontal width is the bounding box size information perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points;

[0055] The guide offset module calculates the position deviation of the center point coordinates and the direction boundary point to match the two key points, obtains the key point pair of the target through the guide offset, and combines the horizontal width information to form all the label parameter information of the rotating bounding box.

[0056] In some embodiments, the data enhancement method includes at least one of affine transformation, noise addition, artificial minority class oversampling method SMOTE (Synthetic Minority Over-sampling Technique), and occlusion.

[0057] In some embodiments, the parameter information of the bounding box can be represented as a data tuple with 7 .in, represents the center of the target bounding box, is the x'-axis dimension of the target, It is the direction boundary point, and the vector between the center point and the direction boundary point is the orientation information of the target.

[0058] The present invention also provides a target detection method based on key point positioning in remote sensing images. By defining a rotating bounding box based on the labels of feature points, compared with existing methods, it avoids the angle periodicity problem and the length and width interchange problem, saves training costs, and can represent the target's orientation information. The model proposed by the present invention adopts an anchor-free frame design to identify the central key point, direction boundary point and horizontal width, and introduces thermal Figure 2 Improvements such as the Vitalik expansion method and the guided offset module reduce quantization errors while improving the model's detection accuracy and lowering the recognition error rate. The proposed algorithm effectively enhances target detection capabilities in large-field-of-view remote sensing imagery, accurately and quickly acquiring the coordinates and rotational orientation information of targets of interest, effectively improving the model's detection efficiency.

[0059] An embodiment of the present invention also provides a method for detecting an object in a remote sensing image based on key point positioning, comprising:

[0060] S201 , cropping an imported remote sensing image with a large pixel size to obtain a plurality of cropped slices, so that the cropped slices conform to an input size of the model.

[0061] S202, constructing a backbone network, using the constructed backbone network to extract features of the remote sensing image, and performing feature fusion on deep features and shallow features;

[0062] S203: The feature maps output by the backbone network are respectively imported into the neck of the network, and the parameter information constituting the rotation bounding box is further calculated through different convolutional layers.

[0063] S204: The detection results are fused and sent to the prediction head to obtain the detection results of the cropped clips of the remote sensing image.

[0064] S205: stitch the cropped clips into the original image, remove redundant objects, and obtain the final detection result.

[0065] It should be noted that the cropped clip corresponds to the sub-remote sensing image in S101.

[0066] In S201, in order to prevent the target from being split due to cropping, there are overlapping pixels on the edge during cropping. After cropping with a suitable size, the image is scaled to the input pixel size of the model.

[0067] The backbone network in S202 adopts a U-shaped network design. The feature information is condensed through the compression path and then parsed through the expansion path. The network connects the shallow network with the deep network to fully extract the feature information of the target.

[0068] The network neck in S203 includes a key point detection module, a lateral width detection module, and a guide offset module, which respectively obtain target key point coordinate information, lateral width information, and key point matching information.

[0069] The key point detection module outputs in the form of a heat map. The output is normalized to between 0 and 1 by the sigmoid function. The extreme value of the response value is the coordinate position of the corresponding key point. Since there are two key coordinates, the center point and the direction boundary point, for an input image of size , need to be tested C The task of classifying objects of interest, the corresponding heat map output size is ,in S is the step size corresponding to the original image.

[0070] During model training for the key point detection module, the true value is calculated using a 2D Gaussian function. The variance σ is related to the length and width of the true bounding box. The resulting 2D Gaussian information is rotated to align with the true bounding box through an affine transformation of the image. The heat map output by the key point detection module calculates the relative coordinates of the key points by calculating the extreme response points. The coordinate values ​​are floating-point numbers calculated using Taylor expansion.

[0071] The horizontal width detection module is responsible for predicting the horizontal width of the target bounding box. The horizontal width refers to the bounding box size information perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points.

[0072] The guided offset module calculates the positional deviation between the center point coordinates and the directional boundary points to match two keypoints, preventing mismatches between keypoints of different targets. The guided offset ultimately yields a keypoint pair for the target, which, combined with the horizontal width information, forms the complete label parameter information for the rotated bounding box.

[0073] In S204, the prediction head uses a joint encoding method of key point pairs (center point coordinates, direction boundary point coordinates) and horizontal dimensions for output. The bounding box encoding method used can not only well describe the angle of the rotating target, but also well indicate the orientation information of the target.

[0074] The bounding box encoding method adopts the method of key point pairs and lateral dimensions, and adopts the method of key point regression to avoid the angular periodicity and length-width interchangeability problems existing in the existing encoding methods, effectively improving the efficiency of model training and the recognition accuracy.

[0075] In the remote sensing image process in S205, due to the existence of overlapping edges, repeated target recognition may occur. The present invention suppresses edge targets through the non-maximum suppression method NMS to reduce the problem of repeated target detection caused by image splicing.

[0076] The present invention also provides a target detection method based on key point positioning in remote sensing images, and provides a new rotating bounding box representation method, which avoids the length and width interchange problems and angle periodicity problems existing in the current rotating bounding box representation method.

[0077] like Figure 2 As shown, it is shown that the existing technology has problems of length and width interchangeability and angle periodicity. The OpenCV representation, long side representation, and corner point definition method of the rotating bounding box make it difficult for the proposal box to learn the real rotating box in the optimal direction, which is not conducive to the training and prediction of target detection tasks in remote sensing images.

[0078] The model provided in the embodiment of the present invention is first trained using a labeled training set to obtain model parameters, and then the data to be detected is input for detection to obtain the location of the target of interest.

[0079] The target detection method based on key point positioning in remote sensing images provided by the present invention uses a target detection model to perform target detection tasks on remote sensing images, and mainly includes the following steps:

[0080] S1: The imported remote sensing image with large pixel size is cropped to the target adaptive size, and the image size is scaled to match the input size of the model.

[0081] S2: The training images are first preprocessed, including denoising, dedistortion, and data enhancement, to construct a training set.

[0082] S3-1: Build a backbone network (BackBone) to extract feature information at different depths of the input image, and perform feature fusion on deep features and shallow features to obtain information about the target of interest.

[0083] S3-2: The feature maps output by the backbone network are respectively imported into the network neck, and the horizontal size information, key point coordinate information, and key point matching information of the rotation bounding box are further calculated through different convolutional layers.

[0084] S3-3: The detection results are fused and sent to the prediction head, and the combined data is used to obtain the detection results of the cropped image.

[0085] S4: Calculate the loss function of the cropped image and use stochastic gradient descent (SGD) optimization to obtain the model parameters.

[0086] S5: For the image to be predicted, use step S1 to crop the image to be detected, and use step S3 to perform target detection on the cropped image.

[0087] S6: Arrange the cropped image detection results output by the model, stitch the cropped images into the original images, remove redundant objects, and obtain the final detection results.

[0088] The rotation frame marking method of the present invention is as follows Figure 3 As shown, for the target k The bounding box of , which can be represented as a rotated bounding box with 7 tuples .in, is the center of the target bounding box, is the target center coordinate output by the heat map, It is the target x'-axis Direction size, Is the orientation boundary point (Orientation Boundary Point), the vector between the center point and the orientation boundary point is the orientation information of the corresponding target. y'-axis The side is wide ( ), parallel to x'-axis The side is high ( ).

[0089] like Figure 4 As shown, the network neck described in step S3-2 includes a key point detection module, a lateral width detection module, and a guide offset module, which respectively obtain target key point coordinate information, lateral width information, and key point matching information.

[0090] The key point detection module described in step S3-2 performs detection using a heat map.

[0091] When the key point detection module is trained, the input visible light remote sensing image Middle k The first category of target m The corresponding key points will first be linearly mapped to the scale of the output feature plane of the prediction network. Above, among them s is the size scaling ratio (stride).

[0092] In order to avoid quantization errors during heatmap training, the quantization process in encoding is canceled during the heatmap generation process. , the present invention adopts 2D Gaussian kernel:

[0093] ;

[0094] in for x'-axisThe variance of the direction, for y'-axis The present invention introduces angle adaptation to expand the Gaussian kernel function and better adapts to the rotation bounding box:

[0095] ;

[0096] in θ is the rotation angle. The key point heat map generated is , C is the number of target types to be detected.

[0097] The calculation process of the heat map loss function can be described as:

[0098] ;

[0099] When predicting, the key point detection module first uses the third order to adjust the heat map to make it approximate Gaussian distribution. The filtering process can be expressed as:

[0100] ;

[0101] ;

[0102] ;

[0103] in M is the normalization parameter, For Variable Gaussian filter kernel of position, variance Determines the weight distribution of feature fusion.

[0104] Then, the present invention adopts the maximum activation point of the heat map Taylor expansion method to calculate Gaussian extreme points , the calculation formula is as follows:

[0105] ;

[0106] Both the first-order partial derivative and the second-order partial derivative can be effectively calculated through the heat map, and the present invention can accurately calculate the information of the key point coordinates.

[0107] The guide offset module described in step S3-2 is as follows Figure 5 As shown, the positional deviation between the center point coordinates and the directional boundary points is calculated to match the two key points, preventing mismatches between key points of different targets. By guiding the offset, the target key point pair is finally obtained, which is further combined with the horizontal width information to form all the label parameter information of the rotated bounding box.

[0108] The guide offset value refers to the vector between the direction boundary point and the center point. For the label value The rotation bounding box of the bounding box is defined as The true value of the bootstrap offset label for:

[0109] ;

[0110] Where s is the size scaling factor of the current layer.

[0111] The present invention indirectly calculates the coordinates of the four grid points surrounding the direction boundary point:

[0112] ;

[0113] Through 4 grid points, It can be expressed as:

[0114] ;

[0115] ;

[0116] The loss value of the guided offset module applies Smooth L1 loss:

[0117] ;

[0118] Where N is the true direction boundary point in the training sample The number of

[0119] The horizontal size information described in step S3-2 is responsible for predicting the horizontal width of the target bounding box. The horizontal width refers to the size information of the bounding box perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points.

[0120] The horizontal size information is directly obtained through model network learning, and the width of the true value of the label Using SmoothL1 loss:

[0121] ;

[0122] Where N is the number of bounding boxes in the training sample, the coefficient (1- w ) facilitates training on small-sized objects.

[0123] Total loss of the present invention By size regression loss , key point localization loss and offset loss Composed of 3 scalar weights:

[0124] .

[0125] The present invention provides a complete method for detecting rotating targets in remote sensing images based on key point positioning. The rotating bounding box annotation method designed in this invention overcomes the issues of angular periodicity and length-width interchangeability, effectively improving model training efficiency and reducing boundary issues. The anchor-free detection model, designed based on the fact that targets of interest in remote sensing images do not overlap, eliminates the computational complexity associated with predicting anchor boxes, enabling real-time, high-precision model detection. Taylor expansion is used during the model's calculations to avoid quantization errors, further improving the model's detection accuracy.

[0126] Simulations show that the present invention is practical and effective, achieving the expected results, with higher detection efficiency and accuracy than existing detection methods and greater robustness.

[0127] Combine Figure 6 As shown, accordingly, an embodiment of the present invention provides a target detection device based on key point positioning in a remote sensing image, comprising:

[0128] An acquisition unit 601 is configured to acquire an optical remote sensing image and pre-process the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images;

[0129] The recognition unit 602 is configured to input the sub-remote sensing image into a pre-trained target detection model, obtain parameter information of the bounding box through a network neck, and obtain a sub-target recognition result of the sub-remote sensing image;

[0130] The splicing unit 603 is configured to splice the plurality of sub-target recognition results to obtain a target detection result of the optical remote sensing image.

[0131] The target detection device based on key point positioning in remote sensing images provided in the embodiment of the present invention defines a rotating bounding box based on the labels of feature points. Compared with the existing methods, it avoids the angle periodicity problem and the length and width interchange problem, saves training costs, and can represent the orientation information of the target. The model proposed by the present invention adopts an anchor-free frame design to identify the central key point, direction boundary point and horizontal width, and introduces thermal Figure 2 Improvements such as the Vitalik expansion method and the guided offset module reduce quantization errors while improving the model's detection accuracy and lowering the recognition error rate. The proposed algorithm effectively enhances target detection capabilities in large-field-of-view remote sensing imagery, accurately and quickly acquiring the coordinates and rotational orientation information of targets of interest, effectively improving the model's detection efficiency.

[0132] Accordingly, according to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0133] Figure 7 FIG. 1 is a structural diagram of an electronic device 12 provided in an embodiment of the present invention. Figure 7 A block diagram of an exemplary electronic device 12 suitable for implementing embodiments of the present invention is shown. Figure 7 The electronic device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0134] like Figure 7 As shown, electronic device 12 is represented in the form of a general-purpose computing device. Electronic device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0135] Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16 , a system memory 28 , and a bus 18 that connects various system components, including system memory 28 and processing unit 16 .

[0136] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0137] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0138] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 7 Not shown, often called a "hard drive"). Although Figure 7 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0139] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0140] The electronic device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 22. Furthermore, the electronic device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0141] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the target detection method based on key point positioning in remote sensing images provided by an embodiment of the present invention.

[0142] An embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein when the program is executed by a processor, the target detection method based on key point positioning in remote sensing images provided in all the inventive embodiments of this application is implemented.

[0143] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device.

[0144] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] The program code that comprises on the computer-readable medium can be transmitted with any appropriate medium, includes but not limited to wireless, electric wire, optical cable, RF etc., or above-mentioned any suitable combination.Can write the computer program code that is used to carry out the operation of the present invention with one or more programming languages ​​or its combination, described programming language comprises object-oriented programming language such as Java, Smalltalk, C++, also comprises conventional procedural programming language--such as " C " language or similar programming language.Program code can be carried out on user's computer completely, partly on user's computer, carry out as an independent software package, partly on user's computer partly on remote computer, or carry out completely on remote computer or server.In the situation that relates to remote computer, remote computer can comprise local area network (LAN) or wide area network (WAN) to be connected to user's computer by the network of any kind, perhaps, can be connected to external computer (for example, utilize Internet service provider to come to connect by Internet).

[0146] An embodiment of the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned target detection method based on key point positioning in remote sensing images.

[0147] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0148] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A target detection method based on key point positioning in remote sensing images, characterized in that: include: Acquiring an optical remote sensing image, and preprocessing the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images; The sub-remote sensing image is input into the pre-trained target detection model, and the feature maps output by the U-shaped backbone network are respectively imported into the network neck. The parameter information of the bounding box is obtained through the network neck. The prediction head adopts the joint encoding method of the center point coordinates, the direction boundary point coordinates and the lateral size to output and obtain the sub-target recognition result of the sub-remote sensing image; the target detection model is divided into a key point detection module, a lateral width detection module and a guide offset module. The heat map output by the key point detection module obtains the relative coordinate information of the key points by calculating the response extreme points. The coordinate values ​​are floating-point numbers obtained by Taylor expansion. During model training, the true value of the key point detection module is calculated by a 2D Gaussian function. The variance σ is related to the length and width information of the true bounding box. The obtained two-dimensional Gaussian information is rotated to the same angle as the true bounding box through the affine transformation of the image. When the key point detection module is trained, the first k The first category of target m The corresponding key points are first linearly mapped to the scale of the output feature plane of the prediction network. When generating the heatmap, the quantization process in encoding is canceled. For the mapped feature points, a 2D Gaussian kernel is used to calculate the key point heat map, and the Gaussian kernel function is expanded by introducing angle adaptation. When predicting, the key point detection module first adjusts the heat map to make it approximate to a Gaussian distribution. The Gaussian extreme point is calculated using the Taylor expansion method at the maximum activation point of the heat map. The horizontal width detection module is used to predict the horizontal width of the target bounding box. The horizontal width is the size information of the bounding box perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points. The vector between the center point and the direction boundary points is the orientation information of the target. The guide offset module calculates the position deviation of the center point coordinates and the direction boundary point to match the two key points, obtains the key point pair of the target through the guide offset, and combines the horizontal width information to form all the label parameter information of the rotating bounding box; The target detection result of the optical remote sensing image is obtained by splicing the multiple sub-target recognition results, and the edge targets are suppressed by the non-maximum suppression method NMS.

2. The target detection method based on key point positioning in remote sensing images according to claim 1, characterized in that: Before inputting the sub-remote sensing image into a pre-trained target detection model, obtaining parameter information of a bounding box through a network neck, and obtaining a sub-target recognition result of the sub-remote sensing image, the method further includes: The training set is used to train the detection model to be trained to obtain the target detection model with optimized model parameters.

3. The target detection method based on key point positioning in remote sensing images according to claim 2, characterized in that: Before using the training set to train the detection model to be trained to obtain the target detection model with optimized model parameters, the method further includes: Obtain an optical remote sensing training image with a target of interest marked thereon, and crop and segment the optical remote sensing training image according to the target pixel size to obtain a plurality of sub-remote sensing training images; Data enhancement processing is performed on the multiple sub-remote sensing training images to obtain a training set.

4. The target detection method based on key point positioning in remote sensing images according to claim 1, characterized in that: The target detection model adopts a deep feature fusion network, shallow feature information and deep feature information are interconnected and calculated, and a feature pyramid structure is used to detect targets of different pixel sizes.

5. The target detection method based on key point positioning in remote sensing images according to claim 3, characterized in that: The data enhancement method includes at least one of affine transformation, noise addition, artificial minority oversampling method SMOTE (Synthetic Minority Over-sampling Technique), and occlusion.

6. The target detection method based on key point positioning in remote sensing images according to claim 1, characterized in that: The parameter information of the bounding box can be represented as data with 7 tuples ,in, represents the center of the target bounding box, is the x'-axis dimension of the target, is the direction boundary point.

7. A target detection device based on key point positioning in remote sensing images, characterized in that: include: an acquisition unit, configured to acquire an optical remote sensing image and preprocess the optical remote sensing image according to a target pixel size to obtain a plurality of sub-remote sensing images; The recognition unit is used to input the sub-remote sensing image into a pre-trained target detection model, import the feature maps output by the U-shaped backbone network into the network neck respectively, obtain the parameter information of the bounding box through the network neck, and output the prediction head using a joint encoding method of the center point coordinates, the direction boundary point coordinates and the lateral size to obtain the sub-target recognition result of the sub-remote sensing image; the target detection model is divided into a key point detection module, a lateral width detection module, and a guide offset module. The heat map output by the key point detection module obtains the relative coordinate information of the key points by calculating the response extreme points. The coordinate values ​​are floating-point numbers obtained by Taylor expansion. During model training, the true value of the key point detection module is calculated by a 2D Gaussian function. The variance σ is related to the length and width information of the true bounding box. The obtained two-dimensional Gaussian information is rotated to the same angle as the true bounding box through the affine transformation of the image. When the key point detection module is trained, the first k The first category of target m The corresponding key points are first linearly mapped to the scale of the output feature plane of the prediction network. When generating the heatmap, the quantization process in encoding is canceled. For the mapped feature points, a 2D Gaussian kernel is used to calculate the key point heat map, and the Gaussian kernel function is expanded by introducing angle adaptation. When predicting, the key point detection module first adjusts the heat map to make it approximate to a Gaussian distribution. The Gaussian extreme point is calculated using the Taylor expansion method at the maximum activation point of the heat map. The horizontal width detection module is used to predict the horizontal width of the target bounding box. The horizontal width is the size information of the bounding box perpendicular to the direction of the line connecting the center point coordinates and the direction boundary points. The vector between the center point and the direction boundary points is the orientation information of the target. The guide offset module calculates the position deviation of the center point coordinates and the direction boundary point to match the two key points, obtains the key point pair of the target through the guide offset, and combines the horizontal width information to form all the label parameter information of the rotating bounding box; The splicing unit is used to splice the multiple sub-target recognition results to obtain the target detection result of the optical remote sensing image, and suppress edge targets by using the non-maximum suppression method NMS.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target detection method based on key point positioning in remote sensing images according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the target detection method based on key point positioning in a remote sensing image according to any one of claims 1 to 6.

Images