Target object state detection method and device and electronic equipment

Abstract

The invention discloses a target object state detection method and device and electronic equipment. The method comprises the following steps: acquiring a first remote sensing image of a target area in a first time period and a second remote sensing image of the target area in a second time period; inputting the first remote sensing image into an object recognition model to obtain a first recognition result, and inputting the second remote sensing image into the object recognition model to obtain a second recognition result; registering the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determining first position information of a target object in the first remote sensing image and the second remote sensing image according to a registration result; and determining position change information of the target object according to the first identification result, the second identification result and the first position information. Through the method and the device, the problem of relatively low accuracy of a state detection method based on pixel difference in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, device and electronic device for detecting the state of a target object. Background Art

[0002] With the development of computer technology and remote sensing technology, the spatial resolution and temporal resolution of remote sensing images have been greatly improved, providing richer information for image analysis using remote sensing images and time series. In some scenarios, remote sensing images can be used to identify objects such as aircraft and vehicles to determine whether the object has changed position.

[0003] Currently, when detecting changes in target objects, the commonly used method is the detection method based on pixel differences, such as the frame difference method. However, when detecting small-sized targets such as aircraft and vehicles, the optical remote sensing images may be affected by clouds and lighting changes, resulting in changes in the characteristics of the target objects such as aircraft and vehicles, or the characteristics of the target objects cannot be detected, thereby affecting the accuracy of detection.

[0004] With regard to the problem of low accuracy of the state detection method based on pixel difference in the related art, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a method, device and electronic device for detecting the state of a target object to solve the problem of low accuracy of the state detection method based on pixel difference in the related art.

[0006] According to one aspect of the present application, a method for detecting the state of a target object is provided. The method includes: obtaining a first remote sensing image of a target area in a first time period, and a second remote sensing image of the target area in a second time period; inputting the first remote sensing image into an object recognition model to obtain a first recognition result, and inputting the second remote sensing image into an object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target objects in the first remote sensing image and the second remote sensing image through multiple preset recognition scales; aligning the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determining the first position information of the target object in the first remote sensing image and the second remote sensing image according to the alignment result; determining the position change information of the target object by using the first recognition result, the second recognition result and the first position information.

[0007] Optionally, inputting the first remote sensing image into the object recognition model to obtain the first recognition result includes: performing data enhancement operation on the first remote sensing image to obtain a processed first remote sensing image; processing the processed first remote sensing image by sliding window clipping to obtain multiple first initial images, wherein each first initial image carries the second position information of the first initial image in the first remote sensing image; inputting each first initial image into the object recognition model respectively to obtain multiple initial recognition results, wherein the initial recognition result contains at least one recognition frame, and the recognition frame is used to mark the target object; combining the multiple initial recognition results according to the second position information of each first initial image to obtain a candidate recognition result; determining the target confidence of each recognition frame in the candidate recognition result, and filtering the multiple recognition frames in the candidate recognition result according to the target confidence to obtain a filtered recognition frame; determining the candidate recognition result containing the filtered recognition frame as the first recognition result.

[0008] Optionally, determining the target confidence of each recognition frame in the candidate recognition result includes: for any recognition frame, determining whether there is an overlapping recognition frame that overlaps with the recognition frame; in the absence of an overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence, wherein the initial confidence is generated by the object recognition model; in the presence of an overlapping recognition frame, comparing the initial confidence of the recognition frame with the initial confidence of the overlapping recognition frame; in the case where the initial confidence of the recognition frame is greater than or equal to the initial confidence of the overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence; in the case where the initial confidence of the recognition frame is less than the initial confidence of the overlapping recognition frame, reducing the initial confidence of the recognition frame according to the initial confidence of the overlapping recognition frame to obtain the target confidence.

[0009] Optionally, multiple recognition boxes in the candidate recognition results are screened according to the target confidence to obtain the screened recognition box, including: determining whether the target confidence is greater than a first preset value; when the target confidence is greater than or equal to the first preset value, retaining the recognition box corresponding to the target confidence; when the target confidence is less than the first preset value, deleting the recognition box corresponding to the target confidence; and determining the retained recognition box as the screened recognition box.

[0010] Optionally, registering the first remote sensing image with the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image includes: sequentially obtaining the Euclidean distance between each first feature point in the first feature point set and each second feature point in the second feature point set, and determining the second feature point corresponding to each first feature point according to the Euclidean distance to obtain M pairs of feature points, where M is a positive integer; selecting N pairs of feature points from the M pairs of feature points, and constructing a transformation model through the N pairs of feature points, where the transformation model is used to transform the N pairs of feature points, where N is less than M; selecting P pairs of feature points from the M pairs of feature points according to the transformation model, and registering the first remote sensing image with the second remote sensing image through the correspondence between the P pairs of feature points, where P is greater than N.

[0011] Optionally, selecting P pairs of feature points from M pairs of feature points according to the transformation model includes: for any pair of feature points, calculating the Euclidean distance between the feature points to obtain a first distance value; inputting the pair of feature points into the transformation model for transformation operation to obtain an updated pair of feature points; calculating the Euclidean distance of the updated pair of feature points to obtain a second distance value; when the difference between the first distance value and the second distance value is less than a second preset value, selecting the pair of feature points as P pairs of feature points.

[0012] Optionally, determining the second feature point corresponding to each first feature point according to the Euclidean distance to obtain M pairs of feature points includes: for any first feature point, calculating the Euclidean distance between the first feature point and each second feature point to obtain multiple third distance values; sorting the multiple third distance values ​​in order from small to large, and obtaining the two candidate distance values ​​with the smallest serial numbers among the multiple third distance values; calculating the ratio of the two candidate distance values, and determining whether the ratio is less than a preset threshold; when the ratio is less than the preset threshold, determining that the second feature point to which the candidate distance value with the smallest serial number belongs is a pair of feature points with the first feature point, and obtaining M pairs of feature points; when the ratio is greater than or equal to the preset threshold, determining that the first feature point has no corresponding second feature point.

[0013] Optionally, using the first recognition result, the second recognition result and the first position information to determine the position change information of the target object includes: obtaining the first position information of each target object in the first recognition result to obtain a first position set; obtaining the first position information of each target object in the second recognition result to obtain a second position set; determining whether the first position set and the second position set are the same; if the first position set and the second position set are not the same, determining that the position change information of the target object is that the position has changed; if the first position set and the second position set are the same, determining that the position change information of the target object is that the position has not changed.

[0014] According to another aspect of the present application, a state detection device for a target object is provided. The device includes: a first acquisition unit, used to acquire a first remote sensing image of a target area in a first time period, and a second remote sensing image of the target area in a second time period; a first recognition unit, used to input the first remote sensing image into an object recognition model to obtain a first recognition result, and input the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target objects in the first remote sensing image and the second remote sensing image through multiple preset recognition scales; a first configuration unit, used to align the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determine the first position information of the target object in the first remote sensing image and the second remote sensing image according to the alignment result; a first determination unit, used to determine the position change information of the target object by using the first recognition result, the second recognition result and the first position information.

[0015] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements a state detection method of a target object provided in the aforementioned embodiment of the present application.

[0016] According to another aspect of the present invention, an electronic device is also provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein when the computer-readable instructions are executed, a state detection method of a target object provided in the aforementioned embodiment is executed.

[0017] Through this application, the following steps are adopted: obtaining a first remote sensing image of a target area in a first time period, and a second remote sensing image of a target area in a second time period; inputting the first remote sensing image into an object recognition model to obtain a first recognition result, and inputting the second remote sensing image into an object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target object in the first remote sensing image and the second remote sensing image through a plurality of preset recognition scales; registering the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determining the first position information of the target object in the first remote sensing image and the second remote sensing image according to the registration result; determining the position change information of the target object according to the first recognition result, the second recognition result and the first position information. The problem of low accuracy of the state detection method based on pixel difference in the related art is solved. By acquiring remote sensing images of the target area in two time periods, and identifying the target objects contained in the two remote sensing images through a recognition model containing multiple preset recognition scales, the target objects can be accurately identified from the remote sensing images, and by aligning the images containing the identified objects, the position error between the images is reduced. After the alignment, it is determined whether the position of the target object has changed based on the position information of the target object in the two images, thereby achieving the effect of accurately determining whether the position state of the target object has changed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flow chart of a method for detecting a state of a target object provided in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of optional recognition results provided according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a target object state detection device provided according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] It should be noted that the target object state detection method, device and electronic device determined in the present disclosure can be used in the field of computer vision, and can also be used in any field other than the field of computer vision. The application field of the target object state detection method, device and electronic device determined in the present disclosure is not limited.

[0027] It should be noted that the collected information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) used in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated. Corresponding operation entrances are provided for users to choose to authorize use or refuse use. If the user chooses to refuse, the expert decision-making process is entered. For example, an interface is set between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0028] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0029] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0030] Remote sensing image: image data of ground locations obtained through remote sensing technology.

[0031] YOLOv5: You Only Look Once v5, an object detection algorithm.

[0032] SIFT algorithm: Scale-Invariant Feature Transform, scale-invariant feature transform algorithm, is an algorithm used for image feature extraction and matching. It can extract unique key points in the image and perform image matching and target recognition through these key points.

[0033] PANet: Path Aggregation Network, path aggregation network explanation, is a network structure used to more effectively aggregate features at different levels in the feature pyramid network to enhance the performance of target detection and segmentation tasks.

[0034] Mosaic Data Augmentation: Mosaic Data Augmentation is an image preprocessing technology used in target detection training. It increases the diversity of training data and improves the generalization ability of the model by randomly cropping and splicing multiple images into a new image.

[0035] Conv: Convolution layer is the core component in convolutional neural networks. It slides the convolution kernel on the input image to detect spatial hierarchical structural features, such as edges and textures, for image feature extraction.

[0036] CSPNet: Cross Stage Partial Networks, a neural network structure that dynamically splits and reorganizes feature maps between different stages, improves learning efficiency and feature expression capabilities, and reduces the memory consumption of the model.

[0037] C3: CSP Bottleneck Block, a cross-stage bottleneck block, is a structure in the CSPNet network module, consisting of multiple Bottleneck layers, which aims to reduce the number of parameters and calculations while maintaining the diversity of features and optimizing the feature extraction process.

[0038] Boottleneck: Bottleneck layer is a design pattern in deep learning networks. It forms a "bottleneck" by reducing the number of channels in the middle layer to reduce the computational cost while maintaining the expressiveness of the model.

[0039] CNN: Convolutional neural network is a deep neural network specially designed to process grid-structured data (such as images). Through convolution, pooling and fully connected layers, it can automatically learn the local correlation features of input data.

[0040] SPPF: Spatial Pyramid Pooling with Fixed Grid Size, a variant of SPP, generates a fixed-length feature vector by performing pooling operations on fixed grids of different scales, which helps to process input images of variable sizes and extract multi-scale features.

[0041] According to an embodiment of the present application, a method for detecting a state of a target object is provided.

[0042] Figure 1 is a flow chart of a method for detecting the state of a target object provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0043] Step S101, acquiring a first remote sensing image of a target area in a first time period, and a second remote sensing image of the target area in a second time period.

[0044] Specifically, the first time period and the second time period are two different time points, and the target area may be the area where the target object that needs to be identified and monitored is located. By acquiring the first remote sensing image in the first time period and the second remote sensing image in the second time period, the target object in the remote sensing image can be identified to determine whether the state of the target object has changed.

[0045] It should be noted that since the acquisition of remote sensing images may be affected by many factors such as satellite revisit period and weather conditions, after obtaining the remote sensing images, preliminary quality inspection and preprocessing of the images are required to ensure the consistency and availability of the images.

[0046] Step S102, inputting the first remote sensing image into the object recognition model to obtain a first recognition result, and inputting the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target objects in the first remote sensing image and the second remote sensing image through multiple preset recognition scales.

[0047] It should be noted that the object recognition model can be a deep learning-based model, such as YOLOv5, which is used to automatically identify and locate target objects in remote sensing images. The preset recognition scale can be the model's ability to detect targets at different resolutions, including small-scale, medium-scale, and large-scale detection grids to accommodate targets of different sizes.

[0048] Specifically, when remote sensing images of two time periods are obtained, the first remote sensing image and the second remote sensing image can be respectively input into an object recognition model, such as a YOLOv5 multi-scale target recognition model, to perform target detection. The model can effectively identify target objects from remote sensing images by performing detection on feature maps of different scales, thereby improving the comprehensiveness and accuracy of target detection.

[0049] It should be noted that when the object recognition model is trained, the processing flow used is as follows:

[0050] 1) Preparation (performed in Input): Perform data enhancement operations on remote sensing images (such as Mosaic data enhancement). On this basis, different sizes of anchor boxes are given for models with different parameter quantities (that is, the size of the recognition box for identifying objects. For small object detection, a smaller recognition box can be used).

[0051] 2) Feature extraction (performed in Backbone): Use the basic structures of Conv, C3, and SPPF to extract features from the input image. Conv is used to downsample the input; C3 is used to extract and fuse features from the input to enrich the semantic information of the features. In this process, Boottleneck is used to reduce the number of parameters and calculations, and the CSPNet network is used to enhance the learning ability of CNN; SPPF uses pooling and feature fusion to enrich the semantic information of the features, so that the deepest feature map has extremely rich semantic information.

[0052] 3) Processing features (performed in Neck): The shallow features are fused to the feature maps of the three scales for target detection (shallow features are conducive to detection). PANet is used to fuse the shallow features to the extracted feature maps, so that the feature maps have both rich semantic information and accurate location information of the object.

[0053] 4) Predict target (performed in the Head): Identify the processed feature map to obtain the recognition result of the target object, and adjust the parameters according to the recognition result to obtain the object recognition model.

[0054] It should be noted that targets such as aircraft and ships in remote sensing images usually have small physical sizes and occupy a limited number of pixels in the image. This makes feature extraction and detection of small targets more difficult because they are easily affected by pixel-level noise and image blur. Therefore, a small target detection layer can be added to detect the shallower feature map after splicing it with the deep feature map. Adding a small target detection layer allows the network to pay more attention to the detection of small targets and improve the detection effect. The small target detection layer can be added in the following ways:

[0055] 1) Modify anchors parameters: The anchors parameters of the model are adjusted. Anchors are used to define anchor boxes at different scales so that the model can better match the target. In the original YOLOv5 configuration, the P3 / 8, P4 / 16, and P5 / 32 anchor boxes are retained unchanged, but a new set of anchor boxes suitable for small targets is added. In this way, the model can better adapt to the size and scale of small targets, and the final detection layer is modified to adapt to the new anchors parameters.

[0056] 2) Modify head parameters: The head parameters of the model are adjusted to adapt to the detection of small targets. The head part of YOLOv5 is responsible for feature fusion and target detection operations. In the original feature fusion structure, a new feature fusion section is added to improve the perception of small targets, and the connection method of the feature fusion part is adjusted to better fuse feature information of different scales.

[0057] With these modifications, the model becomes more suitable for detecting small targets, improving the detection precision and accuracy of small targets in remote sensing images.

[0058] It should be noted that, when the object recognition model is used in this embodiment, in order to improve the recognition accuracy of the model, the grid scale is increased to three types, and the grid size and receptive field of each scale are different. Small-scale grids (such as P3 / 8) are suitable for detecting small targets, medium-scale grids (such as P4 / 16) are suitable for medium-sized targets, and large-scale grids (such as P5 / 32) are better at detecting large targets such as buildings and airports, where the values ​​here (such as 8, 16, 32) represent the downsampling multiples relative to the input image. For example, the P3 / 8 feature map indicates that the resolution of the feature map is 1 / 8 of the input image. Such a feature map usually contains richer spatial details and is suitable for detecting smaller targets; the feature maps of P4 / 16 and P5 / 32 have lower resolutions and larger receptive fields, which can capture the features of larger targets and longer-distance context information, thereby greatly reducing missed detections and having the ability of multi-scale detection. Thus, targets of different scales can be detected simultaneously, and remote sensing images of different resolutions can be adapted to better cope with multi-scale changes.

[0059] For example, taking the YOLOv5 network as an example, the model extracts feature maps of different scales from the input image through multiple downsampling layers (such as convolution operations). In the detection stage, the network upsamples and fuses these feature maps of different scales so that each scale of feature maps can contribute to the final detection result. Specifically, the YOLOv5 network may upsample and fuse the P5 / 32 feature map into the P4 / 16 feature map, and then upsample and fuse the fused feature map into the P3 / 8 feature map, thereby forming a feature representation containing multi-scale information for final target detection. Therefore, by fusing shallow features containing more detailed information, the model can better detect small targets in the image.

[0060] Step S103, registering the first remote sensing image and the second remote sensing image according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image, and determining first position information of the target object in the first remote sensing image and the second remote sensing image according to the registration result.

[0061] It should be noted that the key feature points of the image can be extracted by the SIFT algorithm, so that the image registration operation can be performed through the feature points.

[0062] Specifically, when performing registration, the SIFT algorithm can be used to extract feature points of the first remote sensing image and the second remote sensing image to form a first feature point set and a second feature point set. Then, the two remote sensing images are accurately aligned in space by matching these feature points, thereby completing the registration operation.

[0063] After the registration is completed, the position information of the identified target object on the remote sensing image can be obtained, for example, a coordinate system can be constructed on the remote sensing image to determine the position information of the target object on the remote sensing image, and then determine whether the target object has been transformed based on the position information.

[0064] Step S104: Determine the position change information of the target object using the first recognition result, the second recognition result and the first position information.

[0065] Specifically, after obtaining the first recognition result and the second recognition result, it is possible to compare the quantity information of the target objects identified in the recognition results and the position information of each target object, so as to determine whether there are problems with the target objects being added, reduced, or shifted, and then determine the position change information of the target objects, and determine whether the position of the target objects has changed through the position change information, thereby achieving the effect of accurately determining whether the position status of the target objects has changed.

[0066] This embodiment uses SIFT to accurately register dual-phase remote sensing images, eliminates pseudo-changes caused by differences in imaging conditions, and improves the accuracy of change detection. In addition, based on the YOLOv5 network, a small target detection layer is added, feature fusion and target prediction parts are improved, and a multi-scale reasoning mechanism is introduced to ensure that the network can detect targets at small, medium, and large scales, reduce missed detections, and perform better in small target detection.

[0067] Through the comprehensive application of the above methods, the present invention can effectively improve the accuracy and reliability of target change detection in multi-temporal remote sensing images, reduce false alarm rate and missed detection rate, and especially has a significant performance improvement for change detection of small targets.

[0068] The state detection method of the target object provided in the embodiment of the present application is to obtain a first remote sensing image of the target area in a first time period and a second remote sensing image of the target area in a second time period; input the first remote sensing image into the object recognition model to obtain a first recognition result, and input the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target object in the first remote sensing image and the second remote sensing image through multiple preset recognition scales; aligns the first remote sensing image and the second remote sensing image according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image, and determines the first position information of the target object in the first remote sensing image and the second remote sensing image according to the alignment result; and determines the position change information of the target object according to the first recognition result, the second recognition result and the first position information. The problem of low accuracy of the state detection method based on pixel difference in the related art is solved. By acquiring remote sensing images of the target area in two time periods, and identifying the target objects contained in the two remote sensing images through a recognition model containing multiple preset recognition scales, the target objects can be accurately identified from the remote sensing images, and by aligning the images containing the identified objects, the position error between the images is reduced. After the alignment, it is determined whether the position of the target object has changed based on the position information of the target object in the two images, thereby achieving the effect of accurately determining whether the position state of the target object has changed.

[0069] Optionally, in the state detection method of the target object provided in the embodiment of the present application, the first remote sensing image is input into the object recognition model to obtain the first recognition result, including: performing data enhancement operation on the first remote sensing image to obtain a processed first remote sensing image; processing the processed first remote sensing image by sliding window clipping to obtain multiple first initial images, wherein each first initial image carries the second position information of the first initial image in the first remote sensing image; inputting each first initial image into the object recognition model respectively to obtain multiple initial recognition results, wherein the initial recognition result contains at least one recognition box, and the recognition box is used to mark the target object; combining the multiple initial recognition results according to the second position information of each first initial image to obtain a candidate recognition result; determining the target confidence of each recognition box in the candidate recognition result, and filtering the multiple recognition boxes in the candidate recognition result according to the target confidence to obtain the filtered recognition box; determining the candidate recognition result containing the filtered recognition box as the first recognition result.

[0070] It should be noted that in this embodiment, the processing method for the first remote sensing image and the processing method for the second remote sensing image are the same, and only the first remote sensing image is used as an example for description.

[0071] Specifically, after obtaining the first remote sensing image, it is necessary to perform data enhancement operation on the first remote sensing image to improve the content clarity of the remote sensing image and remove abnormal parts of the image, thereby improving the subsequent recognition accuracy of the content of the remote sensing image by processing the remote sensing image.

[0072] Optionally, after obtaining the processed first remote sensing image, the enhanced first remote sensing image can be processed by sliding window clipping, wherein the sliding window technology can divide the large-size remote sensing image into multiple smaller first initial images, which not only helps to reduce the computational burden of a single inference, but also ensures that the model can focus on the local area of ​​the image and improve the detection rate of small targets. When implementing sliding window clipping, it can be ensured that each first initial image carries its position information (second position information) in the original first remote sensing image, so that the detection result can be accurately mapped back to the original image later.

[0073] Optionally, after the remote sensing image is processed by sliding window clipping, each first initial image can be input into a trained object recognition model to obtain multiple initial recognition results, each of which includes one or more recognition boxes that mark the position of the target object in the first initial image, as well as the target confidence associated with it, wherein the confidence can characterize the accuracy of the recognition result.

[0074] For example, Figure 2 is a schematic diagram of optional recognition results provided according to an embodiment of the present application. Figure 2 As shown, Figure 2 The target object in the figure is a person. It can be seen that the model recognizes two target objects, one of which has a confidence level of 0.9 and the other has a confidence level of 0.8.

[0075] Optionally, when each initial recognition result is obtained, multiple initial recognition results can be combined according to the second position information of each first initial image, so that the recognition frames of the target object in each initial recognition result are combined to obtain a candidate recognition result, and the confidence of each target object in the candidate recognition result is recalculated according to the confidence of the target object in each initial recognition result to obtain the target confidence of each recognition frame, and the multiple recognition frames in the candidate recognition result are screened according to the target confidence to obtain the screened recognition frame, so that the candidate recognition result containing the screened recognition frame is determined as the first recognition result, thereby eliminating the recognition frame with low confidence to reduce false alarms and improve recognition accuracy, so as to obtain a more accurate first recognition result.

[0076] This embodiment processes the first remote sensing image by using a sliding window clipping technology to divide it into multiple first initial images, then processes each first initial image using an object recognition model to obtain an initial recognition result, and finally, through the combination of position information and screening according to confidence, ensures the accuracy and reliability of the recognition result, reduces false alarms, and obtains the first recognition result, thereby optimizing the target recognition efficiency in large-size remote sensing images, while ensuring the recognition accuracy and the accuracy of the first recognition result.

[0077] Optionally, in the state detection method of the target object provided in an embodiment of the present application, determining the target confidence of each recognition frame in the candidate recognition result includes: for any recognition frame, judging whether there is an overlapping recognition frame that overlaps with the recognition frame; in the absence of an overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence, wherein the initial confidence is generated by the object recognition model; in the presence of an overlapping recognition frame, comparing the initial confidence of the recognition frame with the initial confidence of the overlapping recognition frame; in the case where the initial confidence of the recognition frame is greater than or equal to the initial confidence of the overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence; in the case where the initial confidence of the recognition frame is less than the initial confidence of the overlapping recognition frame, reducing the initial confidence of the recognition frame according to the initial confidence of the overlapping recognition frame to obtain the target confidence.

[0078] It should be noted that object recognition models (such as YOLOv5) generate an initial confidence score for each recognition box during the detection process, which reflects the model's confidence in the existence of the target object in the detection box.

[0079] Specifically, when calculating the target confidence, it is first necessary to determine any target recognition frame and determine whether the target recognition frame has an overlapping recognition frame. There are two reasons for the overlap. One is that the overlapping recognition frame and the target recognition frame are located in the same first initial image, and the other is that the overlapping recognition frame and the target recognition frame are located in different first initial images, and the target recognition frame is located in the overlapping area of ​​the images corresponding to the two initial recognition results.

[0080] The following three situations are explained:

[0081] 1. When there is no overlapping recognition box for the target recognition box, the initial confidence of the recognition box is directly determined as the target confidence, that is, the prediction result of the model for the target recognition box is considered valid and does not need to be adjusted.

[0082] 2. When there are overlapping target recognition frames and the target recognition frame is located in the overlapping area of ​​the images corresponding to the two initial recognition results, that is, the object recognition model recognizes the object in the target recognition frame in the two first initial images and generates an initial confidence. At this time, the average value of the initial confidence in the two recognition frames can be calculated, and the average value is determined as the target confidence of the target recognition frame.

[0083] 3. When there is an overlapping recognition frame of the target recognition frame, and the overlapping recognition frame and the target recognition frame are located in the same first initial image, the overlapping recognition frame may be an abnormal recognition result, or the target recognition frame may be an abnormal recognition result, or both recognition frames are correct recognition results. In order to determine the accuracy of the recognition result of each recognition frame, the confidence level needs to be adjusted, and then the recognition frames can be screened according to the confidence level in subsequent steps, so as to retain the correct recognition results and delete the abnormal recognition results.

[0084] Optionally, when there is an overlapping recognition frame of the target recognition frame, and the overlapping recognition frame and the target recognition frame are located in a first initial image, it is necessary to compare the initial confidence of the target recognition frame and the overlapping recognition frame. If the initial confidence of the target recognition frame is greater than or equal to the initial confidence of the overlapping recognition frame, that is, the confidence of the target recognition frame is higher, then the initial confidence of the target recognition frame can be directly determined as the final target confidence, indicating that the target recognition frame is a better detection result. Conversely, if the initial confidence of the target recognition frame is less than the initial confidence of the overlapping recognition frame, it indicates that the overlapping recognition frame has a higher confidence and is more likely to be a real target. At this time, the initial confidence of the recognition frame is lowered and adjusted to obtain a new target confidence. This adjustment helps to reduce false alarms and improve the accuracy of the detection results.

[0085] This embodiment ensures the accuracy of target detection by determining the overlap of the recognition frames, and comparing and dynamically adjusting the confidence of the recognition frames in the candidate recognition results according to the overlap. By comparing the confidence of the recognition frame and the overlapping recognition frame, the detection frame that is more likely to be the real target can be identified, and the recognition frame with lower confidence can be adjusted, avoiding misjudgment caused by overlapping multiple detection frames, reducing the false alarm rate, and improving the efficiency and accuracy of target detection.

[0086] Optionally, in the state detection method of the target object provided in an embodiment of the present application, multiple recognition frames in the candidate recognition results are screened according to the target confidence, and the screened recognition frame includes: judging whether the target confidence is greater than a first preset value; when the target confidence is greater than or equal to the first preset value, retaining the recognition frame corresponding to the target confidence; when the target confidence is less than the first preset value, deleting the recognition frame corresponding to the target confidence; and determining the retained recognition frame as the screened recognition frame.

[0087] Specifically, after the object recognition model completes the recognition of the target object, each recognition box is associated with a target confidence, which reflects the model's confidence in the existence of the target object in the recognition box. At this time, it can be determined whether the target confidence of each recognition box is greater than or equal to a set first preset value, where the preset value is used to distinguish between high-confidence and low-confidence recognition boxes, thereby eliminating possible false alarms.

[0088] For the target confidence of each recognition frame, if the target confidence is greater than or equal to the first preset value, it means that the recognition frame is reliable enough and meets the detection criteria, and the recognition frame is retained. On the contrary, if the target confidence of the recognition frame is lower than the first preset value, it means that the model has low confidence in the recognition frame, and there may be false positives or inaccurate detection results. Therefore, these recognition frames with low confidence are deleted to avoid their negative impact on the final detection results.

[0089] After completing the above judgment and screening process, all the retained high-confidence recognition frames are determined as the screened recognition frames, so that the recognition frames retained after screening the candidate recognition results constitute the first recognition results for further analysis and application. Through the above process, it can be ensured that the recognition frames included in the detection results are the results of high-confidence predictions of the model, thereby improving the accuracy and reliability of the entire recognition system and reducing the false alarm rate.

[0090] This embodiment distinguishes between high-confidence and low-confidence recognition frames by setting a threshold, thereby retaining the recognition frame that is most likely to represent the real target and deleting the recognition frame with low confidence and possible false alarm, thereby improving the accuracy of detection and filtering out irrelevant or false alarm detection results, thereby improving the accuracy of change detection of target objects in remote sensing images.

[0091] Optionally, in the state detection method of the target object provided in the embodiment of the present application, the first remote sensing image and the second remote sensing image are aligned according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image, including: sequentially obtaining the Euclidean distance between each first feature point in the first feature point set and each second feature point in the second feature point set, and determining the second feature point corresponding to each first feature point according to the Euclidean distance, to obtain M pairs of feature points, where M is a positive integer; selecting N pairs of feature points from the M pairs of feature points, and constructing a transformation model through the N pairs of feature points, where the transformation model is used to transform the N pairs of feature points, and N is less than M; selecting P pairs of feature points from the M pairs of feature points according to the transformation model, and aligning the first remote sensing image and the second remote sensing image through the correspondence between the P pairs of feature points, where P is greater than N.

[0092] Specifically, when registering two remote sensing images, the SIFT algorithm can be used to extract a first feature point set from the first remote sensing image and a second feature point set from the second remote sensing image, respectively. The SIFT algorithm can detect and describe key feature points in the image, which remain unchanged under scale and rotation changes, and can therefore be effectively used for image registration. For each detected feature point, a descriptor containing information about its surrounding pixels is generated.

[0093] Optionally, after selecting the feature points, the Euclidean distance between each first feature point in the first feature point set and all second feature points in the second feature point set can be calculated in sequence, wherein the Euclidean distance is a standard for measuring the distance between two points, which can intuitively reflect the similarity between the feature points. Based on the calculated Euclidean distance, the second feature point corresponding to each first feature point can be determined, that is, the second feature point with the closest distance can be found, so that the first feature point and the second feature point are combined into a pair of feature points, and M pairs of feature point pairs are obtained.

[0094] It should be noted that after obtaining M pairs of feature point pairs, since there may be inaccurate feature point pairs among the M pairs of feature point pairs, it is necessary to select N pairs of feature point pairs from the M pairs of feature point pairs and build a transformation model based on the N pairs of feature points, that is, the RANSAC (Randoms Sample Consensus) algorithm model, so as to eliminate incorrect matching point pairs according to the model.

[0095] It should be noted that the RANSAC algorithm is essentially to construct an initial model parameter, and then use data to repeatedly test and iterate the initial model to find a model with the greatest correlation among all points and obtain a transformation model. In the RANSAC algorithm, the calculation and optimization of the transformation model is an iterative process. The algorithm first randomly selects a small number of point pairs from the matching feature point pairs as samples and calculates a preliminary transformation model parameter. Then, the algorithm checks the degree of conformity of the remaining feature point pairs with this preliminary model, that is, calculates the distance error of these feature point pairs after applying the model transformation. If the error is less than the preset threshold, the point pair is considered to be an inlier, that is, it has a high correlation with the model and can accurately reflect the geometric relationship between the two images. The RANSAC algorithm retains these inliers and uses them to recalculate the model parameters, and then continues to iterate until the optimal model parameters that maximize the number of inliers are found. This method can effectively eliminate incorrect matching point pairs, ensure the accuracy of the transformation model, and thus improve the accuracy of image registration.

[0096] For example, more than 4 pairs of points can be randomly sampled from the matching feature points of the two images, the initial transformation model parameters can be calculated, and then all matching feature point pairs can be traversed to iterate and search for internal points. It is determined whether other feature point pairs except the sample points satisfy this model. If they satisfy, they are retained. The pending model is recalculated using the selected matching point internal points, and other matching points are searched for iteratively again until the conditions are met, thereby obtaining the transformation model.

[0097] After selecting N pairs of feature point pairs to generate a transformation model, the transformation model can be used to select P pairs of feature point pairs from the M pairs of feature point pairs, so as to use the feature point pairs with higher accuracy for the final image registration, thereby ensuring the generalization ability of the model and the accuracy of the registration. Through the correspondence between the feature points, the first remote sensing image and the second remote sensing image are accurately aligned to eliminate the influence of changes in imaging conditions (such as lighting, viewing angle, atmospheric conditions, etc.).

[0098] This embodiment constructs and optimizes the transformation model by detecting, describing and matching SIFT feature points, combining Euclidean distance calculation and RANSAC algorithm, so as to accurately generate and select the feature point pairs required for registration, and finally achieve accurate registration of two remote sensing images. This method can not only handle image differences caused by different imaging conditions, but also effectively improve registration accuracy and reduce errors.

[0099] Optionally, in the state detection method of the target object provided in the embodiment of the present application, selecting P pairs of feature points from M pairs of feature points according to the transformation model includes: for any pair of feature points, calculating the Euclidean distance between the feature points to obtain a first distance value; inputting the pair of feature points into the transformation model for transformation operation to obtain an updated pair of feature points; calculating the Euclidean distance of the updated pair of feature points to obtain a second distance value; when the difference between the first distance value and the second distance value is less than a second preset value, selecting the pair of feature points as P pairs of feature points.

[0100] Specifically, when selecting P pairs of feature points, for any pair of feature points in the M pairs of feature points (i.e., a first feature point and its corresponding second feature point), the Euclidean distance between them can be calculated to obtain a first distance value, and the pair of feature points can be input into the transformation model to perform a transformation operation to obtain an updated pair of feature points.

[0101] After the feature point pair is transformed, the Euclidean distance between the updated feature point pair can be calculated again to obtain a second distance value. By comparing the difference between the first distance value and the second distance value, it is possible to evaluate the degree of influence of the transformation model on the similarity between the feature points after the feature points are transformed, thereby determining whether the two feature points are corresponding points.

[0102] Optionally, a second preset value can be set as an evaluation criterion for the difference. If the difference between the first distance value and the second distance value is less than the second preset value, it is determined that the registration effect of the pair of feature points is good and meets the accuracy requirement. Therefore, this pair of feature points is taken as part of P pairs of feature points, thereby ensuring the accuracy of the feature point pairs used for the registration operation.

[0103] This embodiment evaluates and optimizes M pairs of feature point pairs by using a transformation model, and determines the matching degree between feature points by calculating the difference in Euclidean distance between feature point pairs before and after alignment, thereby selecting more accurate feature point pairs, and then ensuring the accuracy and reliability of the final alignment result by selecting feature point pairs (P pairs of feature points) that meet the accuracy requirements, effectively avoiding alignment errors caused by inaccurate feature point pairs.

[0104] Optionally, in the state detection method of the target object provided in the embodiment of the present application, the second feature point corresponding to each first feature point is determined according to the Euclidean distance to obtain M pairs of feature points, including: for any first feature point, the Euclidean distance between the first feature point and each second feature point is calculated to obtain multiple third distance values; the multiple third distance values ​​are sorted in order from small to large, and the two candidate distance values ​​with the smallest serial numbers among the multiple third distance values ​​are obtained; the ratio of the two candidate distance values ​​is calculated, and it is determined whether the ratio is less than a preset threshold; when the ratio is less than the preset threshold, it is determined that the second feature point to which the candidate distance value with the smallest serial number belongs is a pair of feature points with the first feature point, and M pairs of feature points are obtained; when the ratio is greater than or equal to the preset threshold, it is determined that the first feature point has no corresponding second feature point.

[0105] Specifically, when performing feature point matching, first, for any first feature point, the Euclidean distance between the feature point and each second feature point in the second feature point set can be calculated to obtain multiple third distance values, and these third distance values ​​are sorted in order from small to large, so as to quickly identify the second feature point closest to the first feature point.

[0106] Furthermore, after the distance values ​​are sorted, the two candidate distance values ​​with the smallest distances can be obtained, that is, the distances from the first feature point to the two nearest points in the second feature point set. Then, the ratio of the two candidate distance values ​​can be calculated, so as to determine whether there is a second feature point corresponding to the first feature point by the calculated ratio.

[0107] It should be noted that after obtaining the distance ratio, if the distance ratio is less than the preset threshold, it can be determined that the second feature point corresponding to the candidate distance value with the smallest sequence number is a pair of matching feature points with the first feature point, and thus it is determined as a pair of M pairs of feature points. On the contrary, if the ratio is greater than or equal to the preset threshold, it is considered that the first feature point has no matching second feature point in the second feature point set, thereby completing the feature point recognition and matching operation.

[0108] After obtaining M pairs of feature point pairs, the subsequent image registration process can be performed according to the feature point pairs, so as to achieve efficient registration of the two images through the feature point pairs and reduce the registration error.

[0109] For example, the feature vectors of all feature points in the first remote sensing image are traversed with the feature vectors of all feature points in the second remote sensing image, and the Euclidean distance d between the two is calculated. The Euclidean distance of the multidimensional space vector is used as the matching strength standard. In the traversal calculation process, the ratio of the nearest neighbor distance and the next nearest neighbor distance is used to eliminate false matches. If the ratio of the nearest neighbor distance and the next nearest neighbor distance is less than a certain threshold R, it is determined to be a pair of matching feature points.

[0110] This embodiment uses Euclidean distance calculation and distance ratio judgment to accurately match feature point pairs, ensuring the registration accuracy of the first remote sensing image and the second remote sensing image. By setting a preset threshold as the judgment standard of the distance ratio, high-confidence feature point pairs are intelligently identified and screened, avoiding the influence of mismatching and invalid matching point pairs on the registration results.

[0111] Optionally, in the state detection method of the target object provided in the embodiment of the present application, the position change information of the target object is determined by the first recognition result, the second recognition result and the first position information, including: obtaining the first position information of each target object in the first recognition result to obtain a first position set; obtaining the first position information of each target object in the second recognition result to obtain a second position set; determining whether the first position set and the second position set are the same; when the first position set and the second position set are not the same, determining that the position change information of the target object is that the position has changed; when the first position set and the second position set are the same, determining that the position change information of the target object is that the position has not changed.

[0112] Specifically, the first recognition result may be a first remote sensing image including an identification box and a confidence level, and the second recognition result may be a second remote sensing image including an identification box and a confidence level. A coordinate system may be generated in the images of the first remote sensing image and the second remote sensing image, and the positions of each identification box may be determined in the coordinate system.

[0113] Optionally, the position set of the identification frame in the first remote sensing image, that is, the first position set, and the position set of the identification frame in the second remote sensing image, that is, the second position set, can be compared to see if they are the same. If the first position set and the second position set are not the same, this indicates that the position of at least one target object has changed between the two shots. At this time, the position change information of the target object can be determined as "position has changed". On the contrary, if the two position sets are exactly the same, this indicates that the positions of all detected target objects have not changed in the first remote sensing image and the second remote sensing image. In this case, the position change information of the target object can be determined as "position has not changed", thereby completing the accurate identification of the position change of the target object.

[0114] This embodiment determines whether the target has moved by comparing the position information of the target object in the first remote sensing image and the second remote sensing image. By aggregating the position information in the first recognition result and the second recognition result and performing a direct comparison, the position change of the target object can be accurately and effectively identified, thereby achieving the effect of accurately determining whether the position state of the target object has changed.

[0115] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0116] The embodiment of the present application also provides a state detection device for a target object. It should be noted that the state detection device for a target object in the embodiment of the present application can be used to execute the state detection method for a target object provided in the embodiment of the present application. The state detection device for a target object provided in the embodiment of the present application is introduced below.

[0117] Figure 3 Schematic diagram of a target object state detection device provided according to an embodiment of the present application. Figure 3 As shown, the device includes: a first acquisition unit 31, a first identification unit 32, a first configuration unit 33, and a first determination unit 34.

[0118] The first acquisition unit 31 is used to acquire a first remote sensing image of a target area in a first time period and a second remote sensing image of the target area in a second time period.

[0119] The first recognition unit 32 is used to input the first remote sensing image into the object recognition model to obtain a first recognition result, and input the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes the target objects in the first remote sensing image and the second remote sensing image through multiple preset recognition scales.

[0120] The first configuration unit 33 is used to register the first remote sensing image and the second remote sensing image according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image, and determine the first position information of the target object in the first remote sensing image and the second remote sensing image according to the registration result.

[0121] The first determining unit 34 is configured to determine the position change information of the target object using the first recognition result, the second recognition result and the first position information.

[0122] The state detection device of the target object provided in the embodiment of the present application obtains the first remote sensing image of the target area in the first time period and the second remote sensing image of the target area in the second time period through the first acquisition unit 31; the first recognition unit 32 inputs the first remote sensing image into the object recognition model to obtain the first recognition result, and inputs the second remote sensing image into the object recognition model to obtain the second recognition result, wherein the object recognition model recognizes the target object in the first remote sensing image and the second remote sensing image through multiple preset recognition scales; the first configuration unit 33 aligns the first remote sensing image and the second remote sensing image according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image, and determines the first position information of the target object in the first remote sensing image and the second remote sensing image according to the alignment result; the first determination unit 34 determines the position change information of the target object according to the first recognition result, the second recognition result and the first position information. The problem of low accuracy of the state detection method based on pixel difference in the related art is solved. By acquiring remote sensing images of the target area in two time periods, and identifying the target objects contained in the two remote sensing images through a recognition model containing multiple preset recognition scales, the target objects can be accurately identified from the remote sensing images, and by aligning the images containing the identified objects, the position error between the images is reduced. After the alignment, it is determined whether the position of the target object has changed based on the position information of the target object in the two images, thereby achieving the effect of accurately determining whether the position state of the target object has changed.

[0123] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first recognition unit 32 includes: a first processing module, which is used to perform data enhancement operation on the first remote sensing image to obtain a processed first remote sensing image; a second processing module, which is used to process the processed first remote sensing image by sliding window clipping to obtain multiple first initial images, wherein each first initial image carries the second position information of the first initial image in the first remote sensing image; a recognition module, which is used to input each first initial image into the object recognition model respectively to obtain multiple initial recognition results, wherein the initial recognition result contains at least one recognition box, and the recognition box is used to mark the target object; a combination module, which is used to combine the multiple initial recognition results according to the second position information of each first initial image to obtain a candidate recognition result; a first determination module, which is used to determine the target confidence of each recognition box in the candidate recognition result, and filter the multiple recognition boxes in the candidate recognition result according to the target confidence to obtain the filtered recognition box; a second determination module, which is used to determine the candidate recognition result containing the filtered recognition box as the first recognition result.

[0124] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first determination module includes: a first judgment submodule, used to determine whether there is an overlapping recognition frame that overlaps with the recognition frame for any recognition frame; a first determination submodule, used to determine the initial confidence of the recognition frame as the target confidence when there is no overlapping recognition frame, wherein the initial confidence is generated by the object recognition model; a comparison submodule, used to compare the initial confidence of the recognition frame with the initial confidence of the overlapping recognition frame when there is an overlapping recognition frame; a second determination submodule, used to determine the initial confidence of the recognition frame as the target confidence when the initial confidence of the recognition frame is greater than or equal to the initial confidence of the overlapping recognition frame; a reduction submodule, used to reduce the initial confidence of the recognition frame according to the initial confidence of the overlapping recognition frame to obtain the target confidence when the initial confidence of the recognition frame is less than the initial confidence of the overlapping recognition frame.

[0125] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first determination module includes: a second judgment submodule, used to judge whether the target confidence is greater than a first preset value; a retention submodule, used to retain the identification box corresponding to the target confidence when the target confidence is greater than or equal to the first preset value; a deletion submodule, used to delete the identification box corresponding to the target confidence when the target confidence is less than the first preset value; and a third determination submodule, used to determine the retained identification box as the filtered identification box.

[0126] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first configuration unit 33 includes: a first acquisition module, used to sequentially acquire the Euclidean distance between each first feature point in the first feature point set and each second feature point in the second feature point set, and determine the second feature point corresponding to each first feature point according to the Euclidean distance, to obtain M pairs of feature points, wherein M is a positive integer; a construction module, used to select N pairs of feature points from the M pairs of feature points, and construct a transformation model through the N pairs of feature points, wherein the transformation model is used to transform the N pairs of feature points, and N is less than M; a registration module, used to select P pairs of feature points from the M pairs of feature points according to the transformation model, and register the first remote sensing image and the second remote sensing image through the correspondence between the P pairs of feature points, wherein P is greater than N.

[0127] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the registration module includes: a first calculation submodule, used to calculate the Euclidean distance between the feature points for any pair of feature points to obtain a first distance value; a transformation submodule, used to input the pair of feature points into the transformation model for transformation operation to obtain an updated feature point pair; a second calculation submodule, used to calculate the Euclidean distance of the updated feature point pair to obtain a second distance value; a selection submodule, used to select the pair of feature points as P pairs of feature points when the difference between the first distance value and the second distance value is less than a second preset value.

[0128] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first acquisition module includes: a third calculation submodule, which is used to calculate the Euclidean distance between the first feature point and each second feature point for any first feature point, and obtain multiple third distance values; a sorting submodule, which is used to sort the multiple third distance values ​​in order from small to large, and obtain the two candidate distance values ​​with the smallest serial numbers among the multiple third distance values; a fourth calculation submodule, which is used to calculate the ratio of the two candidate distance values, and determine whether the ratio is less than a preset threshold; a fourth determination submodule, which is used to determine that the second feature point to which the candidate distance value with the smallest serial number belongs is a pair of feature points with the first feature point when the ratio is less than the preset threshold, and obtain M pairs of feature points; a fifth determination submodule, which is used to determine that the first feature point has no corresponding second feature point when the ratio is greater than or equal to the preset threshold.

[0129] Optionally, in the state detection device of the target object provided in the embodiment of the present application, the first determination unit 34 includes: a second acquisition module, used to acquire the first position information of each target object in the first recognition result to obtain a first position set; a third acquisition module, used to acquire the first position information of each target object in the second recognition result to obtain a second position set; a judgment module, used to judge whether the first position set and the second position set are the same; a third determination module, used to determine that the position change information of the target object is that the position has changed when the first position set and the second position set are different; and a fourth determination module, used to determine that the position change information of the target object is that the position has not changed when the first position set and the second position set are the same.

[0130] The above-mentioned target object state detection device includes a processor and a memory. The above-mentioned first acquisition unit 31, first identification unit 32, first configuration unit 33, first determination unit 34, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0131] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of low accuracy of the state detection method based on pixel difference in the related art is solved by adjusting the kernel parameters.

[0132] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0133] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the state detection method of the target object is implemented.

[0134] An embodiment of the present invention provides a processor, which is used to run a program, wherein the state detection method of the target object is executed when the program is running.

[0135] Figure 4 is a schematic diagram of an electronic device provided according to an embodiment of the present application, such as Figure 4 As shown, an embodiment of the present invention provides an electronic device, the electronic device 40 includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned target object state detection method when executing the program. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0136] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the steps of the above-mentioned target object state detection method.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0142] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0143] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0144] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0145] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting the state of a target object, characterized in that: include: Acquire a first remote sensing image of a target area in a first time period, and a second remote sensing image of the target area in a second time period; Inputting the first remote sensing image into an object recognition model to obtain a first recognition result, and inputting the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes target objects in the first remote sensing image and the second remote sensing image by using a plurality of preset recognition scales; Registering the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determining first position information of the target object in the first remote sensing image and the second remote sensing image according to the registration result; The first recognition result, the second recognition result, and the first position information are used to determine position change information of the target object.

2. The method according to claim 1, characterized in that Inputting the first remote sensing image into an object recognition model to obtain a first recognition result includes: Performing a data enhancement operation on the first remote sensing image to obtain a processed first remote sensing image; Processing the processed first remote sensing image by using a sliding window clipping method to obtain a plurality of first initial images, wherein each first initial image carries second position information of the first initial image in the first remote sensing image; Inputting each first initial image into the object recognition model respectively to obtain a plurality of initial recognition results, wherein the initial recognition results include at least one recognition frame, and the recognition frame is used to mark the target object; Combining the multiple initial recognition results according to the second position information of each first initial image to obtain a candidate recognition result; Determining a target confidence of each recognition frame in the candidate recognition result, and screening multiple recognition frames in the candidate recognition result according to the target confidence to obtain screened recognition frames; The candidate recognition result including the filtered recognition box is determined as the first recognition result.

3. The method according to claim 2, characterized in that Determining the target confidence of each recognition box in the candidate recognition result includes: For any recognition frame, determine whether there is an overlapping recognition frame that overlaps with the recognition frame; In the absence of the overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence, wherein the initial confidence is generated by the object recognition model; In the case where the overlapping recognition frame exists, comparing the initial confidence of the recognition frame with the initial confidence of the overlapping recognition frame; When the initial confidence of the recognition frame is greater than or equal to the initial confidence of the overlapping recognition frame, determining the initial confidence of the recognition frame as the target confidence; In the case that the initial confidence of the recognition frame is less than the initial confidence of the overlapping recognition frame, the initial confidence of the recognition frame is reduced according to the initial confidence of the overlapping recognition frame to obtain the target confidence.

4. The method according to claim 2, characterized in that: The multiple recognition frames in the candidate recognition results are screened according to the target confidence, and the screened recognition frames include: Determining whether the target confidence is greater than a first preset value; When the target confidence is greater than or equal to the first preset value, retaining the identification box corresponding to the target confidence; When the target confidence is less than the first preset value, deleting the identification box corresponding to the target confidence; The retained recognition frame is determined as the filtered recognition frame.

5. The method according to claim 1, characterized in that Registering the first remote sensing image and the second remote sensing image according to the first feature point set in the first remote sensing image and the second feature point set in the second remote sensing image comprises: Sequentially obtain the Euclidean distance between each first feature point in the first feature point set and each second feature point in the second feature point set, and determine the second feature point corresponding to each first feature point according to the Euclidean distance to obtain M pairs of feature points, where M is a positive integer; Selecting N pairs of feature points from the M pairs of feature points, and constructing a transformation model through the N pairs of feature points, wherein the transformation model is used to transform the N pairs of feature points, where N is less than M; P pairs of feature points are selected from the M pairs of feature points according to the transformation model, and the first remote sensing image and the second remote sensing image are registered through the correspondence between the P pairs of feature points, wherein P is greater than N.

6. The method according to claim 5, characterized in that Selecting P pairs of feature points from the M pairs of feature points according to the transformation model includes: For any pair of feature points, calculating the Euclidean distance between the feature points to obtain a first distance value; Inputting the pair of feature points into the transformation model to perform a transformation operation to obtain an updated pair of feature points; Calculating the Euclidean distance of the updated feature point pair to obtain a second distance value; When the difference between the first distance value and the second distance value is smaller than a second preset value, the pair of feature points is selected as the P pair of feature points.

7. The method according to claim 5, characterized in that Determine the second feature point corresponding to each first feature point according to the Euclidean distance, and obtain M pairs of feature points including: For any first feature point, calculate the Euclidean distance between the first feature point and each second feature point to obtain a plurality of third distance values; Sort the multiple third distance values ​​in ascending order, and obtain two candidate distance values ​​with the smallest sequence numbers among the multiple third distance values; Calculating a ratio of the two candidate distance values, and determining whether the ratio is less than a preset threshold; When the ratio is less than the preset threshold, determine that the second feature point to which the candidate distance value with the smallest sequence number belongs and the first feature point are a pair of feature points, and obtain M pairs of feature points; When the ratio is greater than or equal to the preset threshold, it is determined that the first feature point has no corresponding second feature point.

8. The method according to claim 1, characterized in that Determining the location change information of the target object by using the first recognition result, the second recognition result, and the first location information includes: Acquire first position information of each target object in the first recognition result to obtain a first position set; Acquire first position information of each target object in the second recognition result to obtain a second position set; Determining whether the first position set and the second position set are the same; In a case where the first position set and the second position set are different, determining the position change information of the target object as a position change; When the first position set and the second position set are the same, it is determined that the position change information of the target object is that the position has not changed.

9. A state detection device for a target object, characterized in that: include: A first acquisition unit, used to acquire a first remote sensing image of a target area in a first time period, and a second remote sensing image of the target area in a second time period; a first recognition unit, configured to input the first remote sensing image into an object recognition model to obtain a first recognition result, and input the second remote sensing image into the object recognition model to obtain a second recognition result, wherein the object recognition model recognizes target objects in the first remote sensing image and the second remote sensing image by using a plurality of preset recognition scales; a first configuration unit, configured to register the first remote sensing image and the second remote sensing image according to a first feature point set in the first remote sensing image and a second feature point set in the second remote sensing image, and determine first position information of the target object in the first remote sensing image and the second remote sensing image according to the registration result; A first determining unit is configured to determine position change information of the target object using the first recognition result, the second recognition result, and the first position information.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the state detection method of the target object described in any one of claims 1 to 8 when running.

Images