Three-dimensional space change detection method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a three-dimensional space change detection method and device, equipment and a storage medium, and relates to the field of digital image processing. The method comprises the following steps: acquiring image sequences of the same area shot by the unmanned aerial vehicle in two periods; for each image sequence, performing point cloud reconstruction on the region based on the image sequence to obtain the group of point cloud data; point cloud registration is carried out according to the similarity between the feature descriptors of all the points in the two sets of point cloud data, a matching point pair set comprising a plurality of matching point pairs is obtained, and two points in each matching point pair belong to two pieces of point cloud data and are closest to each other; and for each matching point pair, geometric information of two points in the matching point pair in respective point cloud data is acquired, and if it is determined that the difference between the two pieces of geometric information meets a preset difference condition, it is determined that the corresponding position of the matching point pair in the region is changed. According to the embodiment of the invention, the problem of insufficient spatial change detection precision in a complex scene is solved.

Description

Technical Field

[0001] This application relates to the field of digital image processing technology, and more specifically, to a method, apparatus, device, and storage medium for detecting three-dimensional spatial changes. Background Technology

[0002] Three-dimensional spatial detection technology plays an important role in many fields, such as using changes in point cloud data to analyze and monitor changes in the environment, objects or structures. However, related three-dimensional spatial detection technologies suffer from insufficient detection accuracy. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for detecting three-dimensional spatial changes, which addresses the technical problem of insufficient accuracy in spatial change detection.

[0004] According to a first aspect of the embodiments of this application, a method for detecting three-dimensional spatial changes is provided, the method comprising: Acquire image sequences of the same area taken by a drone in two separate periods; For each image sequence, point cloud reconstruction is performed on the region based on the image sequence to obtain a set of point cloud data for the region; The feature descriptors of each point in the two sets of point cloud data are obtained respectively. Point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. The set of matching point pairs includes multiple matching point pairs. The two points in each matching point pair belong to two point cloud data and the two points are the closest points to each other. For each matching point pair, obtain the geometric information of the two points in their respective point cloud data. If the difference between the two geometric information is determined to meet the preset difference condition, then the position of the matching point pair in the region is determined to have changed.

[0005] According to a second aspect of the embodiments of this application, a three-dimensional spatial change detection device is provided, the device comprising: The acquisition module is used to acquire image sequences taken by the UAV in two different periods of the same area. The reconstruction module is used to reconstruct the point cloud of a region based on the image sequence for each image sequence, and obtain a set of point cloud data of the region. The registration module is used to obtain the feature descriptors of each point in the two sets of point cloud data respectively, and to perform point cloud registration based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. The set of matching point pairs includes multiple matching point pairs. The two points in each matching point pair belong to two point cloud data and the two points are the closest points to each other. The determination module is used to obtain the geometric information of the two points in each matching point pair in their respective point cloud data. If the difference between the two geometric information is determined to meet the preset difference condition, then the position of the matching point pair in the region is determined to have changed.

[0006] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the program to implement the steps of the method provided in the first aspect.

[0007] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0008] According to a fifth aspect of the present application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the method provided in the first aspect.

[0009] The beneficial effects of the technical solutions provided in this application are: The three-dimensional spatial change detection method provided in this application acquires image sequences of the same area taken by a UAV at two different times. For each image sequence, point cloud reconstruction of the area is performed based on the image sequence to obtain a set of point cloud data for the area. This allows for the alignment of data from the two time periods, enabling effective identification of changed areas within the region. Feature descriptors of each point in the two sets of point cloud data are acquired, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By acquiring the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between local features, matching is achieved between points at the same location in the two sets of point cloud data at different times, i.e., obtaining matching point pairs. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is acquired. If the difference between the two geometric information values ​​meets a preset difference condition, it is determined that the corresponding position of the matching point pair in the region has changed. By using only the difference in geometric information between the two points in the matching point pair to determine whether the region corresponding to the matching point pair in three-dimensional space has changed, the accuracy of three-dimensional spatial change detection is ensured while saving computational resources. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0011] Figure 1 A schematic diagram of the system architecture for implementing a method for detecting changes in three-dimensional space provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for detecting three-dimensional spatial changes provided in an embodiment of this application; Figure 3 A flowchart illustrating the method for obtaining a target feature map in a three-dimensional spatial change detection method provided in this application embodiment; Figure 4 A schematic diagram illustrating the process of acquiring point cloud data in a method for detecting three-dimensional spatial changes provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating another step in acquiring point cloud data in a method for detecting three-dimensional spatial changes provided in an embodiment of this application. Figure 6 A flowchart illustrating the method for obtaining a set of matching points in a three-dimensional spatial change detection method provided in this application embodiment; Figure 7 A flowchart illustrating the method for obtaining a set of candidate matching points in a three-dimensional spatial change detection method provided in this application embodiment; Figure 8 A flowchart illustrating the method for updating the first transformation matrix in a three-dimensional spatial change detection method provided in this application embodiment; Figure 9 A flowchart illustrating the method for updating the second transformation matrix in a three-dimensional spatial change detection method provided in this application embodiment; Figure 10 A schematic flowchart illustrating the region change detection method in a three-dimensional spatial change detection method provided in this application embodiment; Figure 11 A schematic diagram of the structure of a three-dimensional spatial change detection device provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0013] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0015] The following explains the terminology and related technologies involved in this application: The relevant technologies are explained below: Existing methods for detecting three-dimensional spatial changes mainly include those based on satellite remote sensing imagery and those based on ground-based laser scanning.

[0016] Three-dimensional spatial change detection based on satellite remote sensing primarily utilizes sensors onboard satellites to acquire multi-temporal remote sensing image data. After data acquisition, preprocessing steps such as geometric correction, radiometric correction, and atmospheric correction are required to ensure data accuracy and consistency. During the feature extraction stage, spectral indices, texture features (such as the gray-level co-occurrence matrix), and deformation features (such as the coherence of SAR images) are calculated. Change detection is achieved through direct interpolation, post-classification comparison, or object-level analysis methods, ultimately generating a change map.

[0017] 3D spatial change detection based on terrestrial laser scanning primarily uses a 3D laser scanner to acquire point cloud data of the target area. After data acquisition, processing steps such as registration, denoising, downsampling, and correction are required to generate a high-quality 3D model. Change detection is achieved through differential operations after point cloud registration or by subtraction processing after reconstructing the solid surface model.

[0018] However, the advantages of satellite remote sensing-based 3D spatial change detection lie in its wide coverage, making it suitable for large-area monitoring. Its disadvantages include lower resolution, making it difficult to capture small-scale changes; limited data acquisition frequency, susceptible to cloud cover and weather conditions; and potentially insufficient detection accuracy in complex scenarios. Terrestrial laser scanning-based 3D spatial change detection, on the other hand, offers high precision (up to millimeter-level), suitable for fine measurement and deformation monitoring; high data acquisition efficiency; and dense sampling points, enabling the capture of richer details. Its disadvantages include limited coverage, suitable for small areas; high equipment cost and complex operation; and susceptibility to occlusion and scanning angle limitations, potentially resulting in insufficient scanning coverage for large or complex scenarios.

[0019] To address at least one of the aforementioned technical problems or areas requiring improvement in related technologies, this application proposes a method for detecting changes in three-dimensional space. This method involves acquiring image sequences of the same area taken by a UAV at two different times. For each image sequence, point cloud reconstruction of the area is performed based on the image sequence to obtain a set of point cloud data for the area. This allows for the alignment of data from the two time periods, enabling effective identification of changed areas within the region. Feature descriptors for each point in the two sets of point cloud data are acquired, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By acquiring the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between local features, matching is achieved between points at the same location in the two sets of point cloud data at different times, i.e., obtaining matching point pairs. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is acquired. If the difference between the two geometric information values ​​satisfies a preset difference condition, it is determined that the corresponding position of the matching point pair in the region has changed. By using only the difference in geometric information between two points in a matching point pair, it is possible to determine whether the region corresponding to the matching point pair in three-dimensional space has changed. This saves computational resources while ensuring the accuracy of change detection in three-dimensional space.

[0020] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0021] Figure 1 This is a schematic diagram of a detection system architecture for realizing three-dimensional spatial changes provided in an embodiment of this application, wherein the system architecture includes: a terminal 120 and a server 140.

[0022] Terminal 120 has an application program installed and runs a method for detecting three-dimensional spatial changes. Terminal 120 is used to determine the location of changes in the region based on image sequences of the same region taken at two different times.

[0023] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0024] Server 140 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Illustratively, server 140 includes a processor 144 and a memory 142, the memory 142 including a display module 1421, a control module 1422, and a receiving module 1423. Server 140 is used to provide background services for the application of the method. Optionally, server 140 undertakes the primary computing work, and terminal 120 undertakes secondary computing work; or, server 140 undertakes secondary computing work, and terminal 120 undertakes primary computing work; or, server 140 and terminal 120 collaborate on computing using a distributed computing architecture.

[0025] Optionally, the device type of the terminal includes at least one of the following: smartphone, tablet computer, e-book reader, Moving Picture Experts Group Audio Layer III (MP3) player, Moving Picture Experts Group Audio Layer IV (MP4) player, laptop computer, and desktop computer.

[0026] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0027] This application provides a method for detecting three-dimensional spatial changes, such as... Figure 2 As shown, the method includes: S101, acquire the image sequence of the same area taken by the drone in two separate periods.

[0028] In this embodiment, the image sequence is a set of photographs taken by a drone while flying over the area to be detected. The image sequence includes images of the area from different perspectives, and the images in the image sequence are arranged in the order they were taken by the drone. It should be understood that the area is in three-dimensional space.

[0029] In this embodiment of the application, the time interval between the two periods can be one day or one week, and the specific time interval between the two periods is determined according to the requirements for detecting changes in three-dimensional space.

[0030] In this embodiment of the application, the drone includes a multi-view camera system, an RTK-GPS module, and an IMU sensor. The multi-view camera system is used to capture image data of the area, and the image sequence is taken by the multi-view camera system mounted on the drone. The RTK-GPS module is used to provide accurate positioning for the drone. The IMU sensor is used to monitor the drone's motion state and attitude changes in real time.

[0031] In this embodiment, before using a drone to photograph an area, the drone's flight path and altitude are planned according to the shooting requirements. When planning the flight path, the overlap between adjacent flight strips and the lateral overlap need to be considered. The overlap between adjacent flight strips refers to the overlapping portion of the images between two adjacent flight strips (i.e., the preceding and following flight paths), thereby ensuring that the captured images have sufficient common areas, thus ensuring higher accuracy feature extraction. The lateral overlap refers to the horizontal overlap between two adjacent images, ensuring that the aerial images have sufficient overlap in the horizontal direction, so that images from different perspectives can be accurately matched in the subsequent point cloud reconstruction process. For example, the overlap between adjacent flight strips should be no less than 75%, and the lateral overlap should be no less than 60%.

[0032] In this embodiment of the application, when planning the flight altitude, the altitude is adjusted according to the ground resolution requirements, specifically based on the following formula:

[0033] in, H For flight altitude, P f is the pixel size, and f is the focal length. IW GSD represents the image width, and GSD represents the ground resolution.

[0034] In this application, the UAV ensures a high degree of overlap between adjacent flight path components by alternating reciprocating flight paths, ensuring that sufficient matching points can still be obtained in complex terrain (such as mountains and cities), providing continuous coverage for 3D reconstruction, automatically adjusting the flight path altitude according to terrain undulations to maintain a constant ground resolution and ensure the quality of the output image.

[0035] S102, for each image sequence, point cloud reconstruction is performed on the region based on the image sequence to obtain a set of point cloud data for the region.

[0036] In this embodiment of the application, for each image sequence, the three-dimensional coordinates of the region to be detected can be calculated based on the image information of all images in the image sequence, thereby obtaining point cloud data that can characterize the three-dimensional information of the region to be detected.

[0037] In this embodiment of the application, the point cloud data of a region refers to a set of data points in the region. Each data point has a corresponding three-dimensional coordinate in three-dimensional space, and the three-dimensional coordinates corresponding to each point define the shape of the object or scene in the region.

[0038] S103, obtain the feature descriptors of each point in the two sets of point cloud data respectively, and perform point cloud registration based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs.

[0039] In this embodiment of the application, the feature descriptor of a point can be a Fast Point Feature Histogram (FPFH), which is a feature descriptor that describes the shape of a local set of points in a point cloud. The feature descriptor describes the corresponding shape features through the geometric relationship of the region surrounding each point in the point cloud data.

[0040] In the embodiments of this application, the set of matching point pairs includes multiple matching point pairs, and the two points in each matching point pair belong to two point cloud data, and the two points are the closest points to each other.

[0041] In this embodiment of the application, the point cloud data from the earlier period of the two point cloud data is used as the first point cloud data, and the point cloud data from the later period is used as the second point cloud data. The set of matching point pairs includes multiple matching point pairs. A matching point pair consists of a point from the first point cloud data and a point from the second point cloud data. The two points in a matching point pair represent the same physical location in the region.

[0042] In this embodiment, the feature descriptor of each point in the first point cloud data is compared with the feature descriptor of each point in the second point cloud data. For each feature descriptor of the first point cloud data, the point corresponding to the feature descriptor with the highest similarity to the feature descriptor is found from the feature descriptors of each point in the second point cloud data, and is used as a candidate matching point for that point. For each feature descriptor of the second point cloud data, the point corresponding to the feature descriptor with the highest similarity to the feature descriptor is found from the feature descriptors of each point in the first point cloud data, and is used as a candidate matching point for that point. If two points are each other's candidate matching points, then the two points are considered as a matching point pair.

[0043] In the embodiments of this application, the similarity between two feature descriptors can be determined by comparing the set geometric features, such as comparing the normal direction and curvature of a point. The similarity between two points can also be determined by comparing their positions in three-dimensional space. Alternatively, the similarity between two points can be compared by calculating the density difference between the two points in a local region.

[0044] In one example, point cloud data A contains points A1, A2, and A3, and point cloud data B contains points B1, B2, and B3. By calculating and determining that points A1 and B2 are each other's candidate matching points, and points A2 and B3 are each other's candidate matching points, then points A1 and B1 can be considered as a pair of matching points, and points A2 and B3 can be considered as a pair of matching points.

[0045] S104. For each matching point pair, obtain the geometric information of the two points in their respective point cloud data. If the difference between the two geometric information is determined to meet the preset difference condition, then determine that the corresponding position of the matching point pair in the region has changed.

[0046] In this embodiment, the geometric information of points in point cloud data refers to the position and shape of points in three-dimensional space. Geometric information may include: point position, location, curvature, density direction, etc. By comparing the positional differences of points in a matching point pair, it can be determined whether the object at the corresponding position has been displaced, such as a building sinking; by comparing the angular differences between the normal vectors of points in a matching point pair, it can be determined whether the surface of the object at the corresponding position has been rotated or deformed, such as a building tilting; by comparing the curvature difference between the curvatures of points in a matching point pair, it can be determined whether the surface of the object has been deformed, such as a road sinking; by comparing the density differences of points in a matching point pair, it can be determined whether the ecological environment in the area has changed, such as vegetation increasing or decreasing.

[0047] In this embodiment, by acquiring the difference between two geometric pieces of information, and determining that the difference exceeds a certain level (i.e., a preset difference condition is met), it is determined that the physical location of the matching point pair in the region has undergone changes in shape, orientation, surface, density, etc. For example, changes caused by the increase or decrease of vegetation in a forest area, changes in terrain such as landslides in mountainous areas, and changes caused by the appearance of new buildings or obstacles in cities.

[0048] In the three-dimensional spatial change detection method provided in this application embodiment, image sequences of the same area taken by a UAV in two different periods are acquired. For each image sequence, point cloud reconstruction of the area is performed based on the image sequence to obtain a set of point cloud data of the area. This allows for the alignment of data from two different times to effectively identify the changed areas in the region. Feature descriptors of each point in the two sets of point cloud data are acquired respectively, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By acquiring the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between the local features, matching between points at the same location in the two sets of point cloud data at different times is achieved, i.e., matching point pairs are obtained. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is acquired. If the difference between the two geometric information is determined to meet a preset difference condition, it is determined that the position of the matching point pair in the region has changed. By using only the difference in geometric information between two points in a matching point pair, it is possible to determine whether the region corresponding to the matching point pair in three-dimensional space has changed. This saves computational resources while ensuring the accuracy of change detection in three-dimensional space.

[0049] Based on the above embodiments, this application also provides a method for obtaining a target feature map of an image, such as... Figure 3 As shown, the specific content is as follows: S201, For each image in the two image sequences, n convolutional layers with different dilation rates are used to extract features from the image to obtain the first feature map of the image at n scales; n is an integer greater than 1; S202, concatenate the feature maps of n scales along the channel dimension to obtain a concatenated feature map, and then apply channel attention weighting to the concatenated feature map to obtain a weighted feature map; S203, expand the feature dimension of the weighted feature map to obtain the second feature map, and perform feature enhancement and normalization processing on the second feature map in sequence to obtain the target feature map corresponding to the image.

[0050] In this embodiment, the dilation rate refers to the spacing between elements of the convolution kernel during the convolution process. By increasing the dilation rate, the size of the receptive field of the convolution kernel can be increased. A smaller receptive field can better capture local details in the image, while a larger receptive field can capture a wider range of contextual information in the image. The first feature map at n scales refers to n first feature maps at different scales. The first feature map at n scales includes at least one low-scale first feature map, at least one medium-scale first feature map, and at least one high-scale first feature map. The low-scale first feature map is used to capture local features in the image, such as the edges and textures of the image. The medium-scale first feature map is used to capture features of a larger range in the image, such as the shape and outline of objects in the image. The high-scale feature map is used to capture global contextual information in the image, such as the structure of the entire object extracted from the image.

[0051] In S201 of this application embodiment, the image is first subjected to primary feature extraction through one or more convolutional layers to obtain an initial feature map of the image. Then, convolutional layers with different dilation rates are used to extract features from the initial feature map of the image to obtain first feature maps of different scales. For example, a convolutional layer with a dilation rate of 1 is used to extract features from the initial feature map to obtain a low-scale first feature map. A convolutional layer with a dilation rate of 3 is used to extract features to obtain a medium-scale first feature map. A convolutional layer with a dilation rate of 5 is used to extract features to obtain a high-scale first feature map.

[0052] In one example, the first convolutional layer uses a kernel with a dilation rate of 1, a size of 3×3, a stride of 1, and padding of 1. This kernel slides across the initial feature map of the input image from left to right and top to bottom, with each sliding window covering a 3×3 local region of the input feature map. Microscopic patterns such as corners and edges are extracted through a weighted summation of the weight parameters and corresponding pixel values. Sixteen independent convolutional kernels operate in parallel, each generating a 112×112 feature map, which is then stacked to form a 112×112×16 output tensor, serving as the low-scale first feature map. The second convolutional layer uses a kernel with a dilation rate of 3, a size of 3×3, a stride of 1, and padding of 3. This kernel slides across the initial feature map of the input image from left to right and top to bottom, performing interval sampling. The sampling interval is expanded to 3 pixels, for example, extracting information at positions (0,0), (0,3), (3,0), and (3,3). Each sliding window covers a 7×7 local region of the input feature map. Texture and local structure are extracted through a weighted summation of weight parameters and corresponding pixel values. Sixteen independent convolutional kernels operate in parallel, each generating a 112×112 feature map. These are then stacked to form a 112×112×16 output tensor, serving as the first feature map at the mesoscale. The third convolutional layer uses a kernel with a dilation of 5, a size of 3×3, a stride of 1, and padding of 5. It slides across the initial feature map of the input image from left to right and from top to bottom, performing interval sampling. The interval sampling is expanded to 5 pixels, for example, extracting information at positions (0,0), (0,5), (5,0), and (5,5). Each sliding window covers an 11×11 local region of the input feature map. Object contours and global context are extracted through a weighted summation of weight parameters and corresponding pixel values. Sixteen independent convolutional kernels operate in parallel, each kernel generating a 112×112 feature map, which are finally stacked into an output tensor of size 112×112×16 as a high-scale feature map.

[0053] In S202 of this application embodiment, when stitching feature maps of different scales along the channel dimension of the feature map, the spatial dimension of the first feature map remains unchanged, that is, the length and width remain unchanged. Stitching along the channel dimension means superimposing the number of channels of the three images to obtain a stitched feature map. For example, the first feature map at a low scale, the first feature map at a medium scale, and the first feature map at a high scale are all feature maps of size 112×112×16. After stitching the above first feature maps along the channel dimension, a stitched feature map of size 112×112×48 is obtained.

[0054] In S202 of this embodiment, after obtaining the stitched feature map, global average pooling is used to compress the spatial information of each channel into a scalar that can represent the overall information of each channel. For example, if the size of the input stitched feature map is 112×112×16, then the size of the output scalar is 1×1×16. After obtaining the scalar, a fully connected layer is used to reduce the number of channels of the scalar to a preset number to extract the high-level information of the channels. A second fully connected layer is used to restore the number of channels of the scalar from the preset number to the original number of channels of the scalar, resulting in a value representing the weight of the channel. In order to make the weight value of each channel a probability value, the Sigmoid activation function is used to map the value representing the weight of the channel to the range of [0,1]. Furthermore, the obtained weight of each channel is multiplied by the value of each channel, that is, important channels are strengthened and unimportant channels are suppressed by weighting, thereby obtaining a weighted feature map.

[0055] In this embodiment, for the stitched feature map, global average pooling, channel weight generation, and feature weighting operations are sequentially performed to generate an attention-weighted feature map. For example, if the size of the stitched feature map is 112×112×16, the 112×112 spatial information is compressed into a single value, outputting a vector of size 1×1×24, where each element corresponds to the global feature value of one channel. Then, the 1×1×24 vector is compressed to 6 channels through channel weight generation, and the correlation between channels is captured through linear transformation to condense the core features. Then, the original channel dimension is restored through linear transformation to generate channel weights. Finally, element-wise multiplication is performed on each channel, that is, the channel weight is multiplied by the original feature map to enhance key features and suppress noise, outputting a weighted feature map of size 112×112×16.

[0056] In S203 of this embodiment, the feature dimension of the input weighted feature map is expanded by mapping the number of channels of the input to a larger number of channels through a convolutional layer, thereby obtaining a second feature map and increasing the feature representation capability. Then, the second feature map is nonlinearly transformed by the H-Swish activation function to enhance the feature map. For the enhanced feature map, the modulus of the feature vector of each channel of the feature map is scaled to 1. Specifically, for all pixel values ​​in each channel of the feature map, the L2 norm of the current channel is calculated based on the feature values ​​at each position in the channel. For the feature value at each position, the ratio between the feature value and the L2 norm is used as the feature value of the normalized feature map to obtain the target feature map.

[0057] In this embodiment, for the weighted feature map, three operations are performed sequentially: feature dimension expansion, nonlinear activation, and feature magnitude normalization to generate the target feature map. For example, for a 112×112×16 weighted feature map, the number of channels is expanded from 16 to 128 through a 1×1 standard convolution with zero padding, improving feature representation capability. Then, for each pixel value x in the input feature map, its value is first incremented by 3 (equivalent to shifting it 3 units to the left), then truncated to the range of 0 to 6 using the ReLU6 function (i.e., if the result is less than 0, it is taken as 0; if it is greater than 6, it is taken as 6). The truncated value is then divided by 6 to obtain a smoothed value (range 0 to 1) that approximates the Sigmoid function. Finally, this smoothed value is multiplied by the original pixel value to obtain the activated feature map, maintaining a size of 112×112×128. Finally, for each spatial location (i.e., each pixel) in the input feature map, first calculate the sum of squares of all its channel values, then take the square root of the sum to obtain the vector magnitude (equivalent to the "length" of the vector) at that location; then, divide each channel value at that location by this magnitude to make the normalized vector length 1, and finally obtain the normalized feature map.

[0058] In this application, before extracting the target feature map, the image collected by the UAV is denoised by Gaussian filtering and enhanced by adaptive histogram equalization (CLAHE), and then distortion correction is performed to obtain the preprocessed image.

[0059] In this embodiment, before extracting first feature maps of different scales through convolutional layers with different dilation rates, the original images captured by the UAV are scaled to 224×224 resolution using bilinear interpolation to balance detail preservation and computational cost. The mean [0.485, 0.456, 0.406] is subtracted from each RGB channel, and then divided by the standard deviation [0.229, 0.224, 0.225] to eliminate illumination differences. The standardized image is then output (224×224×3, pixel value distribution approximately N(0,1)).

[0060] In this embodiment, after obtaining the standardized image, the standardized image sequence is input into the convolutional layer for primary feature extraction. Specifically, this includes: firstly, applying a 3×3 convolutional kernel for local feature extraction, generating diverse feature maps by setting 16 output channels (such as capturing edges or textures in different directions), setting the stride to 2 to achieve downsampling (halving the output size to 112×112), then applying the ReLU6 activation function to introduce nonlinearity and limit the output range (between 0 and 6), and finally outputting a primary feature map (112×112×16) as the input to convolutional layers with different dilation rates.

[0061] In this embodiment, an improved MobileNetV3 network is proposed to extract salient features such as corners and edges from images to obtain target feature maps. A compact descriptor (e.g., a 128-dimensional vector) is generated for each feature point in the image to quantify the local texture information of the feature points. Simultaneously, by reducing the number of network layers and optimizing the computation graph, the computational cost and memory usage of the feature extraction process are reduced.

[0062] In this embodiment, the network used to obtain the target feature map includes: a Conv2d (3×3) layer, a Bottleneck layer, an SE-Attention layer, a Conv2d (1×1) layer, and an L2 normalization layer. The specific contents of each layer are shown in Table 1.

[0063] Table 1 In the Bottleneck layer, the first layer has an expansion rate of 1 to capture local features of the image, the second layer has an expansion rate of 3 to expand the receptive field of the image, and the third layer has an expansion rate of 5 to capture multi-scale contextual information.

[0064] In the SE-Attention layer, Squeeze, Excitation, and feature recalibration are performed. Specifically, the Squeeze operation uses global average pooling to generate channel descriptors z∈R; the Excitation operation generates channel weights through two fully connected layers. Where δ is ReLU and σ is Sigmoid; Feature recalibration: Strengthen the characteristics of important channels.

[0065] In the section on activation function optimization, the first three layers use the ReLU6 activation function to improve stability under low-precision calculations; the last layer uses the H-Swish activation function to balance accuracy and latency.

[0066] In one example, the inputs and outputs of each layer in the Bottleneck layer are as follows: First layer: dilation rate = 1 (local feature capture), input: primary feature map (112×112×16) extracted through Conv2d layer, output: 112×112×16 feature map, small receptive field (3×3), focusing on local details (such as corners and edges).

[0067] Second layer: Expansion rate = 3 (medium receptive field expansion), input: the output feature map of the first layer (112×112×16), output: a feature map of 112×112×16, the receptive field is expanded to 7×7, covering a medium range of structures (such as texture areas), and the resolution is maintained by adjusting the padding, where padding = 3.

[0068] Third layer: dilation rate = 5 (local feature capture), input: the output feature map of the second layer (112×112×16), output: 112×112×16 feature map, the receptive field is expanded to 11×11, capturing global context (such as object outline), and maintaining resolution by adjusting padding, where padding = 5.

[0069] In the above scheme, feature extraction is performed using convolutional layers with different dilation rates. This allows for the acquisition of local features, features with larger receptive fields, and global contextual information during the feature extraction process, achieving multi-scale capture from details to the global picture. This significantly improves the expressive power of multi-scale features without increasing network depth. Furthermore, attention weighting of the acquired feature maps enhances the feature representation of important channels and suppresses unimportant features, thereby suppressing noise and strengthening key features.

[0070] Based on the above embodiments, this application also provides a method for acquiring point cloud data of a region, such as... Figure 4 As shown, the specific content is as follows: S301, take the origin of the camera coordinate system of the first frame image in the image sequence as the origin of the world coordinate system, initialize the pose of the first frame image in the world coordinate system to the identity matrix, and determine the pose of each image in the image sequence in the world coordinate system based on the relative pose and identity matrix between two adjacent frames in the image sequence. S302, For any two images in the image sequence, extract feature points from the target feature maps corresponding to the two images respectively, and determine the projection matrix of the two images according to the preset intrinsic parameter matrix and the extrinsic parameter matrix of the two images respectively. S303, Based on the local region features of each feature point at its corresponding position in the image, determine the matching feature point pairs between the two images; S304, For any matching pair of feature points between two images, obtain the pixel coordinates of the two feature points in their respective images. S305: Based on the pixel coordinates of the two feature points in the feature point pairs matched between the two images and the projection matrices of the two images, triangulation is performed to obtain a set of point cloud data for the region.

[0071] In S301 of this application embodiment, for each image sequence, a first frame image is obtained from the image sequence, and the origin of the camera coordinate system of the first frame image is used as the origin of the world coordinate system. The camera coordinate system of the image is a three-dimensional coordinate system used to describe the position and orientation of objects in the image. Using the origin of the camera coordinate system of the first frame image as the origin of the world coordinate system provides a reference base when calculating the coordinates of other images in the world coordinate system. The first frame image is initialized as an identity matrix, that is, the pose of the first frame image is used as a reference pose in the world coordinate system to provide a reference base when calculating the poses of other images, so that the camera coordinate system of the first frame image corresponds to the world coordinate system. Since the pose of the first frame image has been obtained, the pose of each image in the image sequence in the world coordinate system is determined by the relative pose between two adjacent frames.

[0072] In this embodiment, based on the first frame image and a preset camera intrinsic parameter matrix (including focal length, principal point coordinates, and distortion parameters), the origin of the camera coordinate system of the first frame image is directly defined as the origin of the world coordinate system. The pose of the first frame image in the world coordinate system is initialized to an identity matrix (i.e., rotation is 0 and translation is 0).

[0073] In this embodiment, adjacent frames are defined as: the first frame and the second frame, the second frame and the third frame, and so on, with the last adjacent frame being the (m-1)th frame and the mth frame, where m is the number of images in the image sequence. Calculating the relative pose between adjacent frames requires first using matching feature point pairs between the two images to calculate the relative pose. After calculating the relative poses between all adjacent frames, the pose of each image in the world coordinate system is obtained by summing it with the pose of the first frame.

[0074] In the embodiments of this application, for each image, the pose of the image is determined based on the pose of the previous frame image and the relative pose between the two. For example, the pose of the second frame image is determined based on the identity matrix and the relative pose between the first frame image and the second frame image. The pose of the m-th frame image is determined based on the (m-1)-th frame image and the relative pose between the m-th frame image and the (m-1)-th frame image. Therefore, the pose of each frame image is determined by the relative pose between all adjacent frames between the first frame image and the current frame image. For example, the pose of the 4th frame image in the world coordinate system is determined based on the identity matrix of the first frame image, the relative pose between the first frame image and the second frame image, the relative pose between the second frame image and the third frame image, and the relative pose between the third frame image and the fourth frame image.

[0075] In S302 of this application embodiment, feature points are used to describe the feature information of the local region at the corresponding position in the image. Each feature point is obtained from the target feature map of the image. The feature points include key points and descriptors. Key points are used to mark the position and attributes of the feature points in the image. That is, the key points are essentially the positions of the image corresponding to the target feature map. Descriptors are used to describe the features of the local region around the key points. In other words, a descriptor is a mathematical vector used to describe the local region features at the corresponding position of the feature points in the image.

[0076] In this embodiment, the projection matrix of the image describes the projection process of the image from three-dimensional space to a two-dimensional image plane. The projection matrix is ​​composed of a preset intrinsic parameter matrix and the pose of the image, and the formula is as follows: P=K[R|t], where K is the intrinsic parameter matrix and [R|t] is the pose of the image.

[0077] In S303 of this application embodiment, two feature points used to describe the same location can be determined by comparing the similarity of descriptors in two target feature maps, and matching feature point pairs between two images can be determined. For example, the similarity between two descriptors can be determined by calculating the distance between them, such as Euclidean distance, Hamming distance, and Manhattan distance. The similarity between two descriptors can also be determined by the cosine similarity between them.

[0078] In S304 of this application embodiment, the feature point includes pixel coordinates. The pixel coordinate information of the feature point represents the coordinates of the corresponding position of the feature point in the image. For any matching feature point pair between two images, the pixel coordinates of the two feature points in their respective images are obtained.

[0079] In the embodiments of this application, triangulation is a technique used in computer vision for 3D reconstruction. Based on the matching feature point pairs in two or more viewpoints (images), the corresponding points in 3D space are calculated to obtain point cloud data.

[0080] In S305 of this application embodiment, the projection relationship of three-dimensional coordinate points is constructed based on the projection matrix and the pixel coordinates of the two feature points in the matching point pair. That is, the pixel coordinates of the feature points on the image are obtained by the corresponding three-dimensional coordinate points through the camera projection matrix.

[0081] In this embodiment of the application, triangulation based on pixel coordinates and camera projection matrix means that for each matching point pair, the three-dimensional coordinates of the constructed point cloud data are obtained by minimizing the difference between the projection of the two points in the matching point pair in three-dimensional space and the actual points.

[0082] In the above scheme, the origin of the camera coordinate system of the first frame image is used as the origin of the world coordinate system, so that the poses of other images are calculated based on the world coordinate system, ensuring the consistency of the 3D point cloud. The relative pose is determined based on the matching point pairs between adjacent frames. Since the viewpoint changes between adjacent frames are small, the feature matching is more accurate, and the obtained relative pose is more accurate. By incrementally solving the pose of each frame, the problem of poor pose calculation accuracy caused by matching between distant images is avoided. By calculating the projection matrix of each image, point cloud reconstruction can be performed for any matching feature point pairs between two frames, improving the flexibility of point cloud reconstruction.

[0083] Based on the above embodiments, this application further provides a method for obtaining a set of point cloud data of a region, such as... Figure 5 As shown, the specific content is as follows: S401, For any two images in the image sequence, obtain the average disparity between the two images and the first number of matching feature point pairs between the two images; S402, take two images with an average disparity greater than a preset pixel threshold and a first number greater than a first number threshold as an initial image pair; S403, based on the pixel coordinates of the matching feature point pairs between the initial image pairs and the projection matrices of the two images in the initial image pairs, the matching feature point pairs between the initial image pairs are triangulated to obtain the three-dimensional coordinates of the matching feature point pairs and to construct the initial point cloud data of the region. S404, take the triangulated images in the image sequence as keyframes, and sequentially obtain images other than keyframes from the image sequence based on the filtering conditions to update the initial point cloud data until the stopping condition is met; S405, the initial point cloud data updated when the stopping condition is met is used as the point cloud data of the region.

[0084] In S401 of this application embodiment, for all pairwise images, the first number of matching feature point pairs between each pairwise image is obtained, and the average disparity between the pairwise images is obtained. The disparity refers to the difference in horizontal pixel position of the matching feature point pairs between two images, that is, the matching point pairs A1 and A2. The horizontal position of A1 in the first image is X1, and the horizontal position of A2 in the second image is X2. Then the disparity is X1-X2. The average disparity is the average value of the disparity between all matching feature point pairs between the pairwise images.

[0085] In S402 of this application embodiment, the greater the disparity between two pairs of images, the greater the difference in the position of objects between the two viewpoints. When the disparity is large, the position difference of the matched feature point pairs between the two pairs of images is more obvious, which can avoid mismatch and thus improve the accuracy of point cloud reconstruction. The more matching point pairs between the two pairs of images, the more stable the point cloud reconstruction can be guaranteed. Therefore, when selecting the initial image pair, it is necessary to select two images with an average disparity greater than a preset pixel threshold and a first number greater than a first number threshold as the initial image pair. For example, the average disparity is >50 pixels and the number of matched feature points is ≥500 pairs.

[0086] In this embodiment, when multiple image pairs meet the requirements of average disparity and number of matching feature points, the image pair with the largest average disparity and the most matching feature point pairs is selected as the initial image pair; when multiple image pairs meet the requirements of average disparity and number of matching feature points, but there is no image pair with the largest average disparity and the most matching feature point pairs, the weights of the number of matching feature points and average disparity are determined according to the current actual point cloud reconstruction requirements, and the image pair with the largest value is determined as the initial image pair by weighted summation.

[0087] In S403 of this embodiment, for each matching feature point pair between initial image pairs, the pixel coordinates of the feature point pair in their respective images are determined, the projection matrices of the two initial images are determined, and the three-dimensional coordinates of the feature point pair in three-dimensional space are determined based on the pixel coordinates of the feature point pair in their respective images and their respective projection matrices. Thus, the three-dimensional coordinates of the corresponding points in three-dimensional space are calculated for all feature point pairs. The three-dimensional coordinates of the points corresponding to all matching point pairs in the initial image pairs in three-dimensional space constitute the initial point cloud data of the region.

[0088] In the application embodiment, the pixel coordinates are normalized to the camera coordinate system through the intrinsic parameter matrix, and the three-dimensional coordinates of the matching point pair in three-dimensional space are determined based on the normalized pixel coordinates and the projection matrix of the corresponding image.

[0089] In S404 of this application embodiment, in the image sequence, the image that has been triangulated is used as a keyframe. That is, after the initial point cloud data is obtained, both images in the initial image pair are keyframes. Based on the initial point cloud data, the images other than the keyframes are sequentially obtained from the image sequence according to the filtering conditions. This means that the initial point cloud data is updated by sequentially obtaining images that have not been triangulated from the image sequence. That is, the three-dimensional coordinates of the information are generated according to the projection matrix and pixel coordinates of the new matching point pair, thereby updating the coverage of the region in the point cloud data until the stop condition for updating the point cloud data is met.

[0090] In this embodiment, an image is selected each time to update the initial point cloud data based on the filtering conditions. After a new image is acquired, feature point matching is performed between the image and keyframes to find the keyframe that best matches the current image. Then, based on the pixel coordinates of the feature point pairs matched between the current image and the keyframe and the projection matrices of the current image and the keyframe respectively, new three-dimensional coordinates are generated, and the existing three-dimensional coordinate points are updated, thereby realizing the update of the initial point cloud data. Each update of the initial point cloud data is based on the previously updated initial point cloud data, that is, multiple rounds of updates to the initial point cloud data are required until the stopping condition is met.

[0091] In the embodiments of this application, the stopping condition can be that all images in the image sequence have participated in point cloud reconstruction, the stopping condition can also be that the updated initial point cloud data has covered all angles of the area, and the stopping condition can also be that the number of three-dimensional points is stable or increases slowly.

[0092] In this embodiment of the application, the screening criteria include at least one of the following: The second number of feature point pairs matched between the image and at least one keyframe is greater than a second number threshold. The translation magnitude of the image relative to the most recent keyframe is greater than the magnitude threshold; The third number of feature points in the image that are not matched with at least one keyframe is greater than the third number threshold. The image is uniformly divided into multiple image grids, and the fourth number of feature points in each image grid is greater than the fourth number threshold and not less than the fifth number threshold.

[0093] In this embodiment, the image is matched with the existing keyframes for feature points, and images with a second number of matching point pairs greater than a second threshold are selected. By selecting images for point cloud reconstruction based on the second number of matching feature point pairs with existing keyframes, it can be ensured that images with sufficient feature point pairs can provide more perspectives and constraints for points in three-dimensional space, thereby making the three-dimensional coordinates of the points more accurate.

[0094] In this embodiment, the translational magnitude of the image is compared with that of the most recent keyframe. If the translational magnitude is greater than the magnitude threshold, it indicates that the motion between the images is significant and the relative displacement between the cameras is large. This allows the point cloud data update to provide more geometric information. By selecting images with significant motion for updating, it can be ensured that the information difference between the images is large enough during the update, thereby improving the robustness of the point cloud data update.

[0095] In this embodiment, the image is matched with the existing keyframes for feature points. Images with a third number of feature points that are not matched with at least one keyframe are selected, which is greater than a third number threshold. In other words, the features corresponding to the regions with low coverage of the current initial point cloud data are selected from the images that have not participated in the triangulation process. This ensures that more new three-dimensional coordinate points are added to update the initial point cloud data, thereby improving the scene coverage of the point cloud data.

[0096] In this embodiment of the application, the image is uniformly divided into multiple image networks. By determining that the fourth number of feature points in each image grid is greater than the fourth number threshold and not less than the fifth number threshold, it is indicated that the feature points are uniformly distributed in the image. This can avoid the feature points in some areas being too sparse or too dense. The image information content is high when the feature points are uniformly distributed, which can ensure that the reconstruction of each point has sufficient viewpoint information support and improve the accuracy of point cloud data updates.

[0097] In S405 of this application embodiment, after determining that the stopping condition is met, the latest updated initial point cloud data is used as the point cloud data of the region.

[0098] In the above scheme, by selecting image pairs with large disparity and a large number of matching point pairs for point cloud reconstruction first, the generated initial point cloud data can be ensured to be more accurate. Providing high-precision initial point cloud data can reduce the accumulation of errors in subsequent reconstruction processes. Furthermore, selecting image pairs with large disparity for initial reconstruction can quickly determine the depth relationship of objects, reduce complex calculations, and effectively accelerate the entire point cloud reconstruction process. During the construction of the point cloud data of the region, the initial point cloud data is updated by continuously adding new images. The images participating in the point cloud data update are selected based on the uniformity of feature point distribution, scene coverage between the image and the keyframe, and the degree of motion significance between the image and the keyframe. This provides images that can provide effective information for the point cloud data update, improving the accuracy and robustness of the point cloud data update while reducing the computational burden of the point cloud data update.

[0099] Based on the above embodiments, this application also provides a method for obtaining a set of matching point pairs, such as... Figure 6 As shown, the specific content is as follows: S501, take the point cloud data corresponding to the earlier image sequence in the two image sequences as the first point cloud data, and take the point cloud data corresponding to the latest image sequence as the second point cloud data. S502, respectively obtain the feature descriptors of each point in the two point cloud data, perform point cloud registration based on the similarity between the feature descriptors of each point in the two point cloud data, obtain a candidate matching point pair set, and determine the first transformation moment corresponding to the candidate matching point pair set based on the candidate matching point pair set and the two point cloud data. S503, perform multiple rounds of global optimization on the first transformation matrix until the preset conditions are met, and use the updated first transformation matrix when the preset conditions are met as the second transformation matrix; S504: By minimizing the distance between matching points in two sets of point cloud data, the second transformation matrix is ​​optimized in multiple rounds to obtain the third transformation matrix; S505, transform the first point cloud data according to the third transformation matrix to obtain the third point cloud data, match the third point cloud data with the second point cloud data point by point, and select matching point pairs with errors less than the preset error threshold to construct a matching point pair set.

[0100] In S501 of this application embodiment, since the two image sequences are sequences of the same region at different times, the point cloud data corresponding to the earlier image sequence is used as the first point cloud data, and the point cloud data corresponding to the later image sequence is used as the second point cloud data. Since the second point cloud data is later than the first point cloud data, the region that has changed in the three-dimensional space can be determined based on the difference between the first point cloud data and the second point cloud data.

[0101] In this embodiment of the application, the first transformation matrix is ​​used to characterize the rotation angle and translation vector required on each coordinate axis for the transformation of points in the first point cloud data into points in the second point cloud data in each candidate matching point pair.

[0102] In S502 of this application embodiment, the similarity between the feature descriptors of each point in the two point cloud data is calculated, that is, the feature similarity between the points in the two point cloud data is calculated. The higher the similarity between the two points, the greater the probability of them being used to describe the same point in three-dimensional space. Therefore, point cloud registration can be performed based on the similarity between the feature descriptors, that is, the points in the two point cloud data that correspond to the same location in the region are determined. If the similarity between the feature descriptors of the two points is greater than the preset similarity threshold, the two points are regarded as a pair of candidate matching point pairs. Based on all the matching point pairs between the two point cloud data, a set of candidate matching point pairs is constructed. After determining the set of candidate matching point pairs, since the first point cloud data is from an earlier period, the region corresponding to the second point cloud data is obtained by changing the region corresponding to the second point cloud data. Therefore, in order to determine the difference between the two points of the candidate matching point pair with the same point in the corresponding region, for each candidate matching point pair, the rotation angle and translation vector required to convert the points in the first point cloud data in the candidate matching point pair into points in the second point cloud data are determined. The rotation angle and translation vector corresponding to each candidate matching point pair in the set of candidate matching point pairs are obtained, and the first transformation matrix corresponding to the combination of candidate matching point pairs is constructed.

[0103] In S503 of this application embodiment, performing multi-round global optimization on the transformation matrix means optimizing the transformation matrix to calculate a global transformation matrix that maximizes the overlap between all point clouds, that is, all point clouds can be reasonably aligned in the coordinate system of three-dimensional space, avoiding the generation of local optimal solutions. The preset conditions for stopping multi-round global optimization can be reaching the maximum number of iterations, the update amount of the transformation matrix in the current round being less than a preset amount, etc. After stopping the optimization of the transformation matrix, the first transformation matrix updated when the preset conditions are met is used as the second transformation matrix.

[0104] In this embodiment, the transformation matrix can be globally optimized using a global nonlinear least squares method or a graph optimization method. These methods minimize the error between all candidate matching point pairs by adjusting the transformation matrix between each point cloud.

[0105] In S504 of this application embodiment, the two sets of point cloud data are point cloud data obtained by transforming the second point cloud data and the first point cloud data through the second transformation matrix. The matching points between the two sets of point cloud data are determined, the distance between the matching points is determined, and the second transformation matrix is ​​optimized in multiple rounds by minimizing the distance between each matching point, thereby obtaining the third transformation matrix.

[0106] In this embodiment of the application, the second transformation matrix can be optimized in multiple rounds by means of ICP optimization. Through ICP optimization, the optimal rotation and translation transformation, i.e. the transformation matrix, is found between the two sets of point cloud data, thereby minimizing the distance between corresponding points in the two sets of point clouds.

[0107] In S505 of this application embodiment, after obtaining the third transformation matrix, the first point cloud data is transformed according to the third transformation matrix, that is, the third point cloud data is obtained by rotation and translation operations based on the first point cloud data. The third point cloud data and the second point cloud data are matched point by point. The distance between the points in the third point cloud data and the second point cloud data is calculated. That is, for each point in the third point cloud data, the distance from that point to each point in the second point cloud data is calculated. If the distance between two points is less than the error threshold, the two points are regarded as a matching point pair. All matching point pairs are used to construct a matching point pair set.

[0108] In the above scheme, by performing global multi-round optimization on the first transformation matrix, the influence of local optima is reduced, ensuring that the two sets of point cloud data are aligned globally. After performing global multi-round optimization, the error between point clouds is minimized by minimizing the distance between matching points in the point cloud data, effectively improving the accuracy of point cloud registration. Furthermore, by performing both global and local optimization, the robustness of the system to different types of point clouds can be increased. The accuracy of the final point cloud registration can be improved through multi-round optimization and adjustment for various types of point clouds.

[0109] Based on the above embodiments, this application also provides a method for obtaining a set of candidate matching point pairs, such as... Figure 7 As shown, the specific content is as follows: S601, For each point in the two sets of point cloud data, calculate the average distance from the point to a preset number of nearest neighbor points, use it as the local point spacing, and determine the search radius of the point based on the local point spacing; S602: Take the median of the local point spacings of all points in the two point cloud datasets as the global spacing. For each point, take the ratio between the global spacing and the local point spacing as the local density ratio of the point. S603, for each point, obtain the angle between the normal vector of the point cloud and the normal vector of each point within the search radius of the point, obtain the distance distribution between the points within the search radius of the point, and use the angle between the normal vectors and the distance distribution as the feature descriptors of the point; S604, For each point in the first point cloud data, search for the nearest neighbor point in the second point cloud data based on the point's feature descriptor to obtain a first set of matching point pairs; For each point in the second point cloud data, search for the nearest neighbor point in the first point cloud data based on the feature descriptor to obtain a second set of matching point pairs. S605, retain the matching point pairs of the nearest neighbors in the first and second matching point pairs sets as the third matching point pairs; S606, for each third matching point pair, adjust the preset distance threshold according to the local density ratio of the two points in the third matching point pair to determine the first reference distance threshold of the third matching point pair. If the distance between the two points in the third matching point pair is not greater than the first reference distance threshold, then the third matching point pair is taken as a candidate matching point pair.

[0110] In S601 of this embodiment, for each point in the two sets of point cloud data, a preset number of nearest neighbor stores are first determined, the distance from the current point to each nearest neighbor is calculated, and then the average of all distances is used as the local point spacing. The preset number can be selected according to the point density and the actual application scenario. Since the search radius is used to find neighborhood points, setting the search radius of each point based on the local point spacing ensures that the search radius of points in different density areas is more consistent with the actual situation.

[0111] In S602 of this application embodiment, the local point spacing of all points in the two point cloud data are sorted in descending order, and the median of the local point spacing is obtained according to the sorting of the local point spacing as the global spacing. The ratio between the global spacing and the local point spacing of each point is used as the local density ratio of that point.

[0112] In S603 of this embodiment, for each point, the points within the search radius of the point are determined, and the normal angle between the normal vector of the current point and the normal vector of each point within the search radius is obtained. The normal angle is used to describe the geometric relationship between points, that is, the local geometric features of the current point. The distance between the current point and each point within the search radius is determined, thereby determining the distance distribution between the current point and the points within the search radius. The distance distribution and the normal angle are used as feature descriptors.

[0113] In S604 of this embodiment, the feature descriptor of each point in the first point cloud data is compared with the feature descriptor of each point in the second point cloud data. Based on the distance between the feature descriptors of the points in the first point cloud data and the feature descriptors of the points in the second point cloud data, i.e., the similarity is calculated, multiple points matching the current point can be found in the second point cloud data. Then, the nearest neighbor point can be determined from the multiple matching points using the nearest neighbor search method. The current point and its corresponding nearest neighbor point form a first matching point pair. Based on the first matching point pairs formed by each point in the first point cloud data and its corresponding nearest neighbor point, a first matching point pair set is formed. Then, following the same method as above, the feature descriptor of each point in the second point cloud data is compared with each feature descriptor of the first point cloud data to determine the nearest neighbor point of each point in the second point cloud data to form a second matching point pair set.

[0114] In S605 of this application embodiment, the points that are the nearest neighbors of each other in the first set of matching point pairs and the second set of matching point pairs are retained as the third matching point pairs. In other words, it is necessary to filter out the points that are the nearest neighbors of each other in order to form the third matching point pairs.

[0115] In S606 of this embodiment, after selecting the third matching point pair, it is necessary to determine whether the third matching point pair can be used as a candidate matching point pair based on a distance threshold. The distance threshold is used to determine whether the distance between the two points in the third matching point pair meets the requirements for being a candidate matching point pair. Each third matching point pair has a corresponding first reference distance threshold. The first reference distance threshold is obtained by adjusting the preset distance threshold based on the local density ratio of the two points in the third matching point pair. The specific formula is as follows: dij = d base×(1+ α •(1-( DRi + DRj ) / 2)) in, dij The first reference distance threshold, d `base` is a preset distance threshold. α For adjustment coefficients, DRi , DRj These are the local density ratios of points i and j in the third matching point pair, respectively.

[0116] In another example, histogram descriptors of point features are quickly calculated for the first and second point cloud data respectively. The distribution of the angle between the normals of a point and its neighboring points and the distance distribution are statistically analyzed to generate a 33-dimensional feature vector. The dynamic threshold used to filter matching point pairs is adaptively adjusted according to the point cloud density. If the point spacing is small (high-density area), the dynamic threshold is decreased (e.g., 0.05); if the point spacing is large (low-density area), the dynamic threshold is increased (e.g., 0.15). Then, a bidirectional consistency check is performed. For each point in the first point cloud data, the nearest point in the second point cloud data is searched as a candidate match. At the same time, for each point in the second point cloud data, the nearest neighbor is searched in the reverse direction in the first point cloud data. Only the point pairs that are each other's nearest neighbors are retained, and erroneous matches are eliminated. Then, a further filtering is performed, retaining point pairs whose matching score (reciprocal of Euclidean distance) is higher than the dynamic threshold. Finally, a set of candidate matching point pairs is output.

[0117] In the above scheme, before obtaining the feature descriptor of each point, the local point spacing of the point is determined based on the average distance from the point to a preset number of nearest neighbors. This local point spacing is then used as the search radius of the point. The search radius of each point conforms to the actual situation of each point. Within the search radius, the geometric features between the point and other points within the search radius are determined, thus determining the feature descriptor of the point. This ensures the accuracy of the feature descriptors for each point. Furthermore, when obtaining the third matching point pair, the matching point pair between the nearest neighbors in the first and second matching point pair sets is selected as the third matching point pair to reduce false matches. Finally, candidate matching point pairs are filtered. At that time, the global distance is determined based on the local point spacing of all points, and the local density ratio of each point is determined based on the ratio between the global point spacing and the local point spacing. When filtering matching point pairs in the third set of matching point pairs, two points with a distance less than the first reference distance threshold are selected as candidate matching point pairs. For each matching point pair, the first reference distance threshold of the current matching point pair is determined based on the local density ratio of the two points. That is, the first reference distance threshold is adaptively adjusted according to the point cloud density, which ensures that different standards are used to filter candidate matching point pairs for regions with different point cloud densities in the image, so as to make the filtering results more accurate.

[0118] Based on the above embodiments, this application also provides a method for updating the first transformation matrix, such as... Figure 8 As shown, the specific content is as follows: S701, transform the first point cloud data using the newly acquired first transformation matrix to obtain the fourth point cloud data; S702, for each point in the fourth point cloud data, obtain the distance between the point and the nearest point in the second point cloud data, take the median from all distances, and adjust the distance threshold by the absolute deviation of the median to obtain the second reference distance threshold; S703, for each point in the fourth point cloud data, if the distance between the point and the nearest point in the second point cloud data is less than the second reference distance threshold, then the point and the corresponding nearest point are regarded as an inner pair; S704, Construct the error function; the error function represents the sum of the matching errors of all interior points; the matching error is the square of the distance between the two points in a pair of interior points; S705, the error function is minimized through nonlinearization to obtain the first transformation increment; the first transformation increment includes rotation increment and translation increment; S706, update the rotation matrix in the first transformation matrix according to the rotation increment, and update the translation vector in the first transformation matrix according to the translation increment to obtain a new first transformation matrix.

[0119] In S701 of this application embodiment, for the first round of update, the newly obtained first transformation matrix refers to the first transformation matrix corresponding to the candidate matching point pair set. For non-first round updates, the newly obtained first transformation matrix refers to the first transformation matrix obtained in the previous round of update. Transforming the first point cloud data using the newly obtained first transformation matrix means rotating each point in the first point cloud data through the rotation matrix in the first transformation matrix so that each point is aligned with the direction of the corresponding point in the second point cloud data. Adjusting the position of each point in the first point cloud data through the translation matrix in the first transformation matrix and translating the point to the region that coincides with the corresponding point in the second point cloud data, thereby obtaining the fourth point cloud data.

[0120] In S702 of this embodiment, for each point in the fourth point cloud data, the distance between the current point and the nearest point in the second point cloud data is obtained. The median of all distances is taken, and a second reference distance threshold is determined using the median. The second reference distance threshold is used to filter out interior points between the fourth and second point cloud data. The median of all distances is taken as the standard deviation. Each distance is compared with the standard deviation, i.e., the absolute deviation between the distance and the standard error is calculated to measure the degree to which the deviation of each point deviates from the standard error. The median is selected as the median absolute deviation from all absolute deviations. The median absolute deviation, adjusted by a preset coefficient, is added to the standard deviation to obtain the second reference distance threshold.

[0121] In S703 of this application embodiment, points in the fourth point cloud data and points whose distance between the nearest point in the second point cloud data and the nearest point is less than the second reference distance threshold are taken as interior points. The smaller the distance, the smaller the deviation of point cloud registration. Interior points refer to points that can accurately reflect the matching relationship between the two sets of point clouds during the point cloud registration process.

[0122] In S704 of this embodiment, the matching error for each interior point is determined. The matching error is calculated based on the square of the distance between the two points in a pair of interior points. Based on the matching error for each pair of interior points, an error function is constructed. The error function represents the sum of the matching errors for all interior points.

[0123] In S705 of this application embodiment, the error function can be linearized by calculating the Jacobian matrix to obtain the rotation increment and translation increment. Alternatively, a graph optimization model, such as g2o or Ceres, can be constructed, the optimization variable can be defined as the first transformation matrix, and then the graph optimization model can be called to solve for the optimal solution that minimizes the error function to obtain the rotation increment and translation increment.

[0124] In S705 of this application embodiment, the rotation matrix in the first transformation matrix is ​​updated according to the rotation increment, and the translation vector in the first transformation matrix is ​​updated according to the translation increment to obtain a new first transformation matrix, so that the error between the first point cloud data and the nearest point in the second point cloud data after each point is transformed according to the new first transformation matrix is ​​smaller.

[0125] In one example, based on the first transformation matrix (R_init, t_init), the first point cloud data P, and the second point cloud data Q, the graph optimization model is first initialized by constructing an incremental graph optimization model, defining the optimization variables as the rotation matrix R and the translation vector t. An error term (edge) is constructed for each inlier pair, with the error function being the matching error. Next, adaptive inlier selection is performed: for each point in the transformed P, the nearest neighbor is searched in Q, and the Euclidean distance is calculated. The distance threshold is dynamically adjusted based on the inlier error distribution: the median absolute deviation (MAD) of the current inlier distance is calculated, and a threshold is set. Then, incremental optimization is performed, optimizing only the variable block associated with the current inlier, rather than all global variables, reducing computational cost. Finally, the total energy function is minimized through nonlinear optimization to obtain the transformation increments ΔR and Δt. The first transformation matrix is ​​updated, and the inlier selection and optimization are repeated until convergence (transformation increments ΔR and Δt are less than the thresholds) or the maximum number of iterations is reached. Finally, the globally optimized second transformation matrix (R_global, t_global) is output.

[0126] In the above scheme, by performing multiple rounds of global optimization on the first transformation matrix, local minima can be avoided, improving the robustness of point cloud reconstruction. Furthermore, the distance value is dynamically adjusted according to the internal point distribution error, thereby more accurately selecting internal point pairs. Only the variables related to the internal point pairs are corrected, reducing the amount of computation.

[0127] Based on the above embodiments, this application also provides a method for updating the second transformation matrix, such as... Figure 9 As shown, the specific content is as follows: S801, the first point cloud data is transformed using the newly obtained second transformation matrix to obtain the fifth point cloud data; S802, for each point in the fifth point cloud data and the second point cloud data, obtain the normal and curvature of the point; S803, for each point in the fifth point cloud data, search for the nearest point of the point in the second point cloud data, take the nearest point of the point as a candidate point pair, obtain the first included angle between the normals of the candidate point pair and the first curvature difference between the curvatures, and determine the weight of the candidate point pair based on the first included angle and the first curvature; the larger the first included angle, the lower the weight; the larger the first curvature difference, the lower the weight. S804: Based on the 3D coordinates of each point in the fifth point cloud data, obtain the 3D coordinates of the first centroid of the fifth point cloud data, and subtract the 3D coordinates of the first centroid from the 3D coordinates of each point in the fifth point cloud data to obtain the first point set. Based on the 3D coordinates of each point in the second point cloud data, obtain the 3D coordinates of the second centroid of the second point cloud data, and subtract the 3D coordinates of the second centroid from the 3D coordinates of each point in the second point cloud data to obtain the second point set. For each candidate point pair, obtain the 3D coordinates of the two points in the candidate point pair from the first point set and the second point set respectively. Construct a weighted covariance matrix based on the 3D coordinates of the two points and the weight of the candidate point pair. S805, perform singular value decomposition on the covariance matrix, and obtain the second transformation increment based on the decomposition result; the second transformation increment includes: second rotation increment and second translation increment; S806, Update the rotation matrix in the second transformation matrix according to the second rotation increment, and update the translation vector in the second transformation matrix according to the second translation increment to obtain a new second transformation matrix; S807: Take the median of the third Euclidean distance among all candidate point pairs, and adjust the convergence threshold according to the median to obtain the target convergence threshold. S808, if the second transformation increment is greater than the target convergence threshold, then continue to execute the next round of iterations.

[0128] In S801 of this application embodiment, for the first round of update, the latest obtained second transformation matrix is ​​the original obtained second transformation matrix. For non-first round updates, the latest obtained second transformation matrix is ​​the second transformation matrix obtained in the previous round of update. The first point cloud data is transformed by the second transformation matrix to obtain the fifth point cloud data. Since the transformation matrix used in each round is different, the fifth point cloud data obtained in each round is also different.

[0129] In S802 of this application embodiment, since new fifth point cloud data is acquired in each round, the normal and curvature of each point in the fifth point cloud data and the second point cloud data are acquired.

[0130] In S803 of this embodiment, each point in the fifth point cloud data is processed one by one. The nearest point to the current point is searched in the second point cloud data. The current point and the nearest point are selected as candidate point pairs. Based on the normals of the two points in the candidate point pair, a first angle is determined between the normals of the candidate point pair. Based on the curvatures of the two points in the candidate point pair, a first curvature difference is determined between the curvatures of the candidate point pair. Both the first angle and the first curvature difference characterize the degree of matching between points in the fifth and second point cloud data. The larger the first angle and the larger the first curvature difference, the lower the degree of matching between the fifth and second point cloud data. Therefore, the candidate point pairs can be weighted according to the first angle and the first curvature difference. The larger the first angle and the larger the first curvature difference, the smaller the weight of the candidate point pair, thereby adjusting the influence of the candidate point pairs on the optimization.

[0131] In S804 of this embodiment, the three-dimensional coordinates of all points in the fifth point cloud data are obtained. The three-dimensional coordinates μ_s of the first centroid of the fifth point cloud data are obtained based on the mean of the three-dimensional coordinates of all points. The three-dimensional coordinates of all points in the second point cloud data are obtained. The three-dimensional coordinates μ_t of the second centroid of the second point cloud data are obtained based on the mean of the three-dimensional coordinates of all points. μ_s is subtracted from the three-dimensional coordinates of each point in the fifth point cloud data. Based on each point with the subtracted first centroid coordinates, a first point set is constructed, i.e., a decentralized point set is obtained. μ_t is subtracted from each point in the second point cloud data. Based on each point with the subtracted second centroid coordinates, a second point set is constructed, resulting in a centralized point set. For each candidate point pair, the three-dimensional coordinates of the corresponding points are obtained from the first point set and from the second point set. A weighted covariance matrix is ​​constructed based on the weights of the candidate point pair and the three-dimensional coordinates of the two points in the candidate point pair, as shown in the following formula: H = Σ [ w_i (p_i - μ_s) [(q_i - μ_t)^T], where w_i is the weight of the i-th candidate point pair, p_i is the point in the fifth point cloud data, and q_i is its corresponding point in the second point cloud data.

[0132] In S805 of this embodiment, the weighted covariance matrix is ​​used to characterize the sum of errors of all candidate point pairs. Singular value decomposition (SVD) is performed on the weighted covariance matrix H to obtain... Calculate the optimal rotation increment based on the SVD decomposition results. This refers to the second rotation increment. Calculate the optimal translation increment. . R and t That is, the second translation increment, the second rotation increment, and the second translation increment together constitute the second transformation increment; In S806 of this embodiment, the rotation matrix in the second transformation matrix is ​​updated according to the second rotation increment R_inc, and the translation vector in the second transformation matrix is ​​updated according to the second translation increment t_inc, so as to obtain a new second transformation matrix.

[0133] In S807 of this application embodiment, after the transformation matrix is ​​optimized, it is necessary to calculate a target convergence threshold for this round. The target convergence threshold is used to determine whether to perform the next round of iterative optimization. Therefore, the median of the distance between two points calculated from all candidate point pairs in this round is taken, and the median is multiplied by the preset convergence threshold to obtain the target convergence threshold. That is, the target convergence threshold of each round is dynamically adjusted based on the distance between candidate point pairs in the current round.

[0134] In S808 of this application embodiment, when the second transformation increment is greater than the dynamic convergence threshold, it indicates that the second transformation matrix can still be effectively updated. Therefore, the next round of second transformation matrix optimization is entered based on the new second transformation matrix.

[0135] In one example, point-to-point weights are first calculated. For each point in the transformed first point cloud data, the nearest neighbor is searched in the second point cloud data to form candidate point pairs. The local curvature difference between the point pairs (e.g., the angle between the normals of the source and target points) is calculated, and point pairs with large curvature differences are assigned low weights. Then, weighted ICP optimization is performed, constructing a weighted error matrix. The optimal rigid body transformation (rotation R, translation t) is solved through SVD decomposition, ensuring the transformation satisfies the right-hand coordinate system. Finally, a dynamic convergence threshold is adjusted by calculating the Frobenius norm of the transformation quantities in the current iteration (e.g., the rotation angle ΔR and the translation amount Δt). The convergence threshold is dynamically adjusted. : =0.001 × median distance of current inliers, to avoid premature termination or redundant calculations caused by a fixed threshold. If the transformation amount (ΔR, Δt) is less than ϵ, or the maximum number of iterations is reached, the iteration terminates and the final transformation is output. The final output is the finely registered transformation matrix (R_final, t_final).

[0136] In the above scheme, the point pair error calculation for each fifth point cloud data also considers local geometric features, such as curvature and normal vectors. Weights for each candidate point pair are assigned using the first curvature difference and the first included angle, thereby suppressing the influence of abnormal matching point pairs. The pseudo-weighted error matrix is ​​solved using Singular Value Decomposition (SVD) to obtain the second transformation increment, which is then used to update the second transformation matrix. This makes the point cloud registration results more accurate. The convergence threshold is dynamically adjusted based on the distance between candidate point pairs in the current round, and the stopping iteration standard is adjusted in real time to make the point cloud registration results more accurate and better reflect the actual situation of each point. Furthermore, the convergence threshold is adjusted based on the median distance between interior points to avoid premature termination or redundant calculations caused by a fixed threshold.

[0137] Based on the above embodiments, this application also provides a change detection method, such as... Figure 10 As shown, the specific content is as follows: S901 divides the third point cloud data and the second point cloud data into three-dimensional voxel grids, with each grid having the same side length. S902, for the center point of each three-dimensional voxel grid, search for neighboring points within a preset radius in the third point cloud data and the second point cloud data respectively, obtain the first point in the third point cloud data and the second point in the second point cloud data, determine the first weighted distance sum of the three-dimensional voxel grid based on the weighted distance between each first point and the center point, and determine the second weighted distance sum of the three-dimensional voxel grid based on the weighted distance between each second point and the center point; S903, for each three-dimensional voxel grid, the difference value of the three-dimensional voxel grid is obtained based on the absolute difference between the first weighted distance and the second weighted distance. The difference values ​​of each three-dimensional voxel grid are summarized to construct a three-dimensional difference map. The three-dimensional difference map is used to characterize the matching difference between the third point cloud data and the second point cloud data at each voxel position. S904, calculate the histogram of gray values ​​of all three-dimensional voxel grids in the three-dimensional difference map, and determine the gray value range and the first proportion of each gray value to all pixels; S905, for each gray value in the gray value range, determine the maximum inter-class variance corresponding to the gray value based on the second proportion of pixels greater than the gray value in the histogram, the third proportion of pixels not greater than the gray value, and the first proportion of pixels with the gray value. S906: Select the gray value corresponding to the maximum inter-class variance as the threshold, and regard the area in the 3D difference map with a gray value greater than the threshold as the area where changes have occurred.

[0138] In S901 of this application embodiment, the three-dimensional voxel mesh divides space into some small three-dimensional units. Each three-dimensional voxel mesh contains point cloud data within a spatial range. By dividing the third point cloud data and the second point cloud data into three-dimensional voxel meshes with the same side length, the subsequent analysis and calculation can be simplified.

[0139] In S902 of this embodiment, for the center point of each three-dimensional voxel mesh, neighboring points within a preset radius are searched in the third point cloud data. Points within the preset radius of the center point are designated as first points, and points within the preset radius of the center point are designated as second points in the second point cloud data. The distances between each first point and the center point are weighted to obtain weighted distances between each point and the center point. The weighted distances are summed to obtain the first weighted distance sum of the current three-dimensional voxel mesh. The distances between each second point and the center point are weighted to obtain the weighted distance between the second point and the current center point. The weighted distances are summed to obtain the second weighted distance sum of the current three-dimensional voxel mesh. The weights can be adjusted based on the local density of the points.

[0140] In S903 of this embodiment, for each 3D voxel mesh, the difference value of the current 3D voxel mesh is determined based on the absolute difference between the first weighted distance sum and the second weighted distance sum, i.e., the absolute value of the difference between the first weighted distance sum and the second weighted distance sum. After obtaining the difference values ​​of all 3D voxel meshes, all difference values ​​are summarized. A 3D difference map is used to characterize the matching difference between the third point cloud data and the second point cloud data in each 3D voxel mesh. The 3D difference map compares the difference between the third point cloud data and the second point cloud data in a visual manner, and displays the difference between the third point cloud data and the second point cloud data using color coding. In S904 of this application embodiment, for each three-dimensional voxel mesh, the difference value of the three-dimensional voxel mesh is mapped to a gray value. The range of difference values ​​can be linearly mapped to the gray value range of [0, 255], or a preset color mapping can be set for each difference value, thereby obtaining a histogram of gray values ​​of each three-dimensional voxel mesh. Based on the histogram of gray values ​​of each three-dimensional voxel mesh, the range of gray values ​​is determined, the number of pixels corresponding to each gray value is determined, and the first proportion of the number of pixels corresponding to each gray value to the total number of pixels is calculated.

[0141] In S905 of this application embodiment, for each gray value in the gray value range, the corresponding maximum inter-class variance is calculated. Based on the current gray value, all gray values ​​in the gray value range are divided into two categories: gray values ​​greater than the current gray value and gray values ​​not greater than the current gray value. For the above two categories of gray values, the number of pixels corresponding to all gray values ​​greater than the current gray value is summed. The ratio between the summed result and the total number of pixels is used as the second proportion of pixels greater than the gray value in the histogram. The number of pixels corresponding to all gray values ​​not greater than the current gray value is summed. The ratio between the summed result and the total number of pixels is used as the third proportion of pixels not greater than the gray value in the histogram. For each gray value greater than the current gray value, multiply the current gray value by the corresponding first ratio to obtain a first product. The ratio of the sum of each first product to a second ratio is used as the mean gray value of the latter class, meaning the gray value is the average gray value of all pixels above the current gray value. For each gray value not greater than the current gray value, multiply the current gray value by the corresponding first ratio to obtain a first product. The ratio of the sum of each first product to a third ratio is used as the mean gray value of the former class, meaning the gray value is the average gray value of all pixels below (including) the current gray value. The maximum inter-class variance is calculated based on the second ratio, the third ratio, and the squared difference between the mean gray values ​​of the former and latter classes.

[0142] In S906 of this application embodiment, each gray value has a corresponding maximum inter-class variance. When the inter-class variance is the maximum, it indicates that the difference between the mean gray value of the previous class and the mean gray value of the next class is the largest. This means that image segmentation using the current gray value can make the foreground and background of the image most distinct. Therefore, the gray value corresponding to the maximum inter-class variance is selected as the threshold, and the gray value of each pixel in the three-dimensional difference map is compared with the threshold. If the gray value corresponding to the pixel is greater than the threshold, it means that the region corresponding to the pixel has changed.

[0143] In one example, the third and second point cloud data are first voxelized and divided into 3D voxel grids, each with a center x and a side length of 0.1m. Then, a neighborhood weighted distance calculation is performed. For each voxel center x, neighboring points within a radius of 0.15m are searched in both the third and second point cloud data. The weighted distances from the two point cloud datasets to x are calculated. Next, the difference map D(x) is calculated as the absolute difference between the two distances and linearly scaled to the [0,1] interval. Finally, the processed 3D difference map D(x) is output. Treating the difference map D(x) as a grayscale image, the Otsu algorithm is applied to automatically determine the segmentation threshold. Otsu determines the optimal threshold t by maximizing the inter-class variance.

[0144] in, and The second and third proportions are respectively. , These represent the mean gray levels of the first and second classes, respectively. Based on a threshold t, voxels are binarized into changing regions (D(x)>t) and non-changing regions. An 8-neighborhood connected component analysis is performed on the binarized results, and the number of voxels in each connected region is labeled. Connected regions with an area less than 0.5 square meters (corresponding to fewer than 50 voxels) are filtered out to eliminate spurious changes caused by registration errors or dynamic objects (such as vehicles and pedestrians). Then, geometric attributes are extracted. For the retained changing regions, attributes such as volume (number of voxels × 0.01 cubic meters) and surface normal change rate (statistically calculated through the angle between neighboring normals) are calculated. Finally, the final changing region mask and the geometric attributes (volume, normal change rate) of the changing regions are output.

[0145] In the above scheme, combining geometric information to construct a voxel difference map and segmenting it using the maximum inter-class variance method can not only improve the accuracy and robustness of change detection, but also optimize the segmentation results and reduce manual intervention. Using geometric information to construct the voxel difference map, combined with the segmentation using the maximum inter-class variance method, can more efficiently and accurately identify the changed regions, providing strong support for further analysis.

[0146] In this embodiment, during the point cloud data construction process, local image balancing (BA) is used to optimize the pose and 3D coordinates of the most recent frames, reducing accumulated errors. Global BA is performed periodically to optimize the global consistency of all keyframes and improve reconstruction accuracy. Representative and informative images are selected as keyframes, and the feature points and pose information of these keyframes are retained for subsequent reconstruction and optimization. Specifically, this involves: first, constructing a local optimization, fixing the poses of non-common-view keyframes, and optimizing only the poses of the current keyframe and its common-view keyframes, as well as the coordinates of associated points. Through nonlinear optimization (such as the Levenberg-Marquardt algorithm), the poses and 3D coordinates of the points are adjusted to minimize the error between the reprojection position of the feature points and the actual observation. Then, the optimized local poses and 3D coordinates are synchronized to the global map, replacing the original data. Finally, the optimized poses of the common-view keyframes, the 3D coordinates of the local point cloud, and the updated global map are output. After obtaining a new keyframe, this application embodiment determines the proportion of unoptimized points in the global point cloud. If it exceeds a threshold (e.g., 30%), global BA is triggered. Alternatively, the average matching error after local BA is calculated. If the error growth rate exceeds a threshold (e.g., 10% / frame), global BA is triggered. Then, the poses of all keyframes and the global point cloud are jointly optimized to minimize the matching error of all feature points, eliminate accumulated errors, and further replace the poses and point cloud data in the global map to ensure global consistency. The globally optimized poses, 3D coordinates, and error indicators (e.g., average matching error) are output.

[0147] In this embodiment, after acquiring the point cloud data of the region, the process further includes: denoising and filtering the reconstructed point cloud results to improve point cloud quality. For example, outliers or noise points can be removed to make the point cloud cleaner and more accurate. Texture mapping is then performed to assign color information from the original image to the 3D point cloud, generating a more realistic scene representation. Through texture mapping, the reconstructed 3D model can possess richer visual information.

[0148] In this embodiment of the application, a method for optimizing the first transformation matrix is ​​also provided, the details of which are as follows: Step 1: Transform the first matrix Applying the first point cloud data P to generate the transformed fourth point cloud data. Simultaneously initialize the graph optimization model (such as g2o or Ceres), define the optimization variables as the rotation matrix R and the translation vector t, and set the initial values ​​to... and The final output includes the initialized graph optimization model and transformation parameters. .

[0149] Step 2: Based on the transformation matrix from the previous step (The result after the kth iteration, initially) First point cloud data P, second point cloud data Q, distance threshold For each point Calculate its nearest neighbor in the second point cloud. Form candidate point pairs Calculate the transformed points. and Euclidean distance: Filter to meet Construct an interior point set by taking pairs of points as interior points. .

[0150] Step 3: Based on the set of interior points And Huber robust kernel function ρ(), using rotation matrix R and translation vector t as optimization variables (nodes). For each inlier pair Construct error terms (edges), with the error function being... Then, the total energy function is constructed as follows: .

[0151] Step 4: Then, based on the graph optimization model and energy function E(R,t), optimize the algorithm. Specifically, call an optimization library (such as g2o or Ceres) to find the optimal solution that minimizes the energy function E(R,t), obtaining the transformation parameter increments ΔR and Δt. Update the transformation matrix. The final transformation matrix is ​​obtained. .

[0152] Step 5: Simultaneously, based on the set of interior points... and distance threshold Calculate the statistics of the inlier distance error (such as the median or mean μ and standard deviation σ), and update the threshold. (e.g., α=1.0), or decrease by a fixed step size (e.g., ... Re-select point pairs that meet the new threshold and update the interior point set. .

[0153] Step 6: Based on the current iteration number k, determine the maximum iteration number. Convergence threshold ϵ (e.g., transformation amount) Check if the convergence condition is met: If If the iteration fails, then terminate the iteration. If the result is positive, return to step two; otherwise, output the final second transformation matrix. .

[0154] In this embodiment of the application, an optimization method for the second transformation matrix is ​​also provided, the specific content of which is as follows: The second transformation matrix... Applied to the source point cloud P, it generates the fifth point cloud data after the initial transformation. Initialize the iteration counter k=0 and set the current transformation matrix. .

[0155] Step 2: Based on the transformed fifth point cloud data Second point cloud data Q, for each point Search for the nearest neighbor in the second point cloud data Q. Form candidate point pairs The search method uses a KD-tree to accelerate nearest neighbor queries, based on Euclidean distance. The metric is used to output a set of candidate point pairs. .

[0156] Step 3: Based on the input candidate point set The optimal rigid body transformation is calculated, including centroid removal, covariance matrix construction, and SVD decomposition, to obtain the optimal rigid body transformation. .

[0157] Decentrifugation: Calculate the centroid of the fifth cloud point:

[0158] Calculate the centroid of the second cloud point:

[0159] The fifth point of decentrifugation is as follows:

[0160] The second point of decentrifugation:

[0161] Construct the covariance matrix:

[0162] SVD decomposition: Perform singular value decomposition on H:

[0163] Calculate the rotation matrix: (like The last column of V needs to be adjusted to ensure a right-handed coordinate system.

[0164] Calculate the translation vector:

[0165] Step 4: Based on the generated candidate point pair set And the optimal transformation obtained from the above calculation Convergence threshold Maximum number of iterations First, update the transformation matrix. (Calculate the transformation amount by matrix multiplication and cumulative transformation) And extract the rotation component ΔR and the translation component Δt. Then check the convergence condition: if (I is the identity matrix, If the Frobenius norm is true, then the iteration terminates; if... If the iteration is terminated, then the updated transformation matrix is ​​output. Iteration counter , or convergence / termination flag.

[0166] Step 5: Based on the transformation matrix when convergence is not achieved Given the target point cloud Q, return to step two and use the updated second transformation matrix. That is, the third transformation matrix, recalculate the point pair associations, and repeat the third and fourth steps to finally obtain the global optimal solution.

[0167] In this embodiment, the application can be applied to the fields of UAV remote sensing and low-altitude three-dimensional spatial monitoring. The method is directly applied to the dynamic monitoring of three-dimensional spatial information in low-altitude environments. By acquiring image data through UAVs equipped with multi-source sensors, and combining computer vision, three-dimensional reconstruction and real-time analysis technologies, spatial change detection with centimeter-level accuracy can be achieved. It can be widely used in urban planning, environmental monitoring, disaster emergency response and other fields, providing key technical support for smart cities, ecological protection and other scenarios. It has the advantages of multi-source data fusion, real-time detection and monitoring capabilities, as well as high precision and low latency.

[0168] The three-dimensional spatial change detection method provided in this application integrates knowledge from multiple disciplines, including UAV remote sensing technology, computer vision, 3D reconstruction, and spatial analysis. Its core principle is to use a UAV equipped with a high-resolution camera to periodically photograph low-altitude scenes, acquiring continuous image sequences of the scene surface; to preprocess, extract, and match features from the images using computer vision algorithms; to generate a 3D model of the scene using 3D reconstruction technology; and finally, to compare the 3D models at different time points using spatial analysis algorithms, accurately calculating key parameters of scene changes, thereby achieving real-time monitoring and early warning of spatial changes in low-altitude scenes.

[0169] In this embodiment, a drone equipped with LiDAR can cover several square kilometers in a single flight. Multi-view data fusion reduces stitching errors, balancing coverage and data density. Simultaneously, the drone can dynamically adjust its flight trajectory, combining computer vision to track moving targets (such as YOLO Tiny+DeepSORT), and using multispectral sensors to penetrate lightweight obstructions (such as fog).

[0170] In this embodiment, the solution utilizes a drone flying at low altitude (50-1000 meters) equipped with a high-resolution sensor (such as a 4K optical camera or a 16-line LiDAR), combined with edge computing to achieve centimeter-level accuracy and sub-second latency, directly addressing the spatial resolution and real-time performance bottlenecks of satellite remote sensing. Simultaneously, the drone can flexibly select flight time and route, avoid adverse weather conditions, and achieve sub-meter-level positioning accuracy through the fusion of GPS / RTK and inertial navigation, reducing the impact of geometric distortion.

[0171] The three-dimensional spatial change detection method provided in this application integrates knowledge from multiple disciplines, including UAV remote sensing technology, computer vision, 3D reconstruction, and spatial analysis. Its core principle is to use a UAV equipped with a high-resolution camera to periodically photograph low-altitude scenes, acquiring continuous image sequences of the scene surface; to preprocess, extract, and match features from the images using computer vision algorithms; to generate a 3D model of the scene using 3D reconstruction technology; and finally, to compare the 3D models at different time points using spatial analysis algorithms, accurately calculating key parameters of scene changes, thereby achieving real-time monitoring and early warning of spatial changes in low-altitude scenes.

[0172] In this embodiment, the lightweight network design (128-dimensional descriptors) reduces computation by 50%, while capturing global context through multi-scale expansion. The SE module strengthens key feature channels and suppresses noise interference (H-Swish improves the stability of low-precision computation). The dynamic BA optimization strategy (local + global) reduces the matching error to below 0.2 pixels (compared to approximately 0.5 pixels in traditional methods), solving the problems of high computational cost and significant cumulative error in traditional feature extraction networks.

[0173] In this embodiment, the improved ICP algorithm (KD tree + covariance SVD) increases registration speed by 3 times with millimeter-level accuracy. It solves the noise sensitivity problem of existing Pei Zun algorithms. Spatial variations are quantified using voxel difference maps to suppress local registration errors. The Otsu algorithm automatically segments changing regions, improving accuracy to 95% (compared to approximately 85% for traditional methods). Connected component filtering eliminates dynamic interference (such as vehicles), reducing the false alarm rate to below 5%. It also addresses the issue that traditional threshold segmentation relies on manual settings and cannot adapt to complex scenarios.

[0174] This application provides a device for detecting changes in three-dimensional space, such as... Figure 11 As shown, the three-dimensional spatial change detection device 110 may include: an acquisition module 1101, a reconstruction module 1102, a registration module 1103, and a determination module 1104.

[0175] Specifically, the acquisition module 1101 is used to acquire image sequences of the same area taken by the UAV in two different periods; Reconstruction module 1102 is used to reconstruct the point cloud of the region based on the image sequence for each image sequence, and obtain a set of point cloud data of the region; The registration module 1103 is used to obtain the feature descriptors of each point in the two sets of point cloud data respectively, and to perform point cloud registration based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. The set of matching point pairs includes multiple matching point pairs, and the two points in each matching point pair belong to two point cloud data and are the closest points to each other. The determination module 1104 is used to obtain the geometric information of the two points in each matching point pair in their respective point cloud data. If the difference between the two geometric information is determined to meet the preset difference condition, then the position of the matching point pair in the region is determined to have changed.

[0176] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0177] This application provides an electronic device (computer device / equipment / system) including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a method for detecting changes in three-dimensional space. Compared with related technologies, it can achieve the following: by acquiring image sequences of the same area taken by a drone in two different periods, for each image sequence, point cloud reconstruction of the area is performed based on the image sequence to obtain a set of point cloud data of the area, thereby aligning the data of the two time periods to effectively identify the changed areas in the region. Feature descriptors of each point in the two sets of point cloud data are acquired respectively, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By acquiring the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between the local features, the matching between points at the same location in the two sets of point cloud data at different times is achieved, that is, matching point pairs are obtained. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is acquired. If the difference between the two geometric information is determined to meet a preset difference condition, it is determined that the position of the matching point pair in the region has changed. By using only the difference in geometric information between two points in a matching point pair, it is possible to determine whether the region corresponding to the matching point pair in three-dimensional space has changed. This saves computational resources while ensuring the accuracy of change detection in three-dimensional space.

[0178] In one alternative embodiment, an electronic device is provided, such as Figure 12 As shown, Figure 12 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0179] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0180] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0181] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0182] The memory 4003 is used to store computer programs that execute the embodiments of this application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0183] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0184] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve the following: by acquiring image sequences of the same area taken by a drone in two different periods, for each image sequence, point cloud reconstruction of the area is performed based on the image sequence to obtain a set of point cloud data of the area, thereby aligning the data of the two time periods to effectively identify the changed areas in the region. Feature descriptors of each point in the two sets of point cloud data are obtained respectively, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By acquiring the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between the local features, matching between points at the same location in the two sets of point cloud data at different times is achieved, that is, obtaining matching point pairs. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is acquired. If the difference between the two geometric information is determined to meet a preset difference condition, it is determined that the position of the matching point pair in the region has changed. By using only the difference in geometric information between two points in a matching point pair, it is possible to determine whether the region corresponding to the matching point pair in three-dimensional space has changed. This saves computational resources while ensuring the accuracy of change detection in three-dimensional space.

[0185] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0186] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve: By acquiring image sequences of the same area captured by a drone at two different times, point cloud reconstruction is performed on the region based on each image sequence to obtain a set of point cloud data for the region. This allows for the alignment of data from the two time periods, enabling effective identification of changed areas within the region. Feature descriptors for each point in the two sets of point cloud data are obtained, and point cloud registration is performed based on the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a set of matching point pairs. By obtaining the feature descriptors of each point, the local features of each point in the two sets of point cloud data can be determined. By comparing the similarity between local features, matching is achieved between points at the same location in the two sets of point cloud data at different times, i.e., obtaining matching point pairs. For each matching point pair, the geometric information of the two points in the matching point pair in their respective point cloud data is obtained. If the difference between the two geometric information values ​​meets a preset difference condition, it is determined that the corresponding position of the matching point pair in the region has changed. By using only the difference in geometric information between the two points in the matching point pair to determine whether the region corresponding to the matching point pair in three-dimensional space has changed, the accuracy of three-dimensional space change detection is ensured while saving computational resources.

[0187] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the illustrations or text descriptions.

[0188] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0189] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A method of detecting three-dimensional spatial variations, characterized by, The application relates to a method for determining the position change of a region, comprising: acquiring image sequences of the same region respectively photographed by a UAV at two time periods; the region is a three-dimensional space; for each image sequence, point cloud reconstruction is performed on the region based on the image sequence, and a set of point cloud data of the region is obtained; feature descriptors of each point in the two sets of point cloud data are respectively acquired, point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data, a set of matching point pairs is acquired, the set of matching point pairs comprises a plurality of matching point pairs, the two points in each matching point pair belong to two point cloud data, and the two points are the nearest points of each other; for each matching point pair, the geometric information of the two points in the matching point pair in the respective point cloud data is acquired, and if it is determined that the difference between the two geometric information satisfies a preset difference condition, it is determined that the corresponding position of the matching point pair in the region changes.

2. The method of claim 1, wherein, The image sequence further comprises a target feature map corresponding to each image, and the method for acquiring the feature map comprises: for each image in the two image sequences, n convolution layers with different dilation rates are used to extract features of the image, and n scale first feature maps of the image are obtained; n is an integer greater than 1; the n scale feature maps are spliced along the channel dimension to obtain a spliced feature map, and the spliced feature map is subjected to channel attention weighting to obtain a weighted feature map; the feature dimension of the weighted feature map is expanded to obtain a second feature map, and the second feature map is sequentially subjected to feature enhancement and normalization processing to obtain the target feature map corresponding to the image.

3. The method of claim 2, wherein, The image sequence further comprises a target feature map corresponding to each image; for each image sequence, point cloud reconstruction is performed on the region based on the image sequence, and a set of point cloud data of the region is obtained, comprising: the origin of the camera coordinate system of the first frame image in the image sequence is taken as the origin of the world coordinate system, the pose of the first frame image in the world coordinate system is initialized as a unit matrix, and the poses of the images in the image sequence in the world coordinate system are determined according to the relative poses between adjacent two frame images in the image sequence and the unit matrix; for any two images in the image sequence, feature points are extracted from the target feature maps corresponding to the two images respectively, and projection matrices of the two images are determined according to a preset intrinsic matrix and extrinsic matrices of the two images; based on the local region features of the corresponding positions of the feature points in the images, matching feature point pairs between the two images are determined; the feature points are used for describing the feature information of the local region of the corresponding positions in the images; for any matching feature point pair between the two images, the pixel coordinates of the two feature points in the respective images are acquired; triangulation processing is performed according to the pixel coordinates of the two feature points in the matching feature point pairs between the images and the projection matrices of the two images, and a set of point cloud data of the region is obtained.

4. The method of claim 3, wherein, The two feature points in the matched feature point pair between any two images in the image sequence are matched, and a set of point cloud data of the region is obtained by triangulation processing according to the pixel coordinates of the two feature points and the projection matrix of the two images. For any two images in the image sequence, the average disparity between the two images and the first number of matched feature point pairs between the two images are obtained. Two images with an average disparity greater than a preset pixel threshold and a first number greater than a first number threshold are taken as an initial image pair. The matched feature point pairs between the initial image pair are triangulated according to the pixel coordinates of the matched feature point pairs and the projection matrix of the two images in the initial image pair, and the three-dimensional coordinates of the matched feature point pairs are obtained to construct the initial point cloud data of the region. The images in the image sequence that have been triangulated are taken as key frames, and the initial point cloud data is updated based on a screening condition from the images in the image sequence except the key frames until a stop condition is met. The initial point cloud data updated when the stop condition is met is taken as the point cloud data of the region. The screening condition includes at least one of the following: The second number of matched feature point pairs between the image and at least one key frame is greater than a second number threshold. The length of the translation between the image and the nearest key frame is greater than a length threshold. The third number of unmatched feature points between the image and at least one key frame is greater than a third number threshold. The image is uniformly divided into multiple image grids, and the fourth number of feature points in each image grid is greater than a fourth number threshold and not less than a fifth number threshold.

5. The method of claim 1, wherein, The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The point cloud data corresponding to the image sequence of the earlier period in the two image sequences is taken as the first point cloud data, and the point cloud data corresponding to the image sequence of the latest period is taken as the second point cloud data. The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets of point cloud data are obtained respectively, and the point cloud registration is performed according to the similarity between the feature descriptors of each point in the two sets of point cloud data to obtain a matched point pair set, including: The feature descriptors of each point in the two sets Transform the first point cloud data according to the third transformation matrix to obtain third point cloud data, and perform point-by-point matching on the third point cloud data and the second point cloud data, and select matching point pairs with an error less than a preset error threshold to construct a matching point pair set.

6. The method of claim 5, wherein, The candidate matching point pair set includes a plurality of candidate matching point pairs. The respective feature descriptors of the points in the two groups of point cloud data are obtained, and point cloud registration is performed according to the similarity between the feature descriptors of the points in the two groups of point cloud data to obtain a candidate matching point pair set, including: For each point in the two groups of point cloud data, the average distance of the point to a preset number of nearest neighbor points is calculated as the local point spacing of the point, and the search radius of the point is determined according to the local point spacing; The median of the local point spacings of all points in the two point cloud data is taken as a global spacing, and for each point, the ratio between the global spacing and the local point spacing of the point is taken as the local density ratio of the point; For each point, the normal vector of the point and the normal vectors of points within the search radius of the point are obtained, the normal angle between the normal vectors is obtained, and the distance distribution between the point and the points within the search radius of the point is obtained, and the normal angle and the distance distribution are taken as the feature descriptor of the point; For each point in the first point cloud data, the nearest neighbor point in the second point cloud data is searched according to the feature descriptor of the point to obtain a first matching point pair set, and for each point in the second point cloud data, the nearest neighbor point in the first point cloud data is searched according to the feature descriptor to obtain a second matching point pair set; The matching point pairs of the points that are nearest neighbors of each other in the first matching point pair set and the second matching point pair set are retained as third matching point pairs; For each third matching point pair, the local density ratios of the two points in the third matching point pair are adjusted according to a preset distance threshold to determine a first reference distance threshold of the third matching point pair, and if the distance between the two points in the third matching point pair is not greater than the first reference distance threshold, the third matching point pair is taken as a candidate matching point pair.

7. The method of claim 5, wherein, One round of iterative updating of the first transformation matrix includes: Transforming the first point cloud data by using the newly obtained first transformation matrix to obtain fourth point cloud data; For each point in the fourth point cloud data, the distance between the point and the nearest point in the second point cloud data is obtained, the median of all distances is taken, the distance threshold is adjusted by the absolute deviation of the median to obtain a second reference distance threshold; For each point in the fourth point cloud data, if the distance between the point and the nearest point in the second point cloud data is less than the second reference distance threshold, the point and the corresponding nearest point are taken as a pair of inliers; An error function is constructed; the error function represents the sum of matching errors of all inliers; the matching error refers to the square of the distance between the two points in a pair of inliers; The error function is minimized by non-linearization to obtain a first transformation increment; the first transformation increment includes a rotation increment and a translation increment; According to the rotation increment, a rotation matrix in the first transformation matrix is updated, and according to the translation increment, a translation vector in the first transformation matrix is updated, to obtain a new first transformation matrix.

8. The method of claim 5, wherein, The one-round nearest point optimization of the second transformation matrix comprises: The first point cloud data is transformed by the newly obtained second transformation matrix to obtain fifth point cloud data; For each point in the fifth point cloud data and the second point cloud data, a normal and a curvature of the point are obtained; For each point in the fifth point cloud data, a nearest point of the point in the second point cloud data is searched, the point and the nearest point of the point are taken as a candidate point pair, a first included angle between normals of the candidate point pair and a first curvature difference between curvatures are obtained, and a weight of the candidate point pair is determined according to the first included angle and the first curvature difference; the greater the first included angle is, the lower the weight is, and the greater the first curvature difference is, the lower the weight is. According to three-dimensional coordinates of each point in the fifth point cloud data, three-dimensional coordinates of a first centroid of the fifth point cloud data are obtained, and the three-dimensional coordinates of each point in the fifth point cloud data are subtracted by the three-dimensional coordinates of the first centroid to obtain a first point set; according to three-dimensional coordinates of each point in the second point cloud data, three-dimensional coordinates of a second centroid of the second point cloud data are obtained, and the three-dimensional coordinates of each point in the second point cloud data are subtracted by the three-dimensional coordinates of the second centroid to obtain a second point set; for each candidate point pair, three-dimensional coordinates of two points in the candidate point pair are respectively obtained from the first point set and the second point set, and a weighted covariance matrix is constructed according to the three-dimensional coordinates of the two points and the weight of the candidate point pair; the weighted covariance matrix is used to represent a sum of errors of all candidate point pairs; singular value decomposition is performed on the covariance matrix, and a second transformation increment is obtained according to a decomposition result; the second transformation increment comprises a second rotation increment and a second translation increment; a rotation matrix in the second transformation matrix is updated according to the second rotation increment, and a translation vector in the second transformation matrix is updated according to the second translation increment, to obtain a new second transformation matrix. A median is taken from distances of all the candidate point pairs, a target convergence threshold is adjusted according to the median, and the target convergence threshold is obtained; If the second transformation increment is greater than the target convergence threshold, a next round of iteration is continued.

9. The method of claim 1, wherein, The obtaining of the geometric information of the two points in the matching point pair in the respective point cloud data, and the determination of the change of the corresponding positions of the matching point pair in the region, if it is determined that the difference between the two geometric information satisfies a preset difference condition, comprises: The third point cloud data and the second point cloud data are respectively divided into three-dimensional voxel grids, and the length of each grid is the same. For each center point of a three-dimensional voxel grid, search for neighboring points within a preset radius in the third point cloud data and the second point cloud data respectively, obtain a first point in the third point cloud data and a second point in the second point cloud data, determine a first weighted distance sum of the three-dimensional voxel grid according to the weighted distances between each first point and the center point, and determine a second weighted distance sum of the three-dimensional voxel grid according to the weighted distances between each second point and the center point; For each three-dimensional voxel grid, obtain a difference value of the three-dimensional voxel grid according to an absolute difference value between the first weighted distance sum and the second weighted distance, and aggregate the difference values of each three-dimensional voxel grid to construct a three-dimensional difference map; the three-dimensional difference map is used to represent the matching difference of the third point cloud data and the second point cloud data in each three-dimensional voxel grid; Calculate a histogram of gray values of all three-dimensional voxel grids in the three-dimensional difference map, and determine a gray value range and a first proportion of pixels of each gray value in all pixels; For each gray value in the gray value range, determine a maximum between-cluster variance corresponding to the gray value according to a second proportion of pixels greater than the gray value in all pixels, a third proportion of pixels not greater than the gray value in all pixels, and the first proportion of pixels of the gray value in all pixels in the histogram; Select the gray value corresponding to the maximum between-cluster variance as a threshold, and regard a region with a gray value greater than the threshold in the three-dimensional difference map as a changed region.

10. A device for detecting a three-dimensional spatial variation, characterized in that Comprise: An acquisition module is used to acquire image sequences respectively photographed by a UAV on a same region in two periods; A reconstruction module is used to perform point cloud reconstruction on the region based on each image sequence, and obtain a set of point cloud data of the region; A registration module is used to acquire feature descriptors of each point in two sets of point cloud data respectively, and perform point cloud registration according to the similarity between the feature descriptors of each point in the two sets of point cloud data, and acquire a set of matching point pairs, the set of matching point pairs includes a plurality of matching point pairs, each matching point pair includes two points belonging to two point cloud data, and the two points are nearest points of each other; A determination module is used to acquire geometric information of two points in each matching point pair in the respective point cloud data, and determine that a corresponding position of the matching point pair in the region has changed if it is determined that the difference between the two geometric information satisfies a preset difference condition.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 10. The processor executes the computer program to implement the method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-9.

Images