Deformation monitoring methods, devices, equipment, media, and products based on feature point detection

Abstract

The application provides a deformation monitoring method and device based on feature point detection, equipment, medium and product, the method comprises the following steps: obtaining a to-be-detected image pair corresponding to a target object, the to-be-detected image pair comprises a first image and a second image, the first image and the second image respectively reflect the state of the target object at different time; inputting the first image and the second image into a trained feature point detection model respectively, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, the first feature point detection result comprises a plurality of first feature points, and the second feature point detection result comprises a plurality of second feature points; matching the first feature points and the second feature points to obtain a feature point pair; and determining deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on the pixel position of the feature point pair. The application can reduce the cost of deformation monitoring on the basis of ensuring the accuracy of deformation monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a deformation monitoring method and device based on feature point detection, equipment, medium and product. BACKGROUND

[0002] Deformation monitoring is of great significance for maintaining and ensuring public safety. It can help engineers and maintenance personnel to understand the health status of building structures in a timely manner, predict and prevent potential structural problems. For example, under the influence of long-term service and natural factors, monitoring the deformation of buildings can assess their safety, thereby deciding whether to repair or reinforce them.

[0003] Traditional deformation monitoring techniques usually rely on physical sensors such as strain gauges, inclinometers, displacement sensors, etc. However, high-precision sensors may be damaged or fail due to environmental influences such as temperature changes, humidity, chemical corrosion, and need to be checked and replaced regularly, with high installation and maintenance costs. SUMMARY

[0004] The present application provides a deformation monitoring method, device, equipment, medium and product based on feature point detection, to solve the defect of high cost of deformation monitoring by physical sensors in the prior art, and to reduce the cost of deformation monitoring.

[0005] The present application provides a deformation monitoring method based on feature point detection, comprising:

[0006] Obtaining a pair of images to be detected corresponding to a target object, the pair of images to be detected including a first image and a second image, the first image and the second image reflecting the state of the target object at different times respectively;

[0007] Inputting the first image and the second image into a trained feature point detection model respectively, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, the first feature point detection result including a plurality of first feature points, and the second feature point detection result including a plurality of second feature points;

[0008] Matching the first feature points and the second feature points to obtain a pair of feature points;

[0009] Based on the pixel position of the pair of feature points, determining the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image;

[0010] The feature point detection model comprises a multi-scale feature module and a fusion module, the multi-scale feature module is configured to construct a feature pyramid based on a target image input to the feature point detection model, and output intermediate feature points corresponding to each layer of feature images of the feature pyramid, and the fusion module is configured to output a feature point detection result of the target image based on the intermediate feature points corresponding to each layer of feature images.

[0011] According to the feature point detection-based deformation monitoring method provided in the application, the multi-scale feature module comprises a sampling layer and a plurality of detection network layers, the detection network layers comprise a feature extraction module and a feature point head, the first image and the second image are input to the trained feature point detection model respectively, and first feature point detection results and second feature point detection results output by the feature point detection model are acquired, which comprises:

[0012] The first image and the second image are respectively taken as the target image, and the target image is input to the sampling layer, and a plurality of times of up-sampling and / or down-sampling operations are performed on the target image to obtain a plurality of scale to-be-processed images;

[0013] Each to-be-processed image is input to the feature extraction module in the corresponding detection network layer, and the feature extraction module is used to perform feature extraction on the to-be-processed image to obtain each layer of feature images of the feature pyramid;

[0014] The feature image is input to the feature point head in the corresponding detection network layer, and the feature point head is used to output the intermediate feature points corresponding to the feature image and the confidence of the intermediate feature points;

[0015] The intermediate feature points corresponding to each feature image and the confidence of the intermediate feature points are input to the fusion module, and a feature point detection result of the target image output by the fusion module is acquired.

[0016] According to the feature point detection-based deformation monitoring method provided in the application, the intermediate feature points corresponding to each feature image and the confidence of the intermediate feature points are input to the fusion module, and a feature point detection result of the target image output by the fusion module is acquired, which comprises:

[0017] The fusion module is used to perform the following operations:

[0018] The intermediate feature points are mapped to the same scale;

[0019] select the intermediate feature point with the highest confidence as a to-be-processed feature point, and obtain an overlap degree of the intermediate feature point in the neighborhood of the to-be-processed feature point and the to-be-processed feature point respectively, the overlap degree reflecting the similarity between the intermediate feature point and the to-be-processed feature point;

[0020] remove the intermediate feature point with the overlap degree higher than the preset threshold;

[0021] select the intermediate feature point with the highest confidence from the intermediate feature points removed from the to-be-processed feature point as a new to-be-processed feature point, and repeat the step of obtaining the overlap degree of the intermediate feature point in the neighborhood of the to-be-processed feature point and the to-be-processed feature point respectively until all the intermediate feature points are traversed;

[0022] detect the intermediate feature point remaining as the feature point detection result of the target image.

[0023] According to the deformation monitoring method based on feature point detection provided by the application, the step of obtaining the overlap degree of the intermediate feature point in the neighborhood of the to-be-processed feature point and the to-be-processed feature point comprises:

[0024] obtain the feature descriptor of the intermediate feature point and the to-be-processed feature point respectively;

[0025] determine the overlap degree of the intermediate feature point and the to-be-processed feature point based on the feature descriptor of the intermediate feature point and the to-be-processed feature point.

[0026] According to the deformation monitoring method based on feature point detection provided by the application, the detection network layer is constructed based on a superpoint network.

[0027] According to the deformation monitoring method based on feature point detection provided by the application, after the step of determining the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image, the method further comprises:

[0028] generate a composite image based on the deformation data and the image of the target object;

[0029] display the composite image.

[0030] The application further provides a deformation monitoring method based on feature point detection, comprising:

[0031] an image acquisition module, configured to acquire a to-be-detected image pair corresponding to a target object, the to-be-detected image pair comprising a first image and a second image, the first image and the second image reflecting the state of the target object at different times respectively;

[0032] The feature point detection module is used to input the first image and the second image into the trained feature point detection model respectively, and obtain the first feature point detection result and the second feature point detection result output by the feature point detection model. The first feature point detection result includes multiple first feature points, and the second feature point detection result includes multiple second feature points.

[0033] The feature point matching module is used to match the first feature point and the second feature point to obtain feature point pairs;

[0034] The deformation analysis module is used to determine the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image, based on the pixel position of the feature point pair.

[0035] The feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature pyramid.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deformation monitoring method based on feature point detection as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deformation monitoring method based on feature point detection as described above.

[0038] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the deformation monitoring method based on feature point detection as described above.

[0039] This invention provides a deformation monitoring method, apparatus, device, medium, and product based on feature point detection. By acquiring a first image and a second image that reflect the state of a target object at different times, feature point detection is performed on the first image and the second image respectively, and deformation measurement is achieved through feature point matching. This method does not rely on physical sensors. During the feature point detection process, a multi-scale processing mechanism is used to improve the accuracy of feature point detection, thereby reducing the cost of deformation monitoring while ensuring the accuracy of deformation monitoring. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating a deformation monitoring method based on feature point detection provided by the present invention.

[0042] Figure 2 This is a schematic diagram of the deformation monitoring device provided by the present invention.

[0043] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] In existing technologies, deformation monitoring relies on physical sensors, such as strain gauges, inclinometers, and displacement sensors. However, high-precision sensors may be damaged or fail due to environmental factors (such as temperature changes, humidity, and chemical corrosion), requiring regular inspection and replacement, which results in high installation and maintenance costs.

[0046] To address the high cost of deformation monitoring using physical sensors in existing technologies, this invention provides a deformation monitoring method, device, equipment, medium, and product based on feature point detection, thereby reducing the cost of deformation monitoring.

[0047] The following is combined Figure 1 The deformation monitoring method based on feature point detection provided by this invention is described as follows: Figure 1 As shown, the present invention provides a deformation monitoring method based on feature point detection, comprising the following steps:

[0048] S110. Obtain the image pair to be detected corresponding to the target object. The image pair to be detected includes a first image and a second image. The first image and the second image respectively reflect the state of the target object at different times.

[0049] S120. Input the first image and the second image into the trained feature point detection model respectively, and obtain the first feature point detection result and the second feature point detection result output by the feature point detection model. The first feature point detection result includes multiple first feature points, and the second feature point detection result includes multiple second feature points.

[0050] S130. Match the first feature point and the second feature point to obtain feature point pairs;

[0051] S140. Based on the pixel positions of feature point pairs, determine the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image.

[0052] The feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and outputs the corresponding intermediate feature points based on the feature image of each layer of the feature pyramid. The fusion module is used to output the feature point detection results of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0053] The method provided by this invention acquires a first image and a second image that respectively reflect the state of the target object at different times, performs feature point detection on the first image and the second image respectively, and realizes deformation measurement through feature point matching. It does not rely on physical sensors. In the process of feature point detection, a multi-scale processing mechanism is used to perform feature point detection, which can improve the accuracy of feature point detection and reduce the cost of deformation monitoring while ensuring the accuracy of deformation monitoring.

[0054] The first and second images in the image pair to be detected are images of the target object obtained at different times. When imaging the target object, a drone equipped with a high-resolution camera can be used to capture images from multiple angles, ensuring coverage of key load-bearing parts and potential deformation areas. The drone's flight path and shooting angle are automatically controlled through preset parameters to obtain optimal image data. Alternatively, fixed image acquisition equipment can be used on-site to capture the first and second images of the target object. The acquisition cycle can be set according to the target object's usage and environmental factors, such as periodic monitoring or timely monitoring of its status after external influences (such as earthquakes, storms, etc.).

[0055] After imaging the target object to obtain the first and second images, preprocessing can be performed. Preprocessing may include grayscale conversion, denoising, and histogram equalization. Grayscale conversion reduces computational complexity and improves the speed of subsequent processing. Denoising can be achieved by applying techniques such as Gaussian filtering or median filtering to eliminate random noise in the image and improve image quality. Histogram equalization enhances image contrast, making feature points in the image more prominent and facilitating subsequent feature extraction.

[0056] After obtaining the image pair to be detected, feature point detection is performed on the first and second images in the image pair. Feature point detection is used to match pixels within the image pair, identifying pixels corresponding to the same actual point, thereby acquiring deformation data. Continuous image acquisition enables highly efficient data acquisition. Compared to traditional monitoring methods relying on physical sensors or manual inspection, the method provided by this invention significantly improves data acquisition efficiency, enabling timely detection and response to potential structural problems. Furthermore, it eliminates the need for expensive physical sensing equipment and its maintenance costs, reducing initial investment and long-term operating expenses. In addition, image-based deformation monitoring reduces the difficulty and cost of monitoring in inaccessible areas, especially in complex or extensive terrain.

[0057] Specifically, the multi-scale feature module includes a sampling layer and multiple detection network layers. The detection network layers include a feature extraction module and a feature point header. The first image and the second image are respectively input into the trained feature point detection model to obtain the first feature point detection result and the second feature point detection result output by the feature point detection model, including:

[0058] The first image and the second image are used as target images, and the target images are input into the sampling layer. Multiple upsampling and / or downsampling operations are performed on the target images to obtain images to be processed at multiple scales.

[0059] Each image to be processed is input into the feature extraction module in the corresponding detection network layer. The feature extraction module extracts features from the image to be processed to obtain the feature image of each layer of the feature pyramid.

[0060] The feature image is input into the feature point head of the corresponding detection network layer, and the intermediate feature points and the confidence of the intermediate feature points are obtained from the feature image output by the feature point head.

[0061] The intermediate feature points and their confidence levels corresponding to each feature image are input into the fusion module to obtain the feature point detection results of the target image output by the fusion module.

[0062] The method provided by this invention employs a multi-scale processing mechanism to work simultaneously on images of different resolutions, effectively extracting feature points at various scales and fusing the detected feature points at various scales. This can effectively improve the accuracy of the detected feature points, ensuring that the detected feature points are representative key points, thereby improving the accuracy of deformation monitoring.

[0063] For each image in the image pair to be detected, it can be used as a target image. The target images are input into the trained feature point detection model. First, a feature pyramid is constructed through the multi-scale feature module in the feature point detection model. The pyramid consists of multiple layers of feature images, each with a different resolution. Specifically, the multi-scale feature module includes multiple detection network layers. Each layer includes an image feature extraction module and a feature point head. The image feature extraction module is used to extract the feature image at the corresponding resolution of the layer, and the feature point head is used to perform feature point detection based on the feature image of the layer, outputting the intermediate feature points corresponding to the feature image of the layer.

[0064] By employing a multi-scale detection module, the model can capture image features at different resolutions. Lower layers capture coarse features, while higher layers capture detailed features, effectively enhancing the stability and robustness of feature points and significantly improving the accuracy of feature matching. This, in turn, improves the accuracy and reliability of deformation analysis. By capturing structural features at various scales, from micro to macro, a more comprehensive and detailed analysis of deformation is achieved, ensuring effective detection of everything from large-scale structures to minute cracks. Furthermore, multi-scale feature detection allows the method provided by this invention to adapt to monitoring needs under various environments and lighting conditions. Whether in low-light environments or with complex background noise, the method provided by this invention can effectively identify key feature points, ensuring the stability and accuracy of deformation monitoring.

[0065] In one possible implementation, the detection network layer can be built based on the superpoint network, which is a feature point detection network. It can be used as the detection network layer in the multi-scale detection module to realize the detection of intermediate feature points at various scales.

[0066] Furthermore, after outputting the intermediate feature points corresponding to each layer of feature images, the intermediate feature points are fused to obtain the feature point detection results of the target image, specifically including:

[0067] Perform the following operations using the fusion module:

[0068] Map intermediate feature points to the same scale;

[0069] The intermediate feature point with the highest confidence is selected as the feature point to be processed. The overlap between the intermediate feature point and the feature point to be processed in the neighborhood of the feature point to be processed is obtained. The overlap reflects the similarity between the intermediate feature point and the feature point to be processed.

[0070] Delete intermediate feature points with an overlap exceeding a preset threshold;

[0071] In the process of removing the feature point to be processed, the intermediate feature point with the highest confidence is selected as the new feature point to be processed. The steps of obtaining the overlap between the intermediate feature point in the neighborhood of the feature point to be processed and the feature point to be processed are repeated until all intermediate feature points are traversed.

[0072] The retained intermediate feature points are used as the feature point detection results of the target image.

[0073] As explained above, the multi-scale feature module outputs feature point detection results at multiple resolutions, mapping intermediate feature points at each resolution to the same scale. This means transforming the position coordinates of intermediate feature points at different resolutions to image coordinates at the same resolution. For example, if the coordinates of an intermediate feature point A detected in a feature image with resolution a*b are (x1, y1), transforming it to a 2a*2b image coordinate system will change the coordinates of intermediate feature point A accordingly, ensuring that the actual content corresponding to the position coordinates of the intermediate feature point is consistent across both scales.

[0074] For each intermediate feature point transformed to the same scale, the intermediate feature point with the highest confidence is first selected as the feature point to be processed. Then, intermediate feature points are searched within the neighborhood of the feature point to be processed. The neighborhood of the feature point to be processed refers to the domain composed of pixels whose pixel distance to the feature point to be processed is within a preset range (e.g., 4 pixels). For each intermediate feature point found in the neighborhood of the feature point to be processed, the overlap degree between them is calculated. The overlap degree reflects the similarity between the intermediate feature point and the feature point to be processed; the higher the overlap degree, the greater the probability that the intermediate feature point and the feature point to be processed are actually detecting the same location. The method provided by this invention, by using overlap degree to filter intermediate feature points within the neighborhood of intermediate feature points with high confidence, can effectively reduce feature point redundancy, optimize feature point representation, and improve the stability and robustness of feature points in deformation analysis.

[0075] Specifically, obtaining the overlap between the intermediate feature point in the neighborhood of the feature point to be processed and the feature point to be processed includes:

[0076] Obtain the feature descriptors of the intermediate feature points and the feature points to be processed, respectively;

[0077] The overlap between intermediate feature points and feature points to be processed is determined based on the feature descriptors of the intermediate feature points and the feature points to be processed.

[0078] The feature descriptors of intermediate feature points can be output along with the detection of intermediate feature points. These descriptors describe the intermediate feature points. In the method provided by this invention, if the similarity between the feature descriptors of an intermediate feature point and the feature point to be processed reaches a preset similarity threshold, then the overlap between the intermediate feature point and the feature point to be processed is determined to be higher than a preset threshold, and the intermediate feature point with an overlap higher than the preset threshold is deleted. Then, among the intermediate feature points removed from the current feature point to be processed, the intermediate feature point with the highest confidence is selected as the new feature point to be processed. The steps described above for finding and processing intermediate feature points in the neighborhood of the feature point to be processed are repeated until all intermediate feature points (excluding deleted intermediate feature points) are used as feature points to be processed. Finally, the remaining intermediate feature points are used as the feature point detection results of the target image.

[0079] Feature point detection models can be trained using multiple sets of training data, each set including sample images and the corresponding feature point detection results. The training data is derived from existing feature point annotation datasets. In one possible implementation, besides directly generating training data from the feature point annotation dataset, data augmentation can be performed on the image data in the dataset, such as randomly rotating and scaling the images to create new training data. This allows the trained feature point detection model to adapt to various environments and deformation types. During training, backpropagation and gradient descent algorithms can be used, cross-entropy loss function can be employed to optimize feature point detection accuracy, and regularization methods such as dropout and L2 regularization can be used to prevent overfitting.

[0080] After obtaining the feature point detection results for the first and second images, the first feature point in the first image is matched with the second feature point in the second image. Feature point matching can employ existing methods, such as Euclidean distance-based matching algorithms, descriptor-based methods (e.g., FLANN, Fast Library for Approximate Nearest Neighbors), or other effective matching algorithms. During the matching process, the RANSAC algorithm can be incorporated to eliminate false matches and improve matching accuracy.

[0081] The first and second feature points that are successfully matched are considered to be the same point on the corresponding target object. The change in the coordinate position between the first and second feature points can reflect the change in position of the same point on the target object at different times, i.e., deformation. Therefore, based on the spatial position change of the feature points after matching, the deformation data of the target object can be calculated using a geometric transformation model (such as affine transformation or perspective transformation).

[0082] In one possible implementation, when determining the deformation data of the target object, it can be calculated based on multiple pairs of images to be detected, which can enhance the accuracy of the deformation data through multi-view analysis.

[0083] Furthermore, after obtaining the deformation data of the target object, the deformation data can be compared with a preset deformation threshold. When the deformation exceeds the preset threshold, an alarm message can be generated to promptly notify maintenance personnel to conduct inspections or collect emergency measures.

[0084] After obtaining the deformation data of the target object, a composite image can be generated and displayed based on the deformation data and the image of the target object.

[0085] After obtaining the deformation data, it can be displayed through a graphical interface, providing intuitive deformation trend graphs and heatmaps. A composite image can be generated and displayed by combining the deformation data with an image of the target object, facilitating engineers to quickly identify potential risk areas within the target object. The deformation data can also be output in report form, including detailed deformation data, the location of the deformation area, and warning levels, for further structural safety assessment and maintenance decision-making.

[0086] Furthermore, the method provided by this invention can be integrated into a real-time monitoring system. This system continuously receives image data from the target object, generates continuous pairs of images to be detected, and achieves real-time deformation analysis by acquiring deformation data from these continuous pairs of images. This deformation monitoring system is easily deployed in various monitoring scenarios, including but not limited to bridges, high-rise buildings, and large machinery. The system's scalability ensures that it can adapt to different monitoring needs, from small buildings to large infrastructure projects.

[0087] The deformation monitoring device based on feature point detection provided by this invention will be described below. The deformation monitoring device described below can be referred to in correspondence with the deformation monitoring method based on feature point detection described above. Figure 2 As shown, the deformation monitoring device based on feature point detection provided by the present invention includes:

[0088] Image acquisition module 210 is used to acquire a pair of images to be detected corresponding to the target object. The pair of images to be detected includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times.

[0089] Feature point detection module 220 is used to input the first image and the second image into the trained feature point detection model respectively, and obtain the first feature point detection result and the second feature point detection result output by the feature point detection model. The first feature point detection result includes multiple first feature points, and the second feature point detection result includes multiple second feature points.

[0090] The feature point matching module 230 is used to match the first feature point and the second feature point to obtain feature point pairs;

[0091] The deformation analysis module 240 is used to determine the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image based on the pixel position of the feature point pair.

[0092] The feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and outputs the corresponding intermediate feature points based on the feature image of each layer of the feature pyramid. The fusion module is used to output the feature point detection results of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0093] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logic instructions in the memory 330 to execute a deformation monitoring method based on feature point detection. The deformation monitoring method based on feature point detection includes: acquiring a pair of images to be detected corresponding to the target object, the pair of images to be detected including a first image and a second image, the first image and the second image reflecting the state of the target object at different times; inputting the first image and the second image into a trained feature point detection model, acquiring the first feature point detection result and the second feature point detection result output by the feature point detection model, the first feature point detection result including multiple first feature points, and the second feature point detection result including multiple second feature points; matching the first feature points and the second feature points to obtain a feature point pair; determining the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image based on the pixel position of the feature point pair; wherein, the feature point detection model includes a multi-scale feature module and a fusion module, the multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature pyramid feature image, the fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0094] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the deformation monitoring method based on feature point detection provided above. The deformation monitoring method based on feature point detection includes: acquiring a pair of images to be detected corresponding to a target object, the pair of images to be detected including a first image and a second image, the first image and the second image respectively reflecting the state of the target object at different times; inputting the first image and the second image respectively into a trained feature point detection model, and acquiring the first feature point detection result and the second feature point detection result output by the feature point detection model. The detection results show that the first feature point detection result includes multiple first feature points, and the second feature point detection result includes multiple second feature points. The first and second feature points are matched to obtain feature point pairs. Based on the pixel positions of the feature point pairs, the deformation data of the target object at the time corresponding to the first image is determined relative to the time corresponding to the second image. The feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and outputs the corresponding intermediate feature points based on the feature image of each layer of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0096] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the aforementioned deformation monitoring method based on feature point detection. The deformation monitoring method based on feature point detection includes: acquiring a pair of images to be detected corresponding to a target object, the pair of images to be detected including a first image and a second image, the first image and the second image respectively reflecting the state of the target object at different times; inputting the first image and the second image respectively into a trained feature point detection model, and acquiring a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes... The detection results include multiple first feature points and multiple second feature points. The first and second feature points are matched to obtain feature point pairs. Based on the pixel positions of the feature point pairs, the deformation data of the target object at the time corresponding to the first image is determined relative to the time corresponding to the second image. The feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on the feature image of each layer of the feature pyramid. The fusion module is used to output the feature point detection results of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature point detection-based deformation monitoring method, characterized by, The method comprises: obtaining a target object corresponding to a to-be-detected image pair, the to-be-detected image pair comprising a first image and a second image, the first image and the second image reflecting states of the target object at different time instants respectively; inputting the first image and the second image into a trained feature point detection model respectively, obtaining first feature point detection results and second feature point detection results output by the feature point detection model, the first feature point detection results comprising a plurality of first feature points, and the second feature point detection results comprising a plurality of second feature points; matching the first feature points and the second feature points to obtain a feature point pair; based on a pixel position of the feature point pair, determining deformation data of the target object at a time instant corresponding to the first image relative to a time instant corresponding to the second image; wherein the feature point detection model comprises a multi-scale feature module and a fusion module, the multi-scale feature module is configured to construct a feature pyramid based on a target image input into the feature point detection model, and output corresponding intermediate feature points based on each layer feature image of the feature pyramid, and the fusion module is configured to output feature point detection results of the target image based on the intermediate feature points corresponding to each layer feature image. The multi-scale feature module comprises a sampling layer and a plurality of detection network layers, each detection network layer comprising a feature extraction module and a feature point head, the feature extraction module is configured to extract a feature image of a corresponding resolution of the layer, the feature point head is configured to extract the feature image of the layer for feature point detection, and output the intermediate feature points corresponding to the feature image of the layer; the step of inputting the first image and the second image into the trained feature point detection model respectively, and obtaining first feature point detection results and second feature point detection results output by the feature point detection model comprises: inputting the first image and the second image as the target image into the sampling layer, and performing multiple up-sampling and / or down-sampling operations on the target image to obtain a plurality of scale to-be-processed images; inputting each to-be-processed image into the feature extraction module in the corresponding detection network layer, extracting features of the to-be-processed image through the feature extraction module to obtain the feature image of each layer of the feature pyramid; inputting the feature image into the feature point head in the corresponding detection network layer, obtaining the intermediate feature points corresponding to the feature image and the confidence of the intermediate feature points output by the feature point head; inputting the intermediate feature points and the confidence of the intermediate feature points corresponding to each feature image into the fusion module to obtain the feature point detection results of the target image output by the fusion module; inputting the intermediate feature points and the confidence of the intermediate feature points corresponding to each feature image into the fusion module to obtain the feature point detection results of the target image output by the fusion module, comprising: performing the following operations through the fusion module: Map the intermediate feature points to the same scale; Select the intermediate feature point with the highest confidence as a to-be-processed feature point, and respectively acquire the overlap degree of the intermediate feature points in the neighborhood of the to-be-processed feature point and the to-be-processed feature point, the overlap degree reflecting the similarity of the intermediate feature points and the to-be-processed feature point; Delete the intermediate feature points with an overlap degree higher than a preset threshold; Select the intermediate feature point with the highest confidence from the intermediate feature points of the to-be-processed feature point as a new to-be-processed feature point, and repeatedly perform the step of respectively acquiring the overlap degree of the intermediate feature points in the neighborhood of the to-be-processed feature point and the to-be-processed feature point until all the intermediate feature points are traversed; The intermediate feature points remaining are taken as the feature point detection result of the target image.

2. The feature point detection-based deformation monitoring method according to claim 1, characterized in that, The step of acquiring the overlap degree of the intermediate feature points in the neighborhood of the to-be-processed feature point and the to-be-processed feature point comprises: Respectively acquire the feature descriptors of the intermediate feature points and the to-be-processed feature point; Determine the overlap degree of the intermediate feature points and the to-be-processed feature point based on the feature descriptors of the intermediate feature points and the to-be-processed feature point.

3. The feature point detection-based deformation monitoring method according to claim 1, wherein, The detection network layer is constructed based on a superpoint network. 4.The feature point detection based deformation monitoring method of claim 1, wherein, After the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image is determined, the method comprises: Generating a composite image based on the deformation data and the image of the target object; Displaying the composite image.

5. A deformation monitoring apparatus based on feature point detection, characterized by, The method comprises: An image acquisition module, configured to acquire a to-be-detected image pair corresponding to a target object, the to-be-detected image pair comprising a first image and a second image, the first image and the second image respectively reflecting the state of the target object at different times; A feature point detection module, configured to input the first image and the second image into a trained feature point detection model respectively, and acquire a first feature point detection result and a second feature point detection result output by the feature point detection model, the first feature point detection result comprising a plurality of first feature points, and the second feature point detection result comprising a plurality of second feature points; A feature point matching module, configured to match the first feature points and the second feature points to obtain a feature point pair; A deformation analysis module, configured to determine deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image based on the pixel positions of the feature point pair; The feature point detection model comprises a multi-scale feature module and a fusion module, the multi-scale feature module is configured to construct a feature pyramid based on a target image input into the feature point detection model, and output corresponding intermediate feature points based on each layer of feature images of the feature pyramid, and the fusion module is configured to output a feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature images. The multi-scale feature module comprises a sampling layer and a plurality of detection network layers, each of the detection network layers comprises a feature extraction module and a feature point head, the feature extraction module is configured to extract a feature image of a corresponding resolution of the layer, and the feature point head is configured to perform feature point detection on the feature image of the layer and output intermediate feature points corresponding to the feature image of the layer; the first image and the second image are input into a trained feature point detection model respectively, and first feature point detection results and second feature point detection results output by the feature point detection model are obtained, comprising: the first image and the second image are input into the sampling layer as target images, and a plurality of times of upsampling and / or downsampling operations are performed on the target images to obtain a plurality of scale to-be-processed images; each of the to-be-processed images is input into the feature extraction module in the corresponding detection network layer, and the feature extraction module is configured to extract features of the to-be-processed image to obtain the feature image of each layer of the feature pyramid; the feature image is input into the feature point head in the corresponding detection network layer, and the intermediate feature points corresponding to the feature image and the confidence of the intermediate feature points output by the feature point head are obtained; the intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points are input into the fusion module, and the feature point detection results of the target image output by the fusion module are obtained; the intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points are input into the fusion module, and the feature point detection results of the target image output by the fusion module are obtained, comprising: the fusion module is configured to perform the following operations: map the intermediate feature points to the same scale; select the intermediate feature point with the highest confidence as a to-be-processed feature point, and obtain the overlap degree of the intermediate feature point in the neighborhood of the to-be-processed feature point and the to-be-processed feature point, the overlap degree reflecting the similarity between the intermediate feature point and the to-be-processed feature point; delete the intermediate feature point with an overlap degree higher than a preset threshold; select the intermediate feature point with the highest confidence from the intermediate feature points of the to-be-processed feature point as a new to-be-processed feature point, and repeat the step of obtaining the overlap degree of the intermediate feature point in the neighborhood of the to-be-processed feature point and the to-be-processed feature point until all the intermediate feature points are traversed; retain the intermediate feature points as the feature point detection results of the target image.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the feature point detection-based deformation monitoring method according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the feature point detection-based deformation monitoring method according to any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the feature point detection-based deformation monitoring method according to any one of claims 1-4.

Images