Deformation monitoring method and device based on feature point detection, equipment, medium and product

Abstract

The invention provides a deformation monitoring method and device based on feature point detection, equipment, a medium and a product, and the method comprises the steps: obtaining a to-be-detected image pair corresponding to a target object, the to-be-detected image pair comprises a first image and a second image, and the first image and the second image respectively reflect the states of the target object at different moments; the first image and the second image are input into a trained feature point detection model, a first feature point detection result and a second feature point detection result output by the feature point detection model are obtained, the first feature point detection result comprises a plurality of first feature points, and the second feature point detection result comprises a plurality of second feature points; matching the first feature point and the second feature point to obtain a feature point pair; and determining deformation data of the target object at the moment corresponding to the first image relative to the moment corresponding to the second image based on the pixel positions of the feature point pairs. According to the invention, the deformation monitoring cost can be reduced on the basis of guaranteeing the deformation monitoring accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a deformation monitoring method, device, equipment, medium and product based on feature point detection. Background Art

[0002] Deformation monitoring is of great significance for maintaining and protecting public safety. It can help engineers and maintenance personnel to understand the health status of building structures in a timely manner and predict and prevent potential structural problems. For example, under the influence of long-term service and natural factors, monitoring the deformation of buildings can assess their safety and determine whether they need to be repaired or reinforced.

[0003] Traditional deformation monitoring technology usually relies on physical sensors such as strain gauges, inclinometers, displacement sensors, etc. However, high-precision sensors may be damaged or fail due to environmental influences (such as temperature changes, humidity, and chemical corrosion), and require regular inspection and replacement, and the installation and maintenance costs are very high. Summary of the invention

[0004] The present invention provides a deformation monitoring method, device, equipment, medium and product based on feature point detection, so as to solve the defect of high cost of deformation monitoring by physical sensors in the prior art and reduce the cost of deformation monitoring.

[0005] The present invention provides a deformation monitoring method based on feature point detection, comprising: Acquire a pair of images to be detected corresponding to the target object, wherein the pair of images to be detected includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; Inputting the first image and the second image into a trained feature point detection model respectively, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; Matching the first feature point and the second feature point to obtain a feature point pair; Determining deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on pixel positions of the feature point pairs; Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input into the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0006] According to the deformation monitoring method based on feature point detection provided by the present invention, the multi-scale feature module includes a sampling layer and a plurality of detection network layers, the detection network layer includes a feature extraction module and a feature point head; the first image and the second image are respectively input into the trained feature point detection model, and the first feature point detection result and the second feature point detection result output by the feature point detection model are obtained, including: The first image and the second image are respectively used as the target images, the target images are input into the sampling layer, and multiple upsampling and / or downsampling operations are performed on the target images to obtain images to be processed at multiple scales; Input each of the to-be-processed images into the feature extraction module in the corresponding detection network layer, and extract features of the to-be-processed images through the feature extraction module to obtain feature images of each layer of the feature pyramid; Input the feature image into the feature point head in the corresponding detection network layer, and obtain the intermediate feature points and the confidence of the intermediate feature points corresponding to the feature image output by the feature point head; The intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points are input into the fusion module to obtain the feature point detection result of the target image output by the fusion module.

[0007] According to the deformation monitoring method based on feature point detection provided by the present invention, the intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points are input into the fusion module, and the feature point detection result of the target image output by the fusion module is obtained, including: Through the fusion module, the following operations are performed: Mapping the intermediate feature points to the same scale; Select the intermediate feature point with the largest confidence as the feature point to be processed, and respectively obtain the overlap between the intermediate feature point and the feature point to be processed in the neighborhood of the feature point to be processed, wherein the overlap reflects the similarity between the intermediate feature point and the feature point to be processed; Deleting the intermediate feature points whose overlap degree is higher than a preset threshold; Selecting the intermediate feature point with the largest confidence from among the intermediate feature points after removing the feature point to be processed as a new feature point to be processed, and repeatedly performing the step of respectively obtaining the degree of overlap between the intermediate feature points in the neighborhood of the feature point to be processed and the feature point to be processed until all the intermediate feature points are traversed; The retained intermediate feature points are used as feature point detection results of the target image.

[0008] According to the deformation monitoring method based on feature point detection provided by the present invention, the step of obtaining the degree of overlap between the intermediate feature point in the neighborhood of the feature point to be processed and the feature point to be processed includes: Respectively obtaining feature descriptors of the intermediate feature points and the feature points to be processed; The degree of overlap between the intermediate feature point and the feature point to be processed is determined based on the feature descriptors of the intermediate feature point and the feature point to be processed.

[0009] According to the deformation monitoring method based on feature point detection provided by the present invention, the detection network layer is constructed based on the superpoint network.

[0010] According to the deformation monitoring method based on feature point detection provided by the present invention, after determining the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image, the method comprises: generating a composite image based on the deformation data and the image of the target object; The composite image is displayed.

[0011] The present invention also provides a deformation monitoring method based on feature point detection, comprising: An image acquisition module is used to acquire a pair of to-be-detected images corresponding to the target object, wherein the pair of to-be-detected images includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; A feature point detection module, used to input the first image and the second image into a trained feature point detection model respectively, and obtain a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; A feature point matching module, used for matching the first feature point with the second feature point to obtain a feature point pair; A deformation analysis module, used to determine deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on the pixel positions of the feature point pairs; Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input into the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a deformation monitoring method based on feature point detection as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the deformation monitoring method based on feature point detection as described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the deformation monitoring methods based on feature point detection as described above.

[0015] The present invention provides a deformation monitoring method, device, equipment, medium and product based on feature point detection. By acquiring a first image and a second image that respectively reflect the state of a target object at different times, feature point detection is performed on the first image and the second image respectively, and deformation measurement is achieved through feature point matching. It does not rely on physical sensors. In the process of feature point detection, feature point detection is performed through a multi-scale processing mechanism, which can improve the accuracy of feature point detection and reduce the cost of deformation monitoring on the basis of ensuring the accuracy of deformation monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of a deformation monitoring method based on feature point detection provided by the present invention.

[0018] Figure 2 It is a structural schematic diagram of the deformation monitoring device provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In the existing technology, deformation monitoring technology relies on physical sensors such as strain gauges, inclinometers, displacement sensors, etc. However, high-precision sensors may be damaged or fail due to environmental influences (such as temperature changes, humidity, chemical corrosion), and require regular inspection and replacement, and the installation and maintenance costs are very high.

[0022] In order to solve the defect of high cost of deformation monitoring using physical sensors in the prior art, the present invention provides a deformation monitoring method, device, equipment, medium and product based on feature point detection, so as to reduce the cost of deformation monitoring.

[0023] Combine the following Figure 1 The deformation monitoring method based on feature point detection provided by the present invention is described as follows: Figure 1 As shown, the present invention provides a deformation monitoring method based on feature point detection, comprising the steps of: S110, obtaining a pair of images to be detected corresponding to the target object, the pair of images to be detected comprising a first image and a second image, the first image and the second image respectively reflecting the state of the target object at different times; S120, inputting the first image and the second image into the trained feature point detection model respectively, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; S130, matching the first feature point and the second feature point to obtain a feature point pair; S140. Determine deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on the pixel positions of the feature point pairs.

[0024] Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of feature images in the feature pyramid. The fusion module is used to output the feature point detection results of the target image based on the intermediate feature points corresponding to each layer of feature images.

[0025] The method provided by the present invention acquires a first image and a second image that respectively reflect the state of the target object at different times, performs feature point detection on the first image and the second image respectively, and realizes deformation measurement through feature point matching. It does not rely on physical sensors. In the process of feature point detection, feature point detection is performed through a multi-scale processing mechanism, which can improve the accuracy of feature point detection and reduce the cost of deformation monitoring on the basis of ensuring the accuracy of deformation monitoring.

[0026] The first image and the second image in the image pair to be detected are images obtained by imaging the target object at different times. When imaging the target object, a drone equipped with a high-resolution camera can be used to shoot the target object from multiple angles to ensure that the key load-bearing parts and possible deformation areas of the target object are covered. The flight path and shooting angle of the drone are automatically controlled by a preset degree to obtain the best image data. In addition, image acquisition equipment can also be fixed on site to image and acquire the target object to obtain the first image and the second image. The acquisition cycle between them can be set according to the use of the target object and environmental factors, such as regular monitoring or timely monitoring of its status after external influences (such as earthquakes, storms, etc.).

[0027] After imaging the target object to obtain the first image and the second image, they can be preprocessed, which can include grayscale processing, denoising processing, histogram equalization processing, etc. Grayscale processing can reduce the computational complexity and improve the subsequent processing speed. Denoising processing can be achieved by applying Gaussian filtering or median filtering and other technologies to eliminate random noise in the image and improve image quality. Histogram equalization processing can enhance the contrast of the image, making the feature points in the image more obvious, which is convenient for subsequent feature extraction.

[0028] After obtaining the image pair to be detected, feature point detection is performed on the first image and the second image in the image pair to be detected respectively, and the pixel points in the image pair to be detected are matched by performing feature point detection on the image pair to be detected, that is, the pixel points corresponding to the same actual point are identified from the image pair to be detected, and then the deformation data is acquired. Highly efficient data acquisition can be achieved through continuous image acquisition. Compared with traditional monitoring methods that rely on physical sensors or manual inspections, the method provided by the present invention can greatly improve data acquisition efficiency, can timely discover and respond to potential structural problems, and does not require expensive physical sensing equipment and its maintenance costs, reducing initial investment and long-term operating costs. In addition, deformation monitoring that relies only on images can reduce the difficulty and cost of monitoring in difficult-to-access areas, especially in complex or vast terrains.

[0029] Specifically, the multi-scale feature module includes a sampling layer and multiple detection network layers, the detection network layer includes a feature extraction module and a feature point head; the first image and the second image are respectively input into the trained feature point detection model, and the first feature point detection result and the second feature point detection result output by the feature point detection model are obtained, including: The first image and the second image are respectively used as target images, the target images are input into the sampling layer, and multiple upsampling and / or downsampling operations are performed on the target images to obtain images to be processed at multiple scales; Each image to be processed is input into the feature extraction module in the corresponding detection network layer, and the feature extraction module is used to extract features of the image to be processed to obtain feature images of each layer of the feature pyramid; Input the feature image into the feature point head in the corresponding detection network layer, and obtain the intermediate feature points and the confidence of the intermediate feature points corresponding to the feature image output by the feature point head; The intermediate feature points corresponding to each feature image and the confidence of the intermediate feature points are input into the fusion module to obtain the feature point detection result of the target image output by the fusion module.

[0030] In the method provided by the present invention, a multi-scale processing mechanism is adopted to work simultaneously on images of different resolutions, effectively extract feature points of various scales, and fuse the feature points detected at various scales, which can effectively improve the accuracy of the detected feature points, ensure that the detected feature points are representative key points, and thus improve the accuracy of deformation monitoring.

[0031] Each image in the image pair to be detected can be used as a target image. The target image is input into the trained feature point detection model. First, a feature pyramid is constructed through the multi-scale feature module in the feature point detection model. The pyramid includes multiple layers of feature images, and the resolution of each layer of feature images is different. Specifically, the multi-scale feature module includes multiple detection network layers, each layer includes an image feature extraction module and a feature point head. The image feature extraction module is used to extract the feature image of the corresponding resolution of the layer, and the feature point head is used to perform feature point detection based on the feature image of the layer, and output the intermediate feature points corresponding to the feature image of the layer.

[0032] Through the multi-scale detection module, the model can capture image features at different resolutions, with the lower layer capturing coarse features and the higher layer capturing detailed features, effectively enhancing the stability and robustness of feature points, significantly improving the accuracy of feature matching, and thus improving the accuracy and reliability of deformation analysis. By capturing structural features of various scales from micro to macro, a more comprehensive and detailed analysis of deformation is performed, ensuring that everything from large-scale structures to tiny cracks can be effectively detected. In addition, multi-scale feature detection enables the method provided by the present invention to adapt to monitoring needs in various environments and under different lighting conditions. Whether it is a low-light environment or complex background noise, the method provided by the present invention can effectively identify key feature points and ensure the stability and accuracy of deformation monitoring.

[0033] In one possible implementation, the detection network layer can be built based on the superpoint network, which is a feature point detection network. The superpoint network can be used as the detection network layer in the multi-scale detection module to realize intermediate feature point detection at each scale.

[0034] Furthermore, after outputting the intermediate feature points corresponding to each layer of feature images, each intermediate feature point is fused to obtain the feature point detection result of the target image, which specifically includes: Through the fusion module, perform the following operations: Map the intermediate feature points to the same scale; Select the middle feature point with the largest confidence as the feature point to be processed, and obtain the overlap between the middle feature point and the feature point to be processed in the neighborhood of the feature point to be processed, where the overlap reflects the similarity between the middle feature point and the feature point to be processed; Delete the intermediate feature points whose overlap is higher than the preset threshold; Select the intermediate feature point with the largest confidence from the intermediate feature points after removing the feature point to be processed as the new feature point to be processed, and repeat the steps of obtaining the overlap between the intermediate feature points in the neighborhood of the feature point to be processed and the feature point to be processed, until all the intermediate feature points are traversed; The retained intermediate feature points are used as the feature point detection results of the target image.

[0035] As can be seen from the previous description, the multi-scale feature module outputs the feature point detection results at multiple resolutions, and maps the intermediate feature points at each resolution to the same scale, that is, converts the position coordinates of the intermediate feature points at each resolution to the image coordinates at the same resolution. For example, based on the feature image with a resolution of a*b, the coordinates of the intermediate feature point A are detected as (x1, y1). When it is converted to the image coordinate system with a resolution of 2a*2b, the coordinates of the intermediate feature point A will change accordingly, so that the actual content corresponding to the position coordinates of the intermediate feature point at two scales is consistent.

[0036] For each intermediate feature point converted to the same scale, first select the intermediate feature point with the largest confidence as the feature point to be processed, and search for the intermediate feature point in the neighborhood of the feature point to be processed. The neighborhood of the feature point to be processed refers to a domain composed of pixels within a preset range (e.g., 4 pixel distances) of the pixel distance from the feature point to be processed. The intermediate feature points found in the field of the feature point to be processed are respectively calculated for overlap with the feature point to be processed. The overlap reflects the similarity between the intermediate feature point and the feature point to be processed. The higher the overlap, the greater the possibility that the intermediate feature point and the feature point to be processed are actually detecting the same position point. The method provided by the present invention can effectively reduce the redundancy of feature points, optimize the representation of feature points, and improve the stability and robustness of feature points in deformation analysis by screening the intermediate feature points in the field of intermediate feature points with higher confidence using the overlap.

[0037] Specifically, obtaining the overlap between the intermediate feature point in the neighborhood of the feature point to be processed and the feature point to be processed includes: Obtain feature descriptors of intermediate feature points and feature points to be processed respectively; The overlap between the intermediate feature points and the feature points to be processed is determined based on the feature descriptors of the intermediate feature points and the feature points to be processed.

[0038] The feature descriptor of the intermediate feature point can be output together when detecting the intermediate feature point. The feature descriptor of the intermediate feature point describes the intermediate feature point. In the method provided by the present invention, if the similarity between the intermediate feature point and the feature descriptor of the feature point to be processed reaches a preset similarity threshold, then it is determined that the overlap between the intermediate feature point and the feature point to be processed is higher than the preset threshold, and the intermediate feature point with the overlap higher than the preset threshold is deleted. After that, the intermediate feature point with the highest confidence is continued to be selected as the new feature point to be processed among the intermediate feature points after removing the current feature point to be processed, and the steps of searching for and processing the intermediate feature points in the field of the feature point to be processed described above are repeated until all the intermediate feature points (excluding the deleted intermediate feature points) are used as feature points to be processed once. Finally, the intermediate feature points that have not been deleted are used as the feature point detection results of the target image.

[0039] The feature point detection model can be trained based on multiple sets of training data, each set of training data includes a sample image and the feature point detection results corresponding to the sample image. The training data is obtained based on an existing feature point annotation dataset. In one possible implementation, in addition to generating training data directly based on the feature point annotation dataset, the image data in the dataset can also be enhanced, such as randomly rotating and scaling the images in the dataset to form new training data. In this way, the trained feature point detection model can adapt to a variety of environments and deformation types. During the training process, the back propagation and gradient descent algorithms can be used for model training, the cross entropy loss function can be used to optimize the accuracy of feature point detection, and regularization methods such as dropout and L2 regularization can be used to prevent overfitting.

[0040] After obtaining the feature point detection results of the first image and the second image, the first feature point in the first image is matched with the second feature point in the second image. Feature point matching can adopt existing feature point matching methods, such as using a matching algorithm based on Euclidean distance, a descriptor-based method (such as FLANN, Fast Library for Approximate Nearest Neighbors) or other effective matching algorithms. In the matching process, the RANSAC algorithm can be combined to eliminate false matches and improve the accuracy of matching.

[0041] The first feature point and the second feature point that are successfully matched are considered to be the same point on the corresponding target object. The change in the coordinate position between the first feature point and the second feature point can reflect the change in position of the same point on the target object at different times, that is, deformation. Therefore, according to the change in the spatial position of the feature points after matching, the deformation data of the target object can be calculated using a geometric change model (such as affine transformation or perspective transformation).

[0042] In a possible implementation, when the deformation data of the target object is determined, it can be calculated based on a plurality of pairs of images to be detected, so that the accuracy of the deformation data can be enhanced through multi-view analysis.

[0043] Furthermore, after the deformation data of the target object is obtained, the deformation data can be compared with a preset deformation threshold. When the preset deformation threshold is exceeded, an alarm message can be generated to promptly notify maintenance personnel to conduct inspections or take emergency measures.

[0044] After the deformation data of the target object is obtained, a composite image may be generated based on the deformation data and the image of the target object and displayed.

[0045] After the deformation data is obtained, it can be displayed through a graphical interface, providing intuitive deformation trend graphs and thermal maps, and generating and displaying a composite image of the deformation data and the image of the target object, so that engineers can quickly identify potential risk areas in the target object. The deformation data can also be output in the form of a report, including detailed data of the deformation amount, the location of the deformation area, the warning level and other information, for further structural safety assessment and maintenance decision-making.

[0046] Furthermore, the method provided by the present invention can be integrated into a real-time monitoring system, which continuously receives image data from the target object and generates continuous image pairs to be detected, thereby achieving real-time deformation analysis by acquiring deformation data of continuous image pairs to be detected. The deformation monitoring system is easy to deploy in a variety of monitoring scenarios, including but not limited to bridges, high-rise buildings, large machinery, etc. The scalability of the system ensures that it can adapt to different monitoring needs from small buildings to large infrastructure projects.

[0047] The following is a description of the deformation monitoring device based on feature point detection provided by the present invention. The deformation monitoring device described below and the deformation monitoring method based on feature point detection described above can be referred to in correspondence with each other. Figure 2 As shown, the deformation monitoring device based on feature point detection provided by the present invention includes: An image acquisition module 210 is used to acquire a pair of to-be-detected images corresponding to the target object, wherein the pair of to-be-detected images includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; A feature point detection module 220 is used to input the first image and the second image into a trained feature point detection model respectively, and obtain a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; A feature point matching module 230 is used to match the first feature point and the second feature point to obtain a feature point pair; A deformation analysis module 240, for determining deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on the pixel positions of the feature point pairs; Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of feature images in the feature pyramid. The fusion module is used to output the feature point detection results of the target image based on the intermediate feature points corresponding to each layer of feature images.

[0048] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor (processor) 310, a communication interface (Communications Interface) 320, a memory (memory) 330 and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call the logic instructions in the memory 330 to execute the deformation monitoring method based on feature point detection, and the deformation monitoring method based on feature point detection includes: obtaining a pair of images to be detected corresponding to the target object, the pair of images to be detected includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; inputting the first image and the second image into the trained feature point detection model respectively, and obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, the first feature point detection result includes multiple first feature points, and the second feature point detection result includes multiple second feature points; matching the first feature point and the second feature point to obtain a feature point pair; based on the pixel position of the feature point pair, determining the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image; wherein the feature point detection model includes a multi-scale feature module and a fusion module, the multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid, and the fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0049] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0050] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the deformation monitoring method based on feature point detection provided above. The deformation monitoring method based on feature point detection includes: obtaining a pair of images to be detected corresponding to the target object, the pair of images to be detected includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; inputting the first image and the second image into a trained feature point detection model respectively, and obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model. The detection result of the first feature point includes a plurality of first feature points, and the detection result of the second feature point includes a plurality of second feature points; the first feature point and the second feature point are matched to obtain a feature point pair; based on the pixel position of the feature point pair, the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image is determined; wherein the feature point detection model includes a multi-scale feature module and a fusion module, the multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid, and the fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0051] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the deformation monitoring method based on feature point detection provided above, the deformation monitoring method based on feature point detection comprising: obtaining a pair of images to be detected corresponding to the target object, the pair of images to be detected comprising a first image and a second image, the first image and the second image respectively reflecting the state of the target object at different times; inputting the first image and the second image respectively into a trained feature point detection model, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, the first feature point detection result comprising The method comprises the following steps: the first feature point detection model comprises a plurality of first feature points, and the second feature point detection result comprises a plurality of second feature points; the first feature points and the second feature points are matched to obtain a feature point pair; based on the pixel position of the feature point pair, the deformation data of the target object at the time corresponding to the first image relative to the time corresponding to the second image is determined; wherein the feature point detection model comprises a multi-scale feature module and a fusion module, the multi-scale feature module is used to construct a feature pyramid based on the target image input to the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid, and the fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

[0052] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0053] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deformation monitoring method based on feature point detection, characterized in that: include: Acquire a pair of images to be detected corresponding to the target object, wherein the pair of images to be detected includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; Inputting the first image and the second image into a trained feature point detection model respectively, obtaining a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; Matching the first feature point and the second feature point to obtain a feature point pair; Determining deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on pixel positions of the feature point pairs; Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input into the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

2. The deformation monitoring method based on feature point detection according to claim 1 is characterized in that: The multi-scale feature module includes a sampling layer and a plurality of detection network layers, the detection network layer includes a feature extraction module and a feature point head; the first image and the second image are respectively input into a trained feature point detection model, and a first feature point detection result and a second feature point detection result output by the feature point detection model are obtained, including: The first image and the second image are respectively used as the target images, the target images are input into the sampling layer, and multiple upsampling and / or downsampling operations are performed on the target images to obtain images to be processed at multiple scales; Input each of the to-be-processed images into the feature extraction module in the corresponding detection network layer, and extract features of the to-be-processed images through the feature extraction module to obtain feature images of each layer of the feature pyramid; Input the feature image into the feature point head in the corresponding detection network layer, and obtain the intermediate feature points and the confidence of the intermediate feature points corresponding to the feature image output by the feature point head; The intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points are input into the fusion module to obtain the feature point detection result of the target image output by the fusion module.

3. The deformation monitoring method based on feature point detection according to claim 2 is characterized in that: Inputting the intermediate feature points corresponding to each of the feature images and the confidence of the intermediate feature points into the fusion module, and obtaining the feature point detection result of the target image output by the fusion module, including: Through the fusion module, the following operations are performed: Mapping the intermediate feature points to the same scale; Select the intermediate feature point with the largest confidence as the feature point to be processed, and respectively obtain the overlap between the intermediate feature point and the feature point to be processed in the neighborhood of the feature point to be processed, wherein the overlap reflects the similarity between the intermediate feature point and the feature point to be processed; Deleting the intermediate feature points whose overlap degree is higher than a preset threshold; Selecting the intermediate feature point with the largest confidence from among the intermediate feature points after removing the feature point to be processed as a new feature point to be processed, and repeatedly performing the step of respectively obtaining the degree of overlap between the intermediate feature points in the neighborhood of the feature point to be processed and the feature point to be processed until all the intermediate feature points are traversed; The retained intermediate feature points are used as feature point detection results of the target image.

4. The deformation monitoring method based on feature point detection according to claim 3 is characterized in that: The obtaining the degree of overlap between the intermediate feature point in the neighborhood of the feature point to be processed and the feature point to be processed includes: Respectively obtaining feature descriptors of the intermediate feature points and the feature points to be processed; The degree of overlap between the intermediate feature point and the feature point to be processed is determined based on the feature descriptors of the intermediate feature point and the feature point to be processed.

5. The deformation monitoring method based on feature point detection according to claim 2 is characterized in that: The detection network layer is built based on the superpoint network.

6. The deformation monitoring method based on feature point detection according to claim 1, characterized in that: After determining the deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image, the method further comprises: generating a composite image based on the deformation data and the image of the target object; The composite image is displayed.

7. A deformation monitoring device based on feature point detection, characterized in that: include: An image acquisition module is used to acquire a pair of to-be-detected images corresponding to the target object, wherein the pair of to-be-detected images includes a first image and a second image, and the first image and the second image respectively reflect the state of the target object at different times; A feature point detection module, used to input the first image and the second image into a trained feature point detection model respectively, and obtain a first feature point detection result and a second feature point detection result output by the feature point detection model, wherein the first feature point detection result includes a plurality of first feature points, and the second feature point detection result includes a plurality of second feature points; A feature point matching module, used for matching the first feature point with the second feature point to obtain a feature point pair; A deformation analysis module, used to determine deformation data of the target object at a time corresponding to the first image relative to a time corresponding to the second image based on the pixel positions of the feature point pairs; Among them, the feature point detection model includes a multi-scale feature module and a fusion module. The multi-scale feature module is used to construct a feature pyramid based on the target image input into the feature point detection model, and output the corresponding intermediate feature points based on each layer of the feature image of the feature pyramid. The fusion module is used to output the feature point detection result of the target image based on the intermediate feature points corresponding to each layer of the feature image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the deformation monitoring method based on feature point detection as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deformation monitoring method based on feature point detection as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the deformation monitoring method based on feature point detection as described in any one of claims 1 to 6 is implemented.

Images