Infrastructure field domain deformation monitoring method and device, electronic equipment and storage medium

Abstract

The invention provides an infrastructure field deformation monitoring method and device, electronic equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: determining a current measuring point deformation monitoring area to be subjected to deformation monitoring from a plurality of deformation monitoring areas of a facility field to be monitored; measuring point coordinate conversion relation information corresponding to the measuring point deformation monitoring area is determined from the coordinate conversion relation information set of the camera system; controlling a camera system to rotate based on the measuring point coordinate conversion relation information so as to monitor the measuring point deformation monitoring area; acquiring a monitoring image acquired by an image acquisition device on the monitoring point deformation monitoring area; and performing deformation monitoring on the measuring point deformation monitoring area based on the monitoring image to obtain a deformation monitoring result of the measuring point deformation monitoring area. According to the invention, one camera system can monitor a plurality of deformation monitoring areas, so that there is no need to deploy a plurality of camera systems, and the hardware cost of infrastructure field domain deformation monitoring is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an infrastructure field deformation monitoring method, device, electronic equipment and storage medium. Background Art

[0002] Under the influence of load, environment and other conditions, the components of infrastructure are aging and their properties are deteriorating, which brings safety risks. Therefore, it is necessary to monitor the infrastructure regularly or over a long period of time. In the traditional way, infrastructure monitoring is carried out through contact sensing of accelerometers and strain gauges. However, this method has discrete measuring points, sparse data, and requires the installation of multiple points. The installation and operation and maintenance costs are high, and it is difficult to optimize the installation position of the measuring points. In addition, the accelerometer needs to obtain the deformation through secondary integration, which has cumulative errors, resulting in insufficient accuracy of the infrastructure field monitoring results.

[0003] At present, infrastructure field monitoring is carried out through computer vision. Specifically, one monitoring camera monitors one monitoring area. However, most infrastructures are large facilities. If one monitoring camera can only monitor a single monitoring area, it will lead to the need to deploy multiple monitoring cameras to monitor the infrastructure field, which will increase the hardware cost. Summary of the invention

[0004] The present invention provides an infrastructure field deformation monitoring method, device, electronic equipment and storage medium, which are used to solve the defect of high cost of infrastructure field monitoring hardware in the prior art and realize low-cost infrastructure field deformation monitoring.

[0005] The present invention provides an infrastructure field deformation monitoring method, comprising: Determine a deformation monitoring area of ​​a measuring point to be currently monitored from a plurality of deformation monitoring areas in the facility field to be monitored; Determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system includes an image acquisition device, the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area; Based on the measurement point coordinate transformation relationship information, controlling the camera system to rotate to monitor the measurement point deformation monitoring area; Acquire a monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point; Based on the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point; Return to the step of determining the deformation monitoring area of ​​the measuring point to be currently deformation monitored from the multiple deformation monitoring areas of the facility field to be monitored, until deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0006] According to a method for monitoring deformation of an infrastructure field provided by the present invention, the deformation monitoring of the deformation monitoring area of ​​the measuring point is performed based on the monitoring image to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, including: Acquire a template image set of the deformation monitoring area of ​​the measuring point, wherein the template image set includes a plurality of template images of different brightness; Based on the brightness of the monitoring image, determining at least one measuring point template image from the template image set; Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point.

[0007] According to a method for monitoring deformation of an infrastructure field provided by the present invention, based on the template image of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, including: Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result in the space of the image acquisition device; Based on the deformation monitoring result in the image acquisition device space and the first mapping relationship, the deformation monitoring result of the measuring point deformation monitoring area is determined; the first mapping relationship is the size mapping relationship between the image acquisition device space and the actual world space; the deformation monitoring result of the measuring point deformation monitoring area is the actual physical deformation monitoring result.

[0008] According to a method for monitoring deformation of an infrastructure field provided by the present invention, the first mapping relationship is determined based on the following method: Determining the first mapping relationship corresponding to the deformation monitoring area of ​​the measuring point from a mapping relationship set of the camera system; the mapping relationship set includes mapping relationships corresponding to the multiple deformation monitoring areas; The first mapping relationship is determined by rotating the camera system to face the electronic scale corresponding to the deformation monitoring area of ​​the measuring point.

[0009] According to a method for monitoring deformation of infrastructure field provided by the present invention, the camera system further comprises a point cloud data acquisition device, and the relative position between the point cloud data acquisition device and the image acquisition device remains unchanged; The step of determining the deformation monitoring result of the deformation monitoring area of ​​the measuring point based on the deformation monitoring result in the image acquisition device space and the first mapping relationship includes: Acquire point cloud data of the deformation monitoring area of ​​the measuring point at the current moment, wherein the point cloud data is acquired by the point cloud data acquisition device; Based on the point cloud data, determining distance data between the camera system and the deformation monitoring area of ​​the measuring point, wherein the distance data includes a plurality of sub-distances, and any of the sub-distances is a distance between the camera system and any sub-area in the deformation monitoring area of ​​the measuring point; Based on the distance data and the first mapping relationship, determine a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point, any of the second mapping relationships being a size mapping relationship between the image acquisition device space and the actual world space; Based on the deformation monitoring result in the image acquisition device space and each of the second mapping relationships, the deformation monitoring result of the deformation monitoring area of ​​the measuring point is determined.

[0010] According to a method for monitoring deformation of an infrastructure field provided by the present invention, determining the distance data between the camera system and the deformation monitoring area of ​​the measuring point based on the point cloud data includes: Acquire the position and posture information of the image acquisition device at the current moment, wherein the position and posture information represents the position and posture of the image acquisition device in the three-dimensional model of the monitoring scene of the facility to be monitored; Based on the positional relationship information between the image acquisition device and the point cloud data acquisition device, the posture information and the point cloud data, the distance data between the camera system and the deformation monitoring area of ​​the measuring point is determined.

[0011] According to a method for monitoring deformation of an infrastructure field provided by the present invention, the step of returning to the step of determining the deformation monitoring area of ​​the current measuring point to be monitored from the multiple deformation monitoring areas of the facility field to be monitored until the deformation monitoring results of the multiple deformation monitoring areas are obtained, further comprising: Based on the deformation monitoring results of the multiple deformation monitoring areas, the three-dimensional model of the monitoring scene is updated.

[0012] According to a method for monitoring deformation of an infrastructure field provided by the present invention, the method of determining a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point based on the distance data and the first mapping relationship comprises: Perform interpolation fitting based on the distance data to obtain a scale change relationship function of the deformation monitoring area of ​​the measuring point; The first mapping relationship is scale-changed based on the scale-change relationship function to obtain a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point.

[0013] According to a method for monitoring deformation of an infrastructure field provided by the present invention, the step of acquiring the monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point further comprises: Determining the focal length of the measuring point corresponding to the deformation monitoring area of ​​the measuring point from the focal length information set of the camera system; the focal length information set includes the focal lengths corresponding to the multiple deformation monitoring areas; The focal length of the image acquisition device is adjusted to the focal length of the measuring point.

[0014] According to an infrastructure field deformation monitoring method provided by the present invention, when the coordinate position of the camera system is the second coordinate position, the measurement point deformation monitoring area is located at the center of the field of view of the camera system.

[0015] The present invention also provides an infrastructure field deformation monitoring device, comprising: an area determination module, an information determination module, a system control module, an image acquisition module, a deformation monitoring module and a step return module.

[0016] The area determination module is used to determine the deformation monitoring area of ​​the measuring point to be currently monitored from multiple deformation monitoring areas in the facility field to be monitored.

[0017] An information determination module is used to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system includes an image acquisition device, the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area.

[0018] The system control module is used to control the rotation of the camera system based on the measurement point coordinate transformation relationship information to monitor the measurement point deformation monitoring area.

[0019] The image acquisition module is used to acquire the monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point.

[0020] The deformation monitoring module is used to perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point.

[0021] The step return module is used to return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0022] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for monitoring deformation of an infrastructure field as described above is implemented.

[0023] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for monitoring deformation of an infrastructure field as described in any one of the above is implemented.

[0024] The infrastructure field deformation monitoring method, device, electronic device and storage medium provided by the present invention determine the current measuring point deformation monitoring area to be monitored from multiple deformation monitoring areas in the facility field to be monitored, so as to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system, and the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from the first coordinate position to the second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area, so as to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system, and the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, and the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from the first coordinate position to the second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area, so as to determine the measuring point coordinate transformation relationship information based on the measuring point coordinate transformation relationship information , control the camera system to rotate to monitor the deformation monitoring area of ​​the measuring point, obtain the monitoring image collected by the image acquisition device on the deformation monitoring area of ​​the measuring point, perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image, obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, and return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of multiple deformation monitoring areas are obtained, thereby realizing multi-field inspection and monitoring by one camera system, that is, deformation monitoring of the infrastructure field can be completed using one camera system. In other words, one camera system can monitor multiple deformation monitoring areas, so there is no need to deploy multiple camera systems, thereby reducing the hardware cost of deformation monitoring of the infrastructure field. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is one of the flow diagrams of the infrastructure field deformation monitoring method provided by the present invention.

[0027] Figure 2 This is the second flow chart of the infrastructure field deformation monitoring method provided by the present invention.

[0028] Figure 3 It is a structural schematic diagram of the infrastructure field deformation monitoring device provided by the present invention.

[0029] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Under the influence of load, environment and other conditions, the components of infrastructure are aging and their properties are deteriorating, which brings safety risks. Therefore, it is necessary to monitor the infrastructure regularly or over a long period of time. In the traditional way, infrastructure monitoring is carried out through contact sensing of accelerometers and strain gauges. However, this method has discrete measuring points, sparse data, and requires the installation of multiple points. The installation and operation and maintenance costs are high, and it is difficult to optimize the installation position of the measuring points. In addition, the accelerometer needs to obtain the deformation through secondary integration, which has cumulative errors, resulting in insufficient accuracy of the infrastructure field monitoring results.

[0032] At present, infrastructure field monitoring is performed through computer vision. Specifically, one monitoring camera monitors one monitoring area, that is, one monitoring camera can only monitor a single field of view. However, most infrastructures are large facilities. If one monitoring camera can only monitor a single monitoring area, it will require the deployment of multiple monitoring cameras to monitor the infrastructure field, which will increase hardware costs.

[0033] In view of the above problems, the present invention proposes the following embodiments. Figure 1-Figure 2 The infrastructure field deformation monitoring method of the present invention is described.

[0034] Figure 1 is one of the flow diagrams of the infrastructure field deformation monitoring method provided by the present invention, such as Figure 1 As shown, the infrastructure field deformation monitoring method includes the following steps 110, 120, 130, 140, 150 and 160.

[0035] Step 110: determine the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored.

[0036] Here, the facility field to be monitored is an infrastructure field to be monitored for deformation, and is usually a large infrastructure field.

[0037] Here, the multiple deformation monitoring areas (monitoring points) are monitoring areas that are divided in advance for the monitoring facility site.

[0038] In some embodiments, multiple components to be monitored included in the facility to be monitored are determined, and multiple deformation monitoring areas are determined based on the multiple components to be monitored, wherein one component to be monitored can be divided into one or more deformation monitoring areas according to size.

[0039] In one embodiment, a number of deformation monitoring areas corresponding to any component to be monitored are determined based on the following method: three-dimensionally reconstructing the monitoring scene of the monitoring facility site to obtain a three-dimensional model of the monitoring scene; determining a three-dimensional component model of the component to be monitored based on the three-dimensional model of the monitoring scene; and segmenting the three-dimensional construction model to obtain a number of deformation monitoring areas of the component to be monitored.

[0040] Among them, the monitoring scene is the scene where the facility to be monitored is located.

[0041] In one embodiment, a monitoring scene is scanned at multiple points at preset intervals by a three-dimensional reconstruction device to obtain multiple scan data; three-dimensional reconstruction is performed based on the multiple scan data to obtain a three-dimensional model of the monitoring scene. The three-dimensional reconstruction device may include, but is not limited to: real-scene digital twin devices such as faro three-dimensional scanners (including panoramic cameras and lasers). Exemplarily, multiple points in the monitoring scene are selected, the real-scene digital twin device is powered on, scanning and data recording are performed at certain intervals, and multiple scan data are saved. Furthermore, the scan data includes RGB information and point cloud information.

[0042] Furthermore, the plurality of scan data are preprocessed, and then three-dimensional reconstruction is performed based on the preprocessed plurality of scan data. The preprocessing method may include but is not limited to at least one of the following: denoising, streamlining, segmentation, splicing, multi-point site cloud registration, adaptive filtering, and the like.

[0043] Furthermore, the monitoring scene three-dimensional model is post-processed to obtain a post-processed monitoring scene three-dimensional model for subsequent processing based on the post-processed monitoring scene three-dimensional model. The post-processing method may include but is not limited to at least one of the following: three-dimensional model smoothing processing, three-dimensional model optimization processing, RGB texture mapping and matching processing, etc. The RGB texture mapping and matching processing can obtain a monitoring scene three-dimensional model with texture, thereby further improving the characterization accuracy of the monitoring scene three-dimensional model.

[0044] It should be noted that the embodiment of the present invention can realize a camera system to perform multi-field inspection and monitoring. Therefore, the deformation monitoring area of ​​the measuring point to be currently monitored is first determined from the multiple deformation monitoring areas of the facility field to be monitored. After the deformation monitoring of the deformation monitoring area of ​​the measuring point is performed to obtain the deformation monitoring result, return to step 110 until the deformation monitoring results of multiple deformation monitoring areas are obtained.

[0045] Step 120: Determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system.

[0046] Wherein, the camera system includes an image acquisition device, the coordinate transformation relationship information set includes coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area.

[0047] Here, the camera system needs to be installed in advance in the monitoring scene of the facility to be monitored, and the camera system needs to be connected to a power source. Further, the camera system can be connected to the execution subject of the method provided in the embodiment of the present invention in a communication manner, for example, the execution subject is a server, and the camera system is connected to the server in a communication manner.

[0048] The image acquisition device is used to acquire monitoring images, for example, it is a camera, a video camera, etc.

[0049] Here, the coordinate transformation relationship information set includes coordinate transformation relationship information corresponding to each deformation monitoring area. The coordinate transformation relationship information set is set in advance.

[0050] In a specific embodiment, a monitoring order of multiple deformation monitoring areas is determined; based on the monitoring order, the coordinate position of the camera system when the camera system monitors the first deformation monitoring area and the coordinate position of the camera system when the camera system monitors the second deformation monitoring area are determined, and then the coordinate transformation relationship information corresponding to the second deformation monitoring area is determined, and so on, the coordinate transformation relationship information corresponding to the subsequent deformation monitoring areas is determined, wherein the first position in the coordinate transformation relationship information corresponding to the first deformation monitoring area can be the initial coordinate position of the camera system.

[0051] In other words, the monitoring camera system rotates from the initial position to the coordinate transformation relationship information of the first deformation monitoring area, and the monitoring camera system rotates from the coordinate position of the camera system when monitoring the first deformation monitoring area to the coordinate transformation relationship information of the second deformation monitoring area, and so on, to determine the coordinate transformation relationship information corresponding to the subsequent deformation monitoring areas.

[0052] Furthermore, when the monitoring camera system rotates, ensure that the rotated camera system faces the electronic ruler, and / or ensure that the rotated camera system places the deformation monitoring area at the center of the field of view, so as to ensure that when the coordinate position of the camera system is the second coordinate position, the deformation monitoring area of ​​the measuring point is located at the center of the field of view of the camera system.

[0053] Step 130: Based on the measurement point coordinate transformation relationship information, control the camera system to rotate to monitor the measurement point deformation monitoring area.

[0054] It should be noted that the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from the first coordinate position to the second coordinate position, and the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area. Based on this, based on the measuring point coordinate transformation relationship information, the camera system can be controlled to rotate to the coordinate position of the deformation monitoring area of ​​the monitoring measuring point to monitor the deformation monitoring area of ​​the measuring point.

[0055] Furthermore, the point cloud data of the deformation monitoring area of ​​the measuring point at the current moment is obtained, so that after controlling the rotation of the camera system based on the coordinate conversion relationship information of the measuring point, the rotation of the camera system is controlled based on the point cloud data to ensure that the camera system looks directly at the deformation monitoring area of ​​the measuring point, thereby improving the accuracy of the deformation monitoring result. It should be understood that the deformation monitoring area of ​​the measuring point may be deformed, so the rotation of the camera system is controlled based on the point cloud data, that is, the motion navigation of the camera system is performed based on the point cloud data, so that even if the deformation monitoring area of ​​the measuring point is deformed, the camera system can be ensured to look directly at the deformation monitoring area of ​​the measuring point.

[0056] Step 140: Acquire a monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point.

[0057] It should be noted that after the camera system is controlled to rotate to the coordinate position of the deformation monitoring area of ​​the monitoring point, the image acquisition device will acquire the monitoring image of the deformation monitoring area of ​​the monitoring point.

[0058] Furthermore, before the above step 140, the brightness of the deformation monitoring area of ​​the measuring point is determined, and based on the brightness, the fill light device included in the camera system is controlled to perform fill light. Specifically, it is possible to control whether the fill light device performs fill light, or to control the fill light intensity of the fill light device. For example, when the light is dim before sunrise and after sunset, the fill light device can be controlled to perform fill light so that the deformation monitoring area of ​​the measuring point reaches the brightness of the selected template image. Based on this, the image acquisition device can acquire clear and accurate monitoring images, thereby improving the accuracy of deformation monitoring of the infrastructure field. Furthermore, it can be ensured that the brightness of the monitoring image is similar to the brightness of the template image.

[0059] Step 150: Based on the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point.

[0060] In some embodiments, a measuring point template image of a deformation monitoring area of ​​a measuring point is obtained, and deformation monitoring is performed on the deformation monitoring area of ​​the measuring point based on the measuring point template image and the monitoring image to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point. The measuring point template image is an image previously obtained by performing image acquisition on the deformation monitoring area of ​​the measuring point, and the deformation monitoring result obtained is the deformation monitoring result from before to now. More specifically, based on the comparison result between the measuring point template image and the monitoring image, the deformation monitoring result of the deformation monitoring area of ​​the measuring point is determined. Exemplarily, according to a preset shape function and correlation criterion, an integer pixel feature search is performed on the measuring point template image and the monitoring image, and sub-pixel matching is performed to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, that is, deformation monitoring is performed by a 2D-DIC (Digital Image Correlation) visual algorithm.

[0061] In one embodiment, at least one measuring point template image of the measuring point deformation monitoring area is obtained, and deformation monitoring is performed on the measuring point deformation monitoring area based on each measuring point template image and the monitoring image to obtain a deformation monitoring result of the measuring point deformation monitoring area. More specifically, each sub-deformation monitoring result of the measuring point deformation monitoring area is determined based on the comparison result between each measuring point template image and the monitoring image; and the deformation monitoring result of the measuring point deformation monitoring area is comprehensively determined based on each sub-deformation monitoring result. Based on this, the number of measuring point template images can be multiple, so that multiple sub-deformation monitoring results can be obtained, thereby improving the accuracy of determining the deformation monitoring result of the measuring point deformation monitoring area.

[0062] In one embodiment, considering the influence of brightness on deformation monitoring, based on this, a template image set of the deformation monitoring area of ​​the measuring point is obtained, and the template image set includes multiple template images of different brightness; based on the brightness of the monitoring image, the template image of the measuring point corresponding to the brightness is determined from the template image set; based on the template image of the measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, thereby improving the accuracy of the determination of the deformation monitoring result. Among them, the template image set includes multiple template images corresponding to brightness, so based on the brightness of the monitoring image, the template image of the measuring point closest to the brightness is determined from the template image set.

[0063] Exemplarily, the template image set can be acquired by the following method: acquiring preset template images at equal time intervals within a preset time period to obtain the template image set. For example, 10 clear images are acquired at equal time intervals between sunrise and sunset in each monitoring area as template images.

[0064] Furthermore, the template image set may be updated every preset time period, for example, once a month. During each update, a monitoring image of the monitoring area is recaptured as a template image.

[0065] In one embodiment, based on the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain the deformation monitoring result in the space of the image acquisition device; based on the deformation monitoring result in the space of the image acquisition device and the first mapping relationship, the deformation monitoring result of the deformation monitoring area of ​​the measuring point is determined. Among them, the first mapping relationship is the size mapping relationship between the space of the image acquisition device and the actual world space; the deformation monitoring result of the deformation monitoring area of ​​the measuring point is the physical actual deformation monitoring result. Based on this, the deformation monitoring result is converted into the physical actual deformation monitoring result, thereby improving the accuracy of deformation monitoring of the infrastructure field.

[0066] Step 160 , returning to the step of determining the deformation monitoring area of ​​the measuring point to be currently deformation monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0067] It should be noted that after obtaining the deformation monitoring result of the deformation monitoring area of ​​the measuring point, return to the above step 110 to perform deformation monitoring on the next deformation monitoring area until all deformation monitoring areas are monitored, thereby realizing multi-field inspection monitoring by one camera system.

[0068] Furthermore, based on the deformation monitoring results of multiple deformation monitoring areas, the three-dimensional model of the monitoring scene can be updated to improve the accuracy of the three-dimensional model of the monitoring scene, and the impact of the deformation changes of the monitored facility area on the three-dimensional model of the monitoring scene can be fully considered to evaluate the spatial structural state of the monitored facility area over time.

[0069] Furthermore, the monitoring scene of the monitored facility field can be reconstructed in three dimensions every preset time period, that is, the monitoring scene three-dimensional model can be updated to improve the accuracy of the monitoring scene three-dimensional model, and fully consider the impact of the deformation change of the monitored facility field on the monitoring scene three-dimensional model. And the new monitoring scene three-dimensional model can be compared with the old monitoring scene three-dimensional model to verify the accuracy of the embodiment of the present invention based on the comparison result, and then make adjustments in real time, and finally further improve the deformation monitoring accuracy of the infrastructure field, that is, evaluate the performance of deformation monitoring of the infrastructure field.

[0070] Furthermore, based on the deformation monitoring results of multiple deformation monitoring areas, the deformation monitoring time series trend of each deformation monitoring area can be counted. Furthermore, the deformation change ranking of all deformation monitoring areas can also be given.

[0071] Furthermore, based on the deformation monitoring results of multiple deformation monitoring areas, a full-field deformation monitoring analysis report of the facility area to be monitored can be generated to provide an emergency disaster prevention plan.

[0072] It can be understood that, through the above method, monitoring equipment can be deployed in a non-contact manner without destroying the structural surface of the infrastructure, thereby saving monitoring costs.

[0073] The infrastructure field deformation monitoring method provided by the embodiment of the present invention determines the current measuring point deformation monitoring area to be monitored from multiple deformation monitoring areas in the facility field to be monitored, so as to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system, and the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from the first coordinate position to the second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area, so as to control the camera system based on the measuring point coordinate transformation relationship information. The system rotates to monitor the deformation monitoring area of ​​the measuring point, to obtain the monitoring image collected by the image acquisition device on the deformation monitoring area of ​​the measuring point, to perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image, to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, and to return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained, thereby realizing multi-field inspection and monitoring by one camera system, that is, deformation monitoring of the infrastructure field can be completed using one camera system. In other words, one camera system can monitor multiple deformation monitoring areas, thereby eliminating the need to deploy multiple camera systems, thereby reducing the hardware cost of deformation monitoring of the infrastructure field.

[0074] Based on any of the above embodiments, Figure 2 This is the second flow chart of the infrastructure field deformation monitoring method provided by the present invention, such as Figure 2 As shown, the above step 150 includes step 151, step 152 and step 153.

[0075] Step 151: Acquire a template image set of the deformation monitoring area of ​​the measuring point, wherein the template image set includes a plurality of template images of different brightness.

[0076] Taking into account the influence of brightness on deformation monitoring, based on this, a template image set of the deformation monitoring area of ​​the measuring point is first obtained.

[0077] Exemplarily, the template image set can be acquired in the following manner: a preset number of template images are acquired at equal time differences within a preset time period to obtain a template image set. For example, 10 clear images are acquired at equal time differences between sunrise and sunset in each monitoring area as template images, based on which multiple template images of different brightness can be obtained.

[0078] Furthermore, the template image set may also include several template images with the same brightness or similar brightness.

[0079] The template image set is collected in advance. Further, the template image set can be updated every preset time period, for example, once a month. During each update, the monitoring image of the monitoring area is re-collected as the template image.

[0080] Step 152: Determine at least one measurement point template image from the template image set based on the brightness of the monitoring image.

[0081] In a specific embodiment, based on the brightness of the monitoring image, at least one measurement point template image closest to the brightness is determined from the template image set.

[0082] Step 153: Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point.

[0083] In a specific embodiment, based on the comparison results between the template images of each measuring point and the monitoring image, each sub-deformation monitoring result of the deformation monitoring area of ​​the measuring point is determined; based on each sub-deformation monitoring result, the deformation monitoring result of the deformation monitoring area of ​​the measuring point is comprehensively determined.

[0084] For example, if the number of the measurement point template images is 3, the deformation monitoring result of the measurement point deformation monitoring area is comprehensively determined based on the 3 sub-deformation monitoring results. In one embodiment, the comprehensive determination method may be an arithmetic average calculation method.

[0085] It should be understood that there may be multiple measuring point template images, so that multiple sub-deformation monitoring results can be obtained, thereby improving the accuracy of determining the deformation monitoring results of the measuring point deformation monitoring area.

[0086] The infrastructure field deformation monitoring method provided by the embodiment of the present invention obtains a template image set of a deformation monitoring area of ​​a measuring point, and the template image set includes multiple template images of different brightness, so as to determine at least one measuring point template image from the template image set based on the brightness of the monitoring image, thereby fully considering the influence of brightness on deformation monitoring, and further improving the deformation monitoring accuracy of the infrastructure field; and the number of measuring point template images can be multiple, so as to perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on each measuring point template image and the monitoring image, and obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, so as to obtain multiple sub-deformation monitoring results, thereby improving the determination accuracy of the deformation monitoring result of the deformation monitoring area of ​​the measuring point, that is, further improving the deformation monitoring accuracy of the infrastructure field.

[0087] Based on any of the above embodiments, in the method, the above step 153 includes step 1531 and step 1532.

[0088] Step 1531: Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result in the space of the image acquisition device.

[0089] It should be noted that, since the measurement point template image and the monitoring image are acquired by the image acquisition device, what is obtained is the deformation monitoring result in the space of the image acquisition device.

[0090] Step 1532: determine the deformation monitoring result of the measuring point deformation monitoring area based on the deformation monitoring result in the image acquisition device space and the first mapping relationship; the first mapping relationship is the size mapping relationship between the image acquisition device space and the actual world space; the deformation monitoring result of the measuring point deformation monitoring area is the actual physical deformation monitoring result.

[0091] Considering that image-based monitoring needs to have physical scale to meet actual needs, and determining the physical actual deformation monitoring results will be more accurate, that is, the physical actual deformation monitoring results will be more considerable, therefore, based on the size mapping relationship between the image acquisition device space and the actual world space, the deformation monitoring results in the image acquisition device space are converted into deformation monitoring results in the actual world space.

[0092] In other words, the first mapping relationship is a size mapping relationship between pixels and physical scales.

[0093] The infrastructure field deformation monitoring method provided by the embodiment of the present invention determines the deformation monitoring results of the measuring point deformation monitoring area based on the deformation monitoring results in the image acquisition device space and the first mapping relationship, and the first mapping relationship is the size mapping relationship between the image acquisition device space and the actual world space, so that the deformation monitoring results of the measuring point deformation monitoring area are the actual physical deformation monitoring results, thereby improving the deformation monitoring accuracy of the infrastructure field.

[0094] Based on any of the above embodiments, in the method, the first mapping relationship is determined based on the following manner: the first mapping relationship corresponding to the measurement point deformation monitoring area is determined from a mapping relationship set of the camera system; the mapping relationship set includes mapping relationships corresponding to the multiple deformation monitoring areas.

[0095] Here, the mapping relationship set is set in advance, that is, the corresponding mapping relationships are set in advance for multiple deformation monitoring areas.

[0096] Furthermore, the first mapping relationship is determined by rotating the camera system to face the electronic scale corresponding to the deformation monitoring area of ​​the measuring point.

[0097] It should be noted that the electronic ruler corresponding to each deformation monitoring area is fixed at a position in advance. It is also necessary to ensure that the electronic ruler is greater than a preset distance from the camera system. The electronic ruler can be connected to a power source.

[0098] In a specific embodiment, after the camera system is rotated to face the electronic ruler corresponding to the deformation monitoring area of ​​the measuring point, the camera system is first zoomed, and then the internal and external parameters of the camera system are calibrated, and the first mapping relationship is measured at the same time.

[0099] The infrastructure field deformation monitoring method provided by an embodiment of the present invention determines a first mapping relationship corresponding to a measuring point deformation monitoring area from a mapping relationship set of a camera system, and the first mapping relationship is determined by rotating the camera system to face the electronic ruler corresponding to the measuring point deformation monitoring area, so that the first mapping relationship can accurately characterize the dimensional mapping relationship between the image acquisition device space and the actual world space, thereby ensuring that the actual physical deformation monitoring results can be accurately obtained, that is, improving the deformation monitoring accuracy of the infrastructure field.

[0100] Based on any of the above embodiments, the camera system further includes a point cloud data acquisition device, and the relative position between the point cloud data acquisition device and the image acquisition device remains unchanged. For example, the point cloud data acquisition device may be a laser.

[0101] Accordingly, the above step 1532 includes steps 15321 to 15324.

[0102] Step 15321: Obtain point cloud data of the deformation monitoring area of ​​the measuring point at the current moment, wherein the point cloud data is acquired by the point cloud data acquisition device.

[0103] Here, the point cloud data is used for motion navigation of the camera system and for determining the distance data between the camera system and the deformation monitoring area of ​​the measuring point, so as to supplement the first mapping relationship to obtain the second mapping relationship, thereby improving the deformation monitoring accuracy of the infrastructure field.

[0104] Step 15322: Based on the point cloud data, determine the distance data between the camera system and the deformation monitoring area of ​​the measuring point, wherein the distance data includes multiple sub-distances, and any sub-distance is the distance between the camera system and any sub-area in the deformation monitoring area of ​​the measuring point.

[0105] It should be noted that, since the point cloud data is the point cloud data of the deformation monitoring area of ​​the measuring point, the distance data between the camera system and the deformation monitoring area of ​​the measuring point can be determined based on the point cloud data.

[0106] Step 15323: Based on the distance data and the first mapping relationship, determine multiple second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point, and any second mapping relationship is a size mapping relationship between the image acquisition device space and the actual world space.

[0107] It should be noted that since the distance data includes multiple sub-distances, any sub-distance is the distance between the camera system and any sub-area in the measuring point deformation monitoring area. Based on this, the first mapping relationship can be converted based on the distance data to obtain multiple second mapping relationships corresponding to different sub-areas in the measuring point deformation monitoring area.

[0108] Step 15324: Determine the deformation monitoring result of the deformation monitoring area of ​​the measuring point based on the deformation monitoring result in the image acquisition device space and each of the second mapping relationships.

[0109] Specifically, based on the size mapping relationship between the image acquisition device space and the actual world space, the deformation monitoring results in the image acquisition device space are converted into deformation monitoring results of multiple sub-areas in the actual world space, and then the deformation monitoring results of the deformation monitoring area of ​​the measuring point are comprehensively determined based on the deformation monitoring results of the multiple sub-areas.

[0110] The infrastructure field deformation monitoring method provided by the embodiment of the present invention obtains the point cloud data of the deformation monitoring area of ​​the measuring point at the current moment, and determines the distance data between the camera system and the deformation monitoring area of ​​the measuring point based on the point cloud data at the current moment, and determines multiple second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point based on the distance data and the first mapping relationship, that is, obtains a high-reliability pixel scale at different positions, that is, updates and adjusts the first mapping relationship based on the point cloud data at the current moment, and then based on the deformation monitoring result in the space of the image acquisition device and the second mapping relationships, the deformation monitoring result of the deformation monitoring area of ​​the measuring point can be determined more accurately, that is, the deformation monitoring result of the infrastructure field is improved.

[0111] Based on any of the above embodiments, in the method, the above step 15322 includes step 153221 and step 153222.

[0112] Step 153221: Obtain the position and posture information of the image acquisition device at the current moment, wherein the position and posture information represents the position and posture of the image acquisition device in the three-dimensional model of the monitoring scene of the facility to be monitored.

[0113] In one embodiment, the posture information can be obtained in the following manner: based on the third mapping relationship of the camera system, the posture information corresponding to the deformation monitoring area of ​​the measuring point (i.e., the posture information of the image acquisition device at the current moment) is determined; the third mapping relationship includes a one-to-one correspondence between multiple deformation monitoring areas and multiple posture information, and any corresponding relationship is used to characterize the correspondence between the deformation monitoring area and the posture information. The third mapping relationship can be set in advance, that is, the posture of the image acquisition device in the three-dimensional model of the monitoring scene is calibrated in advance.

[0114] In another embodiment, the position information of the image acquisition device at the current moment may be determined in real time by other means, for example, by fusing information of a camera system and an IMU (Inertial Measurement Unit).

[0115] Step 153222: Based on the positional relationship information between the image acquisition device and the point cloud data acquisition device, the posture information and the point cloud data, determine the distance data between the camera system and the deformation monitoring area of ​​the measuring point.

[0116] Here, since the relative positions of the point cloud data acquisition device and the image acquisition device remain unchanged, the positional relationship information also remains unchanged. The positional relationship information can be set in advance.

[0117] In one embodiment, the positional relationship information can be determined by joint calibration. Specifically, the positions of the image acquisition device and the point cloud data acquisition device are fixed, and calibration objects are set for calibration or self-calibration, so as to obtain an external parameter matrix between the image acquisition device and the point cloud data acquisition device, and then the positional relationship information is determined based on the external parameter matrix. For example, the joint calibration is performed using a joint calibration tool.

[0118] Since the posture information represents the posture of the image acquisition device in the three-dimensional model of the monitoring scene of the facility to be monitored, the point cloud data can be converted into point cloud data in the three-dimensional model space of the monitoring scene based on the positional relationship information between the image acquisition device and the point cloud data acquisition device and the posture information of the image acquisition device at the current moment. Then, based on the converted point cloud data, the distance data in the three-dimensional model space of the monitoring scene can be determined.

[0119] In one embodiment, the internal parameters of the image acquisition device are calibrated in advance, and the positional relationship information is obtained by calibration, so as to determine the distance data between the camera system and the deformation monitoring area of ​​the measuring point based on the positional relationship information, posture information, internal parameters of the image acquisition device and point cloud data between the image acquisition device and the point cloud data acquisition device. The internal parameters of the image acquisition device can be calibrated using the Zhang Zhengyou calibration method.

[0120] In a specific embodiment, the relationship between the camera system and the deformation monitoring area of ​​the measuring point is determined in combination with the spatial coordinate position of the image acquisition device in the coordinate system. The relationship between the camera system and the deformation monitoring area of ​​the measuring point is as follows: ; In the formula, Indicates the straight-line distance between the camera system and the deformation monitoring area of ​​the measuring point, Indicates the azimuth angle of the deformation monitoring area of ​​the measuring point relative to the camera system, Indicates the pitch angle of the deformation monitoring area of ​​the measuring point relative to the camera system. represents the position coordinates of the camera system, Indicates the location coordinates of the deformation monitoring area of ​​the measuring point.

[0121] The camera system also includes a laser ranging radar and an IMU sensor, which can measure the angular velocity and acceleration values ​​at the current moment through the IMU sensor to monitor the azimuth, pitch angle and other parameters of the camera system in real time. Combined with the three-dimensional model of the monitoring scene, the camera system's own coordinate system and the coordinate system of the measurement point deformation monitoring area can be integrated into the coordinate system of the three-dimensional model of the monitoring scene, and the camera system's own coordinate system and the measurement point deformation monitoring area coordinate system can be managed in a unified manner. The movement of the camera system can be reflected in the three-dimensional model of the monitoring scene in real time, thereby achieving accurate positioning and focusing of the camera system's monitoring field of view.

[0122] The infrastructure field deformation monitoring method provided by the embodiment of the present invention can determine the distance data in the three-dimensional model space of the monitoring scene through the above method, thereby improving the accuracy of determining the distance data, and then more accurately determining the deformation monitoring results of the deformation monitoring area of ​​the measuring point, that is, improving the deformation monitoring results of the infrastructure field.

[0123] Based on any of the above embodiments, after the above step 160 , the method further includes step 170 .

[0124] Step 170: Based on the deformation monitoring results of the multiple deformation monitoring areas, update the three-dimensional model of the monitoring scene.

[0125] The infrastructure field deformation monitoring method provided by the embodiment of the present invention updates the three-dimensional model of the monitoring scene based on the deformation monitoring results of multiple deformation monitoring areas, improves the accuracy of the three-dimensional model of the monitoring scene, and fully considers the impact of the deformation changes of the facility field to be monitored on the three-dimensional model of the monitoring scene, that is, realizes the synchronous change of the scene to be monitored, and provides an accurate basis for subsequent monitoring, thereby improving the accuracy of determining the distance data, and further more accurately determines the deformation monitoring results of the deformation monitoring area of ​​the measuring point, that is, improves the deformation monitoring accuracy of the infrastructure field.

[0126] Based on any of the above embodiments, in the method, the above step 15323 includes step 153231 and step 153232.

[0127] Step 153231: Perform interpolation fitting based on the distance data to obtain a scale change relationship function of the deformation monitoring area of ​​the measuring point.

[0128] It should be noted that since the distance data includes multiple sub-distances, and any sub-distance is the distance between the camera system and any sub-area in the deformation monitoring area of ​​the measuring point, based on this, interpolation fitting is required based on the distance data to obtain the scale change relationship function of the deformation monitoring area of ​​the measuring point.

[0129] Step 153232: scale the first mapping relationship based on the scale change relationship function to obtain multiple second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point.

[0130] It should be noted that since the scale change relationship function is obtained by interpolation fitting based on the distance data, and the distance data includes multiple sub-distances, any sub-distance is the distance between the camera system and any sub-area in the deformation monitoring area of ​​the measuring point. Based on this, the first mapping relationship can be scaled based on the scale change relationship function to obtain multiple second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point.

[0131] The infrastructure field deformation monitoring method provided by an embodiment of the present invention performs interpolation fitting based on distance data to obtain a scale change relationship function of the deformation monitoring area of ​​the measuring point, and performs scale change on the first mapping relationship based on the scale change relationship function to obtain a more accurate plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point, thereby more accurately determining the deformation monitoring results of the deformation monitoring area of ​​the measuring point, that is, improving the deformation monitoring results of the infrastructure field.

[0132] Based on any of the above embodiments, before the above step 140, the method further includes steps 180 and 190.

[0133] Step 180: Determine the focal length of the measuring point corresponding to the measuring point deformation monitoring area from the focal length information set of the camera system; the focal length information set includes the focal lengths corresponding to the multiple deformation monitoring areas; Here, the focal length information set includes the focal length corresponding to each deformation monitoring area. The focal length information set is set in advance.

[0134] In a specific embodiment, a monitoring order of multiple deformation monitoring areas is determined; based on the monitoring order, the focal length of the image acquisition device when the camera system monitors the first deformation monitoring area is determined, and so on, the focal length of the image acquisition device corresponding to the subsequent deformation monitoring area is determined, thereby obtaining a focal length information set.

[0135] In other words, the monitoring camera system rotates from the initial position to the focal length of the image acquisition device monitoring the first deformation monitoring area, and the monitoring camera system rotates from the coordinate position of the camera system when monitoring the first deformation monitoring area to the focal length of the image acquisition device monitoring the second deformation monitoring area, and so on, to determine the focal length of the image acquisition device corresponding to the subsequent deformation monitoring area, and then obtain the focal length information set. That is, the focal length value of the image acquisition device when collecting each deformation monitoring area is recorded.

[0136] It should be understood that zoom can support deformation monitoring of multiple deformation monitoring areas, thereby realizing multi-field inspection and monitoring by one camera system, that is, one camera system can be used to complete deformation monitoring of the infrastructure site. In other words, one camera system can monitor multiple deformation monitoring areas, thus eliminating the need to deploy multiple camera systems and reducing the hardware cost of deformation monitoring in the infrastructure site.

[0137] Step 190: Adjust the focal length of the image acquisition device to the focal length of the measuring point.

[0138] It should be understood that the focal length of the image acquisition device is adjusted to the focal length of the measuring point so that the image acquisition device can acquire clearer and more accurate monitoring images, thereby improving the accuracy of deformation monitoring of the infrastructure field.

[0139] The infrastructure field deformation monitoring method provided in an embodiment of the present invention can realize multi-field inspection and monitoring by one camera system through the above-mentioned method, that is, deformation monitoring of the infrastructure field can be completed using one camera system. In other words, one camera system can monitor multiple deformation monitoring areas, thereby eliminating the need to deploy multiple camera systems and reducing the hardware cost of infrastructure field deformation monitoring; and the focal length of the image acquisition device is adjusted to the focal length of the measuring point, so that the image acquisition device can acquire clearer and more accurate monitoring images, thereby improving the accuracy of deformation monitoring of the infrastructure field.

[0140] Based on any of the above embodiments, in the method, when the coordinate position of the camera system is the second coordinate position, the deformation monitoring area of ​​the measuring point is located at the center of the field of view of the camera system, so that the image acquisition device acquires a clear and accurate monitoring image, thereby improving the accuracy of deformation monitoring of the infrastructure field. Furthermore, the point cloud data acquisition device can also acquire accurate point cloud data, thereby improving the accuracy of deformation monitoring of the infrastructure field.

[0141] The infrastructure field deformation monitoring method provided in an embodiment of the present invention ensures that the deformation monitoring area of ​​the measuring point is located at the center of the field of view of the camera system when the coordinate position of the camera system is the second coordinate position in the above manner, so that the image acquisition device can acquire clear and accurate monitoring images, thereby improving the deformation monitoring accuracy of the infrastructure field.

[0142] Based on the above embodiments, the infrastructure field deformation monitoring method provided by the present invention has the advantages of non-contact, single field of view and multiple measuring points. It can also robustly support the millimeter-level quantitative visual monitoring capability of deformation, thereby achieving the effect of single lens multiple fields of view and single field of view multiple measuring points, and the installation and operation and maintenance costs are significantly superior to traditional methods.

[0143] The infrastructure field deformation monitoring device provided by the present invention is described below. The infrastructure field deformation monitoring device described below and the infrastructure field deformation monitoring method described above can be referred to each other.

[0144] Figure 3 is a schematic diagram of the structure of the infrastructure field deformation monitoring device provided by the present invention, such as Figure 3 As shown, the infrastructure field deformation monitoring device includes an area determination module 310, an information determination module 320, a system control module 330, an image acquisition module 340, a deformation monitoring module 350 and a step return module 360.

[0145] The area determination module 310 is used to determine the deformation monitoring area of ​​the measuring point to be currently monitored from multiple deformation monitoring areas of the facility field to be monitored.

[0146] The information determination module 320 is used to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system includes an image acquisition device, the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, and the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area.

[0147] The system control module 330 is used to control the rotation of the camera system based on the measurement point coordinate transformation relationship information to monitor the measurement point deformation monitoring area.

[0148] The image acquisition module 340 is used to acquire the monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point.

[0149] The deformation monitoring module 350 is used to perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point.

[0150] The step returns to module 360, which is used to return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0151] The infrastructure field deformation monitoring device provided by the embodiment of the present invention determines the current measuring point deformation monitoring area to be monitored from multiple deformation monitoring areas in the facility field to be monitored, so as to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system, and the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from the first coordinate position to the second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area, so as to control the camera system based on the measuring point coordinate transformation relationship information. The system rotates to monitor the deformation monitoring area of ​​the measuring point, to obtain the monitoring image collected by the image acquisition device on the deformation monitoring area of ​​the measuring point, to perform deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image, to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point, and to return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained, thereby realizing multi-field inspection and monitoring by one camera system, that is, deformation monitoring of the infrastructure field can be completed using one camera system. In other words, one camera system can monitor multiple deformation monitoring areas, thereby eliminating the need to deploy multiple camera systems, thereby reducing the hardware cost of deformation monitoring of the infrastructure field.

[0152] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the infrastructure field deformation monitoring method, the method comprising: determining the current measuring point deformation monitoring area to be monitored from multiple deformation monitoring areas of the facility field to be monitored; determining the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system comprises an image acquisition device, the coordinate transformation relationship information set comprises the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship of the first coordinate position to the second coordinate position, the first coordinate position is the last deformation monitoring area monitored by the camera system the coordinate position of the camera system when the second coordinate position is the coordinate position of the camera system when the camera system monitors the deformation monitoring area of ​​the measuring point; based on the measuring point coordinate conversion relationship information, controlling the camera system to rotate to monitor the deformation monitoring area of ​​the measuring point; acquiring a monitoring image of the deformation monitoring area of ​​the measuring point acquired by the image acquisition device; based on the monitoring image, performing deformation monitoring on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point; returning to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0153] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the infrastructure field deformation monitoring method provided by the above-mentioned methods, and the method includes: determining a measuring point deformation monitoring area to be currently monitored from multiple deformation monitoring areas of the facility field to be monitored; determining measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from a coordinate transformation relationship information set of a camera system; the camera system includes an image acquisition device, and the coordinate transformation relationship information set includes coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, and the measuring point coordinate transformation relationship information represents the transformation of a first coordinate position to a second coordinate position The coordinate transformation relationship of the first coordinate position is the coordinate position of the camera system when the camera system monitored the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitored the deformation monitoring area of ​​the measuring point; based on the coordinate transformation relationship information of the measuring point, the camera system is controlled to rotate to monitor the deformation monitoring area of ​​the measuring point; a monitoring image of the deformation monitoring area of ​​the measuring point acquired by the image acquisition device is acquired; based on the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point; and the step of determining the deformation monitoring area of ​​the measuring point to be currently deformed monitored from the multiple deformation monitoring areas of the facility field to be monitored is returned until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the infrastructure field deformation monitoring method provided by the above-mentioned methods, the method comprising: determining a measuring point deformation monitoring area to be currently monitored from a plurality of deformation monitoring areas of the facility field to be monitored; determining measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from a coordinate transformation relationship information set of a camera system; the camera system comprises an image acquisition device, the coordinate transformation relationship information set comprises coordinate transformation relationship information corresponding to the plurality of deformation monitoring areas, the measuring point coordinate transformation relationship information represents a coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate The position is the coordinate position of the camera system when the camera system monitored the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the deformation monitoring area of ​​the measuring point; based on the measuring point coordinate conversion relationship information, the camera system is controlled to rotate to monitor the deformation monitoring area of ​​the measuring point; the monitoring image of the deformation monitoring area of ​​the measuring point acquired by the image acquisition device is acquired; based on the monitoring image, the deformation monitoring of the deformation monitoring area of ​​the measuring point is performed to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point; and the step of determining the deformation monitoring area of ​​the measuring point to be currently deformed monitored from the multiple deformation monitoring areas of the facility field to be monitored is returned until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring deformation of infrastructure field, characterized in that: include: Determine a deformation monitoring area of ​​a measuring point to be currently monitored from a plurality of deformation monitoring areas in the facility area to be monitored; Determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system includes an image acquisition device, the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area; Based on the measurement point coordinate transformation relationship information, controlling the camera system to rotate to monitor the measurement point deformation monitoring area; Acquire a monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point; Based on the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point; Return to the step of determining the deformation monitoring area of ​​the measuring point to be currently deformation monitored from the multiple deformation monitoring areas of the facility field to be monitored, until deformation monitoring results of the multiple deformation monitoring areas are obtained.

2. The infrastructure field deformation monitoring method according to claim 1, characterized in that: The step of performing deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point includes: Acquire a template image set of the deformation monitoring area of ​​the measuring point, wherein the template image set includes a plurality of template images of different brightness; Based on the brightness of the monitoring image, determining at least one measuring point template image from the template image set; Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result of the deformation monitoring area of ​​the measuring point.

3. The infrastructure field deformation monitoring method according to claim 2, characterized in that: The step of performing deformation monitoring on the deformation monitoring area of ​​the measuring point based on the template image of each measuring point and the monitoring image to obtain the deformation monitoring result of the deformation monitoring area of ​​the measuring point includes: Based on the template images of each measuring point and the monitoring image, deformation monitoring is performed on the deformation monitoring area of ​​the measuring point to obtain a deformation monitoring result in the space of the image acquisition device; Based on the deformation monitoring result in the image acquisition device space and the first mapping relationship, the deformation monitoring result of the measuring point deformation monitoring area is determined; the first mapping relationship is the size mapping relationship between the image acquisition device space and the actual world space; the deformation monitoring result of the measuring point deformation monitoring area is the actual physical deformation monitoring result.

4. The infrastructure field deformation monitoring method according to claim 3 is characterized in that: The first mapping relationship is determined based on the following method: Determining the first mapping relationship corresponding to the deformation monitoring area of ​​the measuring point from a mapping relationship set of the camera system; the mapping relationship set includes mapping relationships corresponding to the multiple deformation monitoring areas; The first mapping relationship is determined by rotating the camera system to face the electronic scale corresponding to the deformation monitoring area of ​​the measuring point.

5. The infrastructure field deformation monitoring method according to claim 4, characterized in that: The camera system further comprises a point cloud data acquisition device, and the relative position between the point cloud data acquisition device and the image acquisition device remains unchanged; The step of determining the deformation monitoring result of the deformation monitoring area of ​​the measuring point based on the deformation monitoring result in the space of the image acquisition device and the first mapping relationship includes: Acquire point cloud data of the deformation monitoring area of ​​the measuring point at the current moment, wherein the point cloud data is acquired by the point cloud data acquisition device; Based on the point cloud data, determining distance data between the camera system and the deformation monitoring area of ​​the measuring point, wherein the distance data includes a plurality of sub-distances, and any of the sub-distances is a distance between the camera system and any sub-area in the deformation monitoring area of ​​the measuring point; Based on the distance data and the first mapping relationship, determine a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point, any of the second mapping relationships being a size mapping relationship between the image acquisition device space and the actual world space; Based on the deformation monitoring result in the image acquisition device space and each of the second mapping relationships, the deformation monitoring result of the deformation monitoring area of ​​the measuring point is determined.

6. The infrastructure field deformation monitoring method according to claim 5, characterized in that: The determining, based on the point cloud data, distance data between the camera system and the deformation monitoring area of ​​the measuring point includes: Acquire the position and posture information of the image acquisition device at the current moment, wherein the position and posture information represents the position and posture of the image acquisition device in the three-dimensional model of the monitoring scene of the facility to be monitored; Based on the positional relationship information between the image acquisition device and the point cloud data acquisition device, the posture information and the point cloud data, the distance data between the camera system and the deformation monitoring area of ​​the measuring point is determined.

7. The infrastructure field deformation monitoring method according to claim 6, characterized in that: The step of returning to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas in the field of the facility to be monitored until the deformation monitoring results of the multiple deformation monitoring areas are obtained, further includes: Based on the deformation monitoring results of the multiple deformation monitoring areas, the three-dimensional model of the monitoring scene is updated.

8. The method for monitoring infrastructure field deformation according to claim 5, characterized in that: The determining, based on the distance data and the first mapping relationship, a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point includes: Perform interpolation fitting based on the distance data to obtain a scale change relationship function of the deformation monitoring area of ​​the measuring point; The first mapping relationship is scale-changed based on the scale-change relationship function to obtain a plurality of second mapping relationships corresponding to different sub-areas in the deformation monitoring area of ​​the measuring point.

9. The infrastructure field deformation monitoring method according to claim 1, characterized in that: The step of acquiring the monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point further includes: Determining the focal length of the measuring point corresponding to the deformation monitoring area of ​​the measuring point from the focal length information set of the camera system; the focal length information set includes the focal lengths corresponding to the multiple deformation monitoring areas; The focal length of the image acquisition device is adjusted to the focal length of the measuring point.

10. The infrastructure field deformation monitoring method according to claim 1, characterized in that: When the coordinate position of the camera system is the second coordinate position, the measuring point deformation monitoring area is located at the center of the field of view of the camera system.

11. An infrastructure field deformation monitoring device, characterized in that: include: An area determination module is used to determine the deformation monitoring area of ​​the measuring point to be currently monitored from multiple deformation monitoring areas in the field of the facility to be monitored; An information determination module, used to determine the measuring point coordinate transformation relationship information corresponding to the measuring point deformation monitoring area from the coordinate transformation relationship information set of the camera system; the camera system includes an image acquisition device, the coordinate transformation relationship information set includes the coordinate transformation relationship information corresponding to the multiple deformation monitoring areas, the measuring point coordinate transformation relationship information represents the coordinate transformation relationship from a first coordinate position to a second coordinate position, the first coordinate position is the coordinate position of the camera system when the camera system monitors the previous deformation monitoring area, and the second coordinate position is the coordinate position of the camera system when the camera system monitors the measuring point deformation monitoring area; A system control module, used to control the rotation of the camera system based on the coordinate transformation relationship information of the measuring point to monitor the deformation monitoring area of ​​the measuring point; An image acquisition module, used to acquire a monitoring image acquired by the image acquisition device on the deformation monitoring area of ​​the measuring point; A deformation monitoring module, used for performing deformation monitoring on the deformation monitoring area of ​​the measuring point based on the monitoring image, and obtaining a deformation monitoring result of the deformation monitoring area of ​​the measuring point; The step return module is used to return to the step of determining the deformation monitoring area of ​​the measuring point to be currently monitored from the multiple deformation monitoring areas of the facility field to be monitored, until the deformation monitoring results of the multiple deformation monitoring areas are obtained.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the infrastructure field deformation monitoring method as described in any one of claims 1 to 10 is implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring deformation of an infrastructure site as claimed in any one of claims 1 to 10 is implemented.

Images