Multi-focal-length deformation measurement method and device

Abstract

The invention provides a multi-focal-length deformation measurement method and device, and deformation measurement equipment, and the method employs a multi-focal-length camera to obtain the image data of to-be-measured points with different longitudinal depths at the same time, carries out the fusion of the image data of multiple focal lengths, achieves the shooting of more to-be-measured points at a single monitoring position, improves the single monitoring range, and improves the measurement precision. Meanwhile, the deformation measurement equipment does not need point-by-point parking shooting, thereby reducing the time cost and improving the monitoring efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a multi-focal-length deformation measurement method and device. Background Art

[0002] Currently, mobile highway deformation detection often uses cameras with the same focal length for measurement. The measurement depth coverage is short, making it difficult to obtain data from multiple measurement points at the same time. The inspection vehicle needs to stop and take pictures point by point multiple times, which increases time costs. Summary of the Invention

[0003] The present application provides a multi-focal-length deformation measurement method and device, which uses a multi-focal-length camera to simultaneously acquire image data of monitoring points at different longitudinal depths, and fuses the image data of the multiple focal lengths to achieve the goal of capturing more monitoring points at a single monitoring position, thereby increasing the single monitoring range. At the same time, the deformation measurement equipment does not need to stop to shoot point by point, which reduces time cost and improves monitoring efficiency.

[0004] In a first aspect, an embodiment of the present application provides a multi-focal-length deformation measurement method, which is applied to a deformation measuring device, wherein the deformation measuring device is provided with a camera array and a mobile platform, the camera array including at least two camera groups, each camera group including cameras of at least two focal lengths, and the shooting directions of the two camera groups are opposite; the method includes: during a first patrol process, when the mobile platform travels to a first monitoring position of the area to be measured, the monitoring points in the first area to be measured in the first monitoring position are photographed by a camera of each focal length in the forward-looking camera group in the camera array to obtain at least two first images; the monitoring points in the second area to be measured in the second monitoring position are photographed by a camera of each focal length in the rear-looking camera group in the camera array to obtain at least two second images; wherein the first area to be measured and the second area to be measured are arranged on both sides of the first monitoring position; during a second patrol process, when the mobile platform travels to the second monitoring position of the area to be measured, the monitoring points in the first area to be measured are photographed by a camera of each focal length in the forward-looking camera group to obtain at least two second images; The monitoring points in the area to be measured are photographed to obtain at least two third images; the monitoring points in the second area to be measured are photographed by a camera of each focal length in the rearview camera group to obtain at least two fourth images; a multi-focal-length fusion calculation is performed on the at least two first images to obtain a first front view image; a multi-focal-length fusion calculation is performed on the at least two second images to obtain a first rearview image; a multi-focal-length fusion calculation is performed on the at least two third images to obtain a second front view image; a multi-focal-length fusion calculation is performed on the at least two fourth images to obtain a second rearview image; a plurality of first monitoring points commonly contained in the first front view image and the second front view image are obtained, and a plurality of second monitoring points commonly contained in the first rearview image and the second rearview image are obtained; a three-dimensional shape variable of each first monitoring point in the plurality of first monitoring points is obtained according to the first front view image and the second front view image, and a three-dimensional shape variable of each second monitoring point in the plurality of second monitoring points is obtained according to the first rearview image and the second rearview image.

[0005] In a second aspect, an embodiment of the present application provides a multi-focal-segment deformation measurement device, which is applied to a deformation measurement device, wherein the deformation measurement device is provided with a camera array and a mobile platform, the camera array including two camera groups, each camera group including cameras of at least two focal lengths, and the shooting directions of the two camera groups are opposite; the device includes: an image acquisition unit, during a first patrol process, when the mobile platform travels to a first monitoring position of the area to be measured, the monitoring point in the first area to be measured in the first monitoring position is photographed by a camera of each focal length in the forward-looking camera group in the camera array, to obtain at least two first images; the monitoring point in the second area to be measured in the second monitoring position is photographed by a camera of each focal length in the rear-view camera group in the camera array, to obtain at least two second images; wherein the first area to be measured and the second area to be measured are arranged on both sides of the first monitoring position; and, during a second patrol process, when the mobile platform travels to the second monitoring position of the area to be measured, the monitoring point in the first area to be measured is photographed by a camera of each focal length in the forward-looking camera group the monitoring points in the second area to be measured are photographed by a camera of each focal length in the rearview camera group to obtain at least two fourth images; an image processing unit performs multi-focal-segment fusion calculation on the at least two first images to obtain a first front view image; performs multi-focal-segment fusion calculation on the at least two second images to obtain a first rearview image; performs multi-focal-segment fusion calculation on the at least two third images to obtain a second front view image; performs multi-focal-segment fusion calculation on the at least two fourth images to obtain a second rearview image; and, obtains a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtains a plurality of second monitoring points commonly contained in the first rearview image and the second rearview image; and, obtains a three-dimensional shape variable of each first monitoring point in the plurality of first monitoring points according to the first front view image and the second front view image, and obtains a three-dimensional shape variable of each second monitoring point in the plurality of second monitoring points according to the first rearview image and the second rearview image.

[0006] In a third aspect, an embodiment of the present application provides a deformation measurement device, which is provided with a controller, a camera array and a mobile platform, the camera array includes two camera groups, each camera group includes cameras with at least two focal lengths, and the shooting directions of the two camera groups are opposite, and the controller is used to execute the step instructions in the method as described in any one of the first aspects.

[0007] In a fourth aspect, an embodiment of the present application provides a controller comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes the step instructions in the method as described in any one of the first aspects.

[0008] It can be seen that in the embodiment of the present application, during the first patrol process of the deformation measuring device, when the mobile platform travels to the first monitoring position of the area to be measured, the monitoring point in the first area to be measured in the first monitoring position is photographed by the camera of each focal length in the forward-looking camera group in the camera array, and at least two first images are obtained; the monitoring point in the second area to be measured in the second monitoring position is photographed by the camera of each focal length in the rear-looking camera group in the camera array, and at least two second images are obtained; wherein, the first area to be measured and the second area to be measured are arranged on both sides of the first monitoring position; during the second patrol process, when the mobile platform travels to the second monitoring position of the area to be measured, the monitoring point in the first area to be measured is photographed by the camera of each focal length in the forward-looking camera group, and at least two third images are obtained; the monitoring point in the second area to be measured is photographed by the camera of each focal length in the rear-looking camera group The method comprises the following steps: shooting the monitoring points within the area to obtain at least two fourth images; performing multi-focal section fusion calculation on the at least two first images to obtain a first front view image; performing multi-focal section fusion calculation on the at least two second images to obtain a first rear view image; performing multi-focal section fusion calculation on the at least two third images to obtain a second front view image; performing multi-focal section fusion calculation on the at least two fourth images to obtain a second rear view image; obtaining a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtaining a plurality of second monitoring points commonly contained in the first rear view image and the second rear view image; obtaining a three-dimensional deformation amount of each first monitoring point among the plurality of first monitoring points according to the first front view image and the second front view image, and obtaining a three-dimensional deformation amount of each second monitoring point among the plurality of second monitoring points according to the first rear view image and the second rear view image. It can be seen that in this application, by using a multi-focal-length camera to simultaneously obtain image data of the test points at different longitudinal depths, and fusing the image data of the multi-focal-lengths, a single monitoring position can be used to capture more test points, thereby increasing the single monitoring range. At the same time, the deformation measurement equipment does not need to stop and shoot point by point, which reduces time costs and improves monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 A schematic structural diagram of a deformation measurement device provided in an embodiment of the present application; Figure 2 A structural block diagram of a deformation measurement device provided in an embodiment of the present application; Figure 3A schematic diagram of a deformation measurement scenario provided in an embodiment of the present application; Figure 4 A schematic structural block diagram of a deformation measurement system provided in an embodiment of the present application; Figure 5 A schematic flow chart of a multi-focal-segment deformation measurement method provided in an embodiment of the present application; Figure 6 This is a block diagram of the functional unit structure of a multi-focal-segment deformation measurement device provided in an embodiment of the present application; Figure 7 This is a structural block diagram of a controller provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may, in some embodiments, also include steps or elements not listed, or may, in some embodiments, include other steps or elements inherent to the process, method, product, or apparatus.

[0012] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0013] In the embodiments of this application, "and / or" describes the relationship between associated objects and indicates that three relationships can exist. For example, "A and / or B" can represent the following three situations: A exists alone; A and B exist simultaneously; and B exists alone. A and B can be singular or plural.

[0014] In the embodiments of the present application, the symbol " / " can indicate that the preceding and following objects are in an "or" relationship. In addition, the symbol " / " can also represent a division sign, that is, performing a division operation. For example, A / B can mean A divided by B.

[0015] In the embodiments of the present application, "at least one item" or similar expressions refers to any combination of these items, including any combination of single items or plural items, and refers to one or more, and multiple refers to two or more. For example, at least one item (item) of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0016] In the embodiments of this application, "equal to" can be used in conjunction with "greater than" and is applicable to the technical solution adopted when "greater than" is used, and can also be used in conjunction with "less than" and is applicable to the technical solution adopted when "less than" is used. When "equal to" is used in conjunction with "greater than", it should not be used in conjunction with "less than"; when "equal to" is used in conjunction with "less than", it should not be used in conjunction with "greater than".

[0017] The present application provides a multi-focal-length deformation measurement method and device, which uses a multi-focal-length camera to simultaneously acquire image data of monitoring points at different longitudinal depths, and fuses the image data of the multiple focal lengths to achieve the goal of capturing more monitoring points at a single monitoring position, thereby increasing the single monitoring range. At the same time, the deformation measurement equipment does not need to stop to shoot point by point, which reduces time cost and improves monitoring efficiency.

[0018] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0019] See also Figure 1 、 Figure 2 , Figure 1 This is a structural diagram of a deformation measurement device provided in an embodiment of the present application. Figure 2 This is a structural block diagram of a deformation measurement device provided in an embodiment of the present application.

[0020] A mobile platform 10 is provided on the deformation measurement device 1, and a camera array 20 is provided on the mobile platform 10. The camera array 20 is provided with at least two camera groups with opposite shooting directions, and the camera groups with the same shooting direction include cameras with at least two focal lengths, and each camera in the camera array 20 is fixedly connected to each other.

[0021] For example, Figure 1As shown, the camera array 20 includes a front-view camera group 21 and a rear-view camera group 22. The front-view camera group 21 is used to photograph monitoring points in the direction of travel of the deformation measurement device 1, and the rear-view camera group 22 is used to photograph monitoring points in the direction opposite to the direction of travel of the deformation measurement device 1. Figure 3 , Figure 3 Schematic diagram of a deformation measurement scenario provided in an embodiment of the present application, wherein the front / rear view camera groups each include cameras with three focal lengths: long, medium and short.

[0022] In some embodiments, the front view camera group 21 and the rear view camera group 22 simultaneously photograph the monitoring points in the area to be tested, that is, while the front view camera group 21 photographs the monitoring points in the first area to be tested, the rear view camera group 22 also photographs the monitoring points in the second area to be tested.

[0023] In an embodiment of the present application, the deformation measurement device 1 also includes a controller 30, which is used to send a control signal to the camera array 20 after the mobile platform 10 travels to the monitoring position. After receiving the control signal, the camera array 20 will control the forward-looking camera group 21 and the rear-looking camera group 22 to shoot the monitoring point to obtain a multi-focal-segment image, and calculate the three-dimensional deformation of the monitoring point based on the captured multi-focal-segment image.

[0024] For specific implementation, please refer to Figure 7 , Figure 7 A structural block diagram of a controller provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the controller 30 includes a processor 3001, a memory 3003, a communication interface 3002, and one or more programs 30031. The one or more programs 30031 are stored in the memory 3003 and are configured to be executed by the processor 3001. The one or more programs 30031 include instructions for executing any step in an embodiment of a multi-stage speed control method for powder metering and discharging described below.

[0025] Optional, see Figure 4 , Figure 4 A structural schematic diagram of a deformation measurement system provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the deformation measurement system 4 includes a deformation measurement device 1 and a cloud server 2, wherein a mobile platform 10 is provided on the deformation measurement device 1, a camera array 20 is provided on the mobile platform 10, and at least two camera groups with opposite shooting directions are provided on the camera array 20, each camera in each camera group has a different focal length, and each camera in the camera array 20 is fixedly connected to each other.

[0026] In a specific implementation, a controller is also provided on the mobile platform 10. The controller is used to send a control signal to the camera array 20 after the mobile platform 10 travels to the monitoring position. After receiving the control signal, the camera array 20 will control the camera array 20 to shoot the monitoring point to obtain a multi-focal-segment image, and send the captured multi-focal-segment image to the cloud server 2 for processing. After receiving the multi-focal-segment image, the cloud server 2 analyzes and processes the multi-focal-segment image according to the steps and instructions of the following method embodiment to obtain the three-dimensional deformation of the monitoring point.

[0027] Based on the above hardware structure, a multi-focal-section deformation measurement method provided in an embodiment of the present application is proposed.

[0028] See also Figure 5 , Figure 5 A flow chart of the multi-focal-segment deformation measurement method provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the method is applied to Figure 1 The deformation measuring device shown in FIG. 5 includes the following steps S501 to S505 .

[0029] Step S501: During the first patrol process, when the mobile platform travels to the first monitoring position of the area to be tested, the monitoring points within the first area to be tested in the first monitoring position are photographed by the camera of each focal length in the forward-looking camera group in the camera array to obtain at least two first images; and the monitoring points within the second area to be tested in the second monitoring position are photographed by the camera of each focal length in the rear-looking camera group in the camera array to obtain at least two second images.

[0030] The first area to be tested and the second area to be tested are arranged on both sides of the first monitoring position.

[0031] In the embodiments of the present application, the mobile platform can be a mobile monitoring vehicle, a mobile monitoring ship, a mobile aircraft, etc., and the mobile platform can be installed with a positioning device, such as a mileage encoder, a positioning system, etc., and can also be installed with a posture measurement device, such as an inertial navigation system, a gyroscope, etc. This application does not limit this.

[0032] A camera array refers to a plurality of cameras rigidly connected to one another, but each camera may have a different shooting direction. In the embodiment of the present application, the camera array includes a forward-view camera group and a rear-view camera group, each of which includes cameras with at least two focal lengths. The shooting direction of the forward-view camera group is consistent with the direction of travel of the mobile platform, while the shooting direction of the rear-view camera group is opposite to the direction of travel of the mobile platform.

[0033] In some embodiments, the controller determines at least one monitoring position within the test area based on the focal length range of the front-view camera group and the rear-view camera group; in the process of controlling the movement of the mobile platform, if it is monitored that the mobile platform has moved to the first monitoring position, the mobile platform is controlled to stop, and the front-view camera group and the rear-view camera group are controlled to take pictures; after taking the pictures, the mobile platform is controlled to move to the next monitoring position, and the above steps are repeated.

[0034] The location of the mobile platform can be recorded by manual marking, recording odometer readings, or utilizing a positioning system, and there are no limitations here. Furthermore, the markings at the monitoring points can be, for example, natural features of the monitoring points, or artificial collaborative markings can be set at the monitoring points, and there are no limitations here.

[0035] Step S502: During the second patrol, when the mobile platform travels to the second monitoring position of the area to be tested, the monitoring points in the first area to be tested are photographed by the camera of each focal length in the forward-looking camera group to obtain at least two third images; and the monitoring points in the second area to be tested are photographed by the camera of each focal length in the rear-view camera group to obtain at least two fourth images.

[0036] The second inspection process has the same steps as the first inspection process. However, considering that, in actual situations, the positions of the monitoring points in the first area to be inspected photographed by the front camera group and the monitoring points in the second area to be inspected photographed by the rear camera group during the first inspection process may be different from the positions of the monitoring points in the first area to be inspected photographed by the front camera group and the monitoring points in the second area to be inspected photographed by the rear camera group during the second inspection process, the first monitoring position and the second monitoring position may be different in the embodiment of the present application, thereby causing the deformation measurement device itself to generate six degrees of freedom motion.

[0037] In addition, the monitoring points in the first area to be measured and the monitoring points in the second area to be measured are set in the area to be measured according to the preset measurement specifications. For example, the monitoring points in the first area to be measured and the monitoring points in the second area to be measured can be set as settlement measurement points and / or horizontal displacement measurement points, that is, the set settlement measurement points and horizontal displacement measurement points can be the same point or different points. In addition, this application does not limit the position and shape of the monitoring points.

[0038] The first patrol process can be a baseline patrol or any patrol process from several historical patrols. The time interval between the second patrol process and the first patrol process is not limited in this application. By comparing the first and second patrol processes, the deformation amount of the monitoring point during the second patrol compared to the first patrol is determined.

[0039] Step S503: Perform multi-focal-length fusion calculation on the at least two first images to obtain a first front view image; perform multi-focal-length fusion calculation on the at least two second images to obtain a first rear view image; perform multi-focal-length fusion calculation on the at least two third images to obtain a second front view image; and perform multi-focal-length fusion calculation on the at least two fourth images to obtain a second rear view image.

[0040] Among them, the front-view camera group includes cameras with at least two focal lengths, so the images taken by the front-view camera group are multi-focal length images, and the same applies to the rear-view camera group. The depth of field and shooting distance of a single focal length camera are fixed. For example, a short-focus camera is suitable for close-range shooting (such as 0-30 meters), but for distant targets (such as 100 meters away), details will be lost due to too small an image. A telephoto camera is suitable for long-range shooting (such as more than 100), but close targets may be blurred due to too shallow a depth of field, and the field of view is narrow. According to the above characteristics, if multiple cameras with fixed focal lengths are used to shoot the same picture at the same time, and the multiple focal length images are subjected to multi-focal length fusion calculation, it is possible to simultaneously monitor close-range monitoring points and long-range monitoring points, while a single focal length camera needs to adjust its position multiple times to complete monitoring of the same range. Therefore, this application improves the monitoring range and patrol efficiency.

[0041] In some embodiments, the specific process of performing the multi-focal-segment fusion calculation on the at least two first images is as follows: Steps A1 to A5: Step A1: Select a target focal length from the at least two focal lengths, and project the focal length images of the at least two images except the target focal length to coordinates corresponding to the target focal length to obtain at least two geometrically aligned images.

[0042] In some embodiments, projecting the images of the other focal segments except the target focal segment in the at least two images to the coordinates corresponding to the target focal segment to obtain at least two geometrically aligned images includes: determining the monitoring points contained in the at least two first images as same-name point pairs, and obtaining a plurality of the same-name point pairs; calculating a homography matrix based on the plurality of the same-name point pairs; and projecting the images of the other focal segments except the target focal segment in the at least two first images to the coordinate system corresponding to the target focal segment based on the homography matrix.

[0043] The same-name point pairs refer to the set of image coordinates of monitoring points contained in each first image, for example, monitoring points The pairs of points with the same name are: ,in, For monitoring points Image coordinates in the short-focus image, For monitoring points Image coordinates in the mid-focus image, For monitoring points Image coordinates in the telephoto image.

[0044] For at least two first images, the ORB algorithm is used to extract monitoring points, and multiple pairs of points with the same name are obtained through bidirectional matching (such as FLANN matching + RANSAC to eliminate false matches).

[0045] Among them, a homography matrix is calculated based on the multiple pairs of same-name points, and the focal segment images other than the target focal segment in the at least two first images are projected onto the coordinate system corresponding to the target focal segment according to the homography matrix, including: respectively calculating the homography matrices of the focal segment images other than the target focal segment and the target focal segment image, and using the homography matrix to project the focal segment images other than the target focal segment onto the coordinate system of the target focal segment.

[0046] Among them, the homography matrix is used to describe the projection transformation relationship of the same plane under different viewing angles. The homography matrix can describe the correspondence between the pixel coordinates of the two images. This relationship can be expressed as:

[0047] in, and are homogeneous coordinates of the form [ , , 1] T , is the pixel coordinate system The coordinates below, is the pixel coordinate system The coordinates below, is the homography matrix.

[0048] For example, assuming the target focal length is short focal length, the medium focal length image should be solved first. With short-focus image The first homography matrix between , using the first homography matrix The mid-focus image Projected into the short-focus coordinate system, the geometrically aligned image is obtained, which is recorded as ; Similarly, calculate the telephoto image With short-focus image The second homography matrix between , using the second homography matrix Telephoto image Projected into the short-focus coordinate system, the geometrically aligned image is obtained, which is recorded as Finally, the images of the three focal lengths are aligned to the short focal length coordinate system to obtain the geometrically aligned images. 、 、 .

[0049] Specifically, for the first homography matrix , monitoring points in the short-focus image The monitoring point in the mid-focus image Constitute a pair of points with the same name, homography matrix It can be expressed as: , according to the homogeneous coordinate transformation relationship, the same-name points satisfy: , for each pair of points of the same name, can be transformed into two linear equations: , Arrange the above equation into matrix form .

[0050] in, is an unknown vector, Each row of corresponds to the coefficient of an equation. Points with the same name, The two lines are: .

[0051] Since each pair of points of the same name generates two equations, corresponding to and The constraints, and the homography matrix There are 9 unknown parameters, so, The point will generate 2 equations, forming 2 ×9 matrix . Therefore, the complete matrix for: .

[0052] right To solve, since usually Normalize so that =1, therefore, This is equivalent to having only 8 unknown parameters. In order to ensure that the equation has a solution, the number of pairs of points with the same name ≥4.

[0053] Similarly, for the second homography matrix The calculation process is the same as the first homography matrix The calculation process is consistent with that of , and this application will not go into details.

[0054] Among them, the homography matrix is used to project the images of other focal lengths onto the coordinate system of the target focal length. The specific process is to convert the coordinates of each pixel point in the images of other focal lengths into the coordinates in the coordinate system of the target focal length through the homography matrix to form a set aligned image.

[0055] Step A2: for each geometrically aligned image, assign pixel weights of different focal lengths according to the actual depth of the pixel in the geometrically aligned image, where the actual depth is the distance from the pixel to the optical center of the camera along the optical axis of the camera.

[0056] In some embodiments, the at least two focal lengths include long focus, medium focus, and short focus, the target focal length is short focus, and for each geometrically aligned image, assigning pixel weights of different focal lengths according to actual depths of pixels in the geometrically aligned image includes: The short focus weight is determined by the following formula (1): Formula (1) The telephoto weight is determined by the following formula (2): Formula (2) The center focal weight is determined by the following formula (3): Formula (3) in, is the short focal weight, is the telephoto weight, is the center focal weight, is the actual depth, is the weight decay rate; 、 The minimum and maximum actual depths for a single geometrically aligned image pixel.

[0057] For example, for the geometrically aligned image corresponding to the short-focus image, the short-focus weight = 0.8, the medium-focus weight = 0.2, and the long-focus weight = 0; for the geometrically aligned image corresponding to the long-focus image, the short-focus weight = 0, the medium-focus weight = 0.2, and the long-focus weight = 0.8; for the geometrically aligned image corresponding to the medium-focus image, the short-focus weight = 0.3, the medium-focus weight = 0.6, and the long-focus weight = 0.1.

[0058] Step A3: for each geometrically aligned image, construct Layer Gaussian pyramid, is a positive integer greater than or equal to 1.

[0059] Among them, each geometrically aligned image is constructed independently Layer Gaussian pyramid (such as =5), each layer is generated by: Layer 0 ( =0): original image, with the highest resolution and full frequency band information; No. layer( ≥1): for the first l- The layer is first Gaussian blurred and then downsampled to half the resolution; Finally, the pyramid sequence of each geometrically aligned image is output .

[0060] Step A4: From the lowest layer to the highest layer of the Gaussian pyramid, for pixels at the same position, fuse the pixel values of the corresponding layers of the at least two geometrically aligned images according to the pixel weights to obtain a fusion result of the layer.

[0061] Among them, for each layer of the pyramid, according to the pixel position Calculate the fusion result: , in, For the The fusion result of the layer, is the short focal weight, is the telephoto weight, is the center focal weight, is the actual depth, For the short focus image Coordinates in the layer pyramid The pixel value at The mid-focus image after geometric alignment is Coordinates in the layer pyramid The pixel value at The telephoto image after geometric alignment is Coordinates in the layer pyramid The pixel value at .

[0062] The final output is the fused pyramid .

[0063] Step A5: Perform pyramid inverse operations from the highest layer to the lowest layer of the Gaussian pyramid through upsampling and convolution operations, and reconstruct the fused image layer by layer to obtain the first front-view image.

[0064] For the fused pyramid , starting from the highest layer, reconstructing upwards layer by layer, and finally outputting a full-resolution fused image, namely the first front view image.

[0065] It should be noted that the process of performing multi-focal-length fusion calculation on at least two second images, at least two third images, and at least two fourth images is consistent with the process of performing multi-focal-length fusion calculation on at least two first images. For the specific process, please refer to the contents of steps A1 to A5 above, which will not be repeated here.

[0066] Step S504: Acquire a plurality of first monitoring points commonly included in the first front view image and the second front view image, and acquire a plurality of second monitoring points commonly included in the first rear view image and the second rear view image.

[0067] Step S505: Obtain a three-dimensional deformation of each of the plurality of first monitoring points based on the first front view image and the second front view image, and obtain a three-dimensional deformation of each of the plurality of second monitoring points based on the first rear view image and the second rear view image.

[0068] In some embodiments, obtaining the three-dimensional deformation of each of the plurality of first monitoring points according to the first front-view image and the second front-view image includes the following steps B1 to B3: Step B1, obtaining the first pixel coordinates of each first monitoring point among the multiple first monitoring points in the first forward-looking image, obtaining the second pixel coordinates of each first monitoring point in the second forward-looking image; and obtaining the world coordinates of each first monitoring point during the first patrol process.

[0069] Step B2: Obtain an intrinsic parameter matrix and an extrinsic parameter matrix of the target focal length camera, and obtain a scale factor of each first monitoring point.

[0070] Among them, the scale factor is the projection length of the distance from the target focal length camera to the first monitoring point in the direction of the optical axis of the target focal length camera; the intrinsic parameter matrix is associated with the angle between the optical axis of the target focal length camera and the horizontal plane, and the object plane resolution; the extrinsic parameter matrix is associated with the six-degree-of-freedom motion of the deformation measurement device, and the six-degree-of-freedom motion includes the settlement of the deformation measurement device, the roll change, the longitudinal displacement of the deformation measurement device along the driving direction, the horizontal displacement of the deformation measurement device, the pitch angle change, and the yaw change.

[0071] The object plane resolution is the magnification of the first front view image to the first monitoring point.

[0072] Step B3: determining the three-dimensional deformation of each first monitoring point according to the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point.

[0073] In some embodiments, determining the three-dimensional deformation amount of each first monitoring point according to the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point includes the following steps B31 to B34: Step B31: determining a pixel coordinate deformation amount according to the first pixel coordinate and the second pixel coordinate of each first monitoring point.

[0074] For example, for the first monitoring point , the homogeneous coordinates of the first pixel coordinates are , the homogeneous coordinates of the second pixel coordinates are , then the first monitoring point The pixel coordinate shape variable is , the calculation formula is as follows:

[0075] Step B32: establishing a first equation according to the pixel coordinate deformation variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor.

[0076] In some embodiments, the first equation is established based on the image coordinate deformation variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor, including: obtaining an imaging equation; calculating a six-degree-of-freedom change based on the first six-degree-of-freedom motion during the first patrol process and the second six-degree-of-freedom motion during the second patrol process; and establishing the first equation based on the imaging equation, the image coordinate deformation variable, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the six-degree-of-freedom change, and the scale factor.

[0077] Among them, based on the central view projection model, the following imaging equation can be established during the first deformation survey: Formula (4) Among them, the subscript Refers to the generalized camera station , subscript Refers to the status during the first inspection. Refers to the scale factor, its physical meaning is the generalized camera station at the first survey To the measuring point The projection length of the straight-line distance in the direction of the optical axis of the forward-looking camera. Refers to the measurement point during the first inspection Two-dimensional image coordinates expressed homogeneously in the image coordinate system. Refers to a generalized camera station The intrinsic parameter matrix of the front-view camera. For the same fixed-focus camera, once the intrinsic parameter matrix of the camera is calibrated, it can be considered that it will not change. Refers to the generalized camera station during the first survey The extrinsic parameter matrix of the front-view camera, Refers to the measurement point during the first inspection Three-dimensional coordinates expressed in homogeneous form in the world coordinate system.

[0078] For the current survey or any survey ( ), with generalized camera stations Observation points For example, we can also establish the imaging equation based on the central perspective projection model: Formula (5) Among them, the subscript Refers to the generalized camera station , subscript It refers to the Status during the next patrol. Refers to the scale factor, and its physical meaning is the Generalized camera station during the second survey To the measuring point The projection length of the straight-line distance in the direction of the optical axis of the forward-looking camera. It refers to the Secondary measurement point Two-dimensional image coordinates expressed homogeneously in the image coordinate system. Refers to a generalized camera station The intrinsic parameter matrix of the front-view camera, Refers to Generalized camera station during the second survey The extrinsic parameter matrix of the front-view camera, It refers to the Secondary measurement point Three-dimensional coordinates expressed in homogeneous form in the world coordinate system.

[0079] First, since the camera was not replaced during the multiple inspections, the camera's internal parameters remained unchanged. ; Alternatively, the intrinsic parameters of the camera can be obtained in advance through camera calibration, that is, the intrinsic parameters of the camera can be a known constant.

[0080] Secondly, let's make a reasonable assumption: even if the trajectory of the deformation measurement device during the first inspection is different from that during the current inspection, the deformation measurement device is in continuous motion, and during the motion, the onboard camera continuously captures images. Therefore, it is always possible to find one or more generalized camera stations in the current inspection. , and the generalized camera station during the first survey In other words: even if the generalized camera station of the current survey Compared with the generalized camera station during the initial survey There is a difference in position between them, that is, there is a certain distance, but this distance is smaller than the measuring point To the general camera station Therefore, we can assume that the scale factor is a constant. = = , is a constant.

[0081] Finally, as mentioned above, due to the different motion trajectories of the deformation measurement equipment, the generalized camera station during the first survey The generalized camera station during the current survey The position and attitude change occurs between the two, which can be represented by the change in six degrees of freedom: The change in position and posture expressed by the six degrees of freedom can be established. and The relationship between: Formula (6) Assume that the current patrol ( ) relative to the first patrol ( ), measuring point Cumulative deformation occurs, and the deformation can be expressed as: Formula (7) Further according to the imaging equation of the first survey and the The imaging equation of the second patrol measurement can be obtained as The shape variable is: Formula (8) Where, It is a measuring point The amount of pixel change produced in the image can be obtained through image positioning algorithm.

[0082] Substituting formula (6) and formula (7) into formula (8), we can get the first equation: Formula (9) In the formula, the initial patrol state ( ) Generalized camera station The transformation relationship between the world coordinate system (also known as the external parameter matrix ), and measuring points Coordinates in the world coordinate system , which can be obtained in the initial calibration stage. The amount of six degrees of freedom motion that occurs and measuring points Three-dimensional motion It is the quantity to be sought.

[0083] Step B33: establishing a first set of equations based on the first equations of the plurality of first monitoring points.

[0084] Step B34: Calculate the three-dimensional deformation variable of each first monitoring point according to the first set of equations.

[0085] It should be noted that the specific steps of obtaining the three-dimensional deformation variable of each second monitoring point in the multiple second monitoring points based on the first rear-view image and the second rear-view image are the same as the above-mentioned steps B1 to B3 of "obtaining the three-dimensional deformation variable of each first monitoring point in the multiple first monitoring points based on the first front-view image and the second front-view image", and this application will not repeat them.

[0086] It should be noted that when the forward-looking camera group shoots the test points in the first test area to obtain multiple first monitoring points and the rear-looking camera group shoots the test points in the second test area to obtain multiple second monitoring points, the obtained first set of equations and the second set of equations need to satisfy two basic constraints and one optimization constraint, among which the two basic constraints include a fixed constraint and a same-name constraint. The fixed constraint means that the camera array is fixed on the mobile platform, and all cameras included in the camera array in the deformation measurement device have the same six-degree-of-freedom motion at the same monitoring position. For example, the forward-looking camera group and the rear-looking camera group in the deformation measurement device have the same six-degree-of-freedom motion at the first monitoring position and the second monitoring position; the same-name constraint means that when the same first monitoring point or the same second monitoring point is photographed by different cameras, the vertical settlement of the same first monitoring point is the same physical quantity, and the vertical settlement of the same second monitoring point is the same physical quantity. The horizontal displacement of the same first monitoring point is the same physical quantity, and the horizontal displacement of the same second monitoring point is the same physical quantity. For example, during the same patrol process, the same second monitoring point is photographed by the rear-view camera group at a first monitoring position and by the front-view camera group at another first monitoring position. Then the vertical settlement of the same second monitoring point photographed by the front-view camera group and the rear-view camera group are the same, and the horizontal displacement is also the same. If the above-mentioned constraints of the same name are to be met, it can be optionally assumed that during a patrol process, that is, during the process of the moving platform from the starting point to the end point, all monitoring points in the to-be-tested area will no longer be deformed after being photographed by the cameras in the camera array. This application does not limit this. For example, the second monitoring point will no longer be deformed after being photographed by the rear-view camera group, so that the vertical settlement of the second monitoring point photographed by the front-view camera group is the same as the vertical settlement of the second monitoring point photographed by the rear-view camera group. The optimization constraint means that when the mobile platform is driving, the cameras in the camera array continuously and synchronously shoot, for example, the forward-looking camera group and the rear-looking camera group continuously and synchronously shoot. Then, the same first monitoring point or second monitoring point may be imaged multiple times and measured multiple times to obtain the vertical settlement and horizontal displacement of the same first monitoring point or second monitoring point, which can then be adjusted and optimized.

[0087] In an embodiment of the present application, combined with the actual engineering environment, any one or more of the above six degrees of freedom motion quantities can no longer be treated as unknown parameters, but as known parameters, thereby reducing the number of unknown parameters in the first equation. For example, if the mobile platform is a rail monitoring vehicle, the roll variation in the six degrees of freedom motion of the deformation measuring device will no longer be an unknown parameter, which can be understood as the influence of the roll variation is negligible, that is, at this time, only five degrees of freedom motion of the six degrees of freedom motion of the deformation measuring device are unknown parameters, thereby reducing the number of unknown parameters in the first equation; or, if a positioning device is installed on the mobile platform, the six degrees of freedom motion of the deformation measuring device can be obtained through the positioning device, that is, at this time, these three quantities will no longer be unknown parameters, but known parameters, that is, at this time, only three degrees of freedom motion of the six degrees of freedom motion of the deformation measuring device are unknown parameters, thereby reducing the number of unknown parameters in the first equation; or, if a posture measurement device is installed on the mobile platform, the sum of the six degrees of freedom motion of the deformation measuring device can be obtained through the posture measurement device, that is, at this time, the sum and these three quantities will no longer be unknown parameters, but known parameters, that is, at this time, only three degrees of freedom motion of the six degrees of freedom motion of the deformation measuring device are unknown parameters, thereby reducing the number of unknown parameters in the first equation. It should be noted that the degree of freedom motion in the six degrees of freedom motion is not limited here as a known parameter, or it is not limited whether the positioning device and the posture measurement device are installed on the mobile platform at the same time, or only the positioning device is installed, or only the posture measurement device is installed, etc.; at the same time, it should be clear that in addition to installing a positioning device, a posture measurement device, etc. on the mobile platform so that the corresponding degree of freedom motion in the six degrees of freedom motion is changed from an unknown parameter to a known parameter, other methods can also be used to make the corresponding degree of freedom motion in the six degrees of freedom motion changed from an unknown parameter to a known parameter, which will not be listed one by one in this application.

[0088] It can be seen that in the embodiment of the present application, during the first patrol process of the deformation measuring device, when the mobile platform travels to the first monitoring position of the area to be measured, the monitoring point in the first area to be measured in the first monitoring position is photographed by the camera of each focal length in the forward-looking camera group in the camera array, and at least two first images are obtained; the monitoring point in the second area to be measured in the second monitoring position is photographed by the camera of each focal length in the rear-looking camera group in the camera array, and at least two second images are obtained; wherein, the first area to be measured and the second area to be measured are arranged on both sides of the first monitoring position; during the second patrol process, when the mobile platform travels to the second monitoring position of the area to be measured, the monitoring point in the first area to be measured is photographed by the camera of each focal length in the forward-looking camera group, and at least two third images are obtained; the monitoring point in the second area to be measured is photographed by the camera of each focal length in the rear-looking camera group The method comprises the following steps: shooting the monitoring points within the area to obtain at least two fourth images; performing multi-focal section fusion calculation on the at least two first images to obtain a first front view image; performing multi-focal section fusion calculation on the at least two second images to obtain a first rear view image; performing multi-focal section fusion calculation on the at least two third images to obtain a second front view image; performing multi-focal section fusion calculation on the at least two fourth images to obtain a second rear view image; obtaining a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtaining a plurality of second monitoring points commonly contained in the first rear view image and the second rear view image; obtaining a three-dimensional deformation amount of each first monitoring point among the plurality of first monitoring points according to the first front view image and the second front view image, and obtaining a three-dimensional deformation amount of each second monitoring point among the plurality of second monitoring points according to the first rear view image and the second rear view image. It can be seen that in this application, by using a multi-focal-length camera to simultaneously obtain image data of the test points at different longitudinal depths, and fusing the image data of the multi-focal-lengths, a single monitoring position can be used to capture more test points, thereby increasing the single monitoring range. At the same time, the deformation measurement equipment does not need to stop and shoot point by point, which reduces time costs and improves monitoring efficiency.

[0089] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process of the method side. It is understandable that, in order to realize the above functions, the server includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] The embodiments of the present application can divide the server into functional units according to the above method examples. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into a processing module. The above integrated units can be implemented in the form of hardware or in the form of software program modules. It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, other division methods can be used.

[0091] In the case of integrated units, see Figure 6 , Figure 6 This is a functional unit structure block diagram of a multi-focal-section deformation measurement device provided in an embodiment of the present application, which is applied to a deformation measurement device. The deformation measurement device is provided with a camera array and a mobile platform. The camera array includes two camera groups, each camera group includes cameras of at least two focal lengths, and the two camera groups shoot in opposite directions. The multi-focal-section deformation measurement device 6 includes: The image acquisition unit 601, during the first patrol process, when the mobile platform travels to the first monitoring position of the area to be tested, the monitoring points in the first area to be tested in the first monitoring position are photographed by the camera of each focal length in the forward-looking camera group in the camera array to obtain at least two first images; the monitoring points in the second area to be tested in the second monitoring position are photographed by the camera of each focal length in the rear-looking camera group in the camera array to obtain at least two second images; wherein the first area to be tested and the second area to be tested are arranged on both sides of the first monitoring position; and, during the second patrol process, when the mobile platform travels to the second monitoring position of the area to be tested, the monitoring points in the first area to be tested are photographed by the camera of each focal length in the forward-looking camera group to obtain at least two third images; the monitoring points in the second area to be tested are photographed by the camera of each focal length in the rear-looking camera group to obtain at least two fourth images; The image processing unit 602 performs multi-focal-length fusion calculation on the at least two first images to obtain a first front view image; performs multi-focal-length fusion calculation on the at least two second images to obtain a first rear view image; performs multi-focal-length fusion calculation on the at least two third images to obtain a second front view image; performs multi-focal-length fusion calculation on the at least two fourth images to obtain a second rear view image; and obtains a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtains a plurality of second monitoring points commonly contained in the first rear view image and the second rear view image; and obtains a three-dimensional shape variable of each first monitoring point among the plurality of first monitoring points based on the first front view image and the second front view image, and obtains a three-dimensional shape variable of each second monitoring point among the plurality of second monitoring points based on the first rear view image and the second rear view image.

[0092] It can be seen that in the embodiment of the present application, during the first patrol process of the deformation measuring device, when the mobile platform travels to the first monitoring position of the area to be measured, the monitoring point in the first area to be measured in the first monitoring position is photographed by the camera of each focal length in the forward-looking camera group in the camera array, and at least two first images are obtained; the monitoring point in the second area to be measured in the second monitoring position is photographed by the camera of each focal length in the rear-looking camera group in the camera array, and at least two second images are obtained; wherein, the first area to be measured and the second area to be measured are arranged on both sides of the first monitoring position; during the second patrol process, when the mobile platform travels to the second monitoring position of the area to be measured, the monitoring point in the first area to be measured is photographed by the camera of each focal length in the forward-looking camera group, and at least two third images are obtained; the monitoring point in the second area to be measured is photographed by the camera of each focal length in the rear-looking camera group The method comprises the following steps: shooting the monitoring points within the area to obtain at least two fourth images; performing multi-focal section fusion calculation on the at least two first images to obtain a first front view image; performing multi-focal section fusion calculation on the at least two second images to obtain a first rear view image; performing multi-focal section fusion calculation on the at least two third images to obtain a second front view image; performing multi-focal section fusion calculation on the at least two fourth images to obtain a second rear view image; obtaining a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtaining a plurality of second monitoring points commonly contained in the first rear view image and the second rear view image; obtaining a three-dimensional deformation amount of each first monitoring point among the plurality of first monitoring points according to the first front view image and the second front view image, and obtaining a three-dimensional deformation amount of each second monitoring point among the plurality of second monitoring points according to the first rear view image and the second rear view image. It can be seen that in this application, by using a multi-focal-length camera to simultaneously obtain image data of the test points at different longitudinal depths, and fusing the image data of the multi-focal-lengths, a single monitoring position can be used to capture more test points, thereby increasing the single monitoring range. At the same time, the deformation measurement equipment does not need to stop and shoot point by point, which reduces time costs and improves monitoring efficiency.

[0093] In some embodiments, the specific process of performing the multi-focal segment fusion calculation on the at least two first images is as follows: selecting a target focal segment from the at least two focal segments, projecting the focal segment images other than the target focal segment in the at least two images to the coordinates corresponding to the target focal segment, and obtaining at least two geometrically aligned images; for each geometrically aligned image, assigning pixel weights of different focal segments according to the actual depth of the pixel points in the geometrically aligned image, where the actual depth is the distance from the pixel point to the optical center of the camera along the optical axis of the camera; and constructing, for each geometrically aligned image, Layer Gaussian pyramid, is a positive integer greater than or equal to 1; from the lowest layer to the highest layer of the Gaussian pyramid, the pixel values of the corresponding layers of the at least two geometrically aligned images are fused according to the pixel weights for pixels at the same position to obtain a fusion result of the layer; from the highest layer to the lowest layer of the Gaussian pyramid, an inverse pyramid operation is performed through upsampling and convolution operations to reconstruct the fused image layer by layer to obtain the first front-view image.

[0094] In some embodiments, the image processing unit 602 projects the other focal segment images except the target focal segment in the at least two images to the coordinates corresponding to the target focal segment to obtain at least two geometrically aligned images, including: determining the monitoring points contained in the at least two first images as same-name point pairs, and obtaining a plurality of same-name point pairs; calculating a homography matrix based on the plurality of same-name point pairs; and projecting the other focal segment images except the target focal segment in the at least two first images to the coordinate system corresponding to the target focal segment according to the homography matrix.

[0095] In some embodiments, the at least two focal lengths include long focus, medium focus, and short focus, and the target focal length is short focus. The image processing unit 602 assigns pixel weights of different focal lengths to each geometrically aligned image according to actual depths of pixels in the geometrically aligned image, including: The short focus weight is determined by the following formula (1): Formula (1) The telephoto weight is determined by the following formula (2): Formula (2) The center focal weight is determined by the following formula (3): Formula (3) in, is the short focal weight, is the telephoto weight, is the center focal weight, is the actual depth, is the weight decay rate; 、 The minimum and maximum actual depths for a single geometrically aligned image pixel.

[0096] In some embodiments, the image processing unit 602 obtains the three-dimensional deformation of each first monitoring point among the multiple first monitoring points based on the first front view image and the second front view image, including: obtaining the first pixel coordinates of each first monitoring point among the multiple first monitoring points in the first front view image, obtaining the second pixel coordinates of each first monitoring point in the second front view image; and obtaining the world coordinates of each first monitoring point during the first patrol process; obtaining the intrinsic parameter matrix and the extrinsic parameter matrix of the target focal length camera, and obtaining the scale factor of each first monitoring point; wherein the scale factor is the distance from the target focal length camera to the first monitoring point in the target focal length. The projection length of the focal length camera in the optical axis direction; the intrinsic parameter matrix is associated with the angle between the optical axis of the target focal length camera and the horizontal plane, and the object plane resolution; the extrinsic parameter matrix is associated with the six-degree-of-freedom motion of the deformation measurement device, and the six-degree-of-freedom motion includes the settlement of the deformation measurement device, the roll change, the longitudinal displacement of the deformation measurement device along the driving direction, the horizontal displacement of the deformation measurement device, the pitch angle change, and the yaw change; the three-dimensional deformation of each first monitoring point is determined according to the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point.

[0097] In some embodiments, the image processing unit 602 determines the three-dimensional shape variable of each first monitoring point based on the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point, including: determining the image coordinate shape variable based on the first image coordinates and the second image coordinates of each first monitoring point; establishing a first equation based on the image coordinate shape variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor; establishing a first group of equations based on the first equations of the multiple first monitoring points; and calculating the three-dimensional shape variable of each first monitoring point based on the first group of equations.

[0098] In some embodiments, the image processing unit 602 establishes a first equation based on the image coordinate deformation variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor, including: obtaining an imaging equation; calculating a six-degree-of-freedom change based on the first six-degree-of-freedom motion during the first patrol process and the second six-degree-of-freedom motion during the second patrol process; establishing the first equation based on the imaging equation, the image coordinate deformation variable, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the six-degree-of-freedom change, and the scale factor.

[0099] An embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of any possible embodiment method are implemented.

[0100] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0101] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0103] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0105] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program code.

[0106] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing related hardware. The program can be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0107] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-focal section deformation measurement method, characterized in that: The method is applied to a deformation measurement device, wherein the deformation measurement device is provided with a camera array and a mobile platform, wherein the camera array includes at least two camera groups, each camera group includes cameras with at least two focal lengths, and the two camera groups shoot in opposite directions; the method includes: During the first patrol, when the mobile platform travels to a first monitoring position in the area to be tested, a camera of each focal length in the forward-looking camera group in the camera array photographs a monitoring point in a first area to be tested in the first monitoring position to obtain at least two first images; and a camera of each focal length in the rear-looking camera group in the camera array photographs a monitoring point in a second area to be tested in the second monitoring position to obtain at least two second images; wherein the first area to be tested and the second area to be tested are arranged on both sides of the first monitoring position; During the second patrol, when the mobile platform travels to a second monitoring position of the area to be tested, a camera of each focal length in the forward-looking camera group is used to photograph a monitoring point in the first area to be tested, to obtain at least two third images; and a camera of each focal length in the rear-looking camera group is used to photograph a monitoring point in the second area to be tested, to obtain at least two fourth images; Performing a multi-focal-section fusion calculation on the at least two first images to obtain a first front-view image; performing a multi-focal-section fusion calculation on the at least two second images to obtain a first rear-view image; performing a multi-focal-section fusion calculation on the at least two third images to obtain a second front-view image; and performing a multi-focal-section fusion calculation on the at least two fourth images to obtain a second rear-view image. Acquire a plurality of first monitoring points commonly included in the first front view image and the second front view image, and acquire a plurality of second monitoring points commonly included in the first rear view image and the second rear view image; A three-dimensional deformation amount of each first monitoring point in the plurality of first monitoring points is obtained according to the first front view image and the second front view image, and a three-dimensional deformation amount of each second monitoring point in the plurality of second monitoring points is obtained according to the first rear view image and the second rear view image.

2. The method according to claim 1, characterized in that The specific process of performing the multi-focal-segment fusion calculation on the at least two first images is as follows: Selecting a target focal segment from the at least two focal segments, and projecting the images of the other focal segments of the at least two images except the target focal segment to coordinates corresponding to the target focal segment, to obtain at least two geometrically aligned images; For each geometrically aligned image, pixel weights of different focal lengths are assigned according to the actual depth of the pixel in the geometrically aligned image, where the actual depth is the distance from the pixel to the optical center of the camera along the optical axis of the camera. For each geometrically aligned image, construct Layer Gaussian pyramid, is a positive integer greater than or equal to 1; From the lowest layer to the highest layer of the Gaussian pyramid, for pixels at the same position, fuse the pixel values of the corresponding layers of the at least two geometrically aligned images according to the pixel weights to obtain a fusion result of the layer; From the highest layer to the lowest layer of the Gaussian pyramid, an inverse pyramid operation is performed through upsampling and convolution operations to reconstruct the fused image layer by layer to obtain the first front-view image.

3. The method according to claim 2, characterized in that The step of projecting the images of the other focal segments except the target focal segment in the at least two images to coordinates corresponding to the target focal segment to obtain at least two geometrically aligned images includes: Determine the monitoring points included in the at least two first images as point pairs with the same name, and obtain a plurality of the point pairs with the same name; Calculating a homography matrix based on the plurality of same-name point pairs; Projecting the other focal segment images except the target focal segment in the at least two first images into a coordinate system corresponding to the target focal segment according to the homography matrix.

4. The method according to claim 2, characterized in that The at least two focal lengths include long focus, medium focus, and short focus, the target focal length is short focus, and for each geometrically aligned image, allocating pixel weights of different focal lengths according to actual depths of pixels in the geometrically aligned image includes: The short focus weight is determined by the following formula (1): Formula (1) The telephoto weight is determined by the following formula (2): Formula (2) The center focal weight is determined by the following formula (3): Formula (3) in, is the short focal weight, is the telephoto weight, is the center focal weight, is the actual depth, is the weight decay rate; 、 The minimum and maximum actual depths for a single geometrically aligned image pixel.

5. The method according to claim 1, wherein The obtaining of the three-dimensional deformation amount of each first monitoring point among the plurality of first monitoring points according to the first front-view image and the second front-view image comprises: Obtaining a first pixel coordinate of each of the plurality of first monitoring points in the first forward-looking image, obtaining a second pixel coordinate of each of the first monitoring points in the second forward-looking image; and obtaining a world coordinate of each of the first monitoring points during the first survey process; Obtaining an intrinsic parameter matrix and an extrinsic parameter matrix of a target focal length camera, and obtaining a scale factor of each first monitoring point; wherein the scale factor is the projected length of the distance from the target focal length camera to the first monitoring point in the direction of the optical axis of the target focal length camera; the intrinsic parameter matrix is associated with the angle between the optical axis of the target focal length camera and the horizontal plane and the object plane resolution; the extrinsic parameter matrix is associated with the six-degree-of-freedom motion of the deformation measurement device, wherein the six-degree-of-freedom motion includes the settlement of the deformation measurement device, the roll change, the longitudinal displacement of the deformation measurement device along the driving direction, the horizontal displacement of the deformation measurement device, the pitch angle change, and the yaw change; The three-dimensional deformation amount of each first monitoring point is determined according to the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point.

6. The method according to claim 5, characterized in that The determining of the three-dimensional deformation amount of each first monitoring point according to the first pixel coordinates, the second pixel coordinates, the world coordinates, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor of each first monitoring point includes: determining an image coordinate deformation amount according to the first image coordinates and the second image coordinates of each first monitoring point; Establishing a first equation according to the image coordinate deformation variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor; Establishing a first set of equations according to the first equations of the plurality of first monitoring points; The three-dimensional deformation variable of each first monitoring point is calculated according to the first set of equations.

7. The method according to claim 6, characterized in that The step of establishing a first equation based on the image coordinate deformation variable of any first monitoring point, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the extrinsic parameter matrix, and the scale factor includes: Obtain imaging equations; Calculating a six-degree-of-freedom variation according to the first six-degree-of-freedom motion during the first patrol measurement and the second six-degree-of-freedom motion during the second patrol measurement; The first equation is established according to the imaging equation, the image coordinate deformation variable, the world coordinates of the first monitoring point, the intrinsic parameter matrix, the six-degree-of-freedom variation, and the scale factor.

8. A multi-focal section deformation measurement device, characterized in that: Applicable to deformation measurement equipment, the deformation measurement equipment is provided with a camera array and a mobile platform, the camera array includes two camera groups, each camera group includes cameras with at least two focal lengths, and the two camera groups shoot in opposite directions; the device includes: The image acquisition unit, during a first patrol process, when the mobile platform travels to a first monitoring position of the area to be tested, photographs the monitoring points within the first area to be tested in the first monitoring position by means of a camera of each focal length in the forward-looking camera group in the camera array, and obtains at least two first images; photographs the monitoring points within the second area to be tested in the second monitoring position by means of a camera of each focal length in the rear-looking camera group in the camera array, and obtains at least two second images; wherein the first area to be tested and the second area to be tested are arranged on both sides of the first monitoring position; and, during a second patrol process, when the mobile platform travels to the second monitoring position of the area to be tested, photographs the monitoring points within the first area to be tested by means of a camera of each focal length in the forward-looking camera group, and obtains at least two third images; photographs the monitoring points within the second area to be tested by means of a camera of each focal length in the rear-looking camera group, and obtains at least two fourth images; An image processing unit performs multi-focal-length fusion calculation on the at least two first images to obtain a first front view image; performs multi-focal-length fusion calculation on the at least two second images to obtain a first rear view image; performs multi-focal-length fusion calculation on the at least two third images to obtain a second front view image; performs multi-focal-length fusion calculation on the at least two fourth images to obtain a second rear view image; and obtains a plurality of first monitoring points commonly contained in the first front view image and the second front view image, and obtains a plurality of second monitoring points commonly contained in the first rear view image and the second rear view image; and obtains a three-dimensional shape variable of each first monitoring point among the plurality of first monitoring points based on the first front view image and the second front view image, and obtains a three-dimensional shape variable of each second monitoring point among the plurality of second monitoring points based on the first rear view image and the second rear view image.

9. A deformation measuring device, characterized in that: The deformation measurement device is provided with a controller, a camera array and a mobile platform, the camera array includes two camera groups, each camera group includes cameras with at least two focal lengths, and the shooting directions of the two camera groups are opposite, and the controller is used to execute the step instructions in any one of claims 1-7.

10. A controller, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the step instructions in the method according to any one of claims 1 to 7 are executed.

Images