Structural differential deformation monitoring method and related apparatus

Abstract

The application provides a structural difference deformation monitoring method and related device, the method comprises the following steps: determining a target patrol strategy; executing the target patrol strategy to obtain external parameter information, first coordinate information of a to-be-measured point in a world coordinate system and image information; determining a target difference deformation between a first structural member and a second structural member according to the external parameter information, the first coordinate information, the image information and a distribution of the to-be-measured point in a field of view of a forward-looking camera and a field of view of a rear-looking camera. The embodiment of the application can monitor the deformation of the to-be-measured point on the structural member by moving the mobile measurement device, which is beneficial to improving the measurement efficiency compared with manual measurement. In addition, the distribution of the to-be-measured point on the two structural members in the field of view of the forward-looking camera and the rear-looking camera is considered to determine the difference deformation between the two structural members, which is beneficial to improving the measurement accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical measurement, and particularly relates to a structural difference deformation monitoring method and a related device. BACKGROUND

[0002] The difference deformation of a civil structure refers to the difference in deformation between two structural members, for example, the amount of misalignment between two segments in a subway tunnel, or the amount of differential settlement between two piers of a bridge. At present, the difference deformation is mainly measured by manpower using a vernier caliper, a level and other tools. The worker discovers the structural member with a larger difference deformation by eyes and uses the aforementioned tools to measure whether the difference deformation exceeds a preset threshold. However, the difference deformation measurement by manpower has the following problems. On the one hand, the time cost of manpower consumption is large and the efficiency is poor for completing a round of patrol. On the other hand, the accuracy of the difference deformation measurement by the vernier caliper and other tools is low. SUMMARY

[0003] The embodiments of the present application provide a structural difference deformation monitoring method and a related device, which can monitor the deformation of a to-be-measured point on a structural member by moving a mobile measurement device for patrol measurement, which is beneficial to improve the measurement efficiency compared with manpower measurement, and beneficial to improve the measurement accuracy by considering the distribution of the to-be-measured points on two structural members in the fields of view of a front-view camera and a rear-view camera to determine the difference deformation between the two structural members.

[0004] In a first aspect, the embodiments of the present application provide a structural difference deformation monitoring method, applied to a processing device of a mobile measurement device, wherein the mobile measurement device comprises a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device and the processing device, the perception measurement module comprises a camera array, the camera array comprises a front-view camera and a rear-view camera, the processing device is connected with the front-view camera and the rear-view camera respectively, and the power supply device is connected with the perception measurement module and the processing device respectively; the method comprises the following steps.

[0005] An instruction for difference deformation monitoring of a target structure is acquired, wherein the target structure comprises a plurality of structural members;

[0006] According to the instruction for difference deformation monitoring, a target patrol strategy is determined, wherein the target patrol strategy is used to instruct the mobile measurement device to move to observe to-be-measured points on the target structure;

[0007] The target patrol strategy is executed to obtain extrinsic information, first coordinate information of the to-be-measured points in a world coordinate system and image information;

[0008] According to the extrinsic parameter information, the first coordinate information and the image information, and a distribution of the to-be-measured points in a field of view of the front-view camera and a field of view of the rear-view camera, a target difference deformation between the first structural member and the second structural member is determined, the first structural member and the second structural member being any two of the plurality of structural members.

[0009] In one possible example, the determining of the target difference deformation between the first structural member and the second structural member according to the extrinsic parameter information, the first coordinate information and the image information, and the distribution of the to-be-measured points in the field of view of the front-view camera and the field of view of the rear-view camera includes:

[0010] It is judged whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position.

[0011] If it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position, it is judged whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position.

[0012] If it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position, a first difference deformation measurement equation is determined, and a target difference deformation between the first structural member and the second structural member is determined according to the first difference deformation measurement equation.

[0013] If it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position, a second difference deformation measurement equation is determined, and a target difference deformation between the first structural member and the second structural member is determined according to the second difference deformation measurement equation.

[0014] If it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position, it is judged whether the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position.

[0015] If it is determined that the first to-be-measured point on the first structural member at the current monitoring position of the mobile platform is in the field of view of the front-view camera and the second to-be-measured point on the second structural member at the last monitoring position of the mobile platform is in the field of view of the rear-view camera, a reference to-be-measured point is determined, a third difference deformation measurement equation associated with the reference to-be-measured point is determined, and a target difference deformation amount between the first structural member and the second structural member is determined according to the third difference deformation measurement equation.

[0016] In one possible example, the determining of the reference to-be-measured point comprises:

[0017] determining a field of view overlap region between the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform;

[0018] determining a third to-be-measured point in the field of view overlap region;

[0019] determining the third to-be-measured point as the reference to-be-measured point.

[0020] In one possible example, the first difference deformation measurement equation comprises a first difference settlement measurement equation and a first horizontal difference deformation measurement equation, and when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the front-view camera, the determining of the first difference deformation measurement equation comprises:

[0021] determining a first object plane resolution at which the camera captures the first to-be-measured point;

[0022] determining a second object plane resolution at which the camera captures the second to-be-measured point;

[0023] determining, according to the image information, a first vertical pixel change amount and a first horizontal pixel change amount of the first to-be-measured point in the camera image, and determining, according to the image information, a second vertical pixel change amount and a second horizontal pixel change amount of the second to-be-measured point in the camera image;

[0024] determining, according to the first coordinate information, a third distance between the first to-be-measured point and the camera array and a fourth distance between the second to-be-measured point and the camera array, respectively;

[0025] determining a first included angle between the optical axis of the front-view camera and the horizontal plane;

[0026] determining a change amount of the pitch angle, a change amount of the yaw angle, and a change amount of the roll angle of the camera array;

[0027] determine the second difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch angle variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle.

[0028] determine the second horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

[0029] In one possible example, the second difference deformation measurement equation includes a second difference settlement measurement equation and a second horizontal difference deformation measurement equation; and the determining the second difference deformation measurement equation includes:

[0030] determine a longitudinal variation of the camera array;

[0031] determine a second included angle between the rear-view camera optical axis and the horizontal plane;

[0032] determine the second difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch angle variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle.

[0033] determine the second horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

[0034] In one possible example, the third difference deformation measurement equation includes a second difference settlement measurement equation and a second horizontal difference deformation measurement equation; and the determining the third difference deformation measurement equation associated with the reference measurement point includes:

[0035] determine a third object plane resolution at which the camera captures the reference measurement point;

[0036] determine a third vertical pixel variation and a third horizontal pixel variation of the reference measurement point in the camera image respectively according to the image information;

[0037] determine a fifth distance between the reference measurement point and the camera array according to the first coordinate information;

[0038] determine the third difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third vertical pixel variation, the third distance, the fourth distance, the fifth distance, the pitch variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle;

[0039] determine the third horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third horizontal pixel variation, the third distance, the fourth distance, the fifth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

[0040] In one possible example, after the target patrol strategy is executed to obtain the extrinsic parameter information, the first coordinate information of the to-be-measured point in the world coordinate system and the image information, the method further comprises:

[0041] determining a first measurement coordinate system of the current patrol camera array;

[0042] determining a pixel variation of each to-be-measured point in the camera image according to the image information;

[0043] creating a first imaging equation of the initial patrol to-be-measured point according to the central perspective projection model;

[0044] creating a second imaging equation of the current patrol to-be-measured point according to the central perspective projection model;

[0045] determining a to-be-measured point deformation equation according to the first imaging equation and the second imaging equation;

[0046] bringing the pixel variation, the extrinsic parameter information, the intrinsic parameter information and the first coordinate information into the to-be-measured point deformation equation to obtain a deformation amount of each to-be-measured point;

[0047] determining a target difference deformation amount between the first structure and the second structure according to the deformation amount of the to-be-measured point.

[0048] In one possible example, the determination of the first measurement coordinate system of the current patrol camera array comprises:

[0049] determining a second measurement coordinate system of the initial patrol camera array;

[0050] determining a six-degree-of-freedom variation between the initial patrol camera array and the current patrol camera array;

[0051] According to the second measurement coordinate system and the six-degree-of-freedom variation, the first measurement coordinate system is determined.

[0052] In a second aspect, an embodiment of the present application provides a structural difference deformation monitoring device, applied to a processing device of a mobile measurement device, the mobile measurement device comprising a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device, and the processing device, the perception measurement module comprising a camera array, the camera array comprising a front-view camera and a rear-view camera, the processing device being connected to the front-view camera and the rear-view camera respectively, and the power supply device being connected to the perception measurement module and the processing device respectively; the structural difference deformation monitoring device comprising an acquisition unit, a determination unit, and an execution unit; wherein,

[0053] The acquisition unit is configured to acquire a difference deformation monitoring instruction for a target structure, the target structure comprising a plurality of structural members.

[0054] The determination unit is configured to determine a target patrol strategy according to the difference deformation monitoring instruction, the target patrol strategy being used to instruct the mobile measurement device to move to observe a to-be-measured point on the target structure.

[0055] The execution unit is configured to execute the target patrol strategy to obtain extrinsic information, first coordinate information of the to-be-measured point in a world coordinate system, and image information.

[0056] The determination unit is further configured to determine a target difference deformation between a first structural member and a second structural member according to the extrinsic information, the first coordinate information, and the image information, and a distribution of the to-be-measured point in a field of view of the front-view camera and a field of view of the rear-view camera, the first structural member and the second structural member being any two structural members in the plurality of structural members.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, the one or more programs being stored in the memory and configured to be executed by the processor, and the program comprising instructions for executing the steps in the first aspect of the embodiments of the present application.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, having a computer program / instruction stored thereon, the computer program / instruction being executed by a processor to implement the steps in the first aspect of the embodiments of the present application.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0060] It can be seen that, in the embodiments of the present application, the processing device of the mobile measurement device first acquires the differential deformation monitoring instruction for the target structure, the target structure includes a plurality of structural members, then determines the target patrol strategy according to the differential deformation monitoring instruction, the target patrol strategy is used to instruct the mobile measurement device to move to observe the to-be-measured points on the target structure, and then the target patrol strategy is executed to obtain the extrinsic information, the first coordinate information of the to-be-measured points in the world coordinate system and the image information, and further, according to the extrinsic information, the first coordinate information and the image information, the distribution of the to-be-measured points in the field of view of the front-view camera and the field of view of the rear-view camera, the target differential deformation between the first structural member and the second structural member is determined, and the first structural member and the second structural member are any two structural members in the plurality of structural members. The deformation of the to-be-measured points on the structural member can be monitored by the mobile measurement device moving and patrolling, compared with manual measurement, which is beneficial to improve the measurement efficiency, and considering the distribution of the to-be-measured points on the two structural members in the field of view of the front-view camera and the rear-view camera, the differential deformation between the two structural members is determined, which is beneficial to improve the accuracy of measurement. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0062] Figure 1 is a schematic diagram of the architecture of a mobile measurement device provided by an embodiment of the present application;

[0063] Figure 2 is a flowchart of a structural differential deformation monitoring method provided by an embodiment of the present application;

[0064] Figure 3 is a schematic diagram of a monitoring route of a mobile measurement device provided by an embodiment of the present application;

[0065] Figure 4 is a schematic diagram of the distribution of to-be-measured points in a field of view provided by an embodiment of the present application;

[0066] Figure 5 is another schematic diagram of the distribution of to-be-measured points in a field of view provided by an embodiment of the present application;

[0067] Figure 6 is another distribution diagram of a to-be-measured point in a field of view provided by an embodiment of the present application;

[0068] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present application;

[0069] Figure 8 is a functional unit composition block diagram of a structure difference deformation monitoring device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0071] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0072] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined.

[0073] In the embodiments of the present application, "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships. For example, A and / or B can represent the following three cases: A exists alone; A and B exist simultaneously; B exists alone. Wherein, A and B can be singular or plural.

[0074] In the embodiments of the present application, the symbol " / " can represent that the associated objects before and after it are in an "or" relationship. In addition, the symbol " / " can also represent the division sign, that is, to perform division operation. For example, A / B can represent A divided by B.

[0075] In the embodiments of the present application, "at least one" or similar expressions refer to any combination of the items, including any combination of single item or multiple items, refer to one or more, and multiple refers to two or more. For example, at least one of a, b or c can represent the following seven cases: a, b, c, a and b, a and c, b and c, a, b and c. Wherein, each of a, b, c can be an element or a set containing one or more elements.

[0076] In the embodiments of the present application, "equal to" can be used with greater than, which is applicable to the technical solutions adopted when greater than, or can be used with less than, which is applicable to the technical solutions adopted when less than. When equal to is used with greater than, it is not used with less than; when equal to is used with less than, it is not used with greater than.

[0077] In order to better understand the scheme of the embodiments of the present application, the electronic device, related concepts and background that may be involved in the embodiments of the present application are introduced first.

[0078] The electronic device related to the embodiments of the present application can include various handheld devices with wireless communication function, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to wireless modems, and various forms of user equipment (UE), mobile stations (MS), terminal devices, etc. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

[0079] Please refer to Figure 1 , Figure 1 is a schematic diagram of the architecture of a mobile measurement device provided by the embodiments of the present application. As Figure 1 shown, the mobile measurement device 1 includes a mobile platform 10, a perception measurement module 20 arranged on the mobile platform 10, a power supply device 30 and a processing device 40.

[0080] Among them, the mobile platform 10 can drive the perception measurement module 20, the power supply device 30 and the processing device 40 arranged thereon to observe the to-be-measured points on the target structure according to a certain patrol measurement route.

[0081] Among them, the form of the mobile platform 10 is not limited, for example: it can be a mobile monitoring vehicle, a mobile aircraft, etc.

[0082] Among them, the perception measurement module 20 includes a camera array 21, the camera array 21 includes a front-view camera 22 and a rear-view camera 23, and the processing device 40 is connected to the front-view camera 22 and the rear-view camera 23 respectively.

[0083] The power supply device 30 is connected to the front-view camera 22, the rear-view camera 23 and the processing device 40 of the perception measurement module 20 respectively, and is configured to supply power to the front-view camera 22, the rear-view camera 23 and the processing device 40, so that the front-view camera 22 and the rear-view camera 23 can capture images of the to-be-measured points, and the processing device 40 can analyze the captured images in real time to obtain the deformation measurement result of the to-be-measured points.

[0084] Optionally, the power supply device 30 is connected to the power assembly of the mobile platform 10, and is configured to supply power to the power assembly, so that the power assembly can drive the mobile platform 10 to move.

[0085] Optionally, the processing device 40 can be a computer or other image analysis device.

[0086] Optionally, the field of view of the front-view camera 22 and the field of view of the rear-view camera 23 in the camera array 21 are opposite, and the front-view camera 22 and the rear-view camera 23 are fixedly connected to each other.

[0087] Optionally, the model and focal length of the cameras in the camera array 21 are not limited.

[0088] Optionally, the target structure can be a subway tunnel, a highway tunnel or a bridge.

[0089] In one possible example, the processing device 40 of the mobile measurement device 1 first acquires a differential deformation monitoring instruction for a target structure, the target structure includes a plurality of structural members, then determines a target patrol strategy according to the differential deformation monitoring instruction, the target patrol strategy is used to instruct the mobile measurement device 1 to move to observe the to-be-measured points on the target structure, and then executes the target patrol strategy to obtain the extrinsic information, the first coordinate information of the to-be-measured points in the world coordinate system and the image information, and further, according to the extrinsic information, the first coordinate information and the image information, and the distribution of the to-be-measured points in the field of view of the front-view camera 22 and the field of view of the rear-view camera 23, determines a target differential deformation between a first structural member and a second structural member, the first structural member and the second structural member are any two structural members in the plurality of structural members. The deformation of the to-be-measured points on the structural members can be monitored by moving the mobile measurement device 1 to patrol, compared with manual measurement, the measurement efficiency can be improved, and the differential deformation between the two structural members can be determined by considering the distribution of the to-be-measured points on the two structural members in the field of view of the front-view camera 22 and the rear-view camera 23, so that the measurement accuracy can be improved.

[0090] Please refer to Figure 2 , Figure 2is a flowchart of a structural difference deformation monitoring method provided by an embodiment of the present application, applied to a processing device of a mobile measurement device, the mobile measurement device comprising a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device, and the processing device, the perception measurement module comprising a camera array, the camera array comprising a front-view camera and a rear-view camera, the processing device being connected to the front-view camera and the rear-view camera respectively, and the power supply device being connected to the perception measurement module and the processing device respectively, and the method comprising:

[0091] In step S201, a difference deformation monitoring instruction for a target structure is acquired, the target structure comprising a plurality of structural members.

[0092] Wherein, the target structure can be a subway tunnel, a highway tunnel, or a bridge, which is not limited herein. When the target structure is a tunnel, the structural member can be a pipe segment, the tunnel comprising a plurality of pipe segments spliced together, and a target is arranged on the pipe segment in advance as a to-be-measured point, and one or more targets can be arranged on a single pipe segment. When the target structure is a bridge, the structural member can be a bridge pier, the bridge comprising a plurality of bridge piers, and a target can be arranged on the bridge pier in advance as a to-be-measured point, and one or more targets can be arranged on a single bridge pier.

[0093] Wherein, the front-view camera is used to shoot the target on the structural member in front of the moving direction of the mobile platform, and the rear-view camera is used to shoot the target on the structural member in the direction opposite to the moving direction of the mobile platform.

[0094] In step S202, a target patrol strategy is determined according to the difference deformation monitoring instruction, the target patrol strategy being used to instruct the mobile measurement device to move to observe the to-be-measured points on the target structure.

[0095] Wherein, the target patrol strategy can be that the mobile measurement device moves according to a target patrol route multiple times to observe the to-be-measured points on the plurality of structural members of the target structure multiple times, the target patrol route comprising a starting point, an ending point, a plurality of monitoring positions, and a plurality of to-be-measured points, and the positions of the plurality of monitoring positions and the plurality of to-be-measured points are not limited. When the mobile measurement device moves to each monitoring position, the camera array synchronously shoots the to-be-measured points, specifically, the front-view camera shoots the to-be-measured points on the structural member within a certain range in front of the mobile measurement device, and the rear-view camera shoots the to-be-measured points on the structural member within a certain range behind the mobile measurement device.

[0096] Please refer to Figure 3 , Figure 3 is a schematic diagram of a monitoring route of a mobile measurement device provided by an embodiment of the present application, a tunnel mobile monitoring device moving on the monitoring route, the tunnel mobile monitoring device being provided with a front-view camera and a rear-view camera, and the monitoring route comprising a starting point, an ending point, to-be-measured points (black dot positions in the figure), and monitoring positions (white dot positions in the figure).

[0097] The target patrol strategy comprises camera calibration to obtain the extrinsic information and intrinsic information of the front-view camera and the rear-view camera respectively.

[0098] In step S203, the target patrol strategy is executed to obtain the extrinsic information, the first coordinate information of the to-be-measured point in the world coordinate system, and the image information.

[0099] The image information comprises historical images captured in historical patrol and current images captured in the current patrol.

[0100] Since the camera is not replaced in the multiple patrols, the intrinsic parameters of the camera remain unchanged, and the intrinsic information of the front-view camera and the rear-view camera can be obtained through camera calibration in advance. Meanwhile, the conversion relationship between the coordinate system of the front-view camera and the world coordinate system, i.e., the extrinsic information of the front-view camera, can be calibrated in the initial patrol, and the conversion relationship between the coordinate system of the rear-view camera and the world coordinate system, i.e., the extrinsic information of the rear-view camera, can be calibrated in the initial patrol.

[0101] In the initial calibration stage, the coordinates of each to-be-measured point in the world coordinate system can also be calibrated to obtain the first coordinate information.

[0102] In step S204, according to the extrinsic information, the first coordinate information, and the image information, and the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera, a target difference deformation between the first structural member and the second structural member is determined, the first structural member and the second structural member being any two structural members in the multiple structural members.

[0103] The image information can be used to determine the pixel change of the to-be-measured point in the current image of the current monitoring compared with the historical image of the initial monitoring or a historical monitoring through a positioning algorithm.

[0104] The target difference deformation is used to represent the difference between the vertical deformation of the first to-be-measured point on the first structural member and the vertical deformation of the second to-be-measured point on the second structural member.

[0105] Considering the observation difference of the mobile measurement device at different monitoring positions and the difference between the extrinsic information of the front-view camera and the extrinsic information of the rear-view camera, to determine the target difference deformation between the first structural member and the second structural member, it is necessary to consider whether the first to-be-measured point and the second to-be-measured point can be simultaneously observed at the same monitoring position, and whether they can be observed in the field of view of the same camera when observed at the same monitoring position, i.e., it is necessary to consider the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera.

[0106] It can be seen that in the embodiments of the present application, the processing device of the mobile measurement device first acquires the differential deformation monitoring instruction for the target structure, the target structure includes a plurality of structural members, then according to the differential deformation monitoring instruction, the target patrol strategy is determined, the target patrol strategy is used to instruct the mobile measurement device to move to observe the to-be-measured points on the target structure, then the target patrol strategy is executed to obtain the extrinsic parameter information, the first coordinate information of the to-be-measured points in the world coordinate system and the image information, further, according to the extrinsic parameter information, the first coordinate information and the image information, the distribution of the to-be-measured points in the field of view of the front-view camera and the field of view of the rear-view camera, the target differential deformation between the first structural member and the second structural member is determined, the first structural member and the second structural member are any two structural members in the plurality of structural members. The deformation of the to-be-measured points on the structural member can be monitored by the mobile measurement device moving and patrolling, compared with manual measurement, which is beneficial to improve the measurement efficiency, and considering the distribution of the to-be-measured points on the two structural members in the field of view of the front-view camera and the rear-view camera, the differential deformation between the two structural members is determined, which is beneficial to improve the accuracy of measurement.

[0107] In one possible example, in the aspect of determining the target difference deformation amount between the first structural member and the second structural member according to the extrinsic information, the first coordinate information and the image information, the distribution of the to-be-measured points in the field of view of the front-view camera and the field of view of the rear-view camera, the method comprises: judging whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera, then judging whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera, then determining a first difference deformation measurement equation, and determining the target difference deformation amount between the first structural member and the second structural member according to the first difference deformation measurement equation; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are not simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera, then determining a second difference deformation measurement equation, and determining the target difference deformation amount between the first structural member and the second structural member according to the second difference deformation measurement equation; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member of the mobile platform at the current monitoring position are not simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera, then judging whether the first to-be-measured point on the first structural member of the mobile platform at the current monitoring position is in the field of view of the front-view camera and whether the second to-be-measured point on the second structural member of the mobile platform at the previous monitoring position is in the field of view of the rear-view camera; if it is judged that the first to-be-measured point on the first structural member of the mobile platform at the current monitoring position is in the field of view of the front-view camera and the second to-be-measured point on the second structural member of the mobile platform at the previous monitoring position is in the field of view of the rear-view camera, then determining a reference to-be-measured point, determining a third difference deformation measurement equation associated with the reference to-be-measured point, and determining the target difference deformation amount between the first structural member and the second structural member according to the third difference deformation measurement equation.

[0108] The distribution of the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member in the field of view of the front-view camera and the field of view of the rear-view camera includes the following three cases: in a first distribution case, the mobile measuring device can simultaneously observe the first to-be-measured point and the second to-be-measured point at the same monitoring position, and when the mobile measuring device is at the current monitoring position, the first to-be-measured point and the second to-be-measured point are in the field of view of the same camera, specifically, the first to-be-measured point and the second to-be-measured point can be in the field of view of the front-view camera, or the first to-be-measured point and the second to-be-measured point can be in the field of view of the rear-view camera. For example, refer to Figure 4 , Figure 4 which is a distribution diagram of to-be-measured points in a field of view provided by an embodiment of the present application, Figure 4 in which the mobile platform is at the current monitoring position, and the first to-be-measured point and the second to-be-measured point are in the field of view of the front-view camera. In a second distribution case, the mobile measuring device can simultaneously observe the first to-be-measured point and the second to-be-measured point at the same monitoring position, and when the mobile measuring device is at the current monitoring position, the first to-be-measured point and the second to-be-measured point are in the fields of view of different cameras, specifically, the first to-be-measured point is in the field of view of the rear-view camera, and the second to-be-measured point is in the field of view of the front-view camera. For example, refer to Figure 5 , Figure 5 which is another distribution diagram of to-be-measured points in a field of view provided by an embodiment of the present application, Figure 5 in which the mobile platform is at the current monitoring position, the first to-be-measured point is in the field of view of the rear-view camera, and the second to-be-measured point is in the field of view of the front-view camera. In a third distribution case, the mobile measuring device cannot simultaneously observe the first to-be-measured point and the second to-be-measured point at the same monitoring position, and the mobile measuring device needs to observe the second to-be-measured point and the first to-be-measured point at the current monitoring position and the previous monitoring position, respectively, specifically, when the mobile measuring device is at the previous monitoring position, the first to-be-measured point is in the field of view of the rear-view camera, and when the mobile measuring device is at the current monitoring position, the second to-be-measured point is in the field of view of the front-view camera. For example, refer to Figure 6 , Figure 6 which is still another distribution diagram of to-be-measured points in a field of view provided by an embodiment of the present application, in which the mobile platform is at the current monitoring position, the second to-be-measured point is in the field of view of the front-view camera, and the mobile platform is at the previous monitoring position, the first to-be-measured point is in the field of view of the rear-view camera.

[0109] The first to-be-measured point of the first structural member at the current monitoring position of the mobile platform is in the field of view of the front-view camera, and the second to-be-measured point of the second structural member at the last monitoring position of the mobile platform is in the field of view of the rear-view camera. At this time, since the mobile platform is observing the first to-be-measured point and the second to-be-measured point at different monitoring positions and different camera fields of view, the camera field of view needs to be successively transmitted to establish a connection between the first to-be-measured point and the second to-be-measured point, that is, to establish a connection between the first to-be-measured point and the second to-be-measured point by referring to the to-be-measured point, and to determine a third difference deformation measurement equation associated with the reference to-be-measured point.

[0110] It can be seen that, in the present example, the processing device can consider the distribution of the to-be-measured points on the two structural members in the fields of view of the front-view camera and the rear-view camera to determine the difference deformation between the two structural members, which is beneficial to improve the accuracy of measurement.

[0111] In one possible example, in the determination of the reference to-be-measured point, the method comprises: determining a field of view overlap region between the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform; determining a third to-be-measured point in the field of view overlap region; and taking the third to-be-measured point as the reference to-be-measured point.

[0112] The field of view overlap region between the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform, that is, the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform cover one or more same to-be-measured points. Figure 6 , Figure 6 The field of view overlap region between the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform includes three third to-be-measured points.

[0113] If the number of the third to-be-measured points is one, the third to-be-measured point is taken as the reference to-be-measured point, and if the number of the third to-be-measured points is more than one, any one of the third to-be-measured points is taken as the reference to-be-measured point.

[0114] Optionally, if the first to-be-measured point is not the to-be-measured point observed at the last monitoring position, a plurality of fourth to-be-measured points between the current monitoring position of the mobile platform and the monitoring position of the mobile platform for observing the first to-be-measured point are determined, and any one of the fourth to-be-measured points is taken as the reference to-be-measured point.

[0115] It can be seen that in the present example, the field of view overlap region between the field of view of the rear-view camera when the mobile platform is at the current monitoring position and the field of view of the front-view camera when the mobile platform is at the last monitoring position can be determined first, and a reference to-be-measured point in the field of view overlap region is selected as an intermediate variable for camera field of view succession transmission, thereby establishing a connection between the first to-be-measured point and the second to-be-measured point, and the accuracy of measurement is improved.

[0116] In one possible example, the first differential deformation measurement equation includes a first differential settlement measurement equation and a first horizontal differential deformation measurement equation, and in the aspect of determining the first differential deformation measurement equation, the method includes: determining a first object plane resolution of the camera shooting the first to-be-measured point; determining a second object plane resolution of the camera shooting the second to-be-measured point; determining a first vertical pixel change amount and a first horizontal pixel change amount of the first to-be-measured point in the camera image according to the image information, and determining a second vertical pixel change amount and a second horizontal pixel change amount of the second to-be-measured point in the camera image according to the image information; determining a third distance between the first to-be-measured point and the camera array and a fourth distance between the second to-be-measured point and the camera array according to the first coordinate information; determining a first included angle between the front-view camera optical axis and the horizontal plane; determining a pitch change amount, a yaw change amount and a roll change amount of the camera array; determining the first differential settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel change amount, the second vertical pixel change amount, the third distance, the fourth distance, the pitch change amount, the yaw change amount and the first included angle; and determining the first horizontal differential deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel change amount, the second horizontal pixel change amount, the third distance, the fourth distance, the yaw change amount, the roll change amount and the first included angle.

[0117] In the first differential deformation measurement equation, the first to-be-measured point and the second to-be-measured point can be simultaneously in the field of view of the front-view camera or the rear-view camera.

[0118] In the first differential deformation measurement equation, the object plane resolution of the front-view camera and the object plane resolution of the rear-view camera can be determined respectively, and the object plane resolution = focal length / object distance.

[0119] In the first differential deformation measurement equation, when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the same camera, the first differential settlement measurement equation is as follows:

[0120]

[0121] In the first differential deformation measurement equation, when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the same camera, is the first vertical pixel change amount of the first to-be-measured point in the camera image, a second vertical pixel variation of the second test point in the camera image, a first object plane resolution of the first test point captured by the camera, a second object plane resolution of the second test point captured by the camera, a monitoring position captured by the camera, for example, if the patrol route includes 10 monitoring positions, they can be defined in turn as , a vertical settlement of the first test point in the vertical direction, a vertical settlement of the second test point in the vertical direction, a settlement of the camera array in the vertical direction, a longitudinal displacement of the camera array, an angle between the optical axis of the camera capturing the first test point and the horizontal plane, an angle between the optical axis of the camera capturing the second test point and the horizontal plane, a third distance between the first test point and the monitoring position of the camera array , a fourth distance between the second test point and the monitoring position of the camera array , a yaw variation of the camera array, a pitch angle variation of the camera array, a difference settlement in the vertical direction between the first test point and the second test point, which can be used to represent the difference deformation of the first structure and the second structure in the vertical direction. Since the first test point and the second test point are in the field of view of the same camera at the same time, .

[0122] When the first test point and the second test point are in the field of view of the same camera at the same time, the first horizontal difference deformation measurement equation is as follows:

[0123]

[0124] When the first test point and the second test point are in the field of view of the same camera at the same time, a roll variation of the camera array, a first horizontal pixel variation of the first test point in the camera image, a second horizontal pixel variation of the second test point in the camera image, a displacement of the first test point in the horizontal direction, a displacement of the second test point in the horizontal direction, a displacement of the camera array in the horizontal direction, ​The difference deformation between the first to-be-measured point and the second to-be-measured point in the horizontal direction can be used to represent the difference deformation of the first structure and the second structure in the horizontal direction.

[0125] In an example, when the first to-be-measured point and the second to-be-measured point are both in the field of view of the front-view camera, The object plane resolution of the front-view camera for shooting the first to-be-measured point, The object plane resolution of the front-view camera for shooting the second to-be-measured point, The first angle between the optical axis of the front-view camera and the horizontal plane, The first angle between the optical axis of the front-view camera and the horizontal plane. When the first to-be-measured point and the second to-be-measured point are both in the field of view of the rear-view camera, The object plane resolution of the rear-view camera for shooting the first to-be-measured point, The object plane resolution of the rear-view camera for shooting the second to-be-measured point, The second angle between the optical axis of the rear-view camera and the horizontal plane, The second angle between the optical axis of the rear-view camera and the horizontal plane.

[0126] It can be seen that, in the example, the processing device considers the change of the camera pose in the patrol process when determining the difference deformation between the first to-be-measured point and the second to-be-measured point, which is beneficial to improve the accuracy of measurement.

[0127] In one possible example, the second difference deformation measurement equation includes a second difference settlement measurement equation and a second horizontal difference deformation measurement equation; in the aspect of determining the second difference deformation measurement equation, the method includes: determining a longitudinal change amount of the camera array; determining a second angle between the optical axis of the rear-view camera and the horizontal plane; determining the second difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel change amount, the second vertical pixel change amount, the third distance, the fourth distance, the pitch angle change amount, the yaw change amount, the longitudinal change amount, the first angle and the second angle; and determining the second horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel change amount, the second horizontal pixel change amount, the third distance, the fourth distance, the yaw change amount and the roll change amount, the first angle and the second angle.

[0128] When the first to-be-measured point is in the field of view of the rear-view camera and the second to-be-measured point is in the field of view of the front-view camera, the second difference settlement measurement equation is as follows:

[0129]

[0130] wherein, when the first point to be measured is in the field of view of the rear-view camera and the second point to be measured is in the field of view of the front-view camera, at this time is the object plane resolution of the rear-view camera for shooting the first point to be measured, represents the object plane resolution of the front-view camera for shooting the second point to be measured, is the first included angle between the optical axis of the rear-view camera and the horizontal plane, is the second included angle between the optical axis of the front-view camera and the horizontal plane. Since the rear-view camera and the front-view camera at the same monitoring position are fixed to each other, at this time .

[0131] wherein, when the first point to be measured is in the field of view of the rear-view camera and the second point to be measured is in the field of view of the front-view camera, the second horizontal difference deformation measurement equation is as follows:

[0132]

[0133] It can be seen that, in the present example, the processing device considers the changes in the poses of different cameras in the patrol process when determining the difference deformation amount between the first point to be measured and the second point to be measured, which is conducive to improving the accuracy of measurement.

[0134] In one possible example, the third difference deformation measurement equation includes the second difference subsidence measurement equation and the second horizontal difference deformation measurement equation; in the aspect of determining the third difference deformation measurement equation associated with the reference point to be measured, the method includes: determining a third object plane resolution of a camera for shooting the reference point to be measured; determining a third vertical pixel change amount and a third horizontal pixel change amount of the reference point to be measured in the camera image according to the image information; determining a fifth distance between the reference point to be measured and the camera array according to the first coordinate information; determining the third difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel change amount, the second vertical pixel change amount, the third vertical pixel change amount, the third distance, the fourth distance, the fifth distance, the pitch change amount, the yaw change amount, the longitudinal change amount, the first included angle and the second included angle; and determining the third horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first horizontal pixel change amount, the second horizontal pixel change amount, the third horizontal pixel change amount, the third distance, the fourth distance, the fifth distance, the yaw change amount and the roll change amount, the first included angle and the second included angle.

[0135] ​Wherein, when the mobile platform is at the current monitoring position, the first point to be measured on the first structural component is in the field of view of the forward-looking camera, and when the mobile platform is at the previous monitoring position, the second point to be measured on the second structural component is in the field of view of the rear-looking camera, the third differential settlement measurement equation is as follows:

[0136]

[0137] Specifically, when the mobile platform is at the current monitoring position, the first test point on the first structural component is within the field of view of the forward-looking camera, and when the mobile platform is at the previous monitoring position, the second test point on the second structural component is within the field of view of the rear-looking camera. and For the monitoring positions of two adjacent camera arrays, if Indicates the current monitoring location. This indicates the previous monitoring location of the current monitoring location. Therefore, accordingly, This represents the vertical settlement of the reference measurement point in the vertical direction. This represents the change in the first vertical pixel value of the first test point observed at the previous monitoring location in the camera image. This represents the change in the second vertical pixel value of the second target point observed at the current monitoring location in the camera image. This represents the change in the third vertical pixel of the reference point observed at the previous monitoring location in the camera image. This indicates the resolution of the third object surface of the reference measurement point captured by the forward-looking camera at the previous monitoring position. This indicates the resolution of the third object plane when the rear-view camera captures the reference point at the current monitoring location. This indicates the surface resolution of the first test point captured by the rearview camera at the previous monitoring position. This indicates the resolution of the third object surface when the forward-looking camera captures the second test point at the current monitoring location. This indicates the longitudinal displacement of the camera array at the previous monitoring position. This represents the longitudinal displacement of the camera array at the current monitoring position. This indicates the monitoring position of the first point to be measured at the previous monitoring location and the monitoring position of the camera array. The third distance between them This indicates the reference point between the monitored point and the camera array at the previous monitoring location. The fifth distance between them This indicates the monitoring position of the second point to be measured relative to the camera array at the current monitoring location. The fourth distance between them This indicates the reference position between the point to be measured and the monitoring position of the camera array at the current monitoring location. The fifth distance between them This indicates the change in yaw of the camera array at the previous monitoring position. represents the yaw change amount of the camera array at the current monitoring position, represents the pitch angle change amount of the camera array at the previous monitoring position, represents the pitch angle change amount of the camera array at the current monitoring position, is the first included angle between the rear-view camera optical axis and the horizontal plane, is the second included angle between the front-view camera optical axis and the horizontal plane.

[0138] wherein the third horizontal difference deformation measurement equation is as follows:

[0139]

[0140] wherein, if represents the current monitoring position, represents the previous monitoring position of the current monitoring position, and the corresponding is the roll change amount of the camera array at the previous monitoring position, is the roll change amount of the camera array at the current monitoring position, is the first horizontal pixel change amount of the first to-be-measured point observed at the previous monitoring position in the camera image, is the third horizontal pixel change amount of the reference to-be-measured point observed at the previous monitoring position in the camera image, is the third horizontal pixel change amount of the reference to-be-measured point observed at the current monitoring position in the camera image, is the second horizontal pixel change amount of the second to-be-measured point observed at the current monitoring position in the camera image, represents the displacement amount of the first to-be-measured point in the horizontal direction, represents the displacement amount of the second to-be-measured point in the horizontal direction, represents the displacement amount of the reference to-be-measured point in the horizontal direction.

[0141] It can be seen that in the present example, the processing device introduces the reference to-be-measured point to establish the connection between the first to-be-measured point and the second to-be-measured point when determining the difference deformation amount between the first to-be-measured point and the second to-be-measured point, and considers the changes in the poses of different cameras in the patrol process, which is conducive to improving the accuracy of measurement.

[0142] In one possible example, after the target patrol strategy is executed to obtain the extrinsic information, the first coordinate information of the to-be-measured point in the world coordinate system, and the image information, the method further includes: determining a first measurement coordinate system of the current patrol camera array; determining a pixel change amount of each to-be-measured point in the camera image according to the image information; creating a first imaging equation of the initial patrol to-be-measured point according to the central perspective projection model; creating a second imaging equation of the current patrol to-be-measured point according to the central perspective projection model; determining a to-be-measured point deformation equation according to the first imaging equation and the second imaging equation; bringing the pixel change amount, the extrinsic information, the intrinsic information, and the first coordinate information into the to-be-measured point deformation equation to obtain a deformation amount of each to-be-measured point; and determining a target difference deformation amount between the first structure and the second structure according to the deformation amount of the to-be-measured point.

[0143] In the method, the pixel change amount of the to-be-measured point in the camera image is determined according to the image information, and the pixel change amount of the to-be-measured point in the camera image is a pixel change amount of the to-be-measured point in the image photographed in the current patrol compared with the image photographed in the initial patrol.

[0144] In the method, the extrinsic information includes an extrinsic parameter matrix of the camera, and the intrinsic information includes an intrinsic parameter matrix of the camera.

[0145] In the method, the first imaging equation created based on the central perspective projection model is as follows:

[0146]

[0147] In the method, the second imaging equation created based on the central perspective projection model is as follows: , which means that in the initial patrol, , which means that the scale factor is the projection length of the straight line distance from the monitoring position to the to-be-measured point in the initial patrol in the direction of the camera optical axis. , which means that the two-dimensional image coordinates of the to-be-measured point in the image coordinate system are expressed in the homogeneous form in the initial patrol. , which means the intrinsic parameter matrix of the camera, , which means the extrinsic parameter matrix of the camera in the initial patrol, , which means the three-dimensional coordinates of the to-be-measured point expressed in the homogeneous form in the world coordinate system in the initial patrol.

[0148] In the method, the second imaging equation created based on the central perspective projection model is as follows:

[0149]

[0150] In the method, since the camera is not replaced in the multiple patrols, the intrinsic parameter of the camera is unchanged, and is still The intrinsic parameter can be obtained in advance through camera calibration. , which means that in the current patrol, is referred to as a scale factor, and its physical meaning is the projection length of the straight-line distance from the monitoring position to the point to be measured in the direction of the camera optical axis at the current patrol measurement. is referred to as the two-dimensional image coordinates of the point to be measured in the image coordinate system in homogeneous form at the current patrol measurement. is referred to as the extrinsic parameter matrix of the camera at the current patrol measurement, is referred to as the three-dimensional coordinates of the point to be measured in the world coordinate system in homogeneous form at the current patrol measurement.

[0151] wherein it is assumed that the mobile platform moves on the subway segment and monitors the point to be measured on the subway segment, and due to the deformation of the segment, the motion trajectory of the mobile platform at the initial patrol measurement is different from the motion trajectory of the mobile platform at the current patrol measurement, but the mobile platform is continuously moving, and the camera continuously takes images during the movement. Therefore, there is always a monitoring position of one or more cameras in the current patrol measurement that is close to the monitoring position of the camera at the initial patrol measurement. In other words: even if there is a position difference between the monitoring position of the camera at the current patrol measurement and the monitoring position of the camera at the initial patrol measurement, that is, there is a certain distance, but the distance is negligible compared to the distance from the point to be measured to the monitoring position. Therefore, it can be assumed that the scale factor is a constant, that is, .

[0152] Optionally, determining the first measurement coordinate system of the current patrol camera array comprises the following steps: determining a second measurement coordinate system of the initial patrol camera array; determining a six-degree-of-freedom change between the initial patrol camera array and the current patrol camera array; and determining the first measurement coordinate system according to the second measurement coordinate system and the six-degree-of-freedom change.

[0153] wherein due to the different motion trajectories of the mobile platform, there is a position change and a camera pose change between the monitoring position at the initial patrol measurement and the monitoring position at the current patrol measurement, and the changes of the camera monitoring position and pose can be represented by six degrees of freedom as follows: that is, the six-degree-of-freedom change, by the above six degrees of freedom representing the position and pose change, the relationship between (the second measurement coordinate system) and (the first measurement coordinate system) can be established:

[0154] .

[0155] wherein it is assumed that the point to be measured has accumulated deformation relative to the initial patrol measurement at the current patrol measurement, and the deformation amount can be expressed as:

[0156] .

[0157] wherein the second imaging equation and the first imaging equation are obtained as follows:

[0158]

[0159]

[0160] wherein, is the pixel change amount generated by the to-be-measured point in the camera image, which can be obtained through an image positioning algorithm.

[0161] Further, according to the final to-be-measured point deformation equation is obtained:

[0162]

[0163] wherein, represents the three-dimensional deformation amount of the to-be-measured point.

[0164] wherein, the first vertical settlement amount of the first to-be-measured point on the first structural member, the second vertical settlement amount of the second to-be-measured point on the second structural member, the third vertical settlement amount of the reference to-be-measured point observed at the previous monitoring position, and the fourth vertical settlement amount of the reference to-be-measured point observed at the current monitoring position can be determined based on the deformation amount of the to-be-measured point; then, when the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, the target differential deformation amount = the first vertical settlement amount - the second vertical settlement amount; when the first to-be-measured point on the first structural member is in the field of view of the rear-view camera and the second to-be-measured point on the second structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position, the target differential deformation amount = the first vertical settlement amount - the second vertical settlement amount; when the first to-be-measured point on the first structural member is in the field of view of the front-view camera and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position, the target differential deformation amount = (the first vertical settlement amount - the third vertical settlement amount) + (the fourth vertical settlement amount - the second vertical settlement amount).

[0165] It can be seen that, in the present example, the change of the camera position and pose is considered when measuring the deformation amount of the to-be-measured point, which is beneficial to improve the accuracy of determining the deformation amount of the to-be-measured point. Further, the differential deformation amount of the two structural members is calculated according to the deformation amount of the to-be-measured point, which is beneficial to improve the accuracy of determining the differential deformation amount.

[0166] Please refer to Figure 7 , Figure 7A structural schematic diagram of an electronic device provided by an embodiment of the present application is applied to a processing device of a mobile measurement device, the mobile measurement device comprising a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device and the processing device, the perception measurement module comprising a camera array, the camera array comprising a front-view camera and a rear-view camera, the processing device being connected to the front-view camera and the rear-view camera respectively, and the power supply device being connected to the perception measurement module and the processing device respectively; as shown in Figure 7 The electronic device comprises a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to instruct the processor to perform the following steps:

[0167] Obtain a differential deformation monitoring instruction for a target structure, the target structure comprising a plurality of structural members;

[0168] According to the differential deformation monitoring instruction, determine a target patrol strategy, the target patrol strategy being used to instruct the mobile measurement device to move to observe a to-be-measured point on the target structure;

[0169] Execute the target patrol strategy to obtain extrinsic information, first coordinate information of the to-be-measured point in a world coordinate system and image information;

[0170] According to the extrinsic information, the first coordinate information and the image information, a distribution of the to-be-measured point in a field of view of the front-view camera and a field of view of the rear-view camera, determine a target differential deformation between a first structural member and a second structural member, the first structural member and the second structural member being any two structural members in the plurality of structural members.

[0171] It can be seen that, in the embodiment of the application, the electronic device first acquires a differential deformation monitoring instruction for a target structure, the target structure includes a plurality of structural members, then determines a target patrol measurement strategy according to the differential deformation monitoring instruction, the target patrol measurement strategy is used to instruct a mobile measurement device to move to observe a to-be-measured point on the target structure, and then the target patrol measurement strategy is executed to obtain extrinsic parameter information, first coordinate information of the to-be-measured point in a world coordinate system, and image information. Further, according to the extrinsic parameter information, the first coordinate information, and the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera, a target differential deformation between a first structural member and a second structural member is determined, the first structural member and the second structural member being any two structural members in the plurality of structural members. The deformation of the to-be-measured point on the structural member can be monitored by moving the mobile measurement device to patrol and measure, compared with manual measurement, which is beneficial to improve the measurement efficiency. In addition, the distribution of the to-be-measured point on the two structural members in the field of view of the front-view camera and the rear-view camera is considered to determine the differential deformation between the two structural members, which is beneficial to improve the accuracy of measurement.

[0172] In one possible example, in the aspect of determining the target differential deformation between the first structural member and the second structural member according to the extrinsic parameter information, the first coordinate information, and the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera, the above program includes instructions further used to perform the following steps:

[0173] determining whether a first to-be-measured point on the first structural member and a second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position;

[0174] if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position, determining whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position;

[0175] if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, determining a first differential deformation measurement equation, and determining the target differential deformation between the first structural member and the second structural member according to the first differential deformation measurement equation;

[0176] if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, a second differential deformation measurement equation is determined, and the target differential deformation amount between the first structural member and the second structural member is determined according to the second differential deformation measurement equation;

[0177] if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position, it is judged whether the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position;

[0178] if it is judged that the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position, a reference to-be-measured point is determined, a third differential deformation measurement equation associated with the reference to-be-measured point is determined, and the target differential deformation amount between the first structural member and the second structural member is determined according to the third differential deformation measurement equation.

[0179] In one possible example, in the aspect of determining the reference to-be-measured point, the above procedure includes instructions for performing the following steps:

[0180] determining a field-of-view overlap region between the field of view of the rear-view camera when the mobile platform is at the current monitoring position and the field of view of the front-view camera when the mobile platform is at the previous monitoring position;

[0181] determining a third to-be-measured point in the field-of-view overlap region;

[0182] taking the third to-be-measured point as the reference to-be-measured point.

[0183] In one possible example, the first differential deformation measurement equation includes a first differential settlement measurement equation and a first horizontal differential deformation measurement equation, and when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the front-view camera, in the aspect of determining the first differential deformation measurement equation, the above procedure includes instructions for performing the following steps:

[0184] determining a first object plane resolution at which the camera captures the first to-be-measured point;

[0185] determining a second object plane resolution at which the camera captures the second to-be-measured point;

[0186] determining, according to the image information, a first vertical pixel variation and a first horizontal pixel variation of the first point to be measured in a camera image, and determining, according to the image information, a second vertical pixel variation and a second horizontal pixel variation of the second point to be measured in a camera image;

[0187] determining, according to the first coordinate information, a third distance between the first point to be measured and the camera array and a fourth distance between the second point to be measured and the camera array, respectively;

[0188] determining a first included angle between a front-view camera optical axis and a horizontal plane;

[0189] determining a pitch variation, a yaw variation and a roll variation of the camera array;

[0190] determining the first difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch variation, the yaw variation and the first included angle;

[0191] determining the first horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation and the roll variation and the first included angle.

[0192] In one possible example, the second difference deformation measurement equation comprises a second difference subsidence measurement equation and a second horizontal difference deformation measurement equation; in the aspect of determining the second difference deformation measurement equation, the above program comprises instructions further used for performing the following steps:

[0193] determining a longitudinal variation of the camera array;

[0194] determining a second included angle between a rear-view camera optical axis and a horizontal plane;

[0195] determining the second difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle;

[0196] determine the third difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third vertical pixel variation, the third distance, the fourth distance, the fifth distance, the pitch variation, the yaw variation, the first included angle and the second included angle.

[0197] In one possible example, the third difference deformation measurement equation comprises a second difference subsidence measurement equation and a second horizontal difference deformation measurement equation; in the determining of the third difference deformation measurement equation associated with the reference target point, the above program comprises instructions further used for performing the following steps:

[0198] determine a third object plane resolution at which the camera captures the reference target point;

[0199] determine a third vertical pixel variation and a third horizontal pixel variation of the reference target point in the camera image respectively according to the image information;

[0200] determine a fifth distance between the reference target point and the camera array according to the first coordinate information;

[0201] determine the third difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third vertical pixel variation, the third distance, the fourth distance, the fifth distance, the pitch variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle.

[0202] determine the third horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third horizontal pixel variation, the third distance, the fourth distance, the fifth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

[0203] In one possible example, after the execution of the target patrol strategy, the above program comprises instructions further used for performing the following steps:

[0204] determine a first measurement coordinate system of the current camera array;

[0205] determine a pixel variation of each target point in the camera image according to the image information;

[0206] create a first imaging equation of the initial patrol target point according to the central perspective projection model;

[0207] create a second imaging equation of the to-be-measured points in the current round of the survey according to a central perspective projection model;

[0208] determine a deformation equation of the to-be-measured points according to the first imaging equation and the second imaging equation;

[0209] obtain the deformation of each to-be-measured point by bringing the pixel change, the extrinsic information, the intrinsic information and the first coordinate information into the deformation equation of the to-be-measured points;

[0210] determine the target difference deformation between the first structure and the second structure according to the deformation of the to-be-measured points.

[0211] In one possible example, the determination of the first measurement coordinate system of the current round of the camera array includes instructions further used to perform the following steps:

[0212] determine a second measurement coordinate system of the initial round of the camera array;

[0213] determine a six-degree-of-freedom change between the initial round of the camera array and the current round of the camera array;

[0214] determine the first measurement coordinate system according to the second measurement coordinate system and the six-degree-of-freedom change.

[0215] The above mainly describes the scheme of the embodiments of the present application from the perspective of the method execution process. It can be understood that, in order to implement the above functions, the electronic device contains the hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the unit and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of the present application.

[0216] The embodiments of the present application can divide the functional units of the electronic device according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be implemented in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical functional division. When actually implemented, there can be another division manner.

[0217] In the case of dividing each functional module corresponding to each function,Figure 8 is a functional unit composition block diagram of a structural difference deformation monitoring device provided by an embodiment of the present application, as shown in Figure 8 The processing device is applied to a mobile measurement device, the mobile measurement device includes a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device and the processing device, the perception measurement module includes a camera array, the camera array includes a front-view camera and a rear-view camera, the processing device is connected with the front-view camera and the rear-view camera respectively, and the power supply device is connected with the perception measurement module and the processing device respectively; the structural difference deformation monitoring device includes an acquisition unit 801, a determination unit 802 and an execution unit 803, wherein,

[0218] The acquisition unit 801 is configured to acquire a difference deformation monitoring instruction for a target structure, and the target structure includes a plurality of structural members.

[0219] The determination unit 802 is configured to determine a target patrol strategy according to the difference deformation monitoring instruction, and the target patrol strategy is used to instruct the mobile measurement device to move to observe a to-be-measured point on the target structure.

[0220] The execution unit 803 is configured to execute the target patrol strategy to obtain extrinsic information, first coordinate information of the to-be-measured point in a world coordinate system and image information.

[0221] The determination unit 802 is further configured to determine a target difference deformation amount between a first structural member and a second structural member according to the extrinsic information, the first coordinate information and the image information and a distribution of the to-be-measured point in a field of view of the front-view camera and a field of view of the rear-view camera, and the first structural member and the second structural member are any two structural members in the plurality of structural members.

[0222] It can be seen that the structural difference deformation monitoring device described in the embodiments of the present application can first acquire a difference deformation monitoring instruction for a target structure, the target structure including a plurality of structural members, then determine a target patrol strategy according to the difference deformation monitoring instruction, the target patrol strategy being used to instruct a mobile measurement device to move to observe a to-be-measured point on the target structure, and then execute the target patrol strategy to obtain extrinsic parameter information, first coordinate information of the to-be-measured point in a world coordinate system, and image information. Further, according to the extrinsic parameter information, the first coordinate information, and the image information, a distribution of the to-be-measured point in a field of view of a forward-looking camera and a field of view of a rear-looking camera, a target difference deformation amount between a first structural member and a second structural member is determined, the first structural member and the second structural member being any two structural members in the plurality of structural members. The deformation amount of the to-be-measured point on the structural member can be monitored by moving the mobile measurement device to patrol, compared with manual measurement, which is beneficial to improve the measurement efficiency. In addition, the distribution of the to-be-measured point on the two structural members in the field of view of the forward-looking camera and the field of view of the rear-looking camera is considered to determine the difference deformation amount between the two structural members, which is beneficial to improve the accuracy of measurement.

[0223] In one possible example, in the aspect of determining the target difference deformation amount between the first structural member and the second structural member according to the extrinsic parameter information, the first coordinate information, and the image information, the distribution of the to-be-measured point in the field of view of the forward-looking camera and the field of view of the rear-looking camera, the determining unit 802 is specifically configured to:

[0224] determine whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the forward-looking camera and the field of view of the rear-looking camera when the mobile platform is at the current monitoring position;

[0225] if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the forward-looking camera and the field of view of the rear-looking camera when the mobile platform is at the current monitoring position, determine whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the forward-looking camera or the field of view of the rear-looking camera when the mobile platform is at the current monitoring position;

[0226] if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the forward-looking camera or the field of view of the rear-looking camera when the mobile platform is at the current monitoring position, determine a first difference deformation measurement equation, and determine the target difference deformation amount between the first structural member and the second structural member according to the first difference deformation measurement equation;

[0227] if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, a second difference deformation measurement equation is determined, and the target difference deformation amount between the first structural member and the second structural member is determined according to the second difference deformation measurement equation;

[0228] if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position, it is judged whether the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and whether the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position;

[0229] if it is judged that the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position, a reference to-be-measured point is determined, a third difference deformation measurement equation associated with the reference to-be-measured point is determined, and the target difference deformation amount between the first structural member and the second structural member is determined according to the third difference deformation measurement equation.

[0230] In one possible example, in the aspect of determining the reference to-be-measured point, the determining unit 802 is specifically configured to:

[0231] determine a field of view overlap region between the field of view of the rear-view camera when the mobile platform is at the current monitoring position and the field of view of the front-view camera when the mobile platform is at the previous monitoring position;

[0232] determine a third to-be-measured point in the field of view overlap region;

[0233] determine the third to-be-measured point as the reference to-be-measured point.

[0234] In one possible example, the first difference deformation measurement equation includes a first difference settlement measurement equation and a first horizontal difference deformation measurement equation, and when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the front-view camera, in the aspect of determining the first difference deformation measurement equation, the determining unit 802 is further specifically configured to:

[0235] determine a first object plane resolution at which the camera photographs the first to-be-measured point;

[0236] determine a second object plane resolution at which the camera photographs the second to-be-measured point;

[0237] determine, according to the image information, a first vertical pixel variation and a first horizontal pixel variation of the first to-be-measured point in a camera image, and determine, according to the image information, a second vertical pixel variation and a second horizontal pixel variation of the second to-be-measured point in the camera image;

[0238] determine, according to the first coordinate information, a third distance between the first to-be-measured point and the camera array and a fourth distance between the second to-be-measured point and the camera array, respectively;

[0239] determine a first included angle between a front-view camera optical axis and a horizontal plane;

[0240] determine a pitch variation, a yaw variation and a roll variation of the camera array;

[0241] determine the first difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch variation, the yaw variation and the first included angle;

[0242] determine the first horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation, the roll variation and the first included angle.

[0243] In one possible example, the second difference deformation measurement equation includes a second difference subsidence measurement equation and a second horizontal difference deformation measurement equation; in the aspect of determining the second difference deformation measurement equation, the determining unit 802 is specifically configured to:

[0244] determine a longitudinal variation of the camera array;

[0245] determine a second included angle between a rear-view camera optical axis and the horizontal plane;

[0246] determine the second difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle;

[0247] determine the second horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation, the roll variation, the first included angle and the second included angle.

[0248] In a possible example, the third difference deformation measurement equation comprises a second difference subsidence measurement equation and a second horizontal difference deformation measurement equation; in the determining the third difference deformation measurement equation associated with the reference target point, the determining unit 802 is specifically configured to:

[0249] determine a third object plane resolution at which the camera shoots the reference target point;

[0250] determine a third vertical pixel change amount and a third horizontal pixel change amount of the reference target point in a camera image according to the image information;

[0251] determine a fifth distance between the reference target point and the camera array according to the first coordinate information;

[0252] determine the third difference subsidence measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel change amount, the second vertical pixel change amount, the third vertical pixel change amount, the third distance, the fourth distance, the fifth distance, the pitch change amount, the yaw change amount, the longitudinal change amount, the first included angle and the second included angle;

[0253] determine the third horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first horizontal pixel change amount, the second horizontal pixel change amount, the third horizontal pixel change amount, the third distance, the fourth distance, the fifth distance, the yaw change amount and the roll change amount, the first included angle and the second included angle.

[0254] In a possible example, after the executing the target patrol strategy, obtaining the extrinsic parameter information, the first coordinate information of the target point in the world coordinate system and the image information, the determining unit 802 is specifically configured to:

[0255] determine a first measurement coordinate system of a current camera array;

[0256] determine a pixel change amount of each target point in a camera image according to the image information;

[0257] create a first imaging equation of an initial patrol target point according to a central perspective projection model;

[0258] create a second imaging equation of a current patrol target point according to a central perspective projection model;

[0259] determine a target point deformation equation according to the first imaging equation and the second imaging equation;

[0260] The pixel change amount, the extrinsic information, intrinsic information and first coordinate information are brought into the deformation equation of the to-be-measured point to obtain a deformation amount of each to-be-measured point.

[0261] According to the deformation amount of the to-be-measured point, a target difference deformation amount between the first structure and the second structure is determined.

[0262] In one possible example, in the determination of the first measurement coordinate system of the current secondary patrol camera array, the determination unit 802 is specifically configured to:

[0263] determine a second measurement coordinate system of an initial patrol camera array;

[0264] determine a six-degree-of-freedom change amount between the initial patrol camera array and the current secondary patrol camera array;

[0265] determine the first measurement coordinate system according to the second measurement coordinate system and the six-degree-of-freedom change amount.

[0266] It should be noted that all related contents of each step involved in the above method embodiments can be cited to the function description of the corresponding function module, which will not be repeated here.

[0267] The electronic device provided in the embodiment is used to execute the structure difference deformation monitoring method, and thus the same effect as the implementation method can be achieved.

[0268] In the case of using integrated units, the electronic device can include a processing module, a storage module and a communication module. The processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the acquisition unit 801 and the determination unit 802. The storage module can be used to support the electronic device to execute the storage of program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0269] The processing module can be a processor or a controller. It can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure of the present application. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of digital signal processing (digital signal processing, DSP) and microprocessors, etc. The storage module can be a memory. The communication module can be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0270] The embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all of steps of any method described in the above method embodiments, and the computer includes an electronic device.

[0271] The embodiment of the present application further provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of steps of any method described in the above method embodiments. The computer program product can be a software installation package, and the computer includes a control platform.

[0272] It should be noted that, for the above method embodiments, in order to simply describe, each of the above method embodiments is described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of actions described, because according to the present application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0273] In the above embodiments, the description of each embodiment is focused on, and the part not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0274] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented by other means. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical or other forms.

[0275] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0276] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0277] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory, and includes several instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the above-mentioned methods of various embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0278] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods of the embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0279] The embodiments of the present application are described in detail above, and the specific examples are applied to the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a person of ordinary skill in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method of structural differential deformation monitoring, characterized by, A processing device applied to a mobile measurement device, the mobile measurement device comprising a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device and the processing device, the perception measurement module comprising a camera array, the camera array comprising a front-view camera and a rear-view camera, the processing device being connected to the front-view camera and the rear-view camera respectively, and the power supply device being connected to the perception measurement module and the processing device respectively; the method comprising: obtaining a differential deformation monitoring instruction for a target structure, the target structure comprising a plurality of structural members; determining a target patrol strategy according to the differential deformation monitoring instruction, the target patrol strategy being used to instruct the mobile measurement device to move to observe a to-be-measured point on the target structure; executing the target patrol strategy to obtain extrinsic information, first coordinate information of the to-be-measured point in a world coordinate system and image information; determining a target differential deformation between a first structural member and a second structural member according to the extrinsic information, the first coordinate information and the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera, comprising: judging whether a first to-be-measured point on the first structural member and a second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at a current monitoring position; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position, judging whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, determining a first differential deformation measurement equation and determining the target differential deformation between the first structural member and the second structural member according to the first differential deformation measurement equation; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera or the rear-view camera when the mobile platform is at the current monitoring position, determining a second differential deformation measurement equation and determining the target differential deformation between the first structural member and the second structural member according to the second differential deformation measurement equation; if it is judged that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera and the rear-view camera when the mobile platform is at the current monitoring position, judging whether the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at a previous monitoring position. If it is determined that the first to-be-measured point on the first structural member at the current monitoring position of the mobile platform is in the field of view of the front-view camera and the second to-be-measured point on the second structural member at the last monitoring position of the mobile platform is in the field of view of the rear-view camera, a reference to-be-measured point is determined, a third difference deformation measurement equation associated with the reference to-be-measured point is determined, and a target difference deformation amount between the first structural member and the second structural member is determined according to the third difference deformation measurement equation; the first structural member and the second structural member are any two structural members in a plurality of structural members.

2. The method of claim 1, wherein, The determination of the reference to-be-measured point comprises: determining a field of view overlap region between the field of view of the rear-view camera at the current monitoring position of the mobile platform and the field of view of the front-view camera at the last monitoring position of the mobile platform; determining a third to-be-measured point in the field of view overlap region; determining the third to-be-measured point as the reference to-be-measured point.

3. The method of claim 1, wherein, The first difference deformation measurement equation comprises a first difference settlement measurement equation and a first horizontal difference deformation measurement equation, and when the first to-be-measured point and the second to-be-measured point are simultaneously in the field of view of the front-view camera, the determination of the first difference deformation measurement equation comprises: determining a first object plane resolution of the camera shooting the first to-be-measured point; determining a second object plane resolution of the camera shooting the second to-be-measured point; determining a first vertical pixel change amount and a first horizontal pixel change amount of the first to-be-measured point in the camera image according to the image information, and determining a second vertical pixel change amount and a second horizontal pixel change amount of the second to-be-measured point in the camera image according to the image information; determining a third distance between the first to-be-measured point and the camera array and a fourth distance between the second to-be-measured point and the camera array according to the first coordinate information; determining a first included angle between the optical axis of the front-view camera and the horizontal plane; determining a pitch angle change amount, a yaw change amount and a roll change amount of the camera array; determining the first difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel change amount, the second vertical pixel change amount, the third distance, the fourth distance, the pitch angle change amount, the yaw change amount and the first included angle; determining the first horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel change amount, the second horizontal pixel change amount, the third distance, the fourth distance, the yaw change amount, the roll change amount and the first included angle.

4. The method of claim 3, wherein, The second difference deformation measurement equation comprises a second difference settlement measurement equation and a second horizontal difference deformation measurement equation; The determination of the second difference deformation measurement equation comprises: determining a longitudinal change amount of the camera array; determining a second included angle between the optical axis of the rear-view camera and the horizontal plane; determine the second difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third distance, the fourth distance, the pitch angle variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle; determine the second horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third distance, the fourth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

5. The method of claim 4, wherein, The third difference deformation measurement equation includes the second difference settlement measurement equation and the second horizontal difference deformation measurement equation; The determining of the third difference deformation measurement equation associated with the reference to-be-measured point includes: determining a third object plane resolution of the reference to-be-measured point photographed by the camera; determining a third vertical pixel variation and a third horizontal pixel variation of the reference to-be-measured point in the camera image respectively according to the image information; determining a fifth distance between the reference to-be-measured point and the camera array according to the first coordinate information; determining the third difference settlement measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first vertical pixel variation, the second vertical pixel variation, the third vertical pixel variation, the third distance, the fourth distance, the fifth distance, the pitch angle variation, the yaw variation, the longitudinal variation, the first included angle and the second included angle; determining the third horizontal difference deformation measurement equation according to the first object plane resolution, the second object plane resolution, the third object plane resolution, the first horizontal pixel variation, the second horizontal pixel variation, the third horizontal pixel variation, the third distance, the fourth distance, the fifth distance, the yaw variation and the roll variation, the first included angle and the second included angle.

6. The method of claim 1, wherein, After the executing of the target patrol strategy, the obtaining of the extrinsic parameter information, the first coordinate information of the to-be-measured point in the world coordinate system and the image information, the method further includes: determining a first measurement coordinate system of the current time patrol camera array; determining a pixel variation of each to-be-measured point in the camera image according to the image information; creating a first imaging equation of the initial patrol to-be-measured point according to the central perspective projection model; creating a second imaging equation of the current time patrol to-be-measured point according to the central perspective projection model; determining a to-be-measured point deformation equation according to the first imaging equation and the second imaging equation; bringing the pixel variation, the extrinsic parameter information, the intrinsic parameter information and the first coordinate information into the to-be-measured point deformation equation to obtain a deformation amount of each to-be-measured point; determining a target difference deformation amount between the first structure and the second structure according to the deformation amount of the to-be-measured point.

7. The method of claim 6, wherein, The determining of the first measurement coordinate system of the current time patrol camera array includes: determining a second measurement coordinate system of the initial patrol camera array; Determine a six-degree-of-freedom change between the initial patrol camera array and the current patrol camera array; Determine the first measurement coordinate system according to the second measurement coordinate system and the six-degree-of-freedom change.

8. A structural differential deformation monitoring apparatus, characterized by comprising: The processing device is applied to a mobile measurement device, the mobile measurement device comprises a mobile platform, a perception measurement module arranged on the mobile platform, a power supply device and the processing device, the perception measurement module comprises a camera array, the camera array comprises a forward-looking camera and a rear-looking camera, the processing device is connected with the forward-looking camera and the rear-looking camera respectively, and the power supply device is connected with the perception measurement module and the processing device respectively; the structural difference deformation monitoring device comprises an acquisition unit, a determination unit and an execution unit; wherein, The acquisition unit is used to acquire a difference deformation monitoring instruction for a target structure, and the target structure comprises a plurality of structural members; The determination unit is used to determine a target patrol strategy according to the difference deformation monitoring instruction, and the target patrol strategy is used to instruct the mobile measurement device to move to observe a to-be-measured point on the target structure; The execution unit is used to execute the target patrol strategy to obtain external parameter information, first coordinate information of the to-be-measured point in a world coordinate system and image information. The determining unit is further configured to determine the target difference deformation between the first structural member and the second structural member according to the extrinsic information, the first coordinate information, and the distribution of the to-be-measured point in the field of view of the front-view camera and the field of view of the rear-view camera, including: determining whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position; if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position, determining whether the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position; if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position, determining a first difference deformation measurement equation, and determining the target difference deformation between the first structural member and the second structural member according to the first difference deformation measurement equation; if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera or the field of view of the rear-view camera when the mobile platform is at the current monitoring position, determining a second difference deformation measurement equation, and determining the target difference deformation between the first structural member and the second structural member according to the second difference deformation measurement equation; if it is determined that the first to-be-measured point on the first structural member and the second to-be-measured point on the second structural member are not simultaneously in the field of view of the front-view camera and the field of view of the rear-view camera when the mobile platform is at the current monitoring position, determining whether the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position; if it is determined that the first to-be-measured point on the first structural member is in the field of view of the front-view camera when the mobile platform is at the current monitoring position and the second to-be-measured point on the second structural member is in the field of view of the rear-view camera when the mobile platform is at the previous monitoring position, determining a reference to-be-measured point, determining a third difference deformation measurement equation associated with the reference to-be-measured point, and determining the target difference deformation between the first structural member and the second structural member according to the third difference deformation measurement equation; the first structural member and the second structural member are any two structural members in a plurality of structural members.

9. An electronic device, comprising: A computer program product including a processor, a memory for storing one or more programs, and configured to be executed by the processor, the program including instructions for performing the steps in the method of any one of claims 1-6.

Images