Intelligent machine vision displacement monitoring equipment and method

Abstract

The invention discloses an intelligent machine vision displacement monitoring device and method, and the device comprises an image monitoring terminal and a processing terminal, the image monitoring terminal can monitor a bridge in real time, collects a real-time monitoring image of the bridge, and sends the real-time monitoring image to the processing terminal, thereby achieving the non-contact monitoring of the bridge displacement. The processing terminal is provided with a multi-core CPU, and the CPU is provided with an image processing thread and a plurality of displacement calculation threads and supports multi-thread parallel execution. The image processing thread is used for acquiring a reference image and a monitoring image sent by the image monitoring terminal, and preprocessing the monitoring image and the reference image. And the preprocessed monitoring image and reference image are stored in a cache of the CPU. And each displacement calculation thread can synchronously call the stored monitoring image and reference image, and adopts the same DIC algorithm to respectively calculate the displacement of different measuring points, so that the calculation efficiency of the bridge displacement amount is improved, and the efficient and real-time non-contact monitoring of the bridge displacement is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of measuring solid deformation, and in particular to an intelligent machine vision displacement monitoring device and method. Background Art

[0002] Due to the complexity of the engineering structure design, construction, construction, operation and maintenance of bridges, and the damage caused by dynamic loads such as traffic or wind and natural factors such as geology and climate during service, bridges will inevitably have various internal damages or external deformations. Measuring the deflection of bridges is of great value in understanding their safety performance and service status. There are generally three forms of bridge deflection measurement:

[0003] 1) Precision total station method, the basic principle of which is trigonometric height measurement, that is, measuring the horizontal distance and vertical angle between two points to obtain the height difference between the two points. This method requires manual deployment of targets and then point-by-point measurement, which has low measurement efficiency and cannot perform real-time multi-point synchronous measurement;

[0004] 2) Pre-embedded sensors (such as differential pressure transmitters and static level gauges) need to be fixed at a fixed reference point, and then the sensor contacts are in close contact with the measuring point. When the measuring point moves, the sensor generates a reading. It is required that there is a fixed reference point around the measuring point, which is usually difficult to guarantee at the measurement site;

[0005] 3) Global Positioning System (GPS), although it has the advantages of real-time measurement and long-distance measurement, the communication stability of GPS cannot be fully guaranteed, and there are often interruptions during field use. In addition, it is expensive and difficult to promote on a large scale.

[0006] It can be seen that the above-mentioned bridge deflection measurement methods all have their own limitations or disadvantages. Therefore, a technology that can overcome the limitations or disadvantages of the above-mentioned measurement methods is urgently needed. Summary of the invention

[0007] In view of the shortcomings of the existing technology, the present invention proposes an intelligent machine vision displacement monitoring device and method, which can realize efficient, real-time non-contact monitoring of bridge displacement. The specific technical solution is as follows:

[0008] In a first aspect, an intelligent machine vision displacement monitoring device is provided. In a first implementable manner of the first aspect, the device includes a CPU that supports multi-threaded parallel execution, including:

[0009] An image processing thread, configured to obtain and store a reference image and a monitoring image of a measured target;

[0010] The plurality of displacement calculation threads are configured to respectively calculate the displacements of different measuring points using the same DIC algorithm based on the reference image and the monitoring image.

[0011] In combination with the first implementable manner of the first aspect, in a second implementable manner of the first aspect, the displacement calculation thread uses a Gauss-Newton algorithm to calculate the displacement of the measuring point.

[0012] In combination with the first implementable manner of the first aspect, in a third implementable manner of the first aspect, further comprising:

[0013] The measuring point search thread is configured to search for measuring points according to a preset displacement threshold when the shape function of the DIC algorithm fails to converge.

[0014] In combination with the third implementable manner of the first aspect, in a fourth implementable manner of the first aspect, searching for measuring points according to a preset displacement threshold includes:

[0015] Determine the displacement trend of the measuring point according to the preset displacement threshold;

[0016] Taking the coordinates of the measuring point before the point is lost as the center point, determine the current displacement field, and search whether there is a measuring point in the current displacement field;

[0017] In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

[0018] In combination with the fourth implementable manner of the first aspect, in a fifth implementable manner of the first aspect, if the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the entire image for the measuring point.

[0019] In a second aspect, an intelligent machine vision displacement monitoring method is provided. In a first implementable manner of the second aspect, the method includes:

[0020] Acquire reference images and monitoring images of the target under test;

[0021] Based on the reference image and the monitoring image, the same DIC algorithm is used to calculate the displacement of different measuring points.

[0022] In combination with the first implementable manner of the second aspect, in the first implementable manner of the second aspect, using the DIC algorithm to calculate the displacement of the measuring point includes: using the Gauss-Newton algorithm to calculate the displacement of the measuring point

[0023] In combination with the first implementable manner of the second aspect, in a third implementable manner of the second aspect, the method further includes:

[0024] When the shape function of the DIC algorithm fails to converge, the measurement points are searched according to the preset displacement threshold.

[0025] In combination with the third implementable manner of the second aspect, in a fourth implementable manner of the second aspect, searching for the measuring point according to the preset displacement threshold includes:

[0026] Determine the displacement trend of the measuring point according to the preset displacement threshold;

[0027] Taking the coordinates of the measuring point before the point is lost as the center point, determine the current displacement field, and search whether there is a measuring point in the current displacement field;

[0028] In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

[0029] In combination with the fourth implementable manner of the second aspect, in a fifth implementable manner of the second aspect, if the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the entire image for the measuring point.

[0030] Beneficial effect: By adopting the intelligent machine vision displacement monitoring device and method of the present invention, the monitoring image of the bridge can be obtained in real time through the image processing thread, and the displacements of different measuring points in the monitoring image can be calculated in parallel through multiple displacement calculation threads, thereby improving the calculation efficiency of the bridge displacement and realizing efficient, real-time non-contact monitoring of the bridge displacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the specific implementation of the present invention, the following will briefly introduce the drawings required for use in the specific implementation. In all the drawings, each element or part is not necessarily drawn according to the actual scale.

[0032] Figure 1 A schematic diagram of the COU multi-threading principle of an intelligent machine vision displacement monitoring device provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the principle of a measuring point search thread provided by an embodiment of the present invention;

[0034] Figure 3 A flow chart of an intelligent machine vision displacement monitoring method provided by an embodiment of the present invention;

[0035] Figure 4 The displacement field expansion strategy corresponding to when the displacement trend of the measuring point is directly upward;

[0036] Figure 5 This is the corresponding displacement field expansion mode when the displacement trend of the measuring point is at the lower left. DETAILED DESCRIPTION

[0037] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0038] It should be understood that the "DIC algorithm", also known as the "digital image correlation method", is a non-contact modern optical measurement technology that can calculate the deformation or displacement within the field of view.

[0039] like Figure 1 As shown, an intelligent machine vision displacement monitoring device includes a CPU, and the CPU supports multi-threaded parallel execution, including:

[0040] An image processing thread, configured to obtain and store a reference image and a monitoring image of a measured target;

[0041] The plurality of displacement calculation threads are configured to respectively calculate the displacements of different measuring points using the same DIC algorithm based on the reference image and the monitoring image.

[0042] Specifically, the intelligent machine vision displacement monitoring device includes an image monitoring terminal and a processing terminal. The image monitoring terminal can monitor the bridge in real time, collect real-time monitoring images of the bridge and send them to the processing terminal, thereby realizing non-contact monitoring of the bridge displacement. The processing terminal is equipped with a multi-core CPU, which is configured with an image processing thread and multiple displacement calculation threads, and supports multi-threaded parallel execution.

[0043] Among them, the image processing thread is used to obtain the reference image and the monitoring image sent by the image monitoring terminal, and preprocess the monitoring image and the reference image. The preprocessed monitoring image and reference image are stored in the CPU cache. Each displacement calculation thread can synchronously retrieve the stored monitoring image and reference image, and use the same DIC algorithm to calculate the displacement of different measuring points respectively, thereby improving the calculation efficiency of the bridge displacement and realizing efficient, real-time non-contact monitoring of the bridge displacement.

[0044] In this embodiment, optionally, the displacement calculation thread uses a Gauss-Newton algorithm to calculate the displacement of the measuring point.

[0045] Specifically, before the displacement calculation thread calculates the displacement of the measuring point, the Jacobian matrix parameters corresponding to each displacement calculation thread can be initialized according to the reference image, and each displacement calculation thread can use the DIC algorithm to calculate the displacement of different measuring points according to the corresponding Jacobian matrix parameters. The core algorithms of the DIC algorithm include the sub-pixel search algorithm and the integer pixel search algorithm. Among them, the monitoring image can be preliminarily searched by the integer pixel search algorithm to find the best matching point that matches the reference image. The sub-pixel search algorithm can converge the search results of the integer pixel algorithm to the sub-pixel level according to the initial value provided by the integer pixel search algorithm, and obtain the corresponding measuring point, thereby improving the accuracy of subsequent displacement calculation. After searching for the measuring point, the inverse Gauss-Newton algorithm can be used to estimate the shape function parameters between the monitoring image and the reference image, so as to indirectly measure the displacement of the reference point. In order to improve the accuracy and calculation speed of the calculated measuring point, the Jacobian matrix parameters calculated last time can be introduced.

[0046] In this embodiment, optionally, it further includes:

[0047] The measuring point search thread is configured to search for measuring points according to a preset displacement threshold when the shape function of the DIC algorithm fails to converge.

[0048] Specifically, if Figure 2 As shown, the CPU is also configured with a measuring point search thread, which can use an extended search strategy to search for measuring points when each displacement calculation thread fails to converge in the process of calculating the displacement of the measuring point using the Gauss-Newton algorithm. Specifically, the displacement field can be expanded according to a preset displacement threshold, and the measuring point can be continued to be searched in the expanded displacement field.

[0049] In this embodiment, optionally, searching for measuring points according to a preset displacement threshold includes:

[0050] Determine the displacement trend of the measuring point according to the preset displacement threshold;

[0051] Taking the coordinates of the measuring point before the point is lost as the center point, determine the current displacement field, and search whether there is a measuring point in the current displacement field;

[0052] In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

[0053] Specifically, when searching for measuring points, the measuring point search thread can first determine the direction in which the measuring point was displaced before the point was lost based on the displacement threshold, because there is a high probability that the measuring point will continue to displace in this direction beyond the deformation field retrieval area. The predicted directions include due east, due south, due west, due north, southeast, northeast, southwest, and northwest.

[0054] Then, the coordinates of the measuring point before the point is lost are used as the center point to determine the current displacement field, and the whole pixel search algorithm and sub-pixel search algorithm are used to search whether there is a measuring point in the displacement field before the point is lost.

[0055] If it does not exist, then Figure 4 or Figure 5 The strategy of loop expansion is used to retrieve the corresponding displacement field, and the whole pixel search algorithm and sub-pixel search algorithm are used to find the measuring point in the extended displacement field. If the measuring point is found, the displacement field of the new measuring point is determined with the new measuring point position as the center, and the Jacobian matrix parameters are initialized based on the coordinates of the new measuring point, and the Gauss-Newton algorithm is used to calculate the displacement of the new measuring point.

[0056] In this embodiment, optionally, if the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the measuring point in the entire image.

[0057] Specifically, if no new measuring points are found in the extended displacement field, the full-image search strategy can be used to search for measuring points. Specifically, the image edge algorithm can be used to search the entire monitoring image. Since the image edge algorithm has low computational efficiency and accuracy, it is only used after the above search strategy fails. Moreover, since the computing resources occupied by the measuring point search are large, the search frequency of the measuring points can be reduced to reduce the impact on other threads.

[0058] If both the extended search strategy and the full map search strategy fail, it means that the measuring point is no longer within the field of view of the monitoring image. At this time, the information that the measuring point is completely lost will be recorded in the database and marked with a measuring point loss mark for query.

[0059] like Figure 3 The flow chart of the intelligent machine vision displacement monitoring method shown in FIG. 1 includes:

[0060] Step 1: Obtain a reference image and a monitoring image of the target;

[0061] Step 2: Based on the reference image and the monitoring image, the displacements of different measuring points are calculated using the same DIC algorithm.

[0062] Specifically, first, the processing terminal can obtain the reference image and the monitoring image sent by the image monitoring terminal, and pre-process the monitoring image and the reference image. Then, the processing terminal can use the same DIC algorithm to synchronously calculate the displacement of different measuring points, thereby improving the calculation efficiency of the bridge displacement and realizing efficient, real-time non-contact monitoring of the bridge displacement.

[0063] In this embodiment, optionally, using a DIC algorithm to calculate the displacement of a measuring point includes: using a Gauss-Newton algorithm to calculate the displacement of the measuring point.

[0064] Specifically, the core algorithms of the DIC algorithm include the sub-pixel search algorithm and the integer pixel search algorithm. Among them, the integer pixel search algorithm can be used to perform a preliminary search on the monitoring image to find the best matching point that matches the reference image. The sub-pixel search algorithm can converge the search results of the integer pixel algorithm to sub-pixel accuracy based on the initial value provided by the integer pixel search algorithm, and obtain the corresponding measurement points, thereby improving the accuracy of subsequent displacement calculations. After searching for the measurement point, the inverse Gauss-Newton algorithm can be used to estimate the shape function parameters between the monitoring image and the reference image, thereby indirectly measuring the displacement of the reference point. In order to improve the accuracy and calculation speed of the calculated measurement points, the Jacobian matrix parameters of the last calculation can be introduced.

[0065] In this embodiment, optionally, it further includes:

[0066] When the shape function of the DIC algorithm fails to converge, the measurement points are searched according to the preset displacement threshold.

[0067] Specifically, in the process of calculating the displacement of the measuring point using the Gauss-Newton algorithm, it is possible that the shape function still cannot converge after multiple iterations. When this happens, the displacement field can be expanded according to a preset displacement threshold, and the measuring point can be continued to be searched in the expanded displacement field.

[0068] In this embodiment, optionally, searching for measuring points according to a preset displacement threshold includes:

[0069] Determine the displacement trend of the measuring point according to the preset displacement threshold;

[0070] Taking the coordinates of the measuring point before the point is lost as the center point, determine the current displacement field, and search whether there is a measuring point in the current displacement field;

[0071] In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

[0072] Specifically, we can first determine the direction in which the measuring point was displaced before the point was lost based on the displacement threshold, because there is a high probability that the measuring point will continue to displace in this direction beyond the deformation field retrieval area. The predicted directions include due east, due south, due west, due north, southeast, northeast, southwest, and northwest.

[0073] Then, the coordinates of the measuring point before the point is lost are used as the center point to determine the current displacement field, and the whole pixel search algorithm and sub-pixel search algorithm are used to search whether there is a measuring point in the displacement field before the point is lost.

[0074] If it does not exist, then Figure 3 or Figure 4The strategy of loop expansion is used to retrieve the corresponding displacement field, and the whole pixel search algorithm and sub-pixel search algorithm are used to find the measuring point in the extended displacement field. If the measuring point is found, the displacement field of the new measuring point is determined with the new measuring point position as the center, and the Jacobian matrix parameters are initialized based on the coordinates of the new measuring point, and the Gauss-Newton algorithm is used to calculate the displacement of the new measuring point.

[0075] In this embodiment, optionally, if the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the measuring point in the entire image.

[0076] Specifically, if no new measuring points are found in the extended displacement field, the image edge algorithm can be used to search the entire monitoring image. Since the image edge algorithm has low computational efficiency and accuracy, it is only used after the above search strategy fails. In addition, since the measurement point search occupies a large amount of computing resources, the search frequency of the measurement points can be reduced to reduce the impact on other threads.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An intelligent machine vision displacement monitoring device, comprising a CPU, the CPU supporting multi-threaded parallel execution, characterized in that: The threads executed in parallel by the CPU include: An image processing thread, configured to obtain and store a reference image and a monitoring image of a measured target; The plurality of displacement calculation threads are configured to respectively calculate the displacements of different measuring points using the same DIC algorithm based on the reference image and the monitoring image.

2. The intelligent machine vision displacement monitoring device according to claim 1, characterized in that: The displacement calculation thread uses the Gauss-Newton algorithm to calculate the displacement of the measuring point.

3. The intelligent machine vision displacement monitoring device according to claim 1, characterized in that: Also includes: The measuring point search thread is configured to search for measuring points according to a preset displacement threshold when the shape function of the DIC algorithm fails to converge.

4. The intelligent machine vision displacement monitoring device according to claim 3, characterized in that: Searching for measuring points based on preset displacement thresholds includes: Determine the displacement trend of the measuring point according to the preset displacement threshold; The current displacement field is determined with the coordinates of the measuring point before the point is lost as the center point, and a search is performed in the current displacement field to determine whether there is a measuring point; In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

5. The intelligent machine vision displacement monitoring device according to claim 4, characterized in that: If the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the measuring point in the entire image.

6. An intelligent machine vision displacement monitoring method, characterized in that: include: Acquire reference images and monitoring images of the target under test; Based on the reference image and the monitoring image, the same DIC algorithm is used to calculate the displacement of different measuring points.

7. The intelligent machine vision displacement monitoring method according to claim 6, characterized in that: The DIC algorithm is used to calculate the displacement of the measuring point, including: using a Gauss-Newton algorithm to calculate the displacement of the measuring point.

8. The intelligent machine vision displacement monitoring method according to claim 6, characterized in that: Also includes: When the shape function of the DIC algorithm fails to converge, the measurement points are searched according to the preset displacement threshold.

9. The intelligent machine vision displacement monitoring method according to claim 8, characterized in that: Searching for measuring points based on preset displacement thresholds includes: Determine the displacement trend of the measuring point according to the preset displacement threshold; Taking the coordinates of the measuring point before the point is lost as the center point, determine the current displacement field, and search whether there is a measuring point in the current displacement field; In response to the absence of a measuring point in the current displacement field, the corresponding displacement difference is cyclically expanded according to the displacement trend, and the measuring point is searched in the expanded displacement field.

10. The intelligent machine vision displacement monitoring method according to claim 9, characterized in that: If the measuring point is not found in the expanded displacement field, an image edge algorithm is used to search the measuring point in the entire image.

Images