Tunnel displacement automatic monitoring method and device combining machine vision and passive target

Abstract

The embodiment of the application relates to the image processing field, and provides a tunnel displacement automatic monitoring method and device combining machine vision and a passive target, the method comprising the following steps: extracting an image corresponding to a region where a passive target is located from a to-be-processed image in a to-be-processed image set to obtain a third image set; calculating displacement information corresponding to each pixel point in a passive target in a third image in the third image set according to illumination intensity information corresponding to each pixel point in each third image in the third image set and first speed information in the k first speed information sets; generating early warning information according to target displacement information in the target displacement information set, and early warning according to the early warning information, so that the accuracy of displacement monitoring on a to-be-detected tunnel is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and device for automatic monitoring of tunnel displacement that combines machine vision with a passive target. Background Technology

[0002] As a critical infrastructure in transportation, water conservancy, energy and other fields, the safety and stability of tunnels directly depend on the displacement state of the surrounding rock and structure. During the tunnel construction and operation period (such as long-term loads, environmental erosion, and changes in geological conditions), risks such as surrounding rock collapse, lining cracking, and structural settlement may occur.

[0003] Existing methods for monitoring tunnel displacement typically involve periodically dispatching personnel with measuring tools to collect data from the tunnel under inspection. The collected data is then transmitted to evaluation experts via communication devices. The experts calculate the tunnel's displacement based on this data. However, improper operation by personnel during data collection can lead to inaccurate data, resulting in insufficient accuracy in monitoring and issuing early warnings about tunnel displacement. Summary of the Invention

[0004] This application provides an automatic tunnel displacement monitoring method that combines machine vision with a passive target, which can improve the accuracy of displacement monitoring and early warning of tunnels under inspection.

[0005] A first aspect of this application provides a method for automatic monitoring of tunnel displacement combining machine vision and a passive target, the method comprising:

[0006] An image acquisition device is used to acquire images of a passive target in the tunnel to be inspected, resulting in a set of images to be processed.

[0007] The image corresponding to the region where the passive target is located is extracted from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction on the image to be processed in the image to be processed set.

[0008] Extract the coordinate position information of each pixel in the third image set to obtain k sets of coordinate position information. The third image set includes k third images.

[0009] Extract the brightness value corresponding to each pixel in the third image set to obtain a set of k brightness values;

[0010] Based on the shooting time corresponding to each third image in the third image set, the coordinate position information in the k coordinate position information set, and the brightness value in the k brightness value set, construct the optical flow function corresponding to each pixel in the third image set, and obtain the k optical flow function set;

[0011] Determine the cost function corresponding to each optical flow function in the set of k optical flow functions to obtain the set of k cost functions;

[0012] The least squares method is used to minimize the cost functions in the set of k cost functions and solve the corresponding optical flow functions to obtain the set of k first velocity information. The first velocity information in the first velocity information set represents the optical flow velocity of the pixel in the third image.

[0013] Based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, the displacement information corresponding to each pixel in the passive target in the third image set is calculated to obtain the target displacement information set;

[0014] Early warning information is generated based on the target displacement information in the target displacement information set, and an early warning is issued based on the early warning information.

[0015] In this example, an image acquisition device is used to acquire images of a passive target in the tunnel to be detected, resulting in a set of images to be processed. The image corresponding to the region where the passive target is located is extracted from these images, resulting in a third image set. The third images in this third image set are obtained by performing image enhancement, binarization, and contour extraction on the images to be processed in the set. An improved optical flow method is used to calculate the optical flow velocity corresponding to each pixel in each third image in the third image set, resulting in k sets of first velocity information. Based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the k sets of first velocity information... The first velocity information in the set is used to calculate the displacement information corresponding to each pixel in the passive target in the third image of the third image set, to obtain the target displacement information set. Warning information is generated based on the target displacement information in the target displacement information set, and a warning is issued based on the warning information. Therefore, by calculating the optical flow velocity of the image containing the passive target, and calculating the tunnel displacement based on the optical flow velocity and the light intensity information of the corresponding image, the target displacement information set is obtained. Finally, the warning information generated based on the information in the target displacement information set is used for warning processing, thus eliminating the need for manual processing, avoiding errors caused by manual processing, and improving the accuracy of tunnel displacement monitoring and warning.

[0016] In one possible implementation, a method for acquiring real-time images of a preset passive target in a tunnel to be detected using an image acquisition device to obtain a set of images to be processed includes:

[0017] Adjust the shooting angle of the image acquisition device according to the preset shooting angle;

[0018] Using an image acquisition device with an adjusted shooting angle, real-time images of a preset passive target in the tunnel to be inspected are acquired at preset shooting time intervals to obtain a set of images to be processed.

[0019] In one possible implementation, a method for extracting the image corresponding to the region where the passive target is located from the images to be processed in a set of images to be processed, thereby obtaining a third image set, wherein the third image in the third image set is obtained by performing image enhancement, binarization, and contour extraction processing on the images to be processed in the set of images to be processed, includes:

[0020] Image enhancement processing is performed on each image in the image set to be processed to obtain a first image set;

[0021] The first image in the first image set is binarized to obtain the second image set;

[0022] Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

[0023] In one possible implementation, a method for calculating displacement information of each pixel in a passive target within the third image set based on illumination intensity information of each pixel in each third image in the third image set and first velocity information from the k sets of first velocity information, to obtain a target displacement information set, includes:

[0024] Determine the illumination intensity information corresponding to each pixel in each third image in the third image set to obtain k sets of first illumination intensity information;

[0025] Based on the first light intensity information in the k first light intensity information sets, the first velocity information in the k first velocity information sets is corrected to obtain k second velocity information sets;

[0026] The target displacement information is obtained by integrating the second velocity information in the k sets of second velocity information.

[0027] A second aspect of this application provides an automatic tunnel displacement monitoring device combining machine vision and a passive target, the device comprising:

[0028] The image acquisition unit is used to acquire images of a passive target in the tunnel to be inspected using an image acquisition device, and obtain a set of images to be processed.

[0029] An extraction unit is used to extract the image corresponding to the region where the passive target is located from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction on the image to be processed in the image to be processed set.

[0030] The first calculation unit is used to extract the coordinate position information corresponding to the pixels in each third image of the third image set, to obtain k sets of coordinate position information, wherein the third image set includes k third images; extract the brightness value corresponding to the pixels in each third image of the third image set, to obtain k sets of brightness values; construct the optical flow function corresponding to the pixels in each third image of the third image set based on the shooting time corresponding to each third image of the third image set, the coordinate position information in the k sets of coordinate position information, and the brightness value in the k sets of brightness values, to obtain k sets of optical flow functions; determine the cost function corresponding to each optical flow function in the k sets of optical flow functions, to obtain k sets of cost functions; and use the least squares method to minimize the cost functions in the k sets of cost functions while solving the corresponding optical flow functions to obtain k sets of first velocity information, wherein the first velocity information in the first sets of velocity information represents the optical flow velocity corresponding to the pixels in the third image;

[0031] The second calculation unit is used to calculate the displacement information corresponding to each pixel in the passive target in the third image of the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, so as to obtain the target displacement information set.

[0032] The early warning unit is used to generate early warning information based on the target displacement information in the target displacement information set, and to issue an early warning based on the early warning information.

[0033] In one possible implementation, the image acquisition unit is specifically used for:

[0034] Adjust the shooting angle of the image acquisition device according to the preset shooting angle;

[0035] Using an image acquisition device with an adjusted shooting angle, real-time images of a preset passive target in the tunnel to be inspected are acquired at preset shooting time intervals to obtain a set of images to be processed.

[0036] In one possible implementation, the extraction unit is specifically used for:

[0037] Image enhancement processing is performed on each image in the image set to be processed to obtain a first image set;

[0038] The first image in the first image set is binarized to obtain the second image set;

[0039] Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

[0040] In one possible implementation, the second computing unit is specifically used for:

[0041] Determine the illumination intensity information corresponding to each pixel in each third image in the third image set to obtain k sets of first illumination intensity information;

[0042] Based on the first light intensity information in the k first light intensity information sets, the first velocity information in the k first velocity information sets is corrected to obtain k second velocity information sets;

[0043] The target displacement information is obtained by integrating the second velocity information in the k sets of second velocity information.

[0044] A third aspect of this application provides a terminal including a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0045] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of this application.

[0046] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This application provides a schematic diagram of the network architecture for an automatic tunnel displacement monitoring method that combines machine vision and passive targets.

[0049] Figure 2 This application provides a flowchart illustrating an automatic tunnel displacement monitoring method combining machine vision and a passive target.

[0050] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0051] Figure 4 This application provides a schematic diagram of the structure of an automatic tunnel displacement monitoring device that combines machine vision with a passive target. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0054] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0055] To better understand the automatic tunnel displacement monitoring method combining machine vision and passive targets provided in this application, we will first introduce the existing tunnel early warning monitoring methods. In existing tunnel displacement monitoring methods, personnel are typically dispatched periodically with relevant measuring tools to collect data from the tunnel under inspection. The collected data is then transmitted to evaluation experts via communication devices. The evaluation experts calculate the tunnel displacement information based on the returned data. However, improper operation by personnel when collecting data from the tunnel under inspection can lead to inaccurate data, resulting in insufficient accuracy in tunnel displacement monitoring and early warning.

[0056] To address the aforementioned technical problems, this application provides an automatic tunnel displacement monitoring method combining machine vision and passive targets. By calculating the optical flow velocity of an image containing a passive target, and calculating the tunnel displacement based on the optical flow velocity and the corresponding image's illumination intensity information, a target displacement information set is obtained. Finally, warning information is generated based on the information in this target displacement information set for warning processing. This eliminates the need for manual processing, avoids errors caused by manual processing, and improves the accuracy of tunnel displacement monitoring and warning.

[0057] Please see Figure 1 , Figure 1 This application provides a schematic diagram of a tunnel displacement monitoring network system. The system includes an image acquisition device 1, a passive target 2, a server 3, and a visualization module 4. The passive target 2 is placed in the tunnel to be monitored. The image acquisition device 1 performs image acquisition tasks, capturing real-time images of the area where the passive target 2 is located. The image acquisition device 1 transmits the acquired images to the server 3 via a wireless transmission protocol. The server 3 calculates the optical flow velocity of the image containing the passive target based on the received images, and calculates the tunnel displacement based on the optical flow velocity and the corresponding image's illumination intensity information, obtaining a target displacement information set. Finally, based on the information in this target displacement information set, an early warning message is generated and transmitted to the visualization module 4 for visualization. The image acquisition device can be an industrial camera, etc.

[0058] Please see Figure 2 , Figure 2 This application provides a flowchart illustrating an automatic tunnel displacement monitoring method combining machine vision and a passive target, as described in an embodiment of the present application. Figure 2 As shown, the automatic tunnel displacement monitoring method combining machine vision and passive targets is applied to a tunnel displacement monitoring network system. This method includes:

[0059] 201. Use an image acquisition device to acquire images of the passive target in the tunnel to be inspected, and obtain a set of images to be processed.

[0060] One method involves placing passive targets within the tunnel to be inspected, such as on the top or side walls. An industrial camera or other image acquisition device is then used to capture images of the area containing the passive targets at a preset shooting angle, resulting in a set of images to be processed. The preset angle is an angle adapted for image acquisition and is set based on empirical values ​​or historical data.

[0061] 202. Extract the image corresponding to the region where the passive target is located from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction processing on the image to be processed in the image to be processed set.

[0062] This can be achieved by performing image enhancement and binarization on the images to be processed in the image set to be processed, to obtain a second image set, and then extracting the image corresponding to the region where the passive target is located from the second image in the second image set to obtain a third image set.

[0063] 203. Extract the coordinate position information of each pixel in the third image set to obtain k coordinate position information sets. The third image set includes k third images.

[0064] This can be achieved by extracting the coordinate position information of each pixel in the third image set using a general image pixel coordinate position information extraction method, resulting in k sets of coordinate position information.

[0065] 204. Extract the brightness value corresponding to each pixel in the third image in the third image set to obtain a set of k brightness values.

[0066] This can be achieved by debinding the third image in the third image set using a common debinding method to obtain the fourth image set; and by using common programming languages ​​such as Python or MATLAB, or common visualization tools such as Photoshop or GIMP, to extract the brightness corresponding to each pixel in each fourth image in the fourth image set, thus obtaining a set of k brightness values.

[0067] 205. Based on the shooting time corresponding to each third image in the third image set, the coordinate position information in the k coordinate position information set, and the brightness value in the k brightness value set, construct the optical flow function corresponding to each pixel in the third image set, and obtain the k optical flow function set.

[0068] Since the core assumption of the optical flow method is that the movement of objects or points in the scene does not cause significant changes in brightness in consecutive image frames, that is, the brightness value of the image remains unchanged during the movement of the object (brightness consistency assumption), the motion trajectory of the passive target in the image to be processed can be described by the brightness values ​​in the set of k brightness values ​​and the set of k coordinate position information.

[0069] Specifically, this can be achieved by extracting the shooting time corresponding to each third image in the third image set to obtain the target shooting time set; constructing the displacement-luminance equation corresponding to each pixel of the passive target in each third image in the third image set based on the coordinate position information in the k coordinate position information set, the luminance values ​​in the k luminance value set, and the target shooting time in the target shooting time set, thus obtaining a set of k displacement-luminance functions; and expanding the displacement-luminance functions in the set of k displacement-luminance functions using Taylor expansion to obtain a set of k optical flow functions.

[0070] Specifically, the optical flow function for each pixel in the third image can be constructed using the following formula: based on the shooting time of each third image in the third image set, the coordinate position information in the k coordinate position information sets, and the brightness values ​​in the k brightness value sets, resulting in a set of k optical flow functions.

[0071] ,

[0072] In the formula, I(x,y,T) t The ) represents the brightness value in the set of k brightness values, which can be understood as the brightness value corresponding to the pixel with coordinates (x, y) in the passive target image captured at the t-th target shooting time in the target shooting time set T; x represents the horizontal coordinate in the set of k coordinate position information, which can be understood as the horizontal coordinate of the pixel in the passive target image; y represents the vertical coordinate in the set of k coordinate position information, which can be understood as the vertical coordinate of the pixel in the passive target image; T t The t-th target shooting time in the target shooting time set T can be understood as the shooting time of the t-th image to be processed in the image set to be processed, that is, the shooting time of the t-th third image in the third image set; This represents the change in the x-coordinate of the pixel of the passive target in two adjacent third images in the third image set; This represents the change in the ordinate of the pixel of the passive target in two adjacent third images in the third image set; This indicates the preset shooting time interval; u represents the pixel position of the passive target in the third image. The optical flow rate in the direction can be understood as the component of the optical flow velocity of the passive target pixels in the third image along the x-axis of the coordinate system, with the center point of the third image as the origin, the length extension direction of the third image as the x-axis, and the width extension direction of the third image as the y-axis. v represents the optical flow rate of the passive target pixels in the third image along the y-axis, which can be understood as the component of the optical flow velocity of the passive target pixels in the third image along the y-axis of the coordinate system. The pixels representing the passive target in the third image are... Brightness gradient in direction; The pixels representing the passive target in the third image are... Brightness gradient in direction; This represents the temporal gradient corresponding to the pixel of the passive target in the third image; in the formula... Let represent the optical flow function in the set of k optical flow functions.

[0073] Among them, the optical flow functions in the set of k optical flow functions can be obtained by... The result is obtained by Taylor expansion.

[0074] 206. Determine the cost function corresponding to each optical flow function in the set of k optical flow functions to obtain the set of k cost functions.

[0075] Since the optical flow function in the set of optical flow functions includes two unknowns, u and v, and the optical flow function in the set of optical flow functions only contains one equation, it is not possible to solve the optical flow function directly. In order to solve the optical flow function, it is necessary to establish other constraints to assist in solving the optical flow function in the set of k optical flow functions.

[0076] After obtaining the set of k optical flow functions, the local smoothing assumption theory posits that the motion velocities of adjacent pixels in an image are similar, or that the optical flow field is smooth. This means that the motion differences between adjacent pixels will not be significant. Therefore, the optical flow functions in the set of k optical flow functions can be regularized according to preset weight information using the local smoothing assumption theory to determine the cost function corresponding to each optical flow function in the set of k optical flow functions, thus obtaining a set of k cost functions.

[0077] Specifically, the method shown in the following formula can be used to regularize the optical flow functions in the set of k optical flow functions based on the local smoothing assumption theory and preset weight information, thereby determining the cost function corresponding to each optical flow function in the set of k optical flow functions, and obtaining a set of k cost functions:

[0078] ,

[0079] In the formula represents the cost function in the set of k cost functions; all pixels represents all pixels in each third image in the set of third images; u represents the optical flow rate in the x direction of the pixel with coordinates (x,y) of the passive target in the third image; v represents the optical flow rate in the y direction of the pixel with coordinates (x,y) of the passive target in the third image. This represents the brightness gradient of the pixel with coordinates (x, y) of the passive target in the third image in the x-direction. This represents the brightness gradient in the y-direction of the pixel with coordinates (x, y) of the passive target in the third image. The temporal gradient α represents the pixel with coordinates (x, y) of the passive target in the third image; the preset weight information represents the weight information. This represents the gradient of u, and its purpose is to convert u from a scalar to a vector, i.e., the optical flow velocity; This represents the gradient of v, and its purpose is to convert v from a scalar to a vector, i.e., the optical flow velocity.

[0080] 207. Using the least squares method, while minimizing the cost functions in the set of k cost functions, the corresponding optical flow functions are solved to obtain the set of k first velocity information. The first velocity information in the first velocity information set represents the optical flow velocity corresponding to the pixel in the third image.

[0081] This can be achieved by using a general least squares method to minimize the cost functions in the set of k cost functions while simultaneously solving for the optical flow functions in the set of k optical flow functions, thus obtaining k sets of first velocity information. The first velocity information in these k sets includes the optical flow velocity in the x-direction and the optical flow velocity in the y-direction corresponding to each pixel in the passive target.

[0082] 208. Calculate the displacement information corresponding to each pixel in the passive target in the third image of the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, and obtain the target displacement information set.

[0083] Specifically, this can be achieved by calculating the illumination intensity information corresponding to each image in the image set to be detected when acquiring each image in the image set to be detected, based on each third image in the third image set, to obtain an illumination intensity information set; using the illumination intensity information in the illumination intensity information set to correct the first velocity information in the first velocity information set, to obtain a second velocity information set; and integrating the second velocity information in the second velocity information set to obtain the target displacement information.

[0084] 209. Generate early warning information based on the target displacement information in the target displacement information set, and issue an early warning based on the early warning information.

[0085] This can be achieved by comparing the target displacement information in the target displacement information set with a preset displacement threshold to determine if any target displacement information in the target displacement information set exceeds the preset threshold. If all target displacement information in the target displacement information set is less than or equal to the preset displacement threshold, it indicates that the tunnel under inspection is in normal condition, and the warning information generated based on the target displacement information is empty. If there is target displacement information in the target displacement information set that exceeds the preset displacement threshold, it indicates that the tunnel under inspection has undergone significant deformation, and the cause of the tunnel displacement needs to be investigated immediately. Therefore, this can be achieved by extracting the target position information corresponding to the passive target of the target displacement information. The warning information generated based on the target position information and the target displacement information can be, for example, "At the location indicated by the target position information, the tunnel displacement is abnormal, and the displacement is the target displacement information. Please dispatch personnel to investigate the cause immediately." The specific content of the warning information described above is for illustrative purposes only and does not limit the specific content of the warning information.

[0086] Specifically, when the target displacement information set contains target displacement information exceeding a preset displacement threshold, a dynamic 3D model can be obtained by using a general 3D modeling method to dynamically model the passive target based on the target displacement information in the target displacement information set. This dynamic 3D model reflects the change relationship between the displacement of the passive target and time during monitoring, thereby reflecting the change relationship between the displacement of the tunnel and time during monitoring. The dynamic 3D model is then transmitted to a visualization module for visualization, allowing staff to intuitively obtain the tunnel's displacement status. The 3D model is then input into a pre-trained prediction model, which performs a force analysis on the tunnel at the location indicated by the target location information based on the input 3D model, thereby predicting the subsequent displacement of the tunnel and obtaining prediction information. This prediction information is then integrated into an early warning message, for example: "At the location indicated by the target location information, the tunnel displacement is abnormal, the displacement is the target displacement information, and the subsequent displacement will intensify. Please dispatch personnel promptly to investigate the cause."

[0087] In one possible implementation, a method for acquiring real-time images of a preset passive target in a tunnel to be detected using an image acquisition device to obtain a set of images to be processed includes:

[0088] A1. Adjust the shooting angle of the image acquisition device according to the preset shooting angle;

[0089] A2. Using an image acquisition device with an adjusted shooting angle, real-time image acquisition is performed on a preset passive target in the tunnel to be inspected at preset shooting time intervals to obtain a set of images to be processed.

[0090] Because large vehicles passing through tunnels can cause resonance in the image acquisition device, which in turn changes the shooting angle of the device, resulting in insufficient accuracy of the images acquired, the shooting angle of the image acquisition device needs to be adjusted before acquiring the images.

[0091] Specifically, passive targets in shapes such as rectangles, circles, triangles, crosses, or U-shapes can be placed in areas such as the tunnel arch and sidewalls of the tunnel to be inspected. The image acquisition device is then adjusted to a preset shooting angle before image acquisition is performed on the area containing the passive targets. When initially using the image acquisition device to acquire images of the passive targets, the shooting angle must be adjusted to ensure that the center of the passive target coincides with the center of the image acquired by the device. This shooting angle is then recorded as the preset shooting angle.

[0092] After adjusting the shooting angle of the image acquisition device, the device can maintain the preset shooting angle and perform real-time image acquisition on the passive target in the tunnel according to a preset shooting time interval to obtain a set of images to be processed. The preset shooting time interval can be determined by user input or by system default.

[0093] In one possible implementation, a method for extracting the image corresponding to the region where the passive target is located from the images to be processed in a set of images to be processed, thereby obtaining a third image set, wherein the third image in the third image set is obtained by performing image enhancement, binarization, and contour extraction processing on the images to be processed in the set of images to be processed, includes:

[0094] B1. Perform image enhancement processing on each image in the image set to be processed to obtain the first image set;

[0095] B2. Binarize the first image in the first image set to obtain the second image set;

[0096] B3. Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

[0097] Because the light in the tunnel is dim, the image quality of the images to be processed acquired by the image acquisition device is poor. In order to better extract the optical flow information in the images to be processed in the image set, it is necessary to perform image enhancement processing on the images to be processed in the image set.

[0098] This can be achieved by using a general image enhancement method to enhance the images in the image set to be processed, thereby obtaining the first image set.

[0099] After obtaining the first image set, the first image in the first image set can be binarized using a general image binarization method to obtain the second image set.

[0100] Since passive targets are generally rectangular, circular, triangular, cross-shaped, or U-shaped and are made of reflective materials, the passive targets in each second image in the second image set have a sharp contrast with the tunnel walls in the background. Therefore, the gray values ​​of the area corresponding to the passive targets in each second image in the second image set are significantly different from the gray values ​​of the area corresponding to the tunnel walls.

[0101] After obtaining the second image set, since the passive targets in the second images of the second image set may have slight breaks at their edges due to dust in the air, general image dilation and erosion operations can be applied to the second images in the second image set to eliminate the slight breaks at the edges caused by dust in the air, thus obtaining the second optimized image set. Then, a general global thresholding segmentation algorithm, such as Otsu's Thresholding Method, is used to segment the second optimized images in the second optimized image set, separating the image corresponding to the region where the passive target is located in each second optimized image in the second optimized image set, thus obtaining the third image set.

[0102] In this example, image enhancement processing is performed on the images to be processed in the image set to be processed, thereby improving the image quality of the images to be processed in the image set to be processed, resulting in a first image set; binarization processing is performed on the first image in the first image set to increase the distinction between the passive target and the tunnel wall in the background, thereby improving the accuracy of the image corresponding to the area where the passive target is located in the second image in the extracted second image set, and thus improving the accuracy of tunnel displacement monitoring of the tunnel to be detected.

[0103] In one possible implementation, a method for performing image detection on the third images in the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, and calculating the displacement information corresponding to each pixel in the passive target to obtain a target displacement information set, includes:

[0104] C1. Determine the illumination intensity information corresponding to each pixel in each third image in the third image set to obtain k sets of first illumination intensity information;

[0105] C2. Based on the first light intensity information in the k first light intensity information sets, the first velocity information in the k first velocity information sets is corrected to obtain the k second velocity information sets;

[0106] C3. Integrate the second velocity information in the set of k second velocity information to obtain the target displacement information.

[0107] Specifically, this can be achieved by calculating the reflectance and transmittance of each pixel in each third image in the third image set according to a general atmospheric transmission / reflection model, thus obtaining k sets of first reflectance and k sets of first transmittance; and by calculating the illumination intensity information corresponding to each pixel in each third image in the third image set based on the first reflectance in the k sets of first reflectance, the first transmittance in the k sets of first transmittance, and the brightness values ​​in the k sets of brightness values, thus obtaining k sets of first illumination intensity information.

[0108] Specifically, the illumination intensity information corresponding to each pixel in the third image set can be calculated based on the first reflectance from the k first reflectance sets, the first transmittance from the k first transmittance sets, and the brightness values ​​from the k brightness value sets, as shown in the following formula, thus obtaining k sets of first illumination intensity information:

[0109] ,

[0110] I in the formula n (x, y) represents the first illumination intensity information in the set of k first illumination intensity information, which can be understood as the illumination intensity information corresponding to the pixel with coordinates (x, y) in the nth third image in the third image set; L n (x, y) represents the brightness value in the set of k brightness values, which can be understood as the brightness value corresponding to the pixel with coordinates (x, y) in the nth third image in the third image set; r n (x, y) represents the first reflectance in the set of k first reflectances, which can be understood as the first reflectance of the pixel with coordinates (x, y) in the nth third image of the third image set; mn (x,y) represents the first transmittance in the set of k first transmittances, which can be understood as the first transmittance of the pixel with coordinates (x,y) in the nth third image in the third image set.

[0111] Since steps 203-207 are based on the premise that the light intensity information in the tunnel remains unchanged, but the light intensity information in the actual scene will change, the accuracy of the optical flow velocity obtained by steps 203-207 is insufficient. It is necessary to combine the light intensity information to correct the optical flow velocity obtained by steps 203-207 to further improve the accuracy of the obtained optical flow velocity.

[0112] After obtaining k first illumination intensity information, the first illumination intensity information in the set of k first illumination intensity information and the first velocity information in the set of k first velocity information can be input into a preset optical flow velocity correction model. The preset optical flow velocity correction model can automatically correct the first velocity information in the set of k first velocity information based on the input first illumination intensity information in the set of k first illumination intensity information to obtain k second velocity information sets.

[0113] Since the velocity-time curve of an object can be integrated over a given time interval to represent the displacement of the object within that time interval, the displacement information of a passive target can be determined by integrating the second velocity information from k sets of second velocity information.

[0114] Specifically, a general function curve construction method can be used to construct a velocity-time curve corresponding to each pixel in the passive target based on the second velocity information in the k second velocity information sets and the target shooting time information in the target shooting time information set, thus obtaining a second function set; a general integration processing method can be used to integrate the second velocity information in each second function in the second function set to obtain a target displacement information set.

[0115] In this example, by calculating the illumination intensity information corresponding to each pixel in the third image set, k sets of first illumination intensity information are obtained. Based on the first illumination intensity information in the k sets of first illumination intensity information, the first velocity in the k sets of first velocity information is corrected to obtain k sets of second velocity information. This further improves the accuracy of the optical flow velocity in the x and y directions corresponding to each pixel in the passive target. The second velocity information in the k sets of second velocity information is integrated to obtain the target displacement information set, thereby improving the accuracy of tunnel displacement monitoring in the tunnel to be detected.

[0116] For examples consistent with the above embodiments, please refer to... Figure 3 , Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application, such as... Figure 3 As shown, it includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions. The program includes instructions for performing the following steps.

[0117] An image acquisition device is used to acquire images of a passive target in the tunnel to be inspected, resulting in a set of images to be processed.

[0118] The image corresponding to the region where the passive target is located is extracted from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction on the image to be processed in the image to be processed set.

[0119] Extract the coordinate position information of each pixel in the third image set to obtain k sets of coordinate position information. The third image set includes k third images.

[0120] Extract the brightness value corresponding to each pixel in the third image set to obtain a set of k brightness values;

[0121] Based on the shooting time corresponding to each third image in the third image set, the coordinate position information in the k coordinate position information set, and the brightness value in the k brightness value set, construct the optical flow function corresponding to each pixel in the third image set, and obtain the k optical flow function set;

[0122] Determine the cost function corresponding to each optical flow function in the set of k optical flow functions to obtain the set of k cost functions;

[0123] The least squares method is used to minimize the cost functions in the set of k cost functions and solve the corresponding optical flow functions to obtain the set of k first velocity information. The first velocity information in the first velocity information set represents the optical flow velocity of the pixel in the third image.

[0124] Based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, the displacement information corresponding to each pixel in the passive target in the third image set is calculated to obtain the target displacement information set;

[0125] Early warning information is generated based on the target displacement information in the target displacement information set, and an early warning is issued based on the early warning information.

[0126] The above mainly describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] This application embodiment can divide the terminal into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0128] For those consistent with the above, please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an automatic tunnel displacement monitoring device combining machine vision and a passive target, as described in an embodiment of the present application. Figure 4 As shown, the device includes:

[0129] The image acquisition unit 401 is used to acquire images of a passive target in the tunnel to be detected using an image acquisition device, and obtain a set of images to be processed.

[0130] Extraction unit 402 is used to extract the image corresponding to the region where the passive target is located from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction processing on the image to be processed in the image to be processed set.

[0131] The first calculation unit 403 is used to extract the coordinate position information corresponding to the pixel in each third image of the third image set, to obtain k sets of coordinate position information, wherein the third image set includes k third images; extract the brightness value corresponding to the pixel in each third image of the third image set, to obtain k sets of brightness values; construct the optical flow function corresponding to the pixel in each third image of the third image set based on the shooting time corresponding to each third image in the third image set, the coordinate position information in the k sets of coordinate position information, and the brightness value in the k sets of brightness values, to obtain k sets of optical flow functions; determine the cost function corresponding to each optical flow function in the k sets of optical flow functions, to obtain k sets of cost functions; and use the least squares method to minimize the cost function in the k sets of cost functions while solving the corresponding optical flow function to obtain k sets of first velocity information, wherein the first velocity information in the first sets of velocity information represents the optical flow velocity corresponding to the pixel in the third image;

[0132] The second calculation unit 404 is used to calculate the displacement information corresponding to each pixel in the passive target in the third image of the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, so as to obtain the target displacement information set.

[0133] The early warning unit 405 is used to generate early warning information based on the target displacement information in the target displacement information set, and to issue an early warning based on the early warning information.

[0134] In one possible implementation, the image acquisition unit 401 is specifically used for:

[0135] Adjust the shooting angle of the image acquisition device according to the preset shooting angle;

[0136] Using an image acquisition device with an adjusted shooting angle, real-time images of a preset passive target in the tunnel to be inspected are acquired at preset shooting time intervals to obtain a set of images to be processed.

[0137] In one possible implementation, the extraction unit 402 is specifically used for:

[0138] Image enhancement processing is performed on each image in the image set to be processed to obtain a first image set;

[0139] The first image in the first image set is binarized to obtain the second image set;

[0140] Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

[0141] In one possible implementation, the second computing unit 404 is specifically used for:

[0142] Determine the illumination intensity information corresponding to each pixel in each third image in the third image set to obtain k sets of first illumination intensity information;

[0143] Based on the first light intensity information in the k first light intensity information sets, the first velocity information in the k first velocity information sets is corrected to obtain k second velocity information sets;

[0144] The target displacement information is obtained by integrating the second velocity information in the k sets of second velocity information.

[0145] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the machine vision and passive target combined tunnel displacement automatic monitoring methods described in the above method embodiments.

[0146] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of any of the machine vision and passive target combined tunnel displacement automatic monitoring methods described in the above method embodiments.

[0147] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0152] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0153] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0154] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for automatic monitoring of tunnel displacement combining machine vision and a passive target, characterized in that, The method includes: An image acquisition device is used to acquire images of a passive target in the tunnel to be inspected, resulting in a set of images to be processed. The image corresponding to the region where the passive target is located is extracted from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction on the image to be processed in the image to be processed set. Extract the coordinate position information corresponding to the pixel in each third image in the third image set to obtain k sets of coordinate position information. The third image set includes k third images. Extract the brightness value corresponding to each pixel in the third image set to obtain a set of k brightness values; Based on the shooting time corresponding to each third image in the third image set, the coordinate position information in the k coordinate position information set, and the brightness value in the k brightness value set, construct the optical flow function corresponding to each pixel in the third image set, and obtain the k optical flow function set; Determine the cost function corresponding to each optical flow function in the set of k optical flow functions to obtain the set of k cost functions; The least squares method is used to minimize the cost functions in the set of k cost functions and solve the corresponding optical flow functions to obtain the set of k first velocity information. The first velocity information in the first velocity information set represents the optical flow velocity of the pixel in the third image. Based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, the displacement information corresponding to each pixel in the passive target in the third image set is calculated to obtain the target displacement information set; Early warning information is generated based on the target displacement information in the target displacement information set, and an early warning is issued based on the early warning information.

2. The automatic tunnel displacement monitoring method combining machine vision and passive target as described in claim 1, characterized in that, The image acquisition device is used to acquire real-time images of a preset passive target in the tunnel to be detected, resulting in a set of images to be processed, including: Adjust the shooting angle of the image acquisition device according to the preset shooting angle; Using an image acquisition device with an adjusted shooting angle, real-time images of a preset passive target in the tunnel to be inspected are acquired at preset shooting time intervals to obtain a set of images to be processed.

3. The automatic tunnel displacement monitoring method combining machine vision and passive target according to claim 2, characterized in that, The process involves extracting the image corresponding to the region where the passive target is located from the images to be processed in the image set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization, and contour extraction processing on the images to be processed in the image set to be processed, including: Image enhancement processing is performed on each image in the image set to be processed to obtain a first image set; The first image in the first image set is binarized to obtain the second image set; Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

4. The automatic tunnel displacement monitoring method combining machine vision and passive target according to claim 3, characterized in that, The step of calculating the displacement information corresponding to each pixel in the passive target in the third image of the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, to obtain the target displacement information set, includes: Determine the illumination intensity information corresponding to each pixel in each third image in the third image set to obtain k sets of first illumination intensity information; Based on the first light intensity information in the k first light intensity information sets, the first velocity information in the k first velocity information sets is corrected to obtain k second velocity information sets; The target displacement information is obtained by integrating the second velocity information in the k sets of second velocity information.

5. An automatic tunnel displacement monitoring device combining machine vision and a passive target, characterized in that, The device includes: The image acquisition unit is used to acquire images of a passive target in the tunnel to be inspected using an image acquisition device, and obtain a set of images to be processed. An extraction unit is used to extract the image corresponding to the region where the passive target is located from the image to be processed in the image to be processed set to obtain a third image set. The third image in the third image set is obtained by performing image enhancement, binarization and contour extraction processing on the image to be processed in the image to be processed set. The first calculation unit is used to extract the coordinate position information corresponding to the pixels in each third image of the third image set, to obtain k sets of coordinate position information, wherein the third image set includes k third images; extract the brightness value corresponding to the pixels in each third image of the third image set, to obtain k sets of brightness values; construct the optical flow function corresponding to the pixels in each third image of the third image set based on the shooting time corresponding to each third image of the third image set, the coordinate position information in the k sets of coordinate position information, and the brightness value in the k sets of brightness values, to obtain k sets of optical flow functions; determine the cost function corresponding to each optical flow function in the k sets of optical flow functions, to obtain k sets of cost functions; and use the least squares method to minimize the cost functions in the k sets of cost functions while solving the corresponding optical flow functions to obtain k sets of first velocity information, wherein the first velocity information in the first sets of velocity information represents the optical flow velocity corresponding to the pixels in the third image; The second calculation unit is used to calculate the displacement information corresponding to each pixel in the passive target in the third image of the third image set based on the illumination intensity information corresponding to each pixel in each third image in the third image set and the first velocity information in the k first velocity information sets, so as to obtain the target displacement information set. The early warning unit is used to generate early warning information based on the target displacement information in the target displacement information set, and to issue an early warning based on the early warning information.

6. The tunnel displacement automatic monitoring device combining machine vision and passive target according to claim 5, characterized in that, In the process of using an image acquisition device to acquire real-time images of a preset passive target in the tunnel to be detected, and obtaining a set of images to be processed, the image acquisition unit is specifically used for: Adjust the shooting angle of the image acquisition device according to the preset shooting angle; Using an image acquisition device with an adjusted shooting angle, real-time images of a preset passive target in the tunnel to be inspected are acquired at preset shooting time intervals to obtain a set of images to be processed.

7. The tunnel displacement automatic monitoring device combining machine vision and passive target according to claim 6, characterized in that, The extraction unit extracts the image corresponding to the region where the passive target is located from the images to be processed in the image set to be processed, thus obtaining a third image set. The third image in the third image set is obtained by performing image enhancement, binarization, and contour extraction processing on the images to be processed in the image set to be processed. The extraction unit is specifically used for: Image enhancement processing is performed on each image in the image set to be processed to obtain a first image set; The first image in the first image set is binarized to obtain the second image set; Extract the image corresponding to the region where the passive target is located from the second image in the second image set to obtain the third image set.

8. A terminal, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the automatic tunnel displacement monitoring method combining machine vision and passive targets as described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the automatic tunnel displacement monitoring method combining machine vision and passive target as described in any one of claims 1-4.

Images