Tunnel structure multi-parameter intelligent monitoring system based on machine vision

Abstract

The invention discloses a tunnel structure multi-parameter intelligent monitoring system based on machine vision, and relates to the technical field of tunnel monitoring. The method comprises the following steps: setting an acquisition point and an acquisition unit, acquiring an acquisition image, acquiring acquisition image frames at different moments, generating an acquisition image frame set, acquiring image definition and acquisition parameters of each acquisition image frame, acquiring different structural defects and defect images thereof, and constructing an image recognition model; judging whether each acquired image frame has a structural defect or not by using an image recognition model, performing image fusion on the acquired image frames, acquiring the number of fusion frames when each acquired image frame is accurately recognized by combining with the image recognition model, constructing an image fusion model, and performing image fusion on subsequent acquired image frames by using the image fusion model; the machine vision technology can be used for monitoring the interior of the tunnel in real time, and the tunnel monitoring accuracy can be obviously improved.

Description

Technical Field

[0001] This invention relates to the field of tunnel monitoring technology, specifically a machine vision-based intelligent monitoring system for multiple parameters of tunnel structures. Background Technology

[0002] In tunnel structure monitoring, machine vision technology is a common method. It integrates machine vision technology, Internet of Things technology, data processing technology and intelligent analysis algorithms, which can accurately monitor multiple key parameters of the tunnel structure in real time, identify potential structural defects inside the tunnel in a timely manner, and provide a scientific basis for the safe operation and maintenance management of the tunnel. Due to the dynamic changes in the tunnel's internal environment, the image data collected will also change accordingly. This change will directly affect the accuracy of the final recognition result. However, existing technologies lack solutions to address the impact of this change, making it easy to misjudge the image recognition of structural defects inside the tunnel. To address the shortcomings of existing technologies, this invention provides a machine vision-based intelligent monitoring system for multiple parameters of tunnel structures. Summary of the Invention

[0003] The purpose of this invention is to provide a machine vision-based intelligent monitoring system for multiple parameters of tunnel structures.

[0004] The objective of this invention can be achieved through the following technical solution: a machine vision-based intelligent monitoring system for multiple parameters of tunnel structures, comprising the following modules: The image acquisition module is used to set up several acquisition points and their acquisition units inside the tunnel, and to use the acquisition units to acquire the corresponding images. The image processing module is used to acquire image frames at different times based on the acquired images at the same acquisition point, generate a corresponding set of acquired image frames, and obtain the image clarity of each acquired image frame in the same set of acquired image frames and its acquisition parameters at the corresponding time. The image recognition module is used to acquire different structural defects inside the tunnel and their corresponding defect images, and to build an image recognition model. The image recognition model is used to determine whether there are structural defects in each acquired image frame under different image clarity and acquisition parameters. The image fusion module is used to perform image fusion on each acquired image frame in the same set of acquired image frames, and combine it with the image recognition model to obtain the number of fused frames when each acquired image frame is accurately recognized; The image evaluation module is used to construct an image fusion model based on the acquired image frames under different image sharpness and acquisition parameters and the number of fusion frames when the recognition is accurate. The image fusion model is then used to perform image fusion on subsequent acquired image frames.

[0005] Furthermore, the process of setting up several acquisition points and their acquisition units inside the tunnel, and using the acquisition units to acquire the corresponding images, includes: Different acquisition points are set up at different support structures inside the tunnel, and a corresponding acquisition unit is set up for each acquisition point. Each acquisition point is bound to its corresponding acquisition unit, and the acquisition unit acquires the acquisition image of its corresponding acquisition point in real time.

[0006] Furthermore, the process of acquiring image frames at different times based on images acquired from the same acquisition point and generating a corresponding set of acquired image frames includes: Frame decomposition processing is performed on the images acquired at the same acquisition point. The frame decomposition processing refers to dividing the continuous acquired images into a series of individual acquired image frames, each of which has a timestamp. The acquired image frames with corresponding timestamps at each preset time interval are numbered sequentially. Several acquired image frames with adjacent numbers are included in the same acquired image frame set in chronological order. Acquired image frames with the same acquisition point and number are divided into different acquired image frame sets.

[0007] Furthermore, the process of obtaining the image sharpness of each acquired image frame in the same set of acquired image frames and its acquisition parameters at the corresponding time includes: The average pixel value of each acquired image frame in the same set of acquired image frames is used as the image sharpness of the corresponding acquired image frame; Different monitoring units are set up at each collection point, including light monitoring unit, smoke monitoring unit, and amplitude monitoring unit. The collection parameters of the corresponding collection point are obtained in real time through each collection unit, including light intensity, smoke concentration, and vibration amplitude. Each collection parameter has a timestamp. The image sharpness of each acquired image frame in the same set of acquired image frames is bound to the same acquisition parameters at the same timestamp.

[0008] Furthermore, the process of acquiring different structural defects inside the tunnel and their corresponding defect images, and constructing an image recognition model, includes: The structural defects include cracks, water leakage, spalling, deformation, misalignment, joint opening, and exposed rebar. Defect images of different structural defects are obtained. The defect images refer to multi-angle images of the corresponding area of ​​the structural defect inside the tunnel. An image recognition set is generated based on the defect images of different structural defects and their corresponding names, and then divided into a first training set and a first test set. Construct a first convolutional neural network by using different defect images from the first training set as input data and the corresponding structural defects and their names from the first training set as output data. Train the first convolutional neural network using the first training set to obtain an initial first convolutional neural network. The initial first convolutional neural network is validated using the first test set, and the initial first convolutional neural network whose output is less than or equal to the preset first test error threshold is used as the image recognition model.

[0009] Furthermore, the process of using image recognition models to determine whether structural defects exist in each acquired image frame under different image sharpness and acquisition parameters includes: In practical application scenarios, each image frame from the same set of image frames at the same acquisition point is input into the image recognition model to determine whether there are structural defects in each image frame. When a structural defect is found, the name of the corresponding structural defect is output and feedback information is generated to be sent to relevant personnel to notify them to conduct on-site inspection of the acquisition point with structural defects. When a structural defect is found and the name is consistent, the corresponding acquired image frame is marked as accurately identified. Otherwise, it is marked as incorrectly identified. The acquired image frames marked as accurately identified are then bound to their corresponding image clarity and acquisition parameters.

[0010] Furthermore, the process of performing image fusion on each acquired image frame in the same set of acquired image frames, and then combining this with an image recognition model to obtain the number of fused frames when each acquired image frame is accurately recognized, includes: Image fusion is performed on each of the acquired image frames marked as having identification errors within the same set of acquired image frames, and multiple acquired image frames that are adjacent to each other in terms of timestamp are obtained sequentially for each acquired image frame marked as having identification errors. The single image frame marked as incorrectly identified is fused with multiple adjacent image frames in terms of timestamp to obtain a multi-frame fused image. The image recognition model is used to identify the obtained multi-frame fused image to determine whether it has structural defects. When it is marked as correctly identified, the number of fused frames at this time is recorded.

[0011] Furthermore, an image fusion model is constructed based on the acquired image frames under different image sharpness and acquisition parameters, and the number of fused frames when recognition is accurate. The process of using the image fusion model to perform image fusion on subsequent acquired image frames includes: Based on the acquired image frames under different image sharpness and acquisition parameters, and the number of fused frames marked as accurate recognition, an image fusion set is generated and divided into a second training set and a second test set. Construct a second convolutional neural network by using different acquired image frames, their image sharpness, and acquisition parameters from the second training set as input data for the second convolutional neural network, and using the corresponding fused frame number from the second training set as output data for the second convolutional neural network. Train the second convolutional neural network using the second training set to obtain the initial second convolutional neural network. The initial second convolutional neural network is validated using the second test set. The initial second convolutional neural network whose output is less than or equal to the preset second test error threshold is used as the image fusion model. Each acquired image frame, its image clarity, and acquisition parameters in the subsequent acquired image frame set are first input into the image fusion model to obtain the corresponding number of fusion frames. According to the number of fusion frames, the corresponding number of acquired image frames adjacent to each acquired image frame at its timestamp are obtained, and multi-frame fusion is performed to obtain a multi-frame fusion image. The multi-frame fused image acquired at this time is input into the image recognition model to determine whether there is a structural defect. When it is determined that there is a defect, the name of the corresponding structural defect is output and feedback information is generated and sent to relevant personnel to notify them to conduct on-site inspection of the acquisition point with the structural defect.

[0012] Compared with the prior art, the beneficial effects of the present invention are: This invention deploys acquisition points and acquisition units at key locations inside the tunnel, enabling real-time acquisition of images. By performing frame decomposition processing on the acquired images, frames of the acquired images at different times are obtained as the basis for image recognition. This allows for real-time monitoring of the tunnel interior using machine vision technology while reducing the amount of data processing. By judging the accuracy or error of the recognition results of each acquired image frame based on the on-site inspection results, and combining the corresponding acquisition parameters and image clarity, we can analyze the degree of influence of these two factors on the final recognition result. We use multi-frame fusion technology to process each acquired image frame separately and obtain the number of fused frames when the recognition is accurate. We construct an image fusion model that can output the number of fused frames. In subsequent tunnel monitoring, we first perform image fusion on the acquired image frames to improve their clarity, and then judge whether there are structural defects. This can significantly improve the accuracy of tunnel monitoring. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the modules of the present invention. Detailed Implementation

[0014] like Figure 1 As shown, a machine vision-based intelligent monitoring system for multiple parameters of tunnel structures includes the following modules: The image acquisition module is used to set up several acquisition points and their acquisition units inside the tunnel, and to use the acquisition units to acquire the corresponding images. The image processing module is used to acquire image frames at different times based on the acquired images at the same acquisition point, generate a corresponding set of acquired image frames, and obtain the image clarity of each acquired image frame in the same set of acquired image frames and its acquisition parameters at the corresponding time. The image recognition module is used to acquire different structural defects inside the tunnel and their corresponding defect images, and to build an image recognition model. The image recognition model is used to determine whether there are structural defects in each acquired image frame under different image clarity and acquisition parameters. The image fusion module is used to perform image fusion on each acquired image frame in the same set of acquired image frames, and combine it with the image recognition model to obtain the number of fused frames when each acquired image frame is accurately recognized; The image evaluation module is used to construct an image fusion model based on the acquired image frames under different image sharpness and acquisition parameters and the number of fusion frames when the recognition is accurate. The image fusion model is then used to perform image fusion on subsequent acquired image frames.

[0015] It should be further explained that, in the specific implementation process, the process of setting up several acquisition points and their acquisition units inside the tunnel, and using the acquisition units to acquire the corresponding images includes: A corresponding data collection point is set at different support structures inside the tunnel. The support structures include the arch, sidewalls, and inverted arch. A corresponding data collection unit is set for each data collection point. The data collection unit is a camera device. Each acquisition point is bound to its corresponding acquisition unit. The positions of the acquisition points and their acquisition units are fixed. The acquisition units acquire the acquisition images of their corresponding acquisition points in real time. The acquisition images are specifically in video format.

[0016] It should be further explained that, in the specific implementation process, the process of acquiring image frames at different times based on images acquired from the same acquisition point and generating a corresponding set of acquired image frames includes: Taking a single acquisition point as an example, the acquired image of the acquisition point is subjected to frame decomposition processing. The frame decomposition processing refers to dividing the continuous acquired image into a series of separate segments with a fixed time length, and marking the segmented segments as acquired image frames. Each acquired image frame has its corresponding timestamp. The acquired image frames corresponding to the timestamps in each preset time interval are numbered sequentially and denoted as i, i=1,2,...,n, where n is the number of acquired image frames with numbers. The acquired image frames with numbers are used as representatives of each acquired image frame in the preset time interval. Two numbered image frames with adjacent numbers are marked as adjacent image frames. Each group of adjacent image frames contains two image frames. In chronological order, m adjacent image frames with adjacent numbers are included in the same image frame set. This method is used to divide image frames with numbers from the same acquisition point into several image frame sets.

[0017] It should be further explained that, in the specific implementation process, the process of obtaining the image sharpness of each acquired image frame in the same set of acquired image frames and its acquisition parameters at the corresponding time includes: Taking a single set of acquired image frames as an example, computer vision technology is used to obtain the average pixel value of each acquired image frame in the set. The average pixel value refers to the average pixel value of each pixel in the acquired image frame. The obtained average pixel value is used as the image clarity of the corresponding acquired image frame. Different monitoring units are set up at each collection point, including light monitoring unit, smoke monitoring unit, and amplitude monitoring unit. The collection parameters of the corresponding collection point are obtained in real time through each collection unit, including light intensity, smoke concentration, and vibration amplitude. Each collection parameter has its corresponding timestamp. The image clarity of each collection image frame in the same collection image frame set is bound to the collection parameter with the same timestamp.

[0018] It should be further explained that, in the specific implementation process, the process of acquiring different structural defects inside the tunnel and their corresponding defect images, and constructing an image recognition model, includes: For the interior of the tunnel, structural defects include cracks, water leakage, spalling, deformation, misalignment, joint opening, exposed steel bars, etc., and defect images of different structural defects are obtained. The defect images refer to multi-angle images of the corresponding area of ​​the structural defect inside the tunnel, which can reflect the morphological characteristics of the corresponding structural defects. An image recognition set is generated based on the defect images of different structural defects and their corresponding names, and then divided into a first training set and a first test set. Construct a first convolutional neural network by using different defect images from the first training set as input data and the corresponding structural defects and their names from the first training set as output data. Train the first convolutional neural network using the first training set to obtain an initial first convolutional neural network. The initial first convolutional neural network is validated using the first test set. The initial first convolutional neural network whose output is less than or equal to the preset first test error threshold is used as the image recognition model. The image recognition model can identify the structural defects contained in the input image and output their names.

[0019] It should be further explained that, in the specific implementation process, the process of using image recognition models to determine whether there are structural defects in each acquired image frame under different image sharpness and acquisition parameters includes: In practical application scenarios, the image frames from the same set of image frames at the same acquisition point are input into the image recognition model to determine whether each image frame has structural defects. If no defects are found, no other operations are performed on it. When a structural defect is detected, the name of the corresponding defect is output, and corresponding feedback information is generated and sent to relevant personnel to notify them to conduct on-site inspection of the acquisition point with the structural defect. When a structural defect is found and the name matches, the corresponding acquisition image frame is marked as accurately identified. Otherwise, it is marked as incorrectly identified. The acquisition image frame marked as accurately identified is bound to the image clarity and acquisition parameters at the corresponding time.

[0020] It should be further explained that, in the specific implementation process, the process of performing image fusion on each acquired image frame in the same set of acquired image frames, and then combining this with the image recognition model to obtain the number of fused frames when each acquired image frame is accurately recognized, includes: Image fusion is performed on each of the acquired image frames marked as having identification errors within the same set of acquired image frames. Taking a single acquired image frame marked as having identification errors as an example, multiple acquired image frames that are adjacent to each other in terms of timestamp are obtained sequentially. The acquired image frame marked as having an identification error is fused with multiple acquired image frames that are adjacent to it in terms of timestamp to obtain a multi-frame fused image. The number of acquired image frames used for multi-frame fusion at this time is recorded as the number of fused frames. An image recognition model is used to identify the acquired multi-frame fused images to determine whether there are structural defects. When the acquired multi-frame fused images are marked as accurately identified, the number of fused frames at this time is recorded. This method is used to obtain the number of fused frames when each acquired image frame marked as incorrectly identified is marked as accurately identified.

[0021] It should be further explained that, in the specific implementation process, an image fusion model is constructed based on the acquired image frames under different image sharpness and acquisition parameters, and the number of fusion frames when the recognition is accurate. The process of using the image fusion model to perform image fusion on subsequent acquired image frames includes: Based on the acquired image frames under different image sharpness and acquisition parameters, and the number of fused frames marked as accurate recognition, an image fusion set is generated and divided into a second training set and a second test set. Construct a second convolutional neural network by using different acquired image frames, their image sharpness, and acquisition parameters from the second training set as input data for the second convolutional neural network, and using the corresponding fused frame number from the second training set as output data for the second convolutional neural network. Train the second convolutional neural network using the second training set to obtain the initial second convolutional neural network. The initial second convolutional neural network is validated using the second test set. The initial second convolutional neural network whose output is less than or equal to the preset second test error threshold is used as the image fusion model. Each acquired image frame, its image clarity, and acquisition parameters in the subsequent acquired image frame set are first input into the image fusion model to obtain the corresponding fusion frame number. According to the fusion frame number, the corresponding number of acquired image frames that are adjacent to each other in the timestamp are obtained, and multi-frame fusion is performed to obtain multi-frame fusion image. The multi-frame fused image acquired at this time is input into the image recognition model to determine whether there is a structural defect. If no structural defect is found, no other operation is performed. If a structural defect is found, the name of the corresponding structural defect is output and corresponding feedback information is generated and sent to relevant personnel to notify them to conduct on-site inspection of the collection point with the structural defect.

[0022] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A machine vision-based intelligent monitoring system for multiple parameters of tunnel structures, characterized in that, Includes the following modules: The image acquisition module is used to set up several acquisition points and their acquisition units inside the tunnel, and to use the acquisition units to acquire the corresponding images. The image processing module is used to acquire image frames at different times based on the acquired images at the same acquisition point, generate a corresponding set of acquired image frames, and obtain the image clarity of each acquired image frame in the same set of acquired image frames and its acquisition parameters at the corresponding time. The image recognition module is used to acquire different structural defects inside the tunnel and their corresponding defect images, and to build an image recognition model. The image recognition model is used to determine whether there are structural defects in each acquired image frame under different image clarity and acquisition parameters. The image fusion module is used to perform image fusion on each acquired image frame in the same set of acquired image frames, and combine it with the image recognition model to obtain the number of fused frames when each acquired image frame is accurately recognized; The image evaluation module is used to construct an image fusion model based on the acquired image frames under different image sharpness and acquisition parameters and the number of fusion frames when the recognition is accurate. The image fusion model is then used to perform image fusion on subsequent acquired image frames.

2. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 1, characterized in that, The process of setting up acquisition points and acquisition units, and acquiring images includes: Different acquisition points are set up at different support structures inside the tunnel, and a corresponding acquisition unit is set up for each acquisition point. Each acquisition point is bound to its corresponding acquisition unit, and the acquisition unit acquires the acquisition image of its corresponding acquisition point in real time.

3. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 2, characterized in that, The process of acquiring image frames and generating a set of acquired image frames includes: Frame decomposition processing is performed on the images acquired at the same acquisition point. The frame decomposition processing refers to dividing the continuous acquired images into a series of individual acquired image frames, each of which has a timestamp. The acquired image frames with corresponding timestamps at each preset time interval are numbered sequentially. Several acquired image frames with adjacent numbers are included in the same acquired image frame set in chronological order. Acquired image frames with the same acquisition point and number are divided into different acquired image frame sets.

4. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 3, characterized in that, The process of obtaining the image sharpness and acquisition parameters of each acquired image frame includes: The average pixel value of each acquired image frame in the same set of acquired image frames is used as the image sharpness of the corresponding acquired image frame; Different monitoring units are set up at each collection point, including light monitoring unit, smoke monitoring unit, and amplitude monitoring unit. The collection parameters of the corresponding collection point are obtained in real time through each collection unit, including light intensity, smoke concentration, and vibration amplitude. Each collection parameter has a timestamp. The image sharpness of each acquired image frame in the same set of acquired image frames is bound to the same acquisition parameters at the same timestamp.

5. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 4, characterized in that, The process of acquiring different structural defects and their images, and constructing an image recognition model includes: The structural defects include cracks, water leakage, spalling, deformation, misalignment, joint opening, and exposed rebar. Defect images of different structural defects are obtained. The defect images refer to multi-angle images of the corresponding area of ​​the structural defect inside the tunnel. An image recognition set is generated based on the defect images of different structural defects and their corresponding names, and then divided into a first training set and a first test set. Construct a first convolutional neural network by using different defect images from the first training set as input data and the corresponding structural defects and their names from the first training set as output data. Train the first convolutional neural network using the first training set to obtain an initial first convolutional neural network. The initial first convolutional neural network is validated using the first test set, and the initial first convolutional neural network whose output is less than or equal to the preset first test error threshold is used as the image recognition model.

6. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 5, characterized in that, The process of determining whether there are structural defects in each acquired image frame includes: In practical application scenarios, each image frame from the same set of image frames at the same acquisition point is input into the image recognition model to determine whether there are structural defects in each image frame. When a structural defect is found, the name of the corresponding structural defect is output and feedback information is generated to be sent to relevant personnel to notify them to conduct on-site inspection of the acquisition point with structural defects. When a structural defect is found and the name is consistent, the corresponding acquired image frame is marked as accurately identified. Otherwise, it is marked as incorrectly identified. The acquired image frames marked as accurately identified are then bound to their corresponding image clarity and acquisition parameters.

7. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 6, characterized in that, The process of fusing acquired image frames and obtaining the number of fused frames includes: Image fusion is performed on each of the acquired image frames marked as having identification errors within the same set of acquired image frames, and multiple acquired image frames that are adjacent to each other in terms of timestamp are obtained sequentially for each acquired image frame marked as having identification errors. The single image frame marked as incorrectly identified is fused with multiple adjacent image frames in terms of timestamp to obtain a multi-frame fused image. The image recognition model is used to identify the obtained multi-frame fused image to determine whether it has structural defects. When it is marked as correctly identified, the number of fused frames at this time is recorded.

8. The intelligent monitoring system for multiple parameters of tunnel structures based on machine vision according to claim 7, characterized in that, The process of constructing an image fusion model and fusing subsequent acquired image frames includes: Based on the acquired image frames under different image sharpness and acquisition parameters, and the number of fused frames marked as accurate recognition, an image fusion set is generated and divided into a second training set and a second test set. Construct a second convolutional neural network by using different acquired image frames, their image sharpness, and acquisition parameters from the second training set as input data for the second convolutional neural network, and using the corresponding fused frame number from the second training set as output data for the second convolutional neural network. Train the second convolutional neural network using the second training set to obtain the initial second convolutional neural network. The initial second convolutional neural network is validated using the second test set. The initial second convolutional neural network whose output is less than or equal to the preset second test error threshold is used as the image fusion model. Each acquired image frame, its image clarity, and acquisition parameters in the subsequent acquired image frame set are first input into the image fusion model to obtain the corresponding number of fusion frames. According to the number of fusion frames, the corresponding number of acquired image frames adjacent to each acquired image frame at its timestamp are obtained, and multi-frame fusion is performed to obtain a multi-frame fusion image. The multi-frame fused image acquired at this time is input into the image recognition model to determine whether there is a structural defect. When it is determined that there is a defect, the name of the corresponding structural defect is output and feedback information is generated and sent to relevant personnel to notify them to conduct on-site inspection of the acquisition point with the structural defect.

Images