Underground roadway support monitoring method and system based on video monitoring

Abstract

The present application relates to the technical field of image processing, in particular to a mine roadway support monitoring method and system based on video monitoring, comprising: obtaining a support damage degree in each frame of image according to internal gray scale change and position distribution of a support change region in each frame of image; obtaining an expansion intensive factor of each support change region according to the support damage degree and time interval between adjacent next expansion image sequences of the support change region; obtaining a new support expansion damage property of each frame of image according to damage condition of a new support change region; obtaining a support expansion damage saliency of each frame of image according to the new support expansion damage property and an expansion damage risk factor; and monitoring the mine roadway support based on the support expansion damage saliency. The present application improves the accuracy of mine roadway support monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a mine roadway support monitoring method and system based on video monitoring. BACKGROUND

[0002] The mine roadway support monitoring system plays a crucial role in the safety management of underground engineering such as mines and tunnels. The traditional mine roadway support monitoring method uses video monitoring technology to monitor and record the status of roadway support in real time, which can effectively prevent and eliminate safety hazards. In the mine roadway monitoring system, video signals usually occupy a large amount of storage space, especially in high-resolution and long-time monitoring situations, the data volume is huge and the real-time requirement is high. This puts higher requirements on the storage capacity and image processing capacity of the video monitoring system. In order to achieve efficient data storage and processing under limited storage resources, video stream compression technology is widely used in mine roadway support monitoring systems.

[0003] Since the mine roadway support monitoring video occupies a large amount of storage space, and the monitoring system often uses video stream with fixed compression parameters for compression storage, this method can achieve high compression efficiency when processing unimportant frame images, but it often causes large data loss when processing important frame images. For mine roadway support monitoring video, if a frame image contains key information such as deformation, expansion, and destruction of support structure, these information is crucial for judging the safety of the roadway. Excessive compression of frame images containing these key information may result in loss of important information, which affects the effectiveness and accuracy of monitoring. That is, due to the large amount of real-time video data, traditional compression may cause significant damage to important video data, reducing the accuracy of roadway support monitoring. SUMMARY

[0004] The present application provides a mine roadway support monitoring method and system based on video monitoring to solve the existing problem that due to the large amount of real-time video data, traditional compression may cause significant damage to important video data, reducing the accuracy of roadway support monitoring.

[0005] The mine roadway support monitoring method and system based on video monitoring of the present application adopts the following technical scheme:

[0006] The present application provides a mine roadway support monitoring method and system based on video monitoring to solve the existing problem that due to the large amount of real-time video data, traditional compression may cause significant damage to important video data, reducing the accuracy of roadway support monitoring.

[0007] Obtain the monitoring video of the mine roadway support;

[0008] By analyzing the grayscale changes between adjacent frames in the monitoring video, the support change area in each frame is obtained; based on the internal grayscale changes in the support change area, the damage factor of each support change area is obtained; based on the damage factor and position distribution of the support change area, the support damage degree in each frame is obtained;

[0009] Obtain all sub-expanded image sequences of each support change area; obtain the expansion density factor of each support change area based on the support damage degree and the time interval between adjacent sub-expanded image sequences of the support change area; obtain the expansion damage risk factor of each support change area based on the expansion density factor and the change in the area of ​​the support change area between each expansion image sequence; obtain the newly added support change area in each frame of image, and obtain the newly added support expansion damage of each frame of image based on the damage of the newly added support change area; obtain the support expansion damage significance of each frame of image based on the newly added support expansion damage and the expansion damage risk factor;

[0010] Monitoring of underground tunnel support based on support extension damage significance.

[0011] Preferably, the specific method of obtaining the support change area in each frame image by analyzing the grayscale changes between adjacent frame images in the monitoring video is:

[0012] The monitoring video of underground tunnel support Frame image The coordinate position of the pixel point is recorded as the target position; The gray value of the pixel is The absolute value of the difference between the grayscale values ​​of the pixels at the target position in the frame image is recorded as The pixel at the Frame image and The grayscale difference between the frame images; The pixel at the Frame image and Frame image and The normalized value of the mean grayscale difference between frame images is recorded as Frame image Grayscale change value of each pixel;

[0013] Preset a neighborhood grayscale parameter , if Frame image The grayscale change value of the pixel is greater than or equal to the neighborhood grayscale parameter , will The pixel point is recorded as the changed pixel point; Each closed area formed by all the changed pixels in the frame image is recorded as a changed area; All the changed areas in the frame image are input into the trained neural network to obtain the first All support change areas in the frame image.

[0014] Preferably, the specific method for obtaining the damage factor of each support change region according to the internal grayscale change of the support change region is:

[0015] Preset a grayscale parameter , will The mean of the grayscale change values ​​of all pixels in the support change area and the grayscale parameter The ratio of The normalized value of the product of the number of all pixels in the support change area is recorded as The damage factor of each support change area.

[0016] Preferably, the specific method for obtaining the support damage degree in each frame of image based on the damage factor and position distribution of the support change area is:

[0017] The first The number of all support change areas in the frame image is the same as the The product of the mean values ​​of the loss factors of all support change areas in the frame image is recorded as the first product; the All support change regions in the frame image are combined in any two ways to obtain several support change region combinations; The inversely proportional normalized value of the cumulative sum of the Euclidean distances between the centroid positions of two support change regions in all support change region combinations in the frame image is recorded as the support distribution value;

[0018] The normalized value of the product of the first product and the support distribution value is used as the first The support damage degree in the frame image.

[0019] Preferably, the specific method for obtaining all sub-extended image sequences of each support change area is:

[0020] Preset a neighborhood parameter In the video of underground tunnel support monitoring, Frame image and The nearest frame image The image sequence composed of frame images is The neighborhood range image sequence of the frame image; using the KLT tracking algorithm, obtain the first Frame image The support change area is in The support change area corresponding to each frame image in the neighborhood range image sequence of the frame image;

[0021] Preset an area change parameter , in In the neighborhood range image sequence of the frame image, traverse in sequence to obtain the first The absolute value of the difference in the number of all pixels in the support change area is calculated until the absolute value of the difference is less than the area change parameter for the first time. When traversing, all the frame images before the last frame image constitute the first extended image sequence; starting from the last frame image traversed, obtain the second extended image sequence according to the method of obtaining the first extended image sequence, and so on, obtain the first extended image sequence. All sub-extended image sequences of the support change area.

[0022] Preferably, the specific method for obtaining the expansion density factor of each support change area according to the support damage degree and the time interval between adjacent sub-expanded image sequences of the support change area is:

[0023] The first The average value of the support damage degree of all frame images in the image sequence of the neighborhood range of the frame image is The neighborhood range of the frame image in the image sequence The product of the number of all sub-extended image sequences in the support change area is recorded as the second product; In the neighborhood range image sequence of the frame image, the first The first support change area The second extended image sequence and the The time interval between the first frame of the extended image sequence is recorded as The time factor of the second extended image sequence; The inverse proportional normalized value of the cumulative sum of the time factors of all sub-expanded image sequences in the support change area is recorded as The expansion interval density of each support change area; The product of the expansion interval density of the support change area and the second product is used as the first Frame image The expansion density factor of each support change area.

[0024] Preferably, the specific method for obtaining the expansion damage risk factor of each support change region based on the expansion density factor and the area change of the support change region between each expansion image sequence is:

[0025] In the The union of all pixel points in the first support change region between the first frame image and the last frame image in the first extended image sequence of the first support change region in the neighborhood range image sequence of the frame image is recorded as a first set. The intersection of all pixel points in the first support change region between the first frame image and the last frame image in the first extended image sequence of the first support change region in the neighborhood range image sequence of the frame image is recorded as a second set. The difference between the number of elements between the first set and the second set is recorded as an expansion factor of the first extended image sequence. The ratio between the expansion factor of the first extended image sequence and the number of all expanded frame images in the first extended image sequence is recorded as an expansion rate factor of the first extended image sequence. The cumulative sum of the expansion rate factors of all extended image sequences of the first support change region is recorded as an expansion trend value of the first support change region. The product of the expansion density factor of the first support change region and the expansion trend value is taken as an expansion damage risk factor of the first support change region. Preferably, the specific method for obtaining the new support change region in each frame image according to the damage condition of the new support change region is as follows: In the neighborhood range image sequence of the first frame image, the remaining support change regions that are not matched by the tracking matching method are all recorded as new support change regions. The product of the number of all new support change regions in the neighborhood range image sequence of the first frame image and the cumulative sum of the damage factors of all new support change regions in the neighborhood range image sequence of the first frame image is normalized to obtain the new support expansion damage of the first frame image. Preferably, the specific method for obtaining the support expansion damage significance of each frame image according to the new support expansion damage and the expansion damage risk factor is as follows: The cumulative sum of the expansion damage risk factors of all support change regions in the first frame image is normalized to obtain the support expansion damage significance of the first frame image.

[0026]

[0027] In the neighborhood range image sequence of the first frame image, the remaining support change regions that are not matched by the tracking matching method are all recorded as new support change regions. The product of the number of all new support change regions in the neighborhood range image sequence of the first frame image and the cumulative sum of the damage factors of all new support change regions in the neighborhood range image sequence of the first frame image is normalized to obtain the new support expansion damage of the first frame image.

[0028] Preferably, the specific method for obtaining the support expansion damage significance of each frame image according to the new support expansion damage and the expansion damage risk factor is as follows: The cumulative sum of the expansion damage risk factors of all support change regions in the first frame image is normalized to obtain the support expansion damage significance of the first frame image.

[0029]

[0030] The cumulative sum of the expansion damage risk factors of all support change regions in the first frame image is normalized to obtain the support expansion damage significance of the first frame image. ​​​​​​​​​The initial support expansion injury of the frame image is multiplied by the first The initial support expansion injury of the frame image is multiplied by the first The normalized value of the product of the newly added support expansion injury of the frame image and the first The support expansion injury significance of the frame image.

[0031] The application also provides an underground roadway support monitoring system based on video monitoring, which comprises a memory and a processor.

[0032] The technical scheme of the application has the beneficial effects that: the support damage degree in each frame image is obtained according to the internal gray scale change and the position distribution of the support change region in each frame image; the expansion intensive factor of each support change region is obtained according to the support damage degree and the time interval between adjacent next expansion image sequences of the support change region; the newly added support expansion injury of each frame image is obtained according to the damage condition of the newly added support change region; the support expansion injury significance of each frame image is obtained according to the newly added support expansion injury and the expansion damage risk factor; the underground roadway support is monitored based on the support expansion injury significance; and the underground roadway support monitoring video is differentially compressed and stored, which can not only ensure the image processing efficiency and storage saving, but also not lose important monitoring information, thereby improving the accuracy of underground roadway support monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical scheme in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0034] Figure 1 The step flow chart of the underground roadway support monitoring method based on video monitoring of the application;

[0035] Figure 2 The feature relationship flow chart of the underground roadway support monitoring method based on video monitoring of the application. DETAILED DESCRIPTION

[0036] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined inventive objectives, the specific implementation, structure, features and effects of the underground roadway support monitoring method and system based on video monitoring according to the present application are described in detail below in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0038] The specific scheme of the underground roadway support monitoring method and system based on video monitoring provided by the present application is described below in combination with the drawings.

[0039] Please refer to Figure 1 which shows the step flowchart of the underground roadway support monitoring method based on video monitoring provided by one embodiment of the present application, which includes the following steps:

[0040] Step S001: Obtain the monitoring video of the underground roadway support.

[0041] In one specific implementation of the embodiment of the present application, the specific method for obtaining the monitoring video of the underground roadway support is as follows:

[0042] Install the underground monitoring camera to collect the underground roadway in real time and obtain the monitoring video of the underground roadway support.

[0043] At this point, the monitoring video of the underground roadway support is obtained through the above method.

[0044] Step S002: Obtain the support change area in each frame image by analyzing the gray level change between adjacent frame images in the monitoring video; obtain the damage factor of each support change area according to the internal gray level change of the support change area; and obtain the support damage degree in each frame image according to the damage factor and the position distribution of the support change area.

[0045] It should be noted that in the video or continuous frames, the scene is usually static, and only when some objects or structures change, there will be obvious gray level changes; for example, when the support structure deforms, damages or personnel moves, the gray value of the related area will change, therefore, by analyzing the gray value change of the pixels between adjacent frames, the pixel points with large gray level changes are identified, and through the neural network, the support change area is identified from these change areas; finally, the support damage degree of each frame image is evaluated according to the change size, area, number and distribution of these support change areas.

[0046] Specifically, the monitoring video of the underground tunnel support is divided into several separate frame images using OpenCV technology; wherein, OpenCV technology is an existing technology and will not be described in detail in this embodiment.

[0047] Preferably, in some implementations of the embodiments of the present invention, by analyzing the grayscale changes between adjacent frames in the surveillance video, a specific method for obtaining the support change area in each frame is:

[0048] The monitoring video of underground tunnel support Frame image The coordinate position of the pixel point is recorded as the target position; The gray value of the pixel is The absolute value of the difference between the grayscale values ​​of the pixels at the target position in the frame image is recorded as The pixel at the Frame image and The grayscale difference between the frame images; The pixel at the Frame image and Frame image and The normalized value of the mean grayscale difference between frame images is recorded as Frame image Grayscale change value of each pixel;

[0049] Among them, for the last frame of the monitoring video of the underground tunnel support, The normalized value of the grayscale value of the pixel in the last frame is recorded as The grayscale change value of each pixel.

[0050] Preset a neighborhood grayscale parameter , wherein this embodiment is based on This example is described as an example, and this embodiment is not specifically limited. Depends on the specific implementation situation;

[0051] Jordi Frame image The grayscale change value of the pixel is greater than or equal to the neighborhood grayscale parameter , will The pixel point is recorded as the changed pixel point; Each closed area formed by all the changed pixels in the frame image is recorded as a changed area;

[0052] The first All the changed areas in the frame image are input into the trained neural network to obtain the first all support change regions in the frame image;

[0053] The support change region includes a support rupture, collapse or deformation region, and the non-support change region includes movement of a worker and movement of equipment. The neural network used in this embodiment is YOLOv3, and the data set used to train the neural network is obtained by:

[0054] A large number of underground roadway support monitoring images are collected, and a support change region is manually marked in each underground roadway support monitoring image using a bounding box. This marking result is recorded as a label of each underground roadway support monitoring image. A large number of underground roadway support monitoring images and their corresponding labels are collected to form a data set. The neural network is trained using the data set, and a mean square error loss function is used as the loss function during the training process. The specific training process is a known content of the neural network, and the specific training process will not be described in this embodiment.

[0055] It should be noted that the greater the gray value change of the pixel points in the support change region and the greater the area, the more obvious the change between adjacent frames of the support change region, reflecting the structural problem of deformation or damage of the support structure. The greater the area of the support change region, the wider the impact range, and the more serious the damage or change degree involved. A support change region with a large area usually means that the change of the support structure is not local but large-scale, with a greater risk of damage. A large number of support change regions mean that the support structure has undergone extensive changes, which may cause damage in a larger range. Therefore, the more concentrated the distribution of support change regions, the more likely the damage occurs at the same location or adjacent area, which may pose a greater threat to the integrity of the support structure.

[0056] Preferably, in some implementations of the embodiment of the present application, the specific method for obtaining the support damage degree in each frame image according to the internal gray value change and the position distribution of the support change region in each frame image is:

[0057] According to the internal gray value change of the support change region, the damage factor of each support change region is obtained.

[0058] A gray parameter is preset , which is taken as an example for description in this embodiment, which is not specifically limited, and is determined according to the specific implementation.

[0059] The ratio of the mean value of the gray value change of all pixel points in the first support change region to the gray parameter is recorded as the first ratio. The first ratio is compared with the second ratio. ​​The normalized value of the product of the number of all pixels in the support change area is recorded as Damage factor of each support change area;

[0060] The specific formula is:

[0061]

[0062] Where, The damage factor representing the support change area; Represents the mean value of the grayscale change values ​​of all pixels in the support change area; Indicates the number of all pixels in the support change area; represents the linear normalization function.

[0063] The first The number of all support change areas in the frame image is the same as the The product of the mean values ​​of the loss factors of all support change areas in the frame image is recorded as the first product;

[0064] The first All support change regions in the frame image are combined in any two ways to obtain several support change region combinations; The inversely proportional normalized value of the cumulative sum of the Euclidean distances between the centroid positions of two support change regions in all support change region combinations in the frame image is recorded as the support distribution value;

[0065] The normalized value of the product of the first product and the support distribution value is used as the first The degree of support damage in the frame image;

[0066] The specific formula is:

[0067]

[0068] Where, Indicates the The degree of support damage in the frame image; Indicates the The number of all support change areas in the frame image; Indicates the The mean of the loss factors of all support change areas in the frame image; Indicates the The number of all support change area combinations in the frame image; Indicates the Frame image The Euclidean distance between the centroid positions of two support change areas within a support change area combination; represents the linear normalization function; Represents an exponential function with a natural constant as the base, and the embodiment adopts Model to present inverse proportional relationship and normalization processing, As the input of the model, the implementer can choose the inverse proportional function and normalization function according to the actual situation.

[0069] So far, the support damage degree in each frame of image is obtained through the above method.

[0070] Step S003: Obtain all sub-extended image sequences of each support change area; obtain the expansion density factor of each support change area according to the support damage degree and the time interval between adjacent sub-extended image sequences of the support change area; obtain the expansion destruction risk factor of each support change area according to the expansion density factor and the area change of the support change area between each extended image sequence; obtain the newly added support change area in each frame of image, and obtain the newly added support expansion damage of each frame of image according to the damage of the newly added support change area; obtain the support expansion damage significance of each frame of image according to the newly added support expansion damage and the expansion destruction risk factor.

[0071] It should be noted that if the support change area in the image may expand intermittently in the future, or new support change areas may be added, these all indicate that the impact of the image on the subsequent stage of tunnel support monitoring becomes particularly critical, because it reflects the major changes that may occur in the support structure in certain key areas; the larger the expansion area of ​​the support change area and the faster the rate, the stronger the support damage expansibility and the more serious the structural damage to the support, that is, large-scale changes and expansions in a short period of time are dangerous signals, indicating that the support damage is rapidly expanding and will have a serious impact on the support structure; therefore, by combining the area changes and time intervals of different types of support change areas between consecutive frames, the support damage expansibility of each frame image is obtained.

[0072] Preferably, in some implementations of the embodiments of the present invention, the specific method for obtaining all sub-extended image sequences of each support change area is:

[0073] Preset a neighborhood parameter , wherein this embodiment is based on This example is described as an example, and this embodiment is not specifically limited. Depends on the specific implementation situation;

[0074] In the video of underground tunnel support monitoring, Frame image and The nearest frame image The image sequence composed of frame images is The neighborhood range image sequence of the frame image; use the KLT (Kanade-Lucas-Tomasi) tracking algorithm to track according to time until it can no longer be tracked, and obtain the first Frame image The support change area is in The support change area corresponding to each frame image in the neighborhood range image sequence of the frame image;

[0075] Among them, if All the frames after the frame image do not meet the When Frame image and All frames after the frame image constitute the first A neighborhood range image sequence of a frame image; the KLT tracking algorithm is a prior art and will not be described in detail in this embodiment;

[0076] Preset an area change parameter , wherein this embodiment is based on This example is described as an example, and this embodiment is not specifically limited. Depends on the specific implementation situation;

[0077] In the In the neighborhood range image sequence of the frame image, traverse in sequence to obtain the first The absolute value of the difference in the number of all pixels in the support change area is calculated until the absolute value of the difference is less than the area change parameter for the first time. When traversing, all the frame images before the last frame image constitute the first extended image sequence; starting from the last frame image traversed, obtain the second extended image sequence according to the method of obtaining the first extended image sequence, and so on, obtain the first extended image sequence. All sub-extended image sequences of the support change area;

[0078] The specific method is:

[0079] In the In the neighborhood image sequence of the frame image, if the The number of all pixels in the support change area is Frame image and The absolute value of the difference between the frame images is greater than or equal to the area change parameter , will The frame image is recorded as An extended frame image of the support change area; if the The number of all pixels in the support change area is Frame image and The absolute value of the difference between the frame images is greater than or equal to the area change parameter The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The absolute value of the difference between the frame images is less than the area change parameter The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region

[0080] Preferably, in some implementations of the embodiments of the present application, the shorter the time interval between the expansion image sequences of the support change region and the greater the degree of support damage in these frame images, it is indicated that the support change region is undergoing multiple and intensive expansion in the neighborhood range image sequence, that is, the support structure may be experiencing rapid deformation or damage, which has a huge impact on the safety and stability of the support structure; therefore, according to the degree of support damage and the time interval between the adjacent expansion image sequences of the support change region, the specific method for obtaining the expansion intensity factor of each support change region is as follows:

[0081] The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region The first frame image is recorded as the first frame image of the first support change region

[0082] The specific formula is:

[0083]

[0084] In the formula, represents the first expansion density factor of the first support change region in the frame image; represents the first support change region in the frame image; represents the first support change region in the frame image; represents the number of all the secondary expansion image sequences of the first support change region in the frame image; represents the first support change region in the frame image; represents the time interval between the first frame image of the first secondary expansion image sequence and the first frame image of the first represents an exponential function with a natural constant as a base number, and the embodiment adopts a model to present an inverse proportional relationship and normalization processing, is an input of the model, and the implementer can select an inverse proportional function and a normalization function according to actual conditions.

[0085] Preferably, in some implementations of the embodiment of the present application, if the expansion density factor of the support change region is small, and the number of the expansion frame images in each expansion image sequence of the support change region is small, it indicates that each expansion of the support change region occurs faster; and the rapid change of the area of the support change region in a short time interval indicates that the expansion rate of the support change region is high, and the risk of damage to the support structure is greater; therefore, according to the expansion density factor and the area change of the support change region between each expansion image sequence, the specific method for obtaining the expansion damage risk factor of each support change region is as follows:

[0086] In the frame image sequence in the neighborhood range of the first frame image, the union of all the pixel points in the first support change region between the first frame image and the last frame image in the first secondary expansion image sequence of the first support change region is recorded as a first set; the intersection of all the pixel points in the first support change region between the first frame image and the last frame image in the first secondary expansion image sequence of the first support change region is recorded as a second set; the difference between the number of elements in the first set and the number of elements in the second set is recorded as the first The expansion factor of the second expanded image sequence; The expansion factor of the second expanded image sequence is the same as that of the first The ratio between the number of all extended frame images in the extended image sequence is recorded as The expansion rate factor of the second expanded image sequence; The cumulative sum of the expansion rate factors of all sub-expanded image sequences in the support change area is recorded as The expansion trend value of the support change area; The product of the expansion density factor and the expansion trend value of the support change area is used as the The expansion damage risk factor of each support change area;

[0087] The specific formula is:

[0088]

[0089] Where, Indicates the Frame image The expansion damage risk factor of each support change area; Indicates the Frame image The expansion density factor of each support change area; Indicates the The neighborhood range of the frame image in the image sequence The number of all sub-extended image sequences in the support change area; 、 Respectively represent The neighborhood range of the frame image in the image sequence The first support change area The number of frames between the first and last frames in the extended image sequence The number of elements in the union and intersection of all pixels in the support change area; Indicates the The neighborhood range of the frame image in the image sequence The first support change area The number of extended frame images in the secondary extended image sequence; Indicates taking the absolute value;

[0090] in, Indicates the The support change area is in The expansion factor for the second expansion, Indicates the The area of ​​the support change zone is The expansion rate factor for the second expansion.

[0091] Preferably, in In the neighborhood image sequence of the frame image, the remaining support change areas that are not matched by the tracking matching method are recorded as newly added support change areas;

[0092] Preferably, in some implementations of the embodiments of the present invention, if There is a new change area between the frame image and any frame image, and the change area belongs to the support change area, the larger the damage factor is; In the future stage of the frame image, the expansion of the existing support change area is newly added on the basis of the original one, which means that the damage expansion of the support in the future stage will continue or intensify. Therefore, based on the loss factor of the newly added support change area, the specific method for obtaining the damage of the newly added support expansion in each frame image is as follows:

[0093] The first The number of all newly added support change areas in the neighborhood range image sequence of the frame image is the same as the number of the first The normalized value of the product of the cumulative sum of the damage factors of all newly added support change areas in the image sequence of the neighborhood range of the frame image is used as the first New support extension damage resistance of frame images;

[0094] The specific formula is:

[0095]

[0096] Where, Indicates the New support extension damage resistance of frame images; Indicates the The number of all newly added support change areas in the image sequence within the neighborhood range of the frame image; Indicates the The neighborhood range of the frame image in the image sequence The damage factor of the newly added support change area; represents the linear normalization function.

[0097] Preferably, in some implementations of the embodiments of the present invention, if The more the frame image is damaged, the more The larger the expansion damage risk factor of the support change area in the frame image, the greater the In the future stage of the frame image, there is not only the damage extension in the existing support change area, but also the damage extension in the newly added support change area, which indicates that the first The influence of the frame image on the subsequent stage of the roadway support monitoring becomes particularly critical, because it reflects the significant and obvious expansion damage changes that the support structure can have in some key frame images; therefore, according to the expansion damage nature and the expansion damage risk factor of the new support, the specific method for obtaining the support expansion damage obviousness of each frame image is as follows:

[0098] The normalized value of the cumulative sum of the expansion damage risk factors of all support change regions in the frame image is taken as the initial support expansion damage nature of the frame image; The normalized value of the product of the initial support expansion damage nature of the frame image and the new support expansion damage nature of the frame image is taken as the support expansion damage obviousness of the frame image. The normalized value of the product of the initial support expansion damage nature of the frame image and the new support expansion damage nature of the frame image is taken as the support expansion damage obviousness of the frame image. The normalized value of the product of the initial support expansion damage nature of the frame image and the new support expansion damage nature of the frame image is taken as the support expansion damage obviousness of the frame image.

[0099] The specific formula is as follows:

[0100]

[0101] In the formula, S represents the support expansion damage obviousness of the frame image; S represents the new support expansion damage nature of the frame image; S represents the support expansion damage obviousness of the frame image; S represents the new support expansion damage nature of the frame image; S represents the initial support expansion damage nature of the frame image; S represents the linear normalization function. Up to now, the support expansion damage obviousness of each frame image is obtained through the above method.

[0102] Step S004: monitoring the underground roadway support based on the support expansion damage obviousness.

[0103] It should be noted that for any frame image in the monitoring video of the underground roadway support, the greater the support expansion damage obviousness of the frame image, the more rapid and severe the damage expansion of the support in the frame image, and a lower compression weight should be given to retain a greater degree of detail.

[0104] Preferably, in some implementations of the embodiments of the present application, the specific method for monitoring the underground roadway support based on the support expansion damage obviousness is as follows:

[0105] For any frame image in the monitoring video of the underground roadway support, the product of the reciprocal of the support damage obviousness of the frame image and the initial quantization parameter in the H.264 video compression algorithm is taken as the modified quantization parameter of the frame image.

[0106] For any frame image in the monitoring video of the underground roadway support, the product of the reciprocal of the support damage obviousness of the frame image and the initial quantization parameter in the H.264 video compression algorithm is taken as the modified quantization parameter of the frame image.

[0107] ​​​The modified quantization parameter of each frame of image is input into an H.264 video compression algorithm, and the underground roadway support monitoring video is compressed to obtain a compressed underground roadway support monitoring video; the compressed underground roadway support monitoring video is transmitted to a ground control center or a monitoring platform through an underground wireless network or a wired network and is decompressed to obtain a plurality of underground roadway support monitoring images;

[0108] A quality threshold parameter is preset The embodiment is described by taking as an example, and the embodiment is not limited specifically, wherein The specific implementation is determined according to the specific implementation.

[0109] The underground roadway support monitoring image is input into SLAM technology, three-dimensional modeling of the underground roadway support is performed, and detailed three-dimensional point cloud data or a grid model is generated; according to the generated three-dimensional model, a structure quality evaluation value of the support is obtained; if the structure quality evaluation value of the support is less than the quality threshold parameter , it is indicated that the underground roadway support has a risk, and a worker needs to be notified in time to arrange maintenance.

[0110] The SLAM technology and the H.264 video compression algorithm are prior art, and the embodiment is not described in detail here.

[0111] Please refer to Figure 2 , which shows a feature relationship flowchart of the underground roadway support monitoring method based on video monitoring.

[0112] Through the above steps, the underground roadway support monitoring method based on video monitoring is completed.

[0113] Another embodiment of the present application provides an underground roadway support monitoring system based on video monitoring, which comprises a memory and a processor, and the processor executes the computer program stored in the memory to execute the above method steps S001 to S004.

[0114] The above only describes the preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. The underground tunnel support monitoring method based on video monitoring is characterized by: The method comprises the following steps: Obtain monitoring video of underground tunnel support; By analyzing the grayscale changes between adjacent frames in the monitoring video, the support change area in each frame is obtained; based on the internal grayscale changes in the support change area, the damage factor of each support change area is obtained; based on the damage factor and position distribution of the support change area, the support damage degree in each frame is obtained; Obtain all sub-expanded image sequences of each support change area; obtain the expansion density factor of each support change area based on the support damage degree and the time interval between adjacent sub-expanded image sequences of the support change area; obtain the expansion damage risk factor of each support change area based on the expansion density factor and the change in the area of ​​the support change area between each expansion image sequence; obtain the newly added support change area in each frame of image, and obtain the newly added support expansion damage of each frame of image based on the damage of the newly added support change area; obtain the support expansion damage significance of each frame of image based on the newly added support expansion damage and the expansion damage risk factor; Monitoring of underground tunnel support based on support extension damage significance.

2. The underground tunnel support monitoring method based on video monitoring according to claim 1 is characterized in that: The specific method of obtaining the support change area in each frame image by analyzing the grayscale changes between adjacent frame images in the monitoring video is: The monitoring video of underground tunnel support Frame image The coordinate position of the pixel point is recorded as the target position; The gray value of the pixel is The absolute value of the difference between the grayscale values ​​of the pixels at the target position in the frame image is recorded as The pixel at the Frame image and The grayscale difference between the frame images; The pixel at the Frame image and Frame image and The normalized value of the mean grayscale difference between frame images is recorded as Frame image Grayscale change value of each pixel; Preset a neighborhood grayscale parameter , if Frame image The grayscale change value of the pixel is greater than or equal to the neighborhood grayscale parameter , will The pixel point is recorded as the changed pixel point; Each closed area formed by all the changed pixels in the frame image is recorded as a changed area; All the changed areas in the frame image are input into the trained neural network to obtain the first All support change areas in the frame image.

3. The underground tunnel support monitoring method based on video monitoring according to claim 2 is characterized in that: The specific method for obtaining the damage factor of each support change area according to the internal grayscale change of the support change area is: Preset a grayscale parameter , will The mean of the grayscale change values ​​of all pixels in the support change area and the grayscale parameter The ratio of is recorded as the first ratio; The first ratio and the The normalized value of the product of the number of all pixels in the support change area is recorded as The damage factor of each support change area.

4. The underground tunnel support monitoring method based on video monitoring according to claim 1 is characterized in that: The specific method for obtaining the support damage degree in each frame image based on the damage factor and position distribution of the support change area is: The first The number of all support change areas in the frame image is the same as the The product of the mean values ​​of the loss factors of all support change areas in the frame image is recorded as the first product; the All support change regions in the frame image are combined in any two ways to obtain several support change region combinations; The inversely proportional normalized value of the cumulative sum of the Euclidean distances between the centroid positions of two support change regions in all support change region combinations in the frame image is recorded as the support distribution value; The normalized value of the product of the first product and the support distribution value is used as the first The support damage degree in the frame image.

5. The underground tunnel support monitoring method based on video monitoring according to claim 1 is characterized in that: The specific method for obtaining all sub-extended image sequences of each support change area is: Preset a neighborhood parameter In the video of underground tunnel support monitoring, Frame image and The nearest frame image The image sequence composed of frame images is The neighborhood range image sequence of the frame image; using the KLT tracking algorithm, obtain the first Frame image The support change area is in The support change area corresponding to each frame image in the neighborhood range image sequence of the frame image; Preset an area change parameter , in In the neighborhood range image sequence of the frame image, traverse in sequence to obtain the first The absolute value of the difference in the number of all pixels in the support change area is calculated until the absolute value of the difference is less than the area change parameter for the first time. When traversing, all the frame images before the last frame image constitute the first extended image sequence; starting from the last frame image traversed, obtain the second extended image sequence according to the method of obtaining the first extended image sequence, and so on, obtain the first extended image sequence. All sub-extended image sequences of the support change area.

6. The underground tunnel support monitoring method based on video monitoring according to claim 5 is characterized in that: The specific method for obtaining the expansion density factor of each support change area according to the support damage degree and the time interval between adjacent sub-expanded image sequences of the support change area is: The first The average value of the support damage degree of all frame images in the image sequence of the neighborhood range of the frame image is The neighborhood range of the frame image in the image sequence The product of the number of all sub-extended image sequences in the support change area is recorded as the second product; In the neighborhood range image sequence of the frame image, the first The first support change area The second extended image sequence and the The time interval between the first frame of the extended image sequence is recorded as The time factor of the second extended image sequence; The inverse proportional normalized value of the cumulative sum of the time factors of all sub-expanded image sequences in the support change area is recorded as The expansion interval density of each support change area; The product of the expansion interval density of the support change area and the second product is used as the first Frame image The expansion density factor of each support change area.

7. The underground tunnel support monitoring method based on video monitoring according to claim 5 is characterized in that: The specific method for obtaining the expansion damage risk factor of each support change area based on the expansion density factor and the area change of the support change area between each expansion image sequence is: In the In the neighborhood range image sequence of the frame image, the first The first support change area The number of frames between the first and last frames in the extended image sequence The union of all pixels in the support change area is recorded as the first set; The first support change area The number of frames between the first and last frames in the extended image sequence The intersection of all pixels in the support change area is recorded as the second set; the difference in the number of elements between the first set and the second set is recorded as the first set. The expansion factor of the second expanded image sequence; The expansion factor of the second expanded image sequence is the same as that of the first The ratio between the number of all extended frame images in the extended image sequence is recorded as The expansion rate factor of the second expanded image sequence; The cumulative sum of the expansion rate factors of all sub-expanded image sequences in the support change area is recorded as The expansion trend value of each support change area; The first The product of the expansion density factor and the expansion trend value of the support change area is used as the The expansion damage risk factor of each support change area.

8. The underground tunnel support monitoring method based on video monitoring according to claim 5 is characterized in that: The specific method for obtaining the newly added support change area in each frame of image and obtaining the newly added support extension damage of each frame of image according to the damage condition of the newly added support change area is: In the In the neighborhood image sequence of the frame image, the remaining support change areas that are not matched by the tracking matching method are recorded as newly added support change areas; The first The number of all newly added support change areas in the neighborhood range image sequence of the frame image is the same as the number of the first The normalized value of the product of the cumulative sum of the damage factors of all newly added support change areas in the image sequence of the neighborhood range of the frame image is used as the first New support for extended damage resistance of frame images.

9. The underground tunnel support monitoring method based on video monitoring according to claim 1 is characterized in that: The specific method for obtaining the support extension damage significance of each frame image based on the newly added support extension damage and extension failure risk factors is: The first The normalized value of the cumulative sum of the expansion damage risk factors of all support change areas in the frame image is used as the first Initial support extension damage of frame images; The first The initial support extension damage of the frame image and the The normalized value of the product of the newly added support extension damage of the frame image is used as the first Support extension damage significance of frame images.

10. An underground tunnel support monitoring system based on video monitoring, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the underground tunnel support monitoring method based on video monitoring as described in any one of claims 1 to 9 are implemented.

Images