A method for continuous monitoring of internal rock mass changes and failures in a similar material model

Abstract

A kind of similar material model internal stratum change and continuous monitoring method of failure, comprising the following steps: continuous shooting is carried out to monitoring target, and a plurality of time continuous video segments are formed in turn;While video acquisition, key frame extraction is carried out to each video segment collected in turn, and the key frame extracted is saved, and new video dataset is generated, wherein, key frame extraction is carried out to the video segment collected, including the following steps: extracting the key frame index of each image frame in video segment;According to key frame index list L_index, 2 frame difference and 3 frame difference are calculated, and non-key frame is deleted in combination with 2 frame difference and 3 frame difference in video segment, to accurately locate key frame.The design can achieve the effect of real-time shooting, instant processing, accurately obtain the change condition of monitoring target while avoiding a large amount of redundant data.

Description

TECHNICAL FIELD

[0001] The present application relates to a video monitoring method, in particular to a continuous monitoring method for internal rock stratum change and destruction of a similar material model, and is particularly suitable for monitoring internal rock stratum change of a similar material model. BACKGROUND

[0002] Similar material model simulation is a common indoor simulation research method in the field of mining and geotechnical engineering. Similar material model simulation is to use artificial materials similar to the physical and mechanical properties of natural rocks, to follow the three similarity theorems, to make an indoor model by reducing the actual prototype according to a certain scale, and then to carry out working face mining and roadway excavation in the model; by observing and monitoring the deformation and destruction of the model, the situation in the prototype is inversely deduced according to the three similarity theorems. Similar material model is widely used in the research of rock stratum movement, ground subsidence, coal mining under water, coal bed methane extraction and other problems related to rock stratum deformation and destruction. With the increasing progress of mine surveying technology and the increasing understanding of related rock stratum movement problems, more and more attention is paid to the internal deformation and destruction monitoring of rock mass. On the one hand, it is found that the key layer breakage, separation or key layer combination breakage and separation in the overlying rock stratum have a direct relationship with the time sequence process of the surface point subsidence; on the other hand, the deformation and destruction of the key layer or key layer combination in the overlying rock stratum have a strong correlation with the mine pressure and rock burst. Researching the internal deformation and destruction process of rock stratum has important significance for the control of mine pressure and ground subsidence.

[0003] At present, the commonly used similar material model deformation monitoring methods include sketching, photogrammetry, three-dimensional laser scanning, static or quasi-dynamic monitoring of light convex mirror, the change of the model is obtained by periodically monitoring the deformation of the model, and the cycle of deformation monitoring is determined according to the research purpose. The existing model deformation monitoring method is difficult to meet the real-time monitoring demand of the internal deformation and failure of the rock mass, and the main reason is that the rock deformation and breakage often occur instantaneously, and the moment needs to be accurately grasped during the test. The time of one monitoring by using the existing manual sketching, photographing, three-dimensional laser scanning and other deformation monitoring methods is about hours, and it is difficult to reduce the monitoring cycle to minutes from the technical and feasibility aspects. The patent for invention with patent number CN109298158A and publication date of February 1, 2019 discloses a kind of multi-node dynamic surface displacement digital measurement system, which takes pictures of similar coal rock strata continuously, and obtains the corresponding deformation information of the model by processing the images. This method takes pictures of the model at certain time intervals, which makes it difficult to accurately obtain the specific moment of the internal rock failure and other changes of the model. If the accurate moment of the change is to be accurately obtained, the shooting interval needs to be shortened, and the corresponding work intensity and data volume also increase rapidly. If the similar material model is monitored by video, the test cycle of the similar material model is generally 2-3 weeks, and the video volume is about 30G per day. The huge amount of monitoring data brings great difficulties to data storage and processing. At the same time, it is also very difficult to accurately find the rock deformation and failure moment from these video files. SUMMARY

[0004] The purpose of the present application is to overcome the problem that the prior art cannot meet the real-time monitoring demand of the internal deformation and failure of the rock mass, and to provide a video monitoring method for the internal deformation and failure of the rock mass, which can solve the problem of data redundancy in video monitoring by using key frame extraction, and finally form a continuous monitoring method for the internal rock change and failure of one or more similar material models including key frames and working face advancing degree.

[0005] To achieve the above purpose, the technical solution of the present application is as follows:

[0006] A continuous monitoring method for internal rock change and failure of a similar material model, the monitoring method comprising the following steps:

[0007] Step 1, video acquisition:

[0008] The monitoring target is continuously photographed, and a plurality of time-continuous video segments are formed in sequence;

[0009] Step 2, key frame extraction:

[0010] In the video acquisition, key frame extraction is performed on each video segment in sequence, and the extracted key frames are saved.

[0011] The key frame extraction of the video segment includes the following steps:

[0012] S1, key frame indicators of each image frame in the video segment are extracted, a key frame indicator list L_index containing the key frame indicators of each image frame in the video segment is obtained, and then step S2 is entered;

[0013] S2, 2-frame differences are calculated according to the key frame indicator list L_index, a 2-frame difference list L2 containing all 2-frame differences is obtained, 3-frame differences are calculated according to the key frame indicator list L_index, a 3-frame difference list L3 containing all 3-frame differences is obtained, and then step S3 is entered;

[0014] The 2-frame difference is the difference between the key frame indicators of two adjacent image frames in the video segment, and the 3-frame difference is the difference between the key frame indicators of the first image frame and the third image frame in three adjacent image frames in the video segment;

[0015] S3, the minimum value L2[i] in the 2-frame difference list L2 is found, the L2[i] is the difference between the key frame indicator of the i-th image frame in the video segment and the key frame indicator of the i+1-th image frame in the video segment;

[0016] When the i-th image frame in the video segment is the first image frame in the video segment, the second image frame in the video segment is deleted, and the key frame indicator corresponding to the second image frame in the video segment is deleted in the key frame indicator list L_index, and then step S5 is entered;

[0017] When the i+1-th image frame in the video segment is the last image frame in the entire video segment, the second last image frame in the video segment is deleted, and the key frame indicator corresponding to the second last image frame in the video segment is deleted in the key frame indicator list L_index, and then step S5 is entered;

[0018] When the i-th image frame in the video segment is not the first image frame in the video segment, and the i+1-th image frame in the video segment is not the last image frame in the video segment, step S4 is entered;

[0019] S4, L3[i] and L3[i-1] in the 3-frame difference list L3 are found, the L3[i] is the difference between the key frame indicator of the i-th image frame in the video segment and the key frame indicator of the i+2-th image frame in the video segment, and the L3[i-1] is the difference between the key frame indicator of the i-1-th image frame in the video segment and the key frame indicator of the i+1-th image frame in the video segment;

[0020] comparing the size of L3[i] and L3[i-1], if L3[i]>L3[i-1], deleting the i-th image frame in the video segment, and deleting the key frame index corresponding to the i-th image frame in the key frame index list L_index, then entering step S5;

[0021] if L3[i]<=L3[i-1], deleting the i+1-th image frame in the video segment, and deleting the key frame index corresponding to the i+1-th image frame in the key frame index list L_index, then entering step S5;

[0022] S5, judging whether the removal rate of image frames in the video segment is greater than a set value, if the removal rate of image frames in the video segment is greater than the set value, the remaining image frames in the video segment which are not deleted are key frames, and the key frame extraction is completed;

[0023] if the removal rate of image frames in the video segment is less than or equal to the set value, returning to step S2.

[0024] the removal rate of image frames in the video segment is the number of deleted image frames / total number of image frames in the video segment.

[0025] the monitoring target is a similar material model of rock stratum, and an observation surface of the similar material model is arranged to face the shooting device;

[0026] in the step one, the observation surface of the similar material model of rock stratum is continuously shot by using the shooting device, and a plurality of time-continuous video segments are sequentially formed.

[0027] the shooting device is an IP camera, and the distance between the IP camera and the similar material model is between 3 meters and 4 meters.

[0028] in the step S1, the key frame index is the total area of rock stratum cracks inside the similar material model 1 in the image frame;

[0029] in the step S1, the key frame index of the image frame in the video segment includes the following steps:

[0030] A1, obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image by using a Gaussian filter;

[0031] A2, calculating the gradient of the smoothed image in different directions by using a sobel operator, and performing non-maximum suppression on the gradient amplitude;

[0032] A3, determining a gray threshold of the non-maximum suppression image by using an otsu algorithm, segmenting the non-maximum suppression image according to the determined gray threshold, and obtaining a binary image containing edge contours of the internal rock stratum cracks of the similar material model;

[0033] A4, performing inflation and corrosion processing on the obtained binary image;

[0034] A5, extracting the edge contours of the internal rock stratum cracks of the similar material model in the image after the inflation and corrosion processing;

[0035] A6, calculating the total area of the internal rock stratum cracks of the similar material model in the image frame according to the extracted edge contours of the internal rock stratum cracks of the similar material model, and the total area of the internal rock stratum cracks of the similar material model in the image frame is the key frame index of the corresponding image frame in the video segment.

[0036] The surface of the similar material model is provided with a plurality of monitoring points, and a point on the observation surface of the similar material model is the origin of the model coordinate system, the X-axis of the model coordinate system is a straight line on the observation surface of the similar material model passing through the origin and parallel to the horizontal plane, and the Y-axis of the model coordinate system is a straight line on the observation surface of the similar material model passing through the origin and perpendicular to the X-axis;

[0037] In the step S1, the key frame index is the coordinates of the monitoring points in the image frame in the model coordinate system;

[0038] In the step S1, the extraction of the key frame index of the image frame in the video segment includes the following steps:

[0039] B1, obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image frame by Gaussian filtering;

[0040] B2, obtaining the coordinates of the monitoring points in the image frame in the pixel coordinate system after the smoothing processing;

[0041] B3, calculating the coordinates of the monitoring points in the image frame in the model coordinate system according to the coordinates of the monitoring points in the image frame in the pixel coordinate system and the mapping relationship of the model coordinate system relative to the pixel coordinate system, and the coordinates of the monitoring points in the image frame in the model coordinate system are the key frame index of the corresponding image frame in the video segment.

[0042] The mapping relationship of the model coordinate system relative to the pixel coordinate system in the step B3 is obtained by the following steps:

[0043] B301, measuring the coordinates of the monitoring points in the model coordinate system;

[0044] B302, obtaining the corresponding coordinates of the monitoring points in the image frame in the pixel coordinate system;

[0045] B303, calculate the translation parameter and the rotation parameter of the pixel coordinate system relative to the model coordinate system according to the coordinates of the monitoring point in the model coordinate system and the corresponding coordinates of the monitoring point in the pixel coordinate system in the image frame.

[0046] In the step S1, the key frame index is a feature binary image of the image frame.

[0047] In the step S1, the extracting the key frame index of the image frame in the video segment comprises the following steps:

[0048] C1, obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image frame through Gaussian filtering.

[0049] C2, performing binary processing on the smoothed image frame through an otsu automatic threshold segmentation algorithm to obtain a feature binary image of the image frame, and the feature binary image of the image frame is the key frame index of the corresponding image frame in the video segment.

[0050] In the step one, in the video acquisition, the monitoring target is continuously shot, and the shooting time of each image frame is recorded.

[0051] Compared with the prior art, the method has the beneficial effects that:

[0052] 1, in the continuous monitoring method of the internal rock layer change and damage of a similar material model, the similar material model is continuously shot according to the preset video length and frame rate, and the space-time characteristics of the rock layer deformation and damage in the model are accurately obtained; at the same time, since the video sequence obtained by monitoring includes a plurality of video segments with the same length and frame rate arranged in time sequence, the video segments that have been shot are processed at the same time of monitoring and shooting, so that the effect of instant processing is achieved. Therefore, the continuous monitoring method in the application can accurately obtain the space-time characteristics of the rock layer deformation and damage in the model, and can achieve the effects of real-time shooting and instant processing, thereby avoiding a large amount of redundant data.

[0053] 2、The application is a kind of similar material model internal rock layer change and destruction continuous monitoring method, the video segment that has been shot is handled, according to the key frame index calculation 2 frame difference and 3 frame difference, combine 2 frame difference and 3 frame difference in video segment to accurately locate key frame, then the key frame is extracted and saved to form new video segment, relative to the traditional way of only comparing 2 frame difference or 3 frame difference to determine key frame, relative to the key frame extraction method in traditional method by setting threshold to filter image frame, the scheme in this application does not involve frame difference experience threshold, not affected by frame difference experience threshold, the representative of the extracted key frame is better, can more accurately reflect the change of monitoring target, and the data amount of new video segment is smaller, and the redundant data is less.Therefore, the key frame is determined by combining 2 frame difference and 3 frame difference information in the application, the representative of the extracted key frame is better, can more accurately reflect the change of monitoring target, while reducing redundant data, avoid monitoring data too large.

[0054] 3、The application is a kind of similar material model internal rock layer change and destruction continuous monitoring method, different data processing modules can be called, and then the similar material model internal rock layer crack change degree, rock layer deformation or crack and separation comprehensive change amount and other different key frame indexes are used to process video segment, generate video data set of monitoring point displacement data, crack area change data and other data sets, and the generated monitoring data set can accurately reflect the space-time characteristics of rock layer deformation and destruction in the model from multiple angles.Therefore, three kinds of key frame indexes are set in the application, different key frame indexes are used to process video segment and generate different data sets, so that the monitoring data can accurately reflect the space-time characteristics of rock layer deformation and destruction in the model from multiple angles.

[0055] 4、The application is a kind of similar material model internal rock layer change and destruction continuous monitoring method, when the crack change degree of similar material model internal rock layer is used as a key frame index, the crack contour in the image frame is extracted and the total crack area in each image frame is calculated by processing the image frame, the total crack area change in each frame image is monitored and compared, and the image frame with larger total crack area change is extracted as the key frame, which can correspondingly generate a video data set that can reflect the crack development process.Therefore, the crack change degree of material model internal rock layer is used as a key frame index to extract the key frame in the application, which can correspondingly generate a video data set that can reflect the crack development process.

[0056] 5、The similar material model internal rock layer change and destruction continuous monitoring method of the present application, when using the similar material model internal rock layer deformation amount as the key frame index, sets monitoring points on the similar material model, reflects the similar material model internal rock layer deformation amount through the displacement of the monitoring points, and the displacement change of the multiple monitoring points can reflect the real-time change of the material model internal rock layer; meanwhile, when processing the image frames, extracts the image frames with larger position change of the monitoring points as the key frames, and can correspondingly generate the video data set capable of reflecting the monitoring point position. Therefore, in the present application, the monitoring points are set on the similar material model, when processing the image frames, the image frames with larger position change of the monitoring points are extracted as the key frames, and the video data set capable of reflecting the monitoring point position can be correspondingly generated, and the monitoring data can truly reflect the deformation process of the rock layer in the model.

[0057] 6、The similar material model internal rock layer change and destruction continuous monitoring method of the present application, when using the similar material model internal rock layer crack and separation comprehensive change amount as the key frame index, utilizes the otsu automatic threshold segmentation algorithm to perform binaryzation processing on the image frames, obtains the feature binaryzation image of each image frame, and extracts the image frames with larger change of the feature binaryzation image as the key frames, and can correspondingly generate the video data capable of comprehensively reflecting the mining progress. Therefore, in the present application, the binaryzation information of each image frame is obtained, and the image frames with larger change of the binaryzation information are extracted as the key frames, and the video data capable of comprehensively reflecting the mining progress can be correspondingly generated, and the monitoring data can truly reflect the overall change of the monitoring target. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 It is the flow chart of the monitoring method in the present application.

[0059] Figure 2 It is the flow chart of calculating the total area of the similar material model internal rock layer crack.

[0060] Figure 3 It is the flow chart of calculating the coordinates of the monitoring points in the model coordinate system.

[0061] Figure 4 It is the schematic diagram of the IP camera continuously shooting the similar material model internal rock layer change and destruction.

[0062] In the figure: similar material model 1, monitoring point 2, IP camera 3. DETAILED DESCRIPTION

[0063] The present application is further described in detail in combination with the description and specific implementation of the drawings.

[0064] Reference Figures 1 to 4A method for continuously monitoring internal rock stratum changes and destruction of a similar material model, the monitoring method comprising the following steps:

[0065] Step one, image acquisition:

[0066] The monitoring target is continuously photographed, and a plurality of time-continuous video segments are sequentially formed;

[0067] The monitoring target is a similar material model 1 of a rock stratum, an IP camera is placed opposite an observation surface of the similar material model 1, and the IP camera is connected to a computer, the IP camera continuously photographs the similar material model 1 of the rock stratum at a preset frame rate, and the relative position of the IP camera and the monitoring target remains unchanged during the photographing process.

[0068] When the length of the collected video reaches a preset video length n, a video segment is formed. As shown in Figure 1 、 Figure 4 When n image frames are collected, the n image frames form a new video segment, and the video segment can be subjected to key frame extraction processing after being formed.

[0069] Step two, key frame extraction:

[0070] While the images are being collected, key frames are extracted from each video segment collected in turn according to the order of the photographing time, and the extracted key frames are saved, and the saved key frames form a new video data set.

[0071] In the step two, key frame extraction, the key frames of a video segment collected are extracted, comprising the following steps:

[0072] S1, key frame indicators of each image frame in the video segment are extracted, a key frame indicator list L_index containing the key frame indicators of each image frame in the video segment is obtained, and then step S2 is entered;

[0073] Each image frame in the video segment corresponds to a key frame indicator, and all the key frame indicators are stored in the key frame indicator list L_index.

[0074] S2, 2-frame differences are calculated according to the key frame indicator list L_index, a 2-frame difference list L2 containing all the 2-frame differences is obtained, 3-frame differences are calculated according to the key frame indicator list L_index, a 3-frame difference list L3 containing all the 3-frame differences is obtained, and then step S3 is entered;

[0075] The 2-frame difference is the difference between the key frame indicators of two adjacent image frames in the corresponding video segment, and the 3-frame difference is the difference between the key frame indicator of the first image frame and the key frame indicator of the third image frame in the three adjacent image frames in the corresponding video segment.

[0076] The 2-frame difference list L2 contains all 2-frame differences in the corresponding video segment, and if the video segment contains n image frames, the 2-frame difference list L2 contains n-1 2-frame differences in total.

[0077] S3, find the minimum value L2[i] in the 2-frame difference list L2, wherein the L2[i] is the difference between the key frame index of the i-th image in the video segment and the key frame index of the i+1-th image in the video segment;

[0078] When the i-th image in the video segment is the first image in the entire video segment, delete the second image in the video segment, and simultaneously delete the key frame index corresponding to the second image in the key frame index list L_index, and then proceed to step S5;

[0079] When the i+1-th image in the video segment is the last image in the entire video segment, delete the second last image in the video segment, and simultaneously delete the key frame index corresponding to the second last image in the key frame index list L_index, and then proceed to step S5;

[0080] When the i-th image in the video segment is not the first image in the entire video segment, and the i+1-th image in the video segment is not the last image in the entire video segment, proceed to step S4;

[0081] S4, find L3[i] and L3[i-1] in the 3-frame difference list L3, wherein the L3[i] is the difference between the key frame index of the i-th image in the video segment and the key frame index of the i+2-th image in the video segment, and the L3[i-1] is the difference between the key frame index of the i-1-th image in the video segment and the key frame index of the i+1-th image in the video segment;

[0082] Compare the size of L3[i] and L3[i-1], if L3[i]>L3[i-1], delete the i-th image in the video segment, and simultaneously delete the key frame index corresponding to the i-th image in the key frame index list L_index, and then proceed to step S5;

[0083] If L3[i]<=L3[i-1], delete the i+1-th image in the video segment, and simultaneously delete the key frame index corresponding to the i+1-th image in the key frame index list L_index, and then proceed to step S5;

[0084] S5, determining whether the removal rate of the image frames in the video segment reaches a set value, if the removal rate of the image frames in the video segment is greater than the set value, the remaining image frames in the video segment that are not deleted are key frames, and the key frame extraction is completed;

[0085] If the removal rate of the image frames in the video segment is less than or equal to the set value, return to step S2.

[0086] The removal rate of the image frames in the video segment is the number of deleted image frames / total number of frames of the video segment.

[0087] The total number of frames of the video segment is the sum of the number of deleted image frames and the number of remaining image frames that are not deleted.

[0088] The removal rate can be set artificially according to the specific situation of the monitoring target and the data storage requirement.

[0089] When the key frame index is the total area of the internal rock layer cracks of the similar material model 1 in the image frames:

[0090] As shown in Figure 2 The step S1 includes the following steps of extracting the key frame index of the image frames in the video segment:

[0091] A1, obtaining the image frames in the video segment, performing gray scale processing on the image frames in the video segment, and performing smoothing processing on the obtained gray scale image through Gaussian filtering;

[0092] In actual monitoring, the image frames can be smoothed by using the Gaussian filtering function in OpenCV to eliminate image noise interference.

[0093] A2, calculating the gradient of the smoothed image in different directions by using the sobel operator, and performing non-maximum suppression on the gradient amplitude;

[0094] After calculating the gradient of the denoised image in different directions, the non-gradient maximum value direction of the denoised image is suppressed and weakened, and the gray value at the non-maximum value is assigned as 0, so as to facilitate subsequent edge extraction;

[0095] A3, determining the gray threshold of the image after non-maximum suppression by using the otsu algorithm, and segmenting the image after non-maximum suppression according to the determined gray threshold to obtain a binary image containing the edge contour of the internal rock layer cracks of the similar material model 1;

[0096] According to the determination of the gray threshold m of the image frame by the otsu algorithm, the image frame processed by the sobel operator is binarized, the gray value of the pixel with a gray value greater than m in the gray image is set to 255, and the gray value of the pixel with a gray value less than or equal to m in the gray image is set to 0, so as to extract the profile of the internal rock fracture of the similar material model in the image frame.

[0097] A4, dilate and erode the binarized image;

[0098] Because the edge profile of part of the cracks in the image is not closed, the dilatation and erosion processing can make it closed, so as to extract the crack profile and calculate the area of the crack on the image frame in the subsequent process. The corresponding algorithm in OpenCV can be used to dilate and erode the binarized image, so that the edge profile of the internal rock fracture of the similar material model 1 in the binarized image is closed.

[0099] A5, extract the edge profile of the internal rock fracture of the similar material model 1 in the image after dilatation and erosion processing;

[0100] A6, calculate the total area of the internal rock fracture of the similar material model 1 in the image frame according to the extracted edge profile of the internal rock fracture of the similar material model 1, and the total area of the internal rock fracture of the similar material model 1 in the image frame is the key frame index of the corresponding image frame in the video segment.

[0101] The total area of the internal rock fracture of the similar material model 1 in each image frame is calculated, and the total area of the internal rock fracture of the similar material model 1 in each image frame is stored in the key frame index list L_index as a key frame index.

[0102] Because the 2-frame difference is the difference between the key frame indexes of the adjacent two image frames in the video segment, the 2-frame difference is the change amount of the total area of the internal rock fracture of the adjacent two image frames. Because the 3-frame difference is the difference between the key frame index of the first image frame and the key frame index of the third image frame in the adjacent three image frames in the video segment, the 3-frame difference is the change amount of the total area of the internal rock fracture of the first image frame relative to the total area of the internal rock fracture of the third image frame.

[0103] When the key frame index is the coordinate of the monitoring point 2 in the model coordinate system in the image frame:

[0104] A plurality of monitoring points 2 are arranged on the surface of the similar material model 1, and a model coordinate system is established on the observation surface of the similar material model 1, a point on the observation surface of the similar material model 1 is taken as the origin O of the model coordinate system, the X axis of the model coordinate system is a straight line on the observation surface of the similar material model 1 passing through the origin and parallel to the horizontal plane, and the Y axis of the model coordinate system is a straight line on the observation surface of the similar material model 1 passing through the origin and perpendicular to the X axis.

[0105] As shown in Figure 4 , the origin O of the model coordinate system is set at the upper left corner of the observation surface.

[0106] When the key frame index is the coordinate of the monitoring point 2 in the image frame under the model coordinate system, the monitoring target includes the similar material model 1 and the monitoring point 2 set on the surface of the similar material model 1.

[0107] As shown in Figure 3 , in step S1, the key frame index of the image frame in the video segment includes the following steps:

[0108] B1, obtain the image frame in the video segment, perform grayscale processing on the image frame in the video segment, and perform smoothing processing on the obtained grayscale image frame through Gaussian filtering;

[0109] B2, obtain the coordinate of the monitoring point 2 in the smoothed image frame under the pixel coordinate system;

[0110] The origin of the pixel coordinate system is located at the upper left corner of the image, and the horizontal and vertical coordinates are respectively represented as the row and column of the pixel point.

[0111] The number of monitoring points 2 can be set according to the size of the similar material model and the monitoring requirement. As shown in Figure 4 , the shape of the monitoring point 2 is circular, and the Hough circle detection algorithm is used to detect the smoothed image frame, so that the coordinate of the center of the circular monitoring point 2 under the pixel coordinate system can be obtained. The coordinate of the center of the circular monitoring point 2 under the pixel coordinate system is the coordinate of the monitoring point 2 in the smoothed image frame under the pixel coordinate system.

[0112] B3, calculate the coordinate of the monitoring point 2 in the image frame under the model coordinate system according to the coordinate of the monitoring point 2 in the image frame under the pixel coordinate system and the mapping relationship of the model coordinate system relative to the pixel coordinate system. The coordinate of the monitoring point 2 in the image frame under the model coordinate system is the key frame index of the corresponding image frame in the video segment.

[0113] The coordinate of the monitoring point 2 in each image frame under the model coordinate system is calculated, and the coordinate of the monitoring point 2 in each image frame under the model coordinate system is stored in the key frame index list L_index as the key frame index.

[0114] Since the 2-frame difference is the difference between the key frame indicators of two adjacent image frames in the video segment, the 2-frame difference is the change in the coordinates of the monitoring point 2 in the model coordinate system in the two adjacent image frames. Since the 3-frame difference is the difference between the key frame indicators of the first image frame and the third image frame in the three adjacent image frames in the video segment, the 3-frame difference is the change in the coordinates of the monitoring point 2 in the model coordinate system in the first image frame relative to the coordinates of the monitoring point 2 in the model coordinate system in the third image frame.

[0115] The mapping relationship of the model coordinate system relative to the pixel coordinate system in step B3 is obtained by the following steps:

[0116] B301, measuring the coordinates of the monitoring point 2 in the model coordinate system;

[0117] B302, obtaining the corresponding coordinates of the monitoring point 2 in the pixel coordinate system in the image frame;

[0118] B303, calculating the translation parameters and rotation parameters of the pixel coordinate system relative to the model coordinate system according to the coordinates of the monitoring point 2 in the model coordinate system and the corresponding coordinates of the monitoring point 2 in the pixel coordinate system in the image frame.

[0119] The coordinates of the center of the circular monitoring point 2 in the pixel coordinate system can be obtained by using the Hough circle detection algorithm, and then the translation parameters and rotation parameters of the coordinate system can be obtained under the principle of least squares.

[0120] The mapping relationship of the pixel coordinate system relative to the model coordinate system should be obtained in advance before the monitoring starts, so that in the subsequent process of monitoring the internal rock changes and damage of the similar material model, the actual coordinates of the monitoring point 2 in the model coordinate system can be calculated according to the coordinates of the monitoring point 2 in the pixel coordinate system in the image frame.

[0121] When the key frame indicator is a feature binary image of the image frame:

[0122] In step S1, the key frame indicator of the image frame in the video segment includes the following steps:

[0123] C1, obtaining the image frame in the video segment, performing grayscale processing on the image frame in the video segment, and performing smoothing processing on the obtained grayscale image frame by Gaussian filtering;

[0124] C2, performing binaryzation processing on the smoothed image frame by using the otsu automatic threshold segmentation algorithm and obtaining a feature binary image of the image frame, the feature binary image being the key frame indicator of the corresponding image frame in the video segment.

[0125] The feature binary image corresponding to each image frame is calculated, and each feature binary image corresponding to each image frame is stored in the key frame index list L_index as a key frame index.

[0126] Since the image after binaryzation by the automatic threshold otsu algorithm remains unchanged in theory when the monitoring target does not change, and the binaryzation image changes once the monitoring target changes, the change amount can also be used as a frame difference calculation index to calculate the frame difference.

[0127] Since the 2-frame difference is the difference between the key frame indexes of two adjacent image frames in the video segment, the 2-frame difference in the 2-frame difference list L2 is the difference between the feature binary images corresponding to the two adjacent image frames, and the 2-frame difference is calculated by the following formula (1):

[0128] 2FD = h*w-count(a ij m+1 -a ij m ≠ 0) (1)

[0129] In formula (1), 2FD is the 2-frame difference, h is the height of each image frame, w is the width of each image frame, h and w are both in pixels, count() represents counting the functions that meet the conditions in the brackets, and a ij m represents the gray value of the pixel point at the i-th row and the j-th column in the feature binary image corresponding to the m-th image frame in the video segment.

[0130] Since the 3-frame difference is the difference between the key frame index of the first image frame and the key frame index of the third image frame in the three adjacent image frames in the video segment, the 3-frame difference in the 3-frame difference list L3 is the difference between the feature binary image corresponding to the first image frame and the feature binary image corresponding to the third image frame in the three adjacent image frames, and the 3-frame difference is calculated by the following formula (2):

[0131] 3FD = h*w-count(a ij n+2 -a ij n ≠ 0) (2)

[0132] In formula (2), 3FD is the 3-frame difference, h is the height of each image frame, w is the width of each image frame, h and w are both in pixels, count() represents counting the functions that meet the conditions in the brackets, and a ij n represents the gray value of the pixel point at the i-th row and the j-th column in the binary image corresponding to the n-th image frame in the video segment.

[0133] In the step one, the photographing time of each image is recorded while the similar material model is continuously photographed.

[0134] The principle of the present application is explained as follows:

[0135] In the monitoring of the similar material model, different image data processing modules can be set, and the different image data processing modules are respectively used to realize: extracting the total area of the rock stratum cracks in the similar material model 1 in the image frame, extracting the coordinates of the monitoring point 2 in the model coordinate system in the image frame, extracting the feature binary image corresponding to the image frame, and then obtaining different key frame index lists L_index, so as to calculate the 2-frame difference and 3-frame difference in the subsequent steps.

[0136] In the extraction of the key frame, different image data processing modules are selected according to different monitoring emphases. In the monitoring of the rock stratum changes and damages in the similar material model, the first image data processing module is used to extract the total area of the rock stratum cracks in the similar material model 1 in the image frame, the rock stratum crack change degree in the similar material model 1 is used as the key frame index to extract the key frame of the video segment, and the video data set reflecting the crack area change can be correspondingly generated. The second image data processing module is used to extract the coordinates of the monitoring point 2 on the similar material model 1 in the model coordinate system in the image frame, the rock stratum deformation in the similar material model 1 is used as the key frame index to extract the key frame of the video segment, and the video data set reflecting the monitoring point position can be correspondingly generated. The third image data processing module is used to extract the feature binary image of the image frame, and the video data reflecting the mining progress can be correspondingly generated.

[0137] The monitoring target can be the similar material model 1 of the rock stratum, or other models, devices or work sites. In the monitoring of different monitoring targets, it can be determined which kind of key frame index is used according to the characteristics of the detection target, and then the key frame index of each image frame in the monitoring video is extracted.

[0138] Embodiment 1:

[0139] A continuous monitoring method of rock stratum changes and damages in a similar material model, the monitoring method comprising the following steps:

[0140] Step one, video acquisition:

[0141] The monitoring target is continuously photographed, and a plurality of time-continuous video segments are sequentially formed;

[0142] Step two, key frame extraction:

[0143] While the video is being collected, the key frame extraction is sequentially performed on each video segment that has been collected, and the extracted key frame is saved.

[0144] In the step one, the shooting time of each frame image is recorded while the monitoring target is continuously shot in the video shooting.

[0145] In the step two, the key frame extraction of the collected video segment includes the following steps:

[0146] S1, the key frame index of each image frame in the video segment is extracted, and a key frame index list L_index containing the key frame index of each image frame in the video segment is obtained, and then step S2 is entered;

[0147] S2, 2-frame difference is calculated according to the key frame index list L_index, and a 2-frame difference list L2 containing all 2-frame differences is obtained, 3-frame difference is calculated according to the key frame index list L_index, and a 3-frame difference list L3 containing all 3-frame differences is obtained, and then step S3 is entered;

[0148] The 2-frame difference is the difference between the key frame indexes of two adjacent image frames in the video segment, and the 3-frame difference is the difference between the key frame index of the first image frame and the key frame index of the third image frame in three adjacent image frames in the video segment;

[0149] S3, the minimum value L2[i] in the 2-frame difference list L2 is found, and the L2[i] is the difference between the key frame index of the i-th image frame in the video segment and the key frame index of the i+1-th image frame in the video segment;

[0150] When the i-th image frame in the video segment is the first image frame in the entire video segment, the second image frame in the video segment is deleted, and the key frame index corresponding to the second image frame in the video segment is deleted in the key frame index list L_index, and then step S5 is entered;

[0151] When the i+1-th image frame in the video segment is the last image frame in the entire video segment, the second-to-last image frame in the video segment is deleted, and the key frame index corresponding to the second-to-last image frame in the video segment is deleted in the key frame index list L_index, and then step S5 is entered;

[0152] When the i-th image frame in the video segment is not the first image frame in the entire video segment, and the i+1-th image frame in the video segment is not the last image frame in the entire video segment, step S4 is entered;

[0153] S4, find L3[i] and L3[i-1] in the 3-frame difference list L3, the L3[i] is the difference value between the key frame index of the i-th frame image in the video segment and the key frame index of the i+2-th frame image in the video segment, and the L3[i-1] is the difference value between the key frame index of the i-1-th frame image in the video segment and the key frame index of the i+1-th frame image in the video segment;

[0154] Compare the size of L3[i] and L3[i-1], if L3[i]>L3[i-1], delete the i-th frame image in the video segment, and delete the key frame index corresponding to the i-th frame image in the video segment in the key frame index list L_index, then enter step S5;

[0155] If L3[i]≤L3[i-1], delete the i+1-th frame image in the video segment, and delete the key frame index corresponding to the i+1-th frame image in the video segment in the key frame index list L_index, then enter step S5;

[0156] S5, judge whether the removal rate of image frames in the video segment reaches the set value, if the removal rate of image frames in the video segment is greater than the set value, the remaining image frames in the video segment which are not deleted are key frames, and the key frame extraction is completed;

[0157] If the removal rate of image frames in the video segment is less than or equal to the set value, return to step S2.

[0158] The removal rate of image frames in the video segment is the number of deleted image frames / total number of frames of the video segment.

[0159] The monitoring target is a similar material model 1 of rock stratum, and an observation surface of the similar material model 1 faces the shooting device 3;

[0160] In the step one, the observation surface of the similar material model 1 of rock stratum is continuously shot by using the shooting device 3, and a plurality of time-continuous video segments are sequentially formed.

[0161] The shooting device 3 is an IP camera, and the distance between the IP camera and the similar material model 1 is between 3 meters and 4 meters.

[0162] In the step S1, the key frame index is the total area of the rock stratum cracks inside the similar material model 1 in the image frame;

[0163] In the step S1, the key frame index of the image frame in the video segment includes the following steps:

[0164] A1, obtain the image frame in the video segment, perform gray processing on the image frame in the video segment, and perform smoothing processing on the obtained gray image by using Gaussian filtering;

[0165] A2, calculate the gradient of the image after smoothing in different directions by using the sobel operator, and perform non-maximum suppression on the gradient amplitude;

[0166] A3, determine the gray threshold of the image after non-maximum suppression by using the otsu algorithm, and segment the image after non-maximum suppression according to the determined gray threshold to obtain a binary image containing the edge contour of the rock fracture inside the similar material model 1;

[0167] A4, perform inflation and corrosion processing on the obtained binary image;

[0168] A5, extract the edge contour of the rock fracture inside the similar material model 1 in the image after inflation and corrosion processing;

[0169] A6, calculate the total area of the rock fracture inside the similar material model 1 in the image frame according to the extracted edge contour of the rock fracture inside the similar material model 1, and the total area of the rock fracture inside the similar material model 1 in the image frame is the key frame index of the corresponding image frame in the video segment.

[0170] Embodiment 2:

[0171] Embodiment 2 is basically the same as embodiment 1, the difference is:

[0172] A plurality of monitoring points 2 are arranged on the surface of the similar material model 1, a point on the observation surface of the similar material model 1 is the origin of the model coordinate system, the X axis of the model coordinate system is a straight line on the observation surface of the similar material model 1 passing through the origin and parallel to the horizontal plane, and the Y axis of the model coordinate system is a straight line on the observation surface of the similar material model 1 passing through the origin and perpendicular to the X axis;

[0173] In step S1, the key frame index is the coordinates of the monitoring point 2 in the image frame in the model coordinate system;

[0174] In step S1, the key frame index of the image frame in the video segment includes the following steps:

[0175] B1, obtain the image frame in the video segment, perform gray processing on the image frame in the video segment, and perform smoothing processing on the obtained gray image frame by Gaussian filtering;

[0176] B2, obtain the coordinates of the monitoring point 2 in the image frame after smoothing in the pixel coordinate system;

[0177] B3、calculate the coordinates of the monitoring point 2 in the model coordinate system in the image frame according to the coordinates of the monitoring point 2 in the pixel coordinate system in the image frame and the mapping relationship of the model coordinate system relative to the pixel coordinate system, the coordinates of the monitoring point 2 in the model coordinate system in the image frame being the key frame index of the corresponding image frame in the video segment.

[0178] The mapping relationship of the model coordinate system relative to the pixel coordinate system in the step B3 is obtained by the following steps:

[0179] B301、measure the coordinates of the monitoring point 2 in the model coordinate system;

[0180] B302、obtain the corresponding coordinates of the monitoring point 2 in the pixel coordinate system in the image frame;

[0181] B303、calculate the translation parameters and rotation parameters of the pixel coordinate system relative to the model coordinate system according to the coordinates of the monitoring point 2 in the model coordinate system and the corresponding coordinates of the monitoring point 2 in the pixel coordinate system in the image frame.

[0182] Embodiment 3:

[0183] Embodiment 3 is basically the same as Embodiment 2, and the difference is that:

[0184] In the step S1, the key frame index is a feature binary image of the image frame;

[0185] In the step S1, the extraction of the key frame index of the image frame in the video segment includes the following steps:

[0186] C1、obtain the image frame in the video segment, perform grayscale processing on the image frame in the video segment, and perform smoothing processing on the obtained grayscale image frame through Gaussian filtering;

[0187] C2、perform binaryzation processing on the smoothed image frame by using an otsu automatic threshold segmentation algorithm and obtain a feature binary image, the feature binary image being the key frame index of the corresponding image frame in the video segment.

[0188] The above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above embodiments, but any equivalent modification or change made by those skilled in the art according to the disclosed content of the present application shall be included in the protection scope recorded in the claims.

Claims

1. A method for continuously monitoring internal rock layer changes and destruction of a similar material model, characterized in that: the monitoring method comprises the following steps: Step one, video acquisition: Continuous shooting of the monitoring target is performed, and a plurality of time-continuous video segments are sequentially formed; Step two, key frame extraction: Key frame extraction is performed on each video segment that has been collected in sequence while video acquisition is being performed, and the extracted key frames are saved; In the step two, key frame extraction, the key frame extraction on the collected video segment comprises the following steps: S1, extract the key frame index of each image frame in the video segment to obtain a key frame index list L_index containing the key frame index of each image frame in the video segment, and then proceed to step S2; S2, calculate the 2-frame difference according to the key frame index list L_index to obtain a 2-frame difference list L2 containing all 2-frame differences, and calculate the 3-frame difference according to the key frame index list L_index to obtain a 3-frame difference list L3 containing all 3-frame differences, and then proceed to step S3; The 2-frame difference is the difference between the key frame indexes of two adjacent image frames in the video segment, and the 3-frame difference is the difference between the key frame index of the first image frame and the key frame index of the third image frame in three adjacent image frames in the video segment; S3, find the minimum value L2[i] in the 2-frame difference list L2, which is the difference between the key frame index of the i-th image frame in the video segment and the key frame index of the i+1-th image frame in the video segment; When the i-th image frame in the video segment is the first image frame in the video segment, delete the second image frame in the video segment, and simultaneously delete the key frame index corresponding to the second image frame in the video segment in the key frame index list L_index, and then proceed to step S5; When the i+1-th image frame in the video segment is the last image frame in the video segment, delete the second-to-last image frame in the video segment, and simultaneously delete the key frame index corresponding to the second-to-last image frame in the video segment in the key frame index list L_index, and then proceed to step S5; When the i-th image frame in the video segment is not the first image frame in the video segment, and the i+1-th image frame in the video segment is not the last image frame in the video segment, proceed to step S4; S4, find L3[i] and L3[i-1] in the 3-frame difference list L3, wherein L3[i] is the difference between the key frame index of the i-th image frame in the video segment and the key frame index of the i+2-th image frame in the video segment, and L3[i-1] is the difference between the key frame index of the i-1-th image frame in the video segment and the key frame index of the i+1-th image frame in the video segment; Compare the sizes of L3[i] and L3[i-1], if L3[i]>L3[i-1], delete the i-th image frame in the video segment, and simultaneously delete the key frame index corresponding to the i-th image frame in the video segment in the key frame index list L_index, and then proceed to step S5; S5, save the key frame corresponding to the key frame index in the key frame index list L_index as the key frame of the video segment, and then proceed to step S6; S6, repeat the steps S1-S5 until all video segments are collected, and then proceed to step S7; S7, save the key frame of each video segment as the key frame of the monitoring target, and then proceed to step S8; S8, repeat the steps S1-S7 until the monitoring target is completely collected, and then proceed to step S9; S9, save the key frame of the monitoring target as the key frame of the similar material model, and then proceed to step S10; S10, repeat the steps S1-S9 until the similar material model is completely collected, and then proceed to step S11; S11, save the key frame of the similar material model as the key frame of the internal rock layer changes and destruction of the similar material model, and then proceed to step S12; S12, repeat the steps S1-S11 until the internal rock layer changes and destruction of the similar material model are completely collected. If L3[i]≤L3[i-1], the i+1th frame image in the video segment is deleted, and the key frame index corresponding to the i+1th frame image in the video segment is deleted in the key frame index list L_index, and then step S5 is entered; S5, judging whether the removal rate of the image frames in the video segment is greater than a set value, if the removal rate of the image frames in the video segment is greater than the set value, the remaining image frames in the video segment are key frames, and the key frame extraction is completed; If the removal rate of the image frames in the video segment is less than or equal to the set value, return to step S2; In the step S1, when the key frame index is the total area of the rock stratum cracks in the similar material model (1) in the image frame, the step S1 includes the following steps for extracting the key frame index of the image frame in the video segment: A1, obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image by using the Gaussian filter; A2, calculating the gradient of the image after the smoothing processing in different directions by using the sobel operator, and performing non-maximum suppression on the gradient amplitude; A3, determining the gray threshold of the image after the non-maximum suppression by using the otsu algorithm, and performing segmentation on the image after the non-maximum suppression according to the determined gray threshold to obtain a binary image containing the edge contour of the rock stratum cracks in the similar material model (1); A4, performing inflation and corrosion processing on the obtained binary image; A5, extracting the edge contour of the rock stratum cracks in the similar material model (1) in the image after the inflation and corrosion processing; A6, calculating the total area of the rock stratum cracks in the similar material model (1) in the image frame according to the extracted edge contour of the rock stratum cracks in the similar material model (1), and the total area of the rock stratum cracks in the similar material model (1) in the image frame is the key frame index of the corresponding image frame in the video segment; The surface of the similar material model (1) is provided with a plurality of monitoring points (2), a point on the observation surface of the similar material model (1) is the origin of the model coordinate system, the X axis of the model coordinate system is a straight line on the observation surface of the similar material model (1) passing through the origin and parallel to the horizontal plane, and the Y axis of the model coordinate system is a straight line on the observation surface of the similar material model (1) passing through the origin and perpendicular to the X axis; In the step S1, when the key frame index is the coordinates of the monitoring point (2) in the model coordinate system in the image frame, the step S1 includes the following steps for extracting the key frame index of the image frame in the video segment: B1, obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image frame by using the Gaussian filter; B2, obtaining the coordinates of the monitoring point (2) in the pixel coordinate system in the image frame after the smoothing processing; B3, calculating the coordinates of the monitoring point (2) in the model coordinate system in the image frame according to the coordinates of the monitoring point (2) in the pixel coordinate system in the image frame and the mapping relationship of the model coordinate system relative to the pixel coordinate system, and the coordinates of the monitoring point (2) in the model coordinate system in the image frame are the key frame index of the corresponding image frame in the video segment. 2.The method according to claim 1, wherein: the removal rate of the image frames in the video segment is the number of the deleted image frames / the total number of the frames in the video segment. 3.The method according to claim 2, wherein: the monitoring target is a similar material model (1) of the rock stratum, and the observation surface of the similar material model (1) is arranged to face the shooting device (3) ; and in the step one, the shooting device (3) is used to continuously shoot the observation surface of the similar material model (1) of the rock stratum, and a plurality of time-continuous video segments are sequentially formed. 4.The method according to claim 3, wherein: the shooting device (3) is an IP camera, and the distance between the IP camera and the similar material model (1) is between 3 meters and 4 meters. 5.The method according to claim 1, wherein: the mapping relationship between the model coordinate system and the pixel coordinate system in the step B3 is obtained by the following steps: B301.measuring the coordinates of the monitoring point (2) in the model coordinate system; B302.obtaining the corresponding coordinates of the monitoring point (2) in the pixel coordinate system in the image frame; and B303.calculating the translation parameters and the rotation parameters of the pixel coordinate system relative to the model coordinate system according to the coordinates of the monitoring point (2) in the model coordinate system and the corresponding coordinates of the monitoring point (2) in the pixel coordinate system in the image frame. 6.The method according to any one of claims 1-4, wherein: when the key frame indicator is a feature binary image of the image frame in the step S1, the step S1 of extracting the key frame indicator of the image frame in the video segment comprises the following steps: C1.obtaining the image frame in the video segment, performing gray processing on the image frame in the video segment, and performing smoothing processing on the obtained gray image frame by using a Gaussian filter; and C2.carrying out binary processing on the smoothed image frame by using an otsu automatic threshold segmentation algorithm to obtain a feature binary image of the image frame, and the feature binary image of the image frame is the key frame indicator of the corresponding image frame in the video segment. 7.The method according to claim 1, wherein: in the step one, the shooting time of each image is recorded while the monitoring target is continuously shot. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

Images