Belt Coal Conveyor Video Transmission System for Intelligent Coal Mine Monitoring

Abstract

The present invention relates to the field of video compression technology, and specifically relates to a belt coal conveying video transmission system for intelligent coal mine monitoring. The system includes a video data acquisition module for extracting data; a video compression feature extraction module for determining the compression feature indexes corresponding to pixel points in the belt video frames; a video compression priority division module for determining the compression priorities of pixel points in the belt video frames; and a video compression transmission module for compressing and transmitting the belt video frames and the corresponding three-dimensional point cloud data using Huffman coding according to the compression priorities. The present invention improves the compression efficiency of relevant data with coal conveying features in the belt video frames.

Description

Technical Field

[0001] The present invention relates to the technical field of video compression, and particularly to a belt coal conveying video transmission system for intelligent coal mine monitoring. Background Art

[0002] The video compression and storage of belt coal conveying is of great significance in intelligent coal mine monitoring or industrial production. Compressing and storing the video of coal block belt transmission can be used to monitor and record the situation during the transmission process, including possible abnormal situations and accidents. This is very important for timely discovering potential safety problems and quickly taking measures to reduce the accident risk. In addition, once an accident occurs, the compressed video can be used for accident analysis and investigation to help determine the cause of the accident and take measures to prevent recurrence. By compressing and storing the belt coal conveying video, the operating status of the coal block belt transmission can also be monitored regularly or in real time. This helps to detect performance problems, wear levels of the equipment, and components that need maintenance or replacement. Timely maintenance can extend the life of the equipment and reduce production downtime.

[0003] Currently, in traditional video compression methods, only two-dimensional information of images or videos is considered. However, during the belt coal conveying process, the lighting conditions may change, and at the same time, accurate three-dimensional point cloud data and related video data need to be recorded. When traditional Huffman coding encodes video frames, it is usually based on frequency. The larger the frequency, the shorter the code will be used for representation, and the smaller the frequency, the longer the code will be used for representation, so as to achieve efficient compression and decompression of data. However, in the belt coal conveying video, the lighting conditions of the belt coal conveying scene may change greatly, and the frequencies of the data are often small and similar. Therefore, when using traditional Huffman coding, it is very easy to obtain a large number of data with similar and small frequencies, and when compressing them, there will be a problem of poor compression effect. Summary of the Invention

[0004] In order to solve the technical problem that due to the large possible change in the lighting conditions of the belt coal conveying scene, when using traditional Huffman coding, it is very easy to obtain a large number of data with similar and small frequencies, which will lead to poor compression effect, the purpose of the present invention is to provide a belt coal conveying video transmission system for intelligent coal mine monitoring, and the system includes the following modules:

[0005] A video data acquisition module, which is used to acquire the belt video frames and the corresponding three-dimensional point cloud data during the belt coal conveying process, and transmit them to the video compression feature extraction module and the video compression transmission module;

[0006] A video compression feature extraction module, which is used to analyze the motion features of belt video frames, obtain different blocks; determine the correlation sequence of pixel points according to the change conditions of pixel points in different directions in the belt video frames, and combine the blocks to which the pixel points belong, the information richness in the correlation sequence corresponding to the pixel points, and the three-dimensional point cloud data features to determine the compression feature index corresponding to the pixel points in the belt video frames, and transmit the processed data to the video compression priority division module;

[0007] A video compression priority division module, which is used to determine the illumination environment quality index through the contrast between different blocks and the contrast change of belt video frames at different times; combine the compression feature index and the illumination environment quality index to determine the compression priority of pixel points in the belt video frames, and transmit the compression priority to the video compression transmission module;

[0008] A video compression transmission module, which is used to compress the belt video frames and the corresponding three-dimensional point cloud data using Huffman coding according to the compression priority to obtain compressed data; transmit the compressed data.

[0009] Preferably, the analyzing the motion features of the belt video frames and obtaining different blocks includes:

[0010] Construct a distance metric between pixel points according to the motion features corresponding to each pixel point in the belt video frames; combine the distance metric and use the K-means clustering algorithm to divide the pixel points in the belt video frames into two categories; calculate the average value of the motion speeds of the pixel points in the two categories respectively, and take the category corresponding to the maximum average value as the candidate category; the pixel points within the candidate category form an initial belt block;

[0011] Taking any edge point of the initial belt block as the initial center point, perform region growing, and the obtained growing region is used as the belt block not blocked by coal blocks. Except for the belt block not blocked by coal blocks in the initial belt block, other blocks are used as the blocks corresponding to coal blocks.

[0012] Preferably, the calculation formula of the distance metric is:

[0013] ; where D is the distance metric; is the distance between the i-th pixel point and the clustering center a in the belt video frame; is the exponential function with the natural constant e as the base; is the motion speed of the i-th pixel point in the belt video frame; is the motion speed of the clustering center a; is the angle between the motion direction of the i-th pixel point in the belt video frame and the horizontal right direction; is the angle between the motion direction of the clustering center a and the horizontal right direction.

[0014] Preferably, the motion direction and motion speed of each pixel point in the belt video frame are obtained by the optical flow method.

[0015] Preferably, determining the correlation sequence of pixel points according to the change conditions of pixel points in different directions in the belt video frame includes:

[0016] Using gradient operators in different directions, obtaining the gradient values and gradient directions of pixel points in different directions in the belt video frame, taking the gradient direction corresponding to the maximum gradient value of the pixel point as the maximum gradient direction of the pixel point, and taking the gradient direction corresponding to the minimum gradient value of the pixel point as the minimum gradient direction of the pixel point;

[0017] Taking the pixel point as the starting point, obtaining the pixel values of a preset number of pixel points along the maximum gradient direction corresponding to the pixel point to form a first correlation sequence; taking the pixel point as the starting point, obtaining the pixel values of a preset number of pixel points along the minimum gradient direction corresponding to the pixel point to form a second correlation sequence; wherein, the correlation sequence includes the first correlation sequence and the second correlation sequence.

[0018] Preferably, combining the block to which the pixel point belongs, the information richness in the correlation sequence corresponding to the pixel point, and the three-dimensional point cloud data characteristics to determine the compression feature index corresponding to the pixel point in the belt video frame includes:

[0019] When the block to which the pixel point belongs is a block corresponding to a coal block, the block feature value corresponding to the pixel point is 2; when the block to which the pixel point belongs is a belt block not blocked by a coal block, the block feature value corresponding to the pixel point is 1;

[0020] Determining the information feature value of the pixel point according to the block feature value corresponding to the pixel point and the information richness in the correlation sequence of the pixel point;

[0021] Determining the three-dimensional feature value of the pixel point according to the three-dimensional point cloud data characteristics of the pixel point;

[0022] Combining the information feature value and the three-dimensional feature value of the pixel point to determine the compression feature index of the pixel point; wherein, both the information feature value and the three-dimensional feature value are positively correlated with the compression feature index of the pixel point.

[0023] Preferably, the calculation formula of the information feature value of the pixel point is:

[0024] ; where F is the information feature value of the pixel point; A is the block feature value corresponding to the pixel point; u is the number of pixel points with non-zero differences in pixel values between adjacent pixel points in the correlation sequence of the pixel point; I is the number of correlation sequences; J is the number of elements in the correlation sequence; is the jth pixel value in the ith correlation sequence of the pixel point; It is the (j + 1)-th pixel value in the i-th associated sequence of the pixel point.

[0025] Preferably, the calculation formula of the three-dimensional eigenvalue of the pixel point includes:

[0026] ; where P is the three-dimensional eigenvalue of the pixel point; It is the variance of the angles between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point. It is the three-dimensional point cloud data point corresponding to the pixel point The length of the data points in the data point sequence. It is the length of the data points in the data point sequence of the n-th neighboring three-dimensional point cloud data point of the three-dimensional point cloud data point corresponding to the pixel point; N is the number of neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point.

[0027] Preferably, the calculation formula of the illumination environment quality index is:

[0028] ; where Y is the illumination environment quality index; It is the average pixel value of the pixel points in other blocks except the initial belt block. It is the average pixel value of the belt blocks not blocked by coal blocks. It is the average pixel value of the block corresponding to the coal block. It is the average value of the information eigenvalues of the pixel points in the Q-th belt video frame. It is the overall contrast corresponding to the belt video frame at the neighboring b-th moment in the Q-th belt video frame. It is the overall contrast corresponding to the Q-th belt video frame. It is the information eigenvalue of the m-th pixel point in the overlapping block of the belt video frame at the b-th moment and the Q-th belt video frame on the belt video frame at the b-th moment. It is the information eigenvalue of the m-th pixel point in the overlapping block of the belt video frame at the b-th moment and the Q-th belt video frame on the Q-th belt video frame. It is the acquisition time corresponding to the b-th moment. It is the acquisition moment corresponding to the Q-th belt video frame; M is the number of pixel points in the overlapping block of the belt video frame at the b-th moment and the Q-th belt video frame; B is the number of belt video frames in the neighborhood of the Q-th belt video frame.

[0029] Preferably, combining the compression feature index and the illumination environment quality index to determine the compression priority of the pixel points in the belt video frame includes:

[0030] Taking the product of the compression feature index and the illumination environment quality index of the pixel point as the compression priority of the pixel point.

[0031] The embodiments of the present invention have at least the following beneficial effects:

[0032] The present invention relates to the field of video compression technology. The system first obtains the belt video frames and the corresponding three-dimensional point cloud data during the coal conveying process through the video data acquisition module, and regards the pixel points in the belt video frames and the corresponding three-dimensional point cloud data as a whole for compression during subsequent compression; analyzes the motion characteristics of the belt video frames, obtains different blocks, and distinguishes different blocks, so as to analyze the characteristics of coal blocks subsequently and obtain the compression priorities of each pixel point in the belt video frames; it is difficult to describe the shape of coal blocks in the coal conveying video only through two-dimensional images, so the present invention uses the three-dimensional point cloud data corresponding to the belt video frames. The shape of coal blocks can be described through the three-dimensional point cloud data, and the quality of the coal blocks conveyed by the belt can be more effectively reflected through the shape description. Therefore, the association sequence of pixel points is determined according to the change conditions of pixel points in different directions in the belt video frames, and combined with the block to which the pixel points belong, the information richness in the association sequence corresponding to the pixel points, and the three-dimensional point cloud data characteristics, the compression feature index corresponding to the pixel points in the belt video frames is determined; since there may be large changes in the lighting conditions in the belt coal conveying video, the lighting environment quality index is determined through the contrast between different blocks and the contrast change of the belt video frames at different times. The greater the contrast between blocks, the better the lighting environment in the image, and the more the characteristic information of the conveyed coal blocks can be highlighted, and the higher the compression priority of the corresponding pixel points should be; combining the compression feature index and the lighting environment quality index, the compression priority of the pixel points in the belt video frames is determined; the belt video frames and the corresponding three-dimensional point cloud data are compressed and transmitted according to the compression priority. Compressing based on the compression priorities of each pixel point in the belt video frames improves the compression efficiency of the relevant data with coal conveying characteristics in the belt video frames. At the same time, the corresponding coding length is shorter, the probability of loss is smaller, and the efficiency is higher when querying relevant data and during subsequent decompression, improving the compression efficiency of the relevant data with coal conveying characteristics in the belt video frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a system block diagram of a belt coal conveying video transmission system for intelligent coal mine monitoring provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a belt coal conveying video transmission system for intelligent coal mine monitoring according to the present invention, including its specific implementation manner, structure, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0037] The embodiment of the present invention provides a specific implementation method of a belt coal conveying video transmission system for intelligent coal mine monitoring. This system is applicable to the coal conveying scenario. In this scenario, the optical axis of the image acquisition device or video acquisition device is perpendicular to the belt, and it can capture the belt. To solve the technical problem that due to the large possible variation in the illumination conditions of the belt coal conveying scenario, when using the traditional Huffman coding, it is very easy to obtain a large number of data with similar frequencies and small values, resulting in poor compression effects. The present invention analyzes the belt video frames to obtain the compression priority of the pixel points in the image, and compresses them based on the compression priority of each pixel point in the belt video frames, improving the compression efficiency of the relevant data with coal conveying characteristics in the belt video frames. At the same time, the corresponding coding length is shorter, the probability of loss is smaller, and when querying relevant data and decompressing later, the efficiency is higher, thus improving the compression efficiency of the relevant data with coal conveying characteristics in the belt video frames.

[0038] The following specifically describes, in conjunction with the accompanying drawings, the specific solution of a belt coal conveying video transmission system for intelligent coal mine monitoring provided by the present invention.

[0039] Please refer to Figure 1 , which shows the system block diagram of a belt coal conveying video transmission system for intelligent coal mine monitoring provided by an embodiment of the present invention. This system includes the following modules:

[0040] The video data acquisition module 10 is used to acquire the belt video frames and the corresponding three-dimensional point cloud data during the belt coal conveying process, and transmit them to the video compression feature extraction module and the video compression transmission module.

[0041] In the present invention, a video image during the belt coal conveying process is collected by an industrial camera, denoted as the initial video image, and the collected initial video image is an RGB image. As another embodiment of the present invention, a belt coal conveying video during the belt coal conveying process can also be collected by a camera, and the belt coal conveying video is frame-divided to obtain the initial video image. In the present invention, the initial video image is grayscale processed by a weighted grayscaling method to obtain a grayscale image, denoted as the belt video frame. It should be noted that the weighted grayscaling method is a well-known technology and will not be elaborated here. Meanwhile, three-dimensional point cloud data of the belt coal conveying scene is obtained through devices such as a depth sensor or a lidar. And the data obtained by the video data acquisition module is transmitted to the video compression feature extraction module and the video compression transmission module.

[0042] The video compression feature extraction module 20 is used to analyze the motion features of the belt video frame to obtain different blocks; determine the correlation sequence of pixel points according to the change conditions of pixel points in different directions in the belt video frame, and combine the block to which the pixel point belongs, the information richness in the correlation sequence corresponding to the pixel point, and the three-dimensional point cloud data features to determine the compression feature index corresponding to the pixel point in the belt video frame, and transmit the processed data to the video compression priority division module.

[0043] When traditional Huffman coding encodes a video image, it often depends on the frequency. The larger the frequency, the shorter the code is used for representation, and the smaller the frequency, the longer the code is used for representation, so as to achieve efficient compression and decompression of data. However, in the belt coal conveying video, the lighting conditions of the belt coal conveying scene may change greatly, and the frequency of the data is often small and equal. Therefore, when using traditional Huffman coding, it is very easy to obtain a large number of data with equal and small frequencies, and the compression effect is poor during compression.

[0044] Therefore, in the present invention, the video compression priority and the relevant data compression priority are adaptively constructed according to the matching condition of the video image lighting environment and the three-dimensional point cloud, and then Huffman coding is performed using the compression priority to effectively extract the relevant data of coal conveying in the video for priority compression, improving the efficiency and accuracy of data compression and decompression.

[0045] According to the above, the belt video frame during the belt coal conveying process and the three-dimensional point cloud data in the corresponding coal conveying scene are obtained by the video data acquisition module 10.

[0046] According to prior knowledge, since the lighting conditions in the belt coal conveying scenario may vary greatly. In order to further analyze the quality of coal blocks, in the present invention, the three-dimensional point cloud data corresponding to the pixel points in the belt video frames is collected. The shape of the coal blocks can be described through the three-dimensional point cloud data, and through the shape description, the quality of the coal blocks conveyed by the belt can be effectively reflected, which is difficult to describe in a simple planar image. In the present invention, the obtained belt video frames are associated with the corresponding three-dimensional point cloud data through the time stamps or frame numbers corresponding to the belt video frames. When compressing the video, the belt video frames and the corresponding three-dimensional point cloud data are packed and compressed together for subsequent retrieval and analysis.

[0047] The coal conveying videos are often continuous. For any belt video frame Q among them, first, the lighting quality of the belt video frame Q is judged. In the present invention, by analyzing the block contrast within the belt video frame and the registration situation of the three-dimensional point cloud data between consecutive belt video frames, the lighting environment quality index within the belt video frame is adaptively constructed. The logic is as follows: The greater the block contrast in the belt video frame, the better the lighting environment in the image, and the characteristic information of the conveyed coal mine can be more prominent, so the compression priority of this belt video frame should be higher. And the better the matching effect of the three-dimensional point cloud data corresponding to the belt video frame, the more the feature points in the three-dimensional point cloud data can reflect the shape information of the coal blocks, and the corresponding compression priority should be higher.

[0048] For the belt video frames, since they are belt coal conveying videos, the collected images should be moving video images, and the belt and the coal blocks are moving objects. Therefore, first, the motion direction and motion speed of each pixel point in the belt video frame are obtained through the optical flow method.

[0049] According to prior knowledge, there are usually three types of objects in the belt coal conveying video. One is the belt, one is the coal blocks on the belt, and the other is the background. Among them, the motion directions and speeds between the belt and the coal blocks are the same, while the motion data of the background blocks are quite different from those of the belt and the coal blocks. Among them, the block corresponding to the coal block should be located inside the belt block.

[0050] First, according to the motion characteristics corresponding to each pixel point in the belt video frame, the belt video frame is segmented to obtain different blocks. Specifically:

[0051] Construct a distance metric between pixel points based on the motion characteristics corresponding to each pixel point in the belt video frame; combine the distance metric and use the K-means clustering algorithm to divide the pixel points in the belt video frame into two categories; calculate the average values of the motion speeds of the pixel points in the two categories respectively, and take the category corresponding to the maximum average value as the candidate category; form an initial belt block from the pixel points within the candidate category; where the initial belt block includes a coal block corresponding block formed by the coal blocks conveyed on the belt and the belt block not blocked by the coal blocks outside the coal block corresponding block. In the embodiment of the present invention, the number of clusters is set to 2, that is, the K value is 2.

[0052] Among them, when using the K-means clustering algorithm to divide the pixel points in the belt video frame into two categories, first randomly select the clustering center points. In order to better segment the complex background block and the initial belt block, the present invention constructs the motion consistency of the pixel points according to the motion characteristics in the belt video frame and obtains the distance metric between the pixel points. It should be noted that dividing the image into categories according to the K-means clustering algorithm is a well-known technique to those skilled in the art and will not be elaborated here.

[0053] Among them, the calculation formula of the distance metric is:

[0054] ; where D is the distance metric; is the distance between the i-th pixel point in the belt video frame and the clustering center a; is the exponential function with the natural constant e as the base; is the motion speed of the i-th pixel point in the belt video frame; is the motion speed of the clustering center a; is the angle between the motion direction of the i-th pixel point in the belt video frame and the horizontal right direction; is the angle between the motion direction of the clustering center a and the horizontal right direction.

[0055] The smaller the difference in the motion direction and the motion angle, the greater the motion consistency, and the smaller the distance metric value between the two pixel points. The greater the motion consistency, the more likely it is that the two pixel points belong to the same category. The smaller the motion consistency, the less likely it is that the two pixel points belong to the same category.

[0056] After obtaining the initial belt blocks, the initial belt blocks are further segmented. Taking any edge point of the initial belt block as the initial center point, region growing is performed, and the obtained growing region is used as the belt block not blocked by coal blocks. Among the initial belt blocks, other blocks except the belt blocks not blocked by coal blocks are used as the corresponding blocks of coal blocks. It should be noted that when transporting coal by belt, coal blocks are often located at the central position of the belt blocks. Therefore, the edge pixel points of the initial belt block are selected as the initial seed points, and then the obtained growing region is used as the belt block not blocked by coal blocks. As another embodiment of the present invention, it is also possible to compare the average pixel values of the pixel points in the growing region and the non-growing region on the initial belt block, and use the region with the minimum average pixel value in the two regions as the coal block, and use the other region as the belt block not blocked by coal blocks.

[0057] When performing region growing on the initial center point, in the embodiment of the present invention, the growth threshold is set to 10, and in other embodiments, the implementer can adjust this value according to the actual situation. It should be noted that the region growing algorithm is a well-known technology to those skilled in the art and will not be elaborated here.

[0058] Since the belt video frames often collect two-dimensional information of the object, while the three-dimensional point cloud data reflects the three-dimensional information of the object, the pixel points of the corresponding blocks of coal blocks in the belt video frames can be put into one-to-one correspondence with the data points in the three-dimensional point cloud data. For example, for a cube in a space, when collecting the image data of one of its faces, each pixel point of the image data of this face has a corresponding data point at a certain position in the three-dimensional point cloud data. Therefore, the pixel points in the corresponding blocks of coal blocks in the video images obtained in the present invention can all obtain the corresponding three-dimensional data points in the three-dimensional point cloud.

[0059] For example, if there are N pixel points in the corresponding blocks of coal blocks, and they are put into one-to-one correspondence with the three-dimensional point cloud data. Since the object is three-dimensional, each pixel point has its corresponding data point sequence. For example, for the three-dimensional point cloud data point q1 of any pixel point c1 in the corresponding blocks of coal blocks, when the values on the x-axis and y-axis in the three-dimensional space remain unchanged, the value of the z-axis is changed; when the value of the z-axis is changed, multiple three-dimensional point cloud data points of this three-dimensional point cloud data point q1 in the three-dimensional space are obtained. The multiple three-dimensional point cloud data points corresponding to the three-dimensional point cloud data point q1 of this pixel point c1 form the data point sequence of this pixel point c1, and the data point sequence contains three-dimensional point cloud data points. Through this step, the effective three-dimensional point cloud data points related to coal transportation can be effectively extracted from a large amount of three-dimensional point cloud data for compression, reducing the compression of invalid data, thereby reducing the waste of storage space and improving the compression efficiency.

[0060] First, determine the correlation sequence of pixel points according to the change of pixel points in different directions in the belt video frame. Specifically: use gradient operators in different directions to obtain the gradient values and gradient directions of pixel points in different directions in the belt video frame. Take the gradient direction corresponding to the maximum gradient value of the pixel point as the maximum gradient direction of the pixel point, and take the gradient direction corresponding to the minimum gradient value of the pixel point as the minimum gradient direction of the pixel point.

[0061] Starting from the pixel point, obtain the pixel values of a preset number of pixel points along the maximum gradient direction corresponding to the pixel point to form the first correlation sequence; starting from the pixel point, obtain the pixel values of a preset number of pixel points along the minimum gradient direction corresponding to the pixel point to form the second correlation sequence; where the correlation sequence includes the first correlation sequence and the second correlation sequence. In the embodiment of the present invention, the value of the preset number is 10, and in other embodiments, the implementer adjusts this value according to the actual situation.

[0062] Further, combine the block to which the pixel point belongs, the information richness in the correlation sequence corresponding to the pixel point, and the three-dimensional point cloud data characteristics to determine the compression feature index corresponding to the pixel point in the belt video frame. Specifically:

[0063] When the block to which the pixel point belongs is the block corresponding to the coal block, the block feature value corresponding to the pixel point is 2; when the block to which the pixel point belongs is the belt block not blocked by the coal block, the block feature value corresponding to the pixel point is 1.

[0064] Construct the information richness of the pixel point through the change of pixel points in the two pixel point correlation sequences of the pixel point, and construct the compression feature index corresponding to the pixel point through the information richness of the pixel point on the image and the three-dimensional point cloud data point characteristics corresponding thereto. Specifically:

[0065] Determine the information feature value of the pixel point according to the block feature value corresponding to the pixel point and the information richness in the correlation sequence of the pixel point.

[0066] The calculation formula of this information feature value is:

[0067] ; where F is the information feature value of the pixel point; A is the block feature value corresponding to the pixel point; u is the number of pixel points with non-zero differences in pixel values between adjacent pixel points in the correlation sequence of the pixel point; I is the number of correlation sequences; J is the number of elements in the correlation sequence; is the j-th pixel value in the i-th correlation sequence of the pixel point; is the j + 1-th pixel value in the i-th correlation sequence of the pixel point.

[0068] Among them, the number u of pixel points with non-zero differences in pixel values between adjacent pixel points in the correlation sequence can be obtained by statistical methods; The greater the difference between them, the greater the gray-scale difference between adjacent pixel points in the associated sequence, and the greater the information richness of the associated sequence corresponding to the corresponding pixel point.

[0069] Determine the three-dimensional eigenvalue of a pixel point according to the three-dimensional point cloud data characteristics of the pixel point.

[0070] The calculation formula of the three-dimensional eigenvalue is:

[0071] ; where P is the three-dimensional eigenvalue of the pixel point; is the variance of the angles between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point; is the three-dimensional point cloud data point corresponding to the pixel point the length of the data points in the data point sequence; is the length of the data points in the data point sequence of the nth neighboring three-dimensional point cloud data point of the three-dimensional point cloud data point corresponding to the pixel point; N is the number of neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point.

[0072] In the embodiments of the present invention, for each three-dimensional point cloud data point corresponding to a pixel point, 10 pixel points closest to it are obtained as neighboring three-dimensional point cloud data points. Therefore, in the embodiments of the present invention, the value of N, the number of neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point, is 10. In other embodiments, the implementer can adjust this value according to the actual situation.

[0073] Among them, the angle between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point refers to: connecting the three-dimensional point cloud data point a1 corresponding to the pixel point and the neighboring three-dimensional point cloud data point a2 to obtain a connection line b1, connecting the three-dimensional point cloud data point a1 corresponding to the pixel point and another neighboring three-dimensional point cloud data point a3 to obtain a connection line b2, and the angle formed by the connection line b1 and the connection line b2 is one of the angles between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point. The variance of the angles between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point The larger the value, the more scattered the distribution of the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point, and the greater the possibility that the three-dimensional point cloud data point a1 corresponding to the pixel point contains the shape information of the coal block. Therefore, its corresponding three-dimensional eigenvalue is larger. The length of the data points in the data point sequence of the three-dimensional point cloud data point corresponding to the pixel point is the distance length of the data points in the data point sequence corresponding to the three-dimensional point cloud data point of the pixel point in three-dimensional space. The larger the value of the more likely the three-dimensional point cloud data point corresponding to the pixel point is a convex point or a concave point, and the greater the probability that it is a feature point.

[0074] After obtaining the information eigenvalue and three-dimensional eigenvalue of a pixel, the compression feature index of the pixel is determined by combining the information eigenvalue and three-dimensional eigenvalue of the pixel; wherein, both the information eigenvalue and the three-dimensional eigenvalue are positively correlated with the compression feature index of the pixel. In the embodiments of the present invention, the product of the information eigenvalue and the three-dimensional eigenvalue of the pixel is used as the compression feature index of the pixel. In other embodiments, as long as both the information eigenvalue and the three-dimensional eigenvalue are positively correlated with the compression feature index of the pixel. The data processed by the video compression feature extraction module is transmitted to the video compression priority division module.

[0075] The video compression priority division module 30 is configured to determine the illumination environment quality index by the contrast between different blocks and the contrast change of the belt video frames at different times; combine the compression feature index and the illumination environment quality index to determine the compression priority of the pixels in the belt video frame, and transmit the compression priority to the video compression transmission module.

[0076] The calculation formula of the illumination environment quality index is:

[0077] ; where Y is the illumination environment quality index; is the average pixel value of the pixels in other blocks except the initial belt block; is the average pixel value of the belt block not blocked by the coal block; is the average pixel value of the block corresponding to the coal block; is the average value of the information eigenvalues of the pixels in the Qth belt video frame; is the overall contrast corresponding to the belt video frame at the bth adjacent moment in the Qth belt video frame; is the overall contrast corresponding to the Qth belt video frame; is the information eigenvalue of the mth pixel in the overlapping block of the belt video frame at the bth moment and the Qth belt video frame on the belt video frame at the bth moment; is the information eigenvalue of the mth pixel in the overlapping block of the belt video frame at the bth moment and the Qth belt video frame on the Qth belt video frame; is the acquisition time corresponding to the bth moment; is the acquisition moment corresponding to the Qth belt video frame; M is the number of pixels in the overlapping block of the belt video frame at the bth moment and the Qth belt video frame; B is the number of belt video frames in the neighborhood of the Qth belt video frame.

[0078] where reflects the light fluctuation of adjacent belt video frames; since the image acquisition device is fixed, during the continuous transportation of coal blocks by the belt, there must be overlapping corresponding blocks of coal blocks in the images at adjacent times, that is, the number of pixel points within the overlapping block of two belt video frames can be obtained; among them, the overlapping block can be obtained by the difference method, and the block with the smallest difference is the overlapping block. The information feature value of the m-th pixel point on the belt video frame at the b-th moment within the overlapping block of the belt video frame at the b-th moment and the Q-th belt video frame , the greater the difference between the information feature value of the m-th pixel point on the Q-th belt video frame within the overlapping block of the belt video frame at the b-th moment and the information feature value of the m-th pixel point on the belt video frame at the b-th moment, the greater the contrast change it reflects, and the stronger the light fluctuation. and are the acquisition times corresponding to the belt video frames. The smaller the difference in the acquisition times of the two belt video frames, the greater their weights.

[0079] Combining the contrast between different blocks and the contrast change of belt video frames at different times, an illumination environment quality index is obtained. The larger the illumination environment quality index, the better the quality of the belt video frame during acquisition, the more accurate and rich the information contained in the belt video frame, and the higher the corresponding compression priority.

[0080] The compression feature index and the illumination environment quality index are obtained, and further constitute the compression priority of the data point sequence of pixel points and the corresponding three-dimensional point cloud data points.

[0081] That is, the product of the compression feature index of the pixel point and the illumination environment quality index is used as the compression priority of the pixel point. Among them, the larger the illumination environment quality index, the richer and clearer the information in the belt video frame is reflected, and the larger the corresponding illumination environment quality index, the greater the compression priority; the larger the compression feature index, the greater the possibility of reflecting the shape information of the coal block it contains, and the greater the corresponding compression priority.

[0082] So far, the construction of the data compression priority in the Huffman coding process has been realized. And the compression priority obtained by the video compression priority division module is transmitted to the video compression and transmission module.

[0083] The video compression and transmission module 40 is used to compress the belt video frame and the corresponding three-dimensional point cloud data using Huffman coding according to the compression priority to obtain compressed data; and transmit the compressed data.

[0084] The compression priority of each pixel point is obtained. The compression priority corresponding to this pixel point is also the compression priority of the data point sequence of the 3D point cloud data point corresponding to this pixel point. During compression, it is regarded as a whole for compression. According to the compression priority, Huffman coding is used to compress the belt video frame and the corresponding 3D point cloud data to obtain compressed data. Specifically: in the order from the largest to the smallest compression priority, Huffman coding is used to compress the pixel points in the belt video frame and the corresponding relevant 3D point cloud data to generate a Huffman coding table. Through the Huffman coding table, the data can be decompressed to obtain the compressed data. It should be noted that the process of decompressing the data through the Huffman coding table is well-known to those skilled in the art and will not be elaborated here. Finally, the compressed data is transmitted.

[0085] Compressing the pixel points and the corresponding relevant 3D point cloud data in the order of the compression priority improves the compression efficiency of the relevant data with coal conveying characteristics in the belt video frame. At the same time, the corresponding coding length is shorter, the probability of loss is smaller, and the efficiency is higher when querying relevant data and decompressing, which improves the compression efficiency of the belt coal conveying video.

[0086] In summary, the present invention relates to the technical field of video compression. The system first obtains the belt video frame and the corresponding 3D point cloud data during the belt coal conveying process; analyzes the motion characteristics of the belt video frame to obtain different blocks; determines the association sequence of pixel points according to the change situation of pixel points in different directions in the belt video frame, and combines the block to which the pixel point belongs, the information richness in the association sequence corresponding to the pixel point, and the 3D point cloud data characteristics to determine the compression feature index corresponding to the pixel point in the belt video frame; determines the illumination environment quality index through the contrast between different blocks and the contrast change of the belt video frames at different times; combines the compression feature index and the illumination environment quality index to determine the compression priority of the pixel points in the belt video frame; compresses the belt video frame and the corresponding 3D point cloud data according to the compression priority. The present invention compresses the relevant data in the belt video frame in the order of the compression priority, which improves the compression efficiency of the relevant data with coal conveying characteristics in the belt coal conveying video.

[0087] It should be noted that: the above sequence of embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A video transmission system for coal conveying belts for intelligent coal mine monitoring, characterized in that: The system includes the following modules: The video data acquisition module is used to acquire the belt video frames and the corresponding three-dimensional point cloud data during the belt coal transportation process, and transmit them to the video compression feature extraction module and the video compression transmission module; The video compression feature extraction module is used to analyze the motion features of the belt video frame and obtain different blocks; determine the association sequence of the pixels in the belt video frame according to the changes of the pixels in different directions, combine the blocks to which the pixels belong, the information richness in the association sequence corresponding to the pixels and the three-dimensional point cloud data features, determine the compression feature index corresponding to the pixels in the belt video frame, and transmit the processed data to the video compression priority division module; the information richness reflects the grayscale difference between adjacent pixels in the association sequence; The video compression priority division module is used to determine the light environment quality index by comparing the contrast between different blocks and the contrast change of the belt video frame at different times; determine the compression priority of the pixel points in the belt video frame by combining the compression feature index and the light environment quality index, and transmit the compression priority to the video compression transmission module; A video compression transmission module is used to compress the belt video frame and the corresponding three-dimensional point cloud data using Huffman coding according to the compression priority to obtain compressed data; Transmitting the compressed data; The step of determining the associated sequence of pixels according to the change of pixels in different directions in the belt video frame includes: Using gradient operators in different directions, the gradient values ​​and gradient directions of the pixels in the belt video frame in different directions are obtained, and the gradient direction corresponding to the maximum gradient value of the pixel is taken as the maximum gradient direction of the pixel, and the gradient direction corresponding to the minimum gradient value of the pixel is taken as the minimum gradient direction of the pixel; Taking the pixel point as the starting point, the pixel values ​​of a preset number of pixel points are obtained along the direction of the maximum gradient corresponding to the pixel point to form a first associated sequence; taking the pixel point as the starting point, the pixel values ​​of a preset number of pixel points are obtained along the direction of the minimum gradient corresponding to the pixel point to form a second associated sequence; wherein the associated sequence includes the first associated sequence and the second associated sequence.

2. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 1 is characterized in that: The analysis of the motion characteristics of the belt video frame to obtain different blocks includes: The distance metric between pixels is constructed according to the motion features corresponding to each pixel in the belt video frame; the pixels in the belt video frame are divided into two categories using the K-means clustering algorithm in combination with the distance metric; the average values ​​of the motion speeds of the pixels in the two categories are calculated respectively, and the category corresponding to the maximum average value is used as the candidate category; the initial belt block is formed by the pixels in the candidate category; Taking any edge point of the initial belt block as the initial center point, regional growth is performed, and the resulting growth area is used as the belt block not blocked by the coal block. Except for the belt block not blocked by the coal block in the initial belt block, the other blocks are used as the coal block corresponding blocks.

3. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 2 is characterized in that: The calculation formula of the distance metric is: ; Where D is the distance metric; is the distance between the i-th pixel in the belt video frame and the cluster center a; is an exponential function with the natural constant e as base; is the movement speed of the i-th pixel in the belt video frame; is the movement speed of cluster center a; is the angle between the motion direction of the i-th pixel in the belt video frame and the horizontal right direction; It is the angle between the moving direction of cluster center a and the horizontal right direction.

4. The video transmission system for belt conveyor for monitoring smart coal mines according to claim 3 is characterized in that: The optical flow method is used to obtain the movement direction and speed of each pixel in the belt video frame.

5. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 2 is characterized in that: The method combines the block to which the pixel belongs, the information richness in the associated sequence corresponding to the pixel, and the three-dimensional point cloud data features to determine the compression feature index corresponding to the pixel in the belt video frame, including: When the block to which the pixel belongs is the block corresponding to the coal block, the block feature value corresponding to the pixel is 2; when the block to which the pixel belongs is the belt block that is not blocked by the coal block, the block feature value corresponding to the pixel is 1; Determine the information characteristic value of the pixel point according to the block characteristic value corresponding to the pixel point and the information richness in the associated sequence of the pixel point; Determine the three-dimensional feature value of the pixel point according to the three-dimensional point cloud data feature of the pixel point; The compression feature index of the pixel point is determined by combining the information feature value and the three-dimensional feature value of the pixel point; wherein the information feature value and the three-dimensional feature value are both positively correlated with the compression feature index of the pixel point.

6. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 5 is characterized in that: The calculation formula of the information characteristic value of the pixel point is: ; Wherein, F is the information feature value of the pixel; A is the block feature value corresponding to the pixel; u is the number of pixels whose adjacent pixel values ​​have a difference not equal to 0 in the associated sequence of pixels; I is the number of associated sequences; J is the number of elements in the associated sequence; is the jth pixel value in the i-th associated sequence of the pixel point; is the j+1th pixel value in the i-th associated sequence of the pixel point.

7. The video transmission system for belt conveyor for monitoring smart coal mines according to claim 5 is characterized in that: The calculation formula of the three-dimensional characteristic value of the pixel point includes: ; Where P is the three-dimensional feature value of the pixel; is the variance of the angle between the neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point corresponding to the pixel point; The 3D point cloud data point corresponding to the pixel point The length of the data points in the data point sequence; is the length of the data points in the data point sequence of the nth neighboring three-dimensional point cloud data point of the three-dimensional point cloud data point corresponding to the pixel point; N is the number of neighboring three-dimensional point cloud data points of the three-dimensional point cloud data point.

8. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 2 is characterized in that: The calculation formula of the lighting environment quality index is: ; Wherein, Y is the light environment quality index; is the average pixel value of the pixels in other blocks except the initial belt block; is the average pixel value of the belt blocks not blocked by coal blocks; is the average pixel value of the corresponding block of coal; is the average value of the information feature values ​​of each pixel in the Q-th belt video frame; is the overall contrast of the belt video frame at the bth moment adjacent to the Qth belt video frame; is the overall contrast corresponding to the Qth belt video frame; is the information feature value of the mth pixel point in the overlapping block of the belt video frame at the bth moment and the Qth belt video frame on the belt video frame at the bth moment; is the information feature value of the mth pixel point in the overlapping block of the belt video frame at the bth moment and the Qth belt video frame on the Qth belt video frame; is the collection time corresponding to the b-th moment; is the acquisition time corresponding to the Qth belt video frame; M is the number of pixels in the overlapping block between the belt video frame at the bth moment and the Qth belt video frame; B is the number of belt video frames in the neighborhood of the Qth belt video frame.

9. The intelligent coal mine monitoring belt coal conveyor video transmission system according to claim 1 is characterized in that: The step of combining the compression feature index and the lighting environment quality index to determine the compression priority of the pixel points in the belt video frame includes: The product of the compression feature index of the pixel and the lighting environment quality index is used as the compression priority of the pixel.

Images