A multi-source information fusion supervision method for a coal mine underground drilling process

Abstract

The present application relates to the field of coal mine safety and mine system intelligentization, in particular to a kind of multi-source information fusion supervision method for coal mine underground drilling process, comprising: S1: obtaining the continuous synthesis monitoring picture group of coal mine underground drilling process and forming supervision video stream;S2: each frame synthesis monitoring picture in supervision video stream is regionally segmented, and the corresponding static UI area, dynamic data area and real-time video area are obtained;S3: the static UI area, dynamic data area and real-time video area of each frame synthesis monitoring picture in supervision video stream are compressed using corresponding compression strategy respectively, and the corresponding compressed data are obtained;S4: the compressed data of static UI area, dynamic data area and real-time video area are fused, and regional compression supervision video stream is generated;S5: regional compression supervision video stream is stored, to realize subsequent inspection and supervision.The present application can improve the storage efficiency of supervision video, and the efficiency and accuracy of coal mine underground drilling process supervision.

Description

Technical Field

[0001] This invention relates to the field of coal mine safety and intelligent mining system technology, specifically to a multi-source information fusion and monitoring method for underground drilling processes in coal mines. Background Technology

[0002] Underground drilling in coal mines is a crucial step in ensuring safe operations, involving the integrated monitoring of multiple information sources, including equipment operation status monitoring, geological parameter acquisition, and environmental risk early warning. With the development of information and intelligent technologies, modern coal mine safety monitoring systems generally employ a combination of multi-sensor collaboration, video surveillance, and data visualization to dynamically track drilling operations in real time. In this context, composite monitoring images, as a centralized presentation of multi-source information, play a vital role in information integration, status feedback, and decision support. However, due to the complex underground environment and data transmission limitations, existing technologies still face numerous challenges in the compression and storage of composite monitoring images, necessitating targeted improvements.

[0003] like Figure 1 As shown, the composite monitoring screen mainly consists of three functional areas: a static UI area, a dynamic data area, and a real-time video area. The static UI area primarily contains unchanging or infrequently updated elements such as the operating software's menu interface, fixed text labels, and equipment identification boxes, providing operational guidance and a basic interface for supervisors. The dynamic data area displays real-time changing numerical information, such as drill bit rotation speed, pressure parameters, and gas concentration, serving as a core data source reflecting equipment operating status and environmental safety. The real-time video area captures dynamic footage of the downhole operation scene through cameras, providing visual evidence for intuitively judging equipment malfunctions, personnel behavior, and environmental anomalies. These three areas together constitute a complete monitoring screen, and their synergistic effect directly impacts supervisory efficiency and the accuracy of safety decisions.

[0004] However, existing technologies suffer from problems such as a single compression strategy and low storage efficiency when storing and compressing such composite monitoring footage. Current mainstream solutions typically use a uniform compression algorithm (such as H.264 or H.265) to compress the entire image as a whole, without differentiating between the characteristics of different areas. First, the static UI area of ​​the composite monitoring footage contains a large number of text and graphical interface elements. If high compression ratios are used, it can easily lead to blurred text and distorted icons, affecting the recognition and execution of operation commands. Second, the numerical information in the dynamic data area needs to maintain high readability, but existing compression methods struggle to balance data clarity with storage space usage. Frequent numerical updates can also cause data lag due to compression delays. Finally, the real-time video area, under complex lighting and motion scenarios underground, needs to balance image quality and storage efficiency. However, uniform compression strategies often cannot dynamically adapt to the activity level of the image, resulting in wasted storage resources in static scenes and loss of critical details in dynamic scenes.

[0005] Therefore, there is an urgent need for a technical solution that can achieve differentiated compression and dynamic optimization based on the characteristics of each functional area of ​​the synthetic monitoring image, so as to improve the efficiency and accuracy of monitoring the underground drilling process in coal mines. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, the technical problem this invention aims to solve is: how to provide a multi-source information fusion monitoring method for underground coal mine drilling processes, which involves regional segmentation of the composite monitoring images and compression using differentiated compression strategies. This method satisfies the long-term storage requirements of monitoring data while avoiding unnecessary resource consumption, thereby improving the storage efficiency of monitoring videos and the efficiency and accuracy of monitoring underground coal mine drilling processes (in the data storage and compression stages).

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A method for multi-source information fusion and supervision of underground coal mine drilling processes includes:

[0009] S1: Acquire continuous composite monitoring footage during underground drilling in coal mines to form a regulatory video stream;

[0010] S2: Perform region segmentation on each frame of the composite monitoring image in the monitoring video stream to obtain the corresponding static UI area, dynamic data area and real-time video area;

[0011] S3: Compress the static UI area, dynamic data area and real-time video area of ​​each frame of the composite monitoring image in the monitoring video stream using the corresponding compression strategies to obtain the corresponding compressed data.

[0012] S4: Merge the compressed data of the static UI area, dynamic data area and real-time video area of ​​each frame in the surveillance video stream to generate a regionally compressed surveillance video stream;

[0013] S5: Store the compressed surveillance video streams by region for subsequent inspection and supervision.

[0014] Preferably, in step S2, the process of segmenting the synthesized monitoring image into regions includes:

[0015] S201: Obtain a composite monitoring image in pixel matrix format;

[0016] S202: The size used for composite monitoring images is... The sliding window calculates the matching score between each window position and each static UI template; windows with matching scores greater than the static UI threshold are marked as static UI areas;

[0017] The formula for calculating the matching score is expressed as follows:

[0018] ;

[0019] In the formula: This indicates the matching score for the current window. Represents the coordinates of the top-left corner of the window; Indicates the first A static UI template Middle group coordinates Pixel values; Represents the coordinates in the current window pixel values, , ;

[0020] S203: Divide the area outside the static UI area into several candidate areas; extract the HSV spatial histogram of each candidate area. And calculate its HSV histogram with the preset template. KL divergence ;

[0021] The formula is expressed as:

[0022] ;

[0023] S204: Calculate the gradient magnitude of each candidate region using the Sobel operator. ;

[0024] The formula is expressed as:

[0025] ;

[0026] In the formula: This represents the horizontal gradient of the candidate region; This represents the vertical gradient of the candidate region;

[0027] S205: Extract texture histograms of each candidate region using local binary mode. ;

[0028] S206: Calculate the corresponding comprehensive score based on the KL divergence, gradient magnitude and texture histogram of each candidate region, and mark the candidate region with the comprehensive score greater than the threshold of the dynamic data region as the dynamic data region.

[0029] Overall Score The formula is expressed as:

[0030] ;

[0031] In the formula: , , Indicates the weighting coefficient;

[0032] S207: Calculate the inter-frame motion vector field of each pixel in the synthesized surveillance image using the Horn-Schunck optical flow method; calculate the motion intensity of each pixel. and exercise intensity Pixels with a motion intensity higher than the average of all pixels are marked as high-motion pixels;

[0033] ;

[0034] In the formula: Indicates horizontal displacement; Indicates vertical displacement;

[0035] S208: Cluster high-motion pixels using connected components and treat each connected region as a high-motion region;

[0036] S209: Extract the geometric features of each high-motion region and match them with the preset camera resolution template. If the match is successful, mark the corresponding high-motion region as the real-time video area.

[0037] Preferably, in step S3, the compression strategy for the static UI area of ​​the composite monitoring image in each frame of the monitoring video stream includes:

[0038] S301: Calculate the differences in the static UI area between frames of the composite monitoring image in the surveillance video stream;

[0039] The formula is expressed as:

[0040] ;

[0041] In the formula: Indicates the first Frame and the Differences in the static UI area between frames of the frame-synthesized monitoring image; This represents the total number of pixels in the static UI area; Indicates the first Coordinates in the static UI area of ​​the frame-composite monitoring screen Pixel values;

[0042] S302: Calculate the corresponding key frame judgment threshold for each frame of the composite monitoring image;

[0043] The formula is expressed as:

[0044] ;

[0045] In the formula: Indicates the first Key frame determination threshold for frame synthesis monitoring footage; This indicates the initial difference value set. Indicates the previous number Average difference in static UI area between frames of the frame-synthesized monitoring image;

[0046] S303: Differences in static UI areas between frames Greater than its keyframe judgment threshold The composite surveillance footage is used as a keyframe;

[0047] S304: Compress and encode the difference pixels in key frames and non-key frames to obtain the static UI area compressed data of the regulatory video stream.

[0048] Preferably, in step S3, for the dynamic data area of ​​the composite monitoring image in each frame of the monitoring video stream: the encoding strategy is to use the JPEG-LS encoding algorithm to encode the data in the dynamic data area to obtain the compressed data of the dynamic data area of ​​the monitoring video stream.

[0049] Preferably, in step S3, for the real-time video area of ​​the composite monitoring image in each frame of the monitoring video stream: the encoding strategy is to use the H.265 / HEVC encoding algorithm to encode the data of the real-time video area to obtain the compressed data of the real-time video area of ​​the monitoring video stream.

[0050] Preferably, in step S4, after aligning the timestamps and hashing the compressed data of the static UI area, the compressed data of the dynamic data area, and the compressed data of the real-time video area of ​​the regulatory video stream, the compressed data of the static UI area, the compressed data of the dynamic data area, and the compressed data of the real-time video area are merged to generate a regionally compressed regulatory video stream.

[0051] Preferably, in step S4, the compression parameters of the static UI area, dynamic data area and real-time video area in the regional compressed monitoring video stream are optimized by using device sensor data, and new compressed data is generated based on the optimized compression parameters to obtain the dynamically adjusted compressed monitoring video stream.

[0052] Preferably, in step S4, the processing step of dynamically adjusting the compression parameters of the static UI area, dynamic data area, and real-time video area in the regionally compressed monitoring video stream using device sensor data includes:

[0053] S401: Obtain compression parameters for static UI area compression data, dynamic data area compression data, and real-time video area compression data in the regionally compressed surveillance video stream. , and ;

[0054] S402: After standardizing the acquired equipment sensor data, perform state vector fusion with user interaction event encoding to obtain the process state vector. ;

[0055] The formula is expressed as:

[0056] ;

[0057] In the formula: This represents the standardized data from the device's sensor data; Indicates the encoding of user interaction events; , Indicates the weighting coefficient;

[0058] S403: Based on process state vector Calculate the priority weights of the static UI area, dynamic data area, and real-time video area;

[0059] The formula is expressed as:

[0060] ;

[0061] ;

[0062] ;

[0063] In the formula: , , These represent the priority weights of the static UI area, dynamic data area, and real-time video area, respectively. Indicates the attenuation coefficient; This indicates the set vibration threshold.

[0064] S404: Compression parameters based on the priority weights of the static UI area, dynamic data area, and real-time video area. , and Optimize to obtain optimized compression parameters. , and ;

[0065] The formula is expressed as:

[0066] ;

[0067] ;

[0068] ;

[0069] In the formula: Indicates the sensitivity coefficient; Indicates the adjustment range.

[0070] Preferably, in step S404, the compression parameters are optimized based on the static UI area, dynamic data area, and real-time video area. , and The data in the static UI area, dynamic data area, and real-time video area of ​​the surveillance video stream are recompressed to obtain dynamically adjusted compressed data of the static UI area, dynamic data area, and real-time video area, which are then merged to generate a dynamically adjusted compressed surveillance video stream.

[0071] Preferably, regions with priority weights greater than the weight threshold are designated as critical regions, and compression parameter adjustments and recompression are performed only on critical regions.

[0072] Compared with existing technologies, the multi-source information fusion monitoring method for underground coal mine drilling processes in this invention has the following advantages:

[0073] This invention addresses the problems of information redundancy, low storage efficiency, and easy loss of critical information in the processing of monitoring videos during underground coal mine drilling. It proposes a multi-source information fusion monitoring method for underground coal mine drilling processes. By segmenting the composite monitoring image from each frame of the monitoring video stream into regions, targeted compression strategies are applied to the static UI area, dynamic data area, and real-time video area, achieving efficient processing and storage of monitoring video data. First, region segmentation technology divides the composite monitoring image into three functional areas: static UI area, dynamic data area, and real-time video area, avoiding the excessive compression of non-critical information in traditional uniform compression methods. The static UI area uses a lossless compression strategy to ensure the integrity and accuracy of the monitoring interface information, facilitating accurate identification of equipment status and operation interface; the dynamic data area uses a medium compression rate to balance data readability and storage efficiency; and the real-time video area uses a high compression rate to effectively reduce the amount of video data while ensuring the identifiability of key scene content. Second, in the complex environment of underground coal mines, the amount of video data is enormous, and traditional uniform compression methods often lead to wasted storage space or loss of critical information. To address this, the present invention employs a differentiated compression strategy tailored to the characteristics of different regions, scientifically improving storage space utilization. This satisfies the long-term storage needs of regulatory data while avoiding unnecessary resource consumption, thereby enhancing the storage efficiency of regulatory videos. Finally, the generation and storage of regionally compressed regulatory video streams provide a structured and easily searchable data foundation for subsequent regulatory inspections, enabling rapid location and analysis of key information. This significantly improves the efficiency and accuracy of monitoring the underground drilling process in coal mines, providing reliable data support for safe production.

[0074] This invention optimizes the compression parameters of the static UI area, dynamic data area, and real-time video area by using equipment sensor data and user interaction events, enabling intelligent processing and resource allocation of monitoring video streams. First, sensor data and user interaction events in coal mines can reflect the actual state of the monitoring scene in real time, providing a dynamic basis for adjusting compression parameters. For example, when high-frequency operation commands are detected in the dynamic data area, the system can appropriately reduce the compression rate of that area to retain key details; when the real-time video area is static, the compression rate can be increased to save storage space. This dynamic adaptation mechanism avoids excessive resource consumption or loss of key information due to fixed compression strategies. Second, parameter optimization based on sensor data improves the system's adaptability to complex working conditions. In dynamic environments such as underground coal mines, equipment operating status, lighting conditions, or screen content may change frequently. By adjusting compression parameters in real time, the system can ensure that the static UI area always presents interface information in a lossless manner, the dynamic data area is flexibly compressed while maintaining readability, and the real-time video area dynamically balances image quality and storage efficiency based on screen activity, thereby enhancing the reliability and practicality of monitoring data. Finally, dynamically adjusted compressed surveillance video streams offer greater advantages in long-term storage and retrospective analysis. By employing compression strategies precisely tailored to specific scenario requirements, both redundant and invalid data can be reduced while ensuring the integrity of critical information, providing a structured and highly available data foundation for subsequent regulatory inspections, accident tracing, and other scenarios. Attached Figure Description

[0075] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0076] Figure 1 This is a schematic diagram of the composite surveillance footage.

[0077] Figure 2 This is a logical block diagram for a multi-source information fusion and supervision method for underground coal mine drilling processes. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0079] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not mean that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0080] The following detailed explanation illustrates the specific implementation methods:

[0081] Example:

[0082] This embodiment discloses a multi-source information fusion monitoring method for coal mine underground drilling processes.

[0083] like Figure 2 As shown, a multi-source information fusion monitoring method for coal mine underground drilling processes includes:

[0084] S1: Acquire continuous composite monitoring footage during underground drilling in coal mines to form a regulatory video stream;

[0085] S2: Perform region segmentation on each frame of the composite monitoring image in the monitoring video stream to obtain the corresponding static UI area, dynamic data area and real-time video area;

[0086] S3: Compress the static UI area, dynamic data area and real-time video area of ​​each frame of the composite monitoring image in the monitoring video stream using the corresponding compression strategies to obtain the corresponding compressed data.

[0087] S4: Merge the compressed data of the static UI area, dynamic data area and real-time video area of ​​each frame in the surveillance video stream to generate a regionally compressed surveillance video stream;

[0088] S5: Store the compressed monitoring video streams by region for subsequent inspection and supervision during underground coal mine drilling.

[0089] In the specific implementation process, the steps for segmenting the composite monitoring image into regions include:

[0090] S201: Acquire composite monitoring image in pixel matrix format ;

[0091] S202: The size used for composite monitoring images is... The sliding window calculates the matching score between each window position and each static UI template. ; Match score Windows exceeding the static UI threshold are marked as static UI areas;

[0092] The formula for calculating the matching score is expressed as follows:

[0093] ;

[0094] In the formula: This indicates the matching score for the current window. Indicates the coordinates of the top left corner of the window (relative to the composite monitoring screen); Indicates the first A static UI template Middle group coordinates Pixel values ​​(relative to the window); Represents the coordinates in the current window pixel values, , ;

[0095] In this embodiment, static UI elements (such as menus and labels) are extracted from historical composite surveillance footage as a template set. Each template contains geometric features and an HSV histogram.

[0096] S203: Divide the area outside the static UI area into several candidate areas; extract the HSV spatial histogram of each candidate area. And calculate its HSV histogram with the preset template. KL divergence ;

[0097] The formula is expressed as:

[0098] ;

[0099] S204: Calculate the gradient magnitude of each candidate region using the Sobel operator. ;

[0100] The formula is expressed as:

[0101] ;

[0102] In the formula: This represents the horizontal gradient of the candidate region; This represents the vertical gradient of the candidate region;

[0103] S205: Extract texture histograms of each candidate region using Local Binary Pattern (LBP). ;

[0104] S206: Calculate the corresponding comprehensive score based on the KL divergence, gradient magnitude and texture histogram of each candidate region, and mark the candidate region with the comprehensive score greater than the threshold of the dynamic data region as the dynamic data region.

[0105] Overall Score The formula is expressed as:

[0106] ;

[0107] In the formula: , , Indicates the weighting coefficient;

[0108] S207: Calculate the inter-frame motion vector field of each pixel in the synthesized surveillance image using the Horn-Schunck optical flow method. ; Calculate the motion intensity of each pixel and exercise intensity Pixels with a motion intensity higher than the average of all pixels are marked as high-motion pixels;

[0109] ;

[0110] In the formula: Indicates horizontal displacement ( Directional velocity); Indicates vertical displacement ( Directional velocity);

[0111] S208: Cluster high-motion pixels using connected components and treat each connected region (connected component) as a high-motion region;

[0112] S209: Extract the geometric features (such as aspect ratio) of each high motion region and match them with the preset camera resolution template. If the match is successful, mark the corresponding high motion region as the real-time video area.

[0113] This invention uses region segmentation technology to divide the composite monitoring screen into three functional areas: a static UI area, a dynamic data area, and a real-time video area, thus avoiding the excessive compression of non-critical information by traditional uniform compression methods.

[0114] In the specific implementation process, the compression strategy for the static UI area of ​​the composite monitoring image in each frame of the surveillance video stream includes:

[0115] S301: Calculate the differences in the static UI area between frames of the composite monitoring image in the surveillance video stream;

[0116] The formula is expressed as:

[0117] ;

[0118] In the formula: Indicates the first Frame and the Differences in the static UI area between frames of the frame-synthesized monitoring image; This represents the total number of pixels in the static UI area; Indicates the first Coordinates in the static UI area of ​​the frame-composite monitoring screen Pixel values;

[0119] S302: Calculate the corresponding key frame judgment threshold for each frame of the composite monitoring image;

[0120] The formula is expressed as:

[0121] ;

[0122] In the formula: Indicates the first Key frame determination threshold for frame synthesis monitoring footage; This indicates the initial difference value set (to 5). Indicates the previous number Average difference in static UI area between frames of the frame-synthesized monitoring image;

[0123] S303: Differences in static UI areas between frames Greater than its keyframe judgment threshold The composite surveillance footage is used as a keyframe;

[0124] S304: Perform lossless compression encoding on the difference pixels in keyframes and non-keyframes (delta frames) to obtain the static UI area compressed data of the regulatory video stream (including keyframes and delta frames).

[0125] This invention employs a lossless compression strategy for the static UI area, ensuring the integrity and accuracy of the monitoring interface information, and facilitating accurate identification of device status and operating interface.

[0126] In the specific implementation process, for the dynamic data area of ​​the composite monitoring image in each frame of the monitoring video stream: the encoding strategy is to use the JPEG-LS encoding algorithm with a medium compression rate to encode the data in the dynamic data area, so as to obtain the compressed data of the dynamic data area of ​​the monitoring video stream.

[0127] This invention employs a moderate compression rate for the dynamic data area, balancing data readability and storage efficiency.

[0128] In the specific implementation process, for the real-time video area of ​​the composite monitoring image in each frame of the regulatory video stream: the encoding strategy is to use the H.265 / HEVC encoding algorithm with a high compression ratio to encode the data of the real-time video area to obtain the compressed data of the real-time video area of ​​the regulatory video stream.

[0129] This invention employs a high compression rate for the real-time video area, effectively reducing the amount of video data while ensuring the recognizability of key scene content.

[0130] In the specific implementation process, after aligning the timestamps and hashing the compressed data of the static UI area, dynamic data area, and real-time video area of ​​the regulatory video stream, the compressed data of the static UI area, dynamic data area, and real-time video area are merged to generate a regionally compressed regulatory video stream.

[0131] This invention addresses the problems of information redundancy, low storage efficiency, and easy loss of critical information in the processing of monitoring videos during underground coal mine drilling. It proposes a multi-source information fusion monitoring method for underground coal mine drilling processes. By segmenting the composite monitoring image from each frame of the monitoring video stream into regions, targeted compression strategies are applied to the static UI area, dynamic data area, and real-time video area, achieving efficient processing and storage of monitoring video data. First, region segmentation technology divides the composite monitoring image into three functional areas: static UI area, dynamic data area, and real-time video area, avoiding the excessive compression of non-critical information in traditional uniform compression methods. The static UI area uses a lossless compression strategy to ensure the integrity and accuracy of the monitoring interface information, facilitating accurate identification of equipment status and operation interface; the dynamic data area uses a medium compression rate to balance data readability and storage efficiency; and the real-time video area uses a high compression rate to effectively reduce the amount of video data while ensuring the identifiability of key scene content. Second, in the complex environment of underground coal mines, the amount of video data is enormous, and traditional uniform compression methods often lead to wasted storage space or loss of critical information. To address this, the present invention employs a differentiated compression strategy tailored to the characteristics of different regions, scientifically improving storage space utilization. This satisfies the long-term storage needs of regulatory data while avoiding unnecessary resource consumption, thereby enhancing the storage efficiency of regulatory videos. Finally, the generation and storage of regionally compressed regulatory video streams provide a structured and easily searchable data foundation for subsequent regulatory inspections, enabling rapid location and analysis of key information. This significantly improves the efficiency and accuracy of monitoring the underground drilling process in coal mines, providing reliable data support for safe production.

[0132] In the specific implementation process, the compression parameters of the static UI area, dynamic data area, and real-time video area in the regional compressed surveillance video stream are optimized using equipment sensor data. New compressed data is then generated based on the optimized compression parameters to obtain a dynamically adjusted compressed surveillance video stream. Specific processing steps include:

[0133] S401: Obtain compression parameters for static UI area compression data, dynamic data area compression data, and real-time video area compression data in the regionally compressed surveillance video stream. , and ;

[0134] S402: After standardizing the acquired equipment sensor data, perform state vector fusion with user interaction event encoding to obtain the process state vector. ;

[0135] The formula is expressed as:

[0136] ;

[0137] In the formula: This represents the standardized data from the device's sensor data; Indicates the encoding of user interaction events; , Indicates the weighting coefficient;

[0138] In this embodiment, the device sensor data includes data such as temperature, pressure, flow rate, and vibration; user interaction events include events such as clicks and alarm triggers, wherein the user interaction event encoding includes: event occurrence -1, otherwise -0.

[0139] S403: Based on process state vector Calculate the priority weights of the static UI area, dynamic data area, and real-time video area;

[0140] The formula is expressed as:

[0141] ;

[0142] ;

[0143] ;

[0144] In the formula: , , These represent the priority weights of the static UI area, dynamic data area, and real-time video area, respectively. When calculating the priority weights of the dynamic data area and real-time video area, the process state vector is given special consideration. The components related to pressure and vibration (scalar values, range) reflect process risks; Indicates the attenuation coefficient; This indicates the set vibration threshold (e.g., 0.8).

[0145] S404: Compression parameters based on the priority weights of the static UI area, dynamic data area, and real-time video area. , and Optimize to obtain optimized compression parameters. , and ;

[0146] The formula is expressed as:

[0147] ;

[0148] ;

[0149] ;

[0150] In the formula: This represents the sensitivity coefficient (set to 0.2). Indicates the adjustment range (set to 5).

[0151] Specifically, the compression parameters are optimized based on the static UI area, dynamic data area, and real-time video area. , and The data in the static UI area, dynamic data area, and real-time video area of ​​the surveillance video stream are recompressed to obtain dynamically adjusted compressed data of the static UI area, dynamic data area, and real-time video area, which are then merged to generate a dynamically adjusted compressed surveillance video stream.

[0152] Specifically, regions with priority weights greater than the weight threshold are designated as critical regions, and compression parameter adjustments and recompression are performed only on critical regions.

[0153] This invention identifies regions with priority weights greater than a weight threshold as critical regions and performs compression parameter optimization and recompression only on these critical regions. The purpose is to select regions that require priority quality assurance under the current process conditions, rather than directly adjusting compression parameters.

[0154] This invention optimizes the compression parameters of the static UI area, dynamic data area, and real-time video area by using equipment sensor data and user interaction events, enabling intelligent processing and resource allocation of monitoring video streams. First, sensor data and user interaction events in coal mines can reflect the actual state of the monitoring scene in real time, providing a dynamic basis for adjusting compression parameters. For example, when high-frequency operation commands are detected in the dynamic data area, the system can appropriately reduce the compression rate of that area to retain key details; when the real-time video area is static, the compression rate can be increased to save storage space. This dynamic adaptation mechanism avoids excessive resource consumption or loss of key information due to fixed compression strategies. Second, parameter optimization based on sensor data improves the system's adaptability to complex working conditions. In dynamic environments such as underground coal mines, equipment operating status, lighting conditions, or screen content may change frequently. By adjusting compression parameters in real time, the system can ensure that the static UI area always presents interface information in a lossless manner, the dynamic data area is flexibly compressed while maintaining readability, and the real-time video area dynamically balances image quality and storage efficiency based on screen activity, thereby enhancing the reliability and practicality of monitoring data. Finally, dynamically adjusted compressed surveillance video streams offer greater advantages in long-term storage and retrospective analysis. By employing compression strategies precisely tailored to specific scenario requirements, both redundant and invalid data can be reduced while ensuring the integrity of critical information, providing a structured and highly available data foundation for subsequent regulatory inspections, accident tracing, and other scenarios.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for multi-source information fusion and monitoring of underground coal mine drilling processes, characterized in that, include: S1: Acquire continuous composite monitoring footage during underground drilling in coal mines to form a regulatory video stream; S2: Perform region segmentation on each frame of the composite monitoring image in the monitoring video stream to obtain the corresponding static UI area, dynamic data area and real-time video area; Step S2, the process of segmenting the composite monitoring image into regions, includes: S201: Obtain composite monitoring footage; S202: The size used for composite monitoring images is... The sliding window calculates the matching score between each window position and each static UI template; windows with matching scores greater than the static UI threshold are marked as static UI areas; The formula for calculating the matching score is expressed as follows: ； In the formula: This indicates the matching score for the current window. Represents the coordinates of the top-left corner of the window; Indicates the first A static UI template Middle group coordinates Pixel values; Represents the coordinates in the current window pixel values, , ; S203: Divide the area outside the static UI area into several candidate areas; extract the HSV spatial histogram of each candidate area. And calculate its HSV histogram with the preset template. KL divergence ; The formula is expressed as: ； S204: Calculate the gradient magnitude of each candidate region using the Sobel operator. ; The formula is expressed as: ； In the formula: This represents the horizontal gradient of the candidate region; This represents the vertical gradient of the candidate region; S205: Extract texture histograms of each candidate region using local binary mode. ; S206: Calculate the corresponding comprehensive score based on the KL divergence, gradient magnitude and texture histogram of each candidate region, and mark the candidate region with the comprehensive score greater than the threshold of the dynamic data region as the dynamic data region. Overall Score The formula is expressed as: ； In the formula: , , Indicates the weighting coefficient; S207: Calculate the inter-frame motion vector field of each pixel in the synthesized surveillance image using the Horn-Schunck optical flow method; calculate the motion intensity of each pixel. and exercise intensity Pixels with a motion intensity higher than the average of all pixels are marked as high-motion pixels; ； In the formula: Indicates horizontal displacement; Indicates displacement in the vertical direction; S208: Cluster high-motion pixels using connected components and treat each connected region as a high-motion region; S209: Extract the geometric features of each high-motion region and match them with the preset camera resolution template. If the match is successful, mark the corresponding high-motion region as the real-time video area. S3: The static UI area, dynamic data area and real-time video area of ​​each frame of the composite monitoring image in the monitoring video stream are compressed using the corresponding compression strategies to obtain the corresponding compressed data. S4: Merge the compressed data of the static UI area, dynamic data area and real-time video area of ​​each frame in the surveillance video stream to generate a regionally compressed surveillance video stream; S5: Store the compressed surveillance video streams by region for subsequent inspection and supervision.

2. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 1, characterized in that: In step S3, the compression strategy for the static UI area of ​​the composite monitoring image in each frame of the monitoring video stream includes: S301: Calculate the differences in the static UI area between frames of the composite monitoring image in the surveillance video stream; The formula is expressed as: ； In the formula: Indicates the first Frame and the Differences in the static UI area between frames of the frame-synthesized monitoring image; This represents the total number of pixels in the static UI area; Indicates the first Coordinates in the static UI area of ​​the frame-synthesized monitoring screen Pixel values; S302: Calculate the corresponding key frame judgment threshold for each frame of the composite monitoring image; The formula is expressed as: ； In the formula: Indicates the first Key frame determination threshold for frame synthesis monitoring footage; This indicates the initial difference value set. Indicates the previous number Average difference in static UI area between frames of the frame-synthesized monitoring image; S303: Differences in static UI areas between frames Greater than its keyframe judgment threshold The composite surveillance footage is used as keyframes; S304: Compress and encode the difference pixels in key frames and non-key frames to obtain the static UI area compressed data of the regulatory video stream.

3. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 2, characterized in that: In step S3, for the dynamic data area of ​​the composite monitoring image in each frame of the monitoring video stream, the encoding strategy is to use the JPEG-LS encoding algorithm to encode the data in the dynamic data area to obtain the compressed data of the dynamic data area of ​​the monitoring video stream.

4. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 3, characterized in that: In step S3, for the real-time video area of ​​the composite monitoring image in each frame of the monitoring video stream, the encoding strategy is to use the H.265 / HEVC encoding algorithm to encode the data of the real-time video area to obtain the compressed data of the real-time video area of ​​the monitoring video stream.

5. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 4, characterized in that: In step S4, after aligning the timestamps and deduplicating the compressed data of the static UI area, dynamic data area, and real-time video area of ​​the regulatory video stream, the compressed data of the static UI area, dynamic data area, and real-time video area are merged to generate a regionally compressed regulatory video stream.

6. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 1, characterized in that: In step S4, the compression parameters of the static UI area, dynamic data area and real-time video area in the regional compressed surveillance video stream are optimized by using device sensor data, and new compressed data is generated based on the optimized compression parameters to obtain the dynamically adjusted compressed surveillance video stream.

7. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 6, characterized in that: Step S4, which involves dynamically adjusting the compression parameters of the static UI area, dynamic data area, and real-time video area in the regionally compressed monitoring video stream based on device sensor data, includes the following steps: S401: Obtain compression parameters for static UI area compression data, dynamic data area compression data, and real-time video area compression data in the regionally compressed surveillance video stream. , and ; S402: After standardizing the acquired equipment sensor data, perform state vector fusion with user interaction event encoding to obtain the process state vector. ; The formula is expressed as: ； In the formula: This represents the standardized data from the device's sensor data; Indicates the encoding of user interaction events; , Indicates the weighting coefficient; S403: Based on process state vector Calculate the priority weights of the static UI area, dynamic data area, and real-time video area; The formula is expressed as: ； ； ； In the formula: , , These represent the priority weights of the static UI area, dynamic data area, and real-time video area, respectively. Indicates the attenuation coefficient; This indicates the set vibration threshold. S404: Compression parameters based on the priority weights of the static UI area, dynamic data area, and real-time video area. , and Optimize to obtain optimized compression parameters. , and ; The formula is expressed as: ； ； ； In the formula: Indicates the sensitivity coefficient; Indicates the adjustment range.

8. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 7, characterized in that: In step S404, the compression parameters are optimized based on the static UI area, dynamic data area, and real-time video area. , and The data in the static UI area, dynamic data area, and real-time video area of ​​the surveillance video stream are recompressed to obtain dynamically adjusted compressed data of the static UI area, dynamic data area, and real-time video area, which are then merged to generate a dynamically adjusted compressed surveillance video stream.

9. The multi-source information fusion monitoring method for coal mine underground drilling processes as described in claim 8, characterized in that: Regions with priority weights greater than the weight threshold are designated as critical regions, and compression parameter adjustments and recompression are performed only on critical regions.

Images