Coal mine safety production video intelligent analysis system

Abstract

The invention relates to a coal mine safety production video intelligent analysis system, and belongs to the technical field of mine safety. The system comprises a data acquisition module used for acquiring a production video and environmental parameters of a coal mine production area, the environmental parameters comprising a plurality of types of sub-environmental parameters; the image alignment module is used for performing time alignment on the production image sequence of the production video and the environmental parameters to obtain a standard production image sequence containing the environmental parameters; the region-of-interest extraction module is used for generating a feature enhancement strategy and determining an abnormal region-of-interest of the standard production image based on the feature enhancement strategy; the anomaly analysis module is used for performing feature matching on the abnormal region of interest of the standard production image and a preset safety production anomaly type to determine the safety production anomaly of the standard production image; and the result generation module is used for generating a safety intelligent analysis result of the production video. According to the invention, the precision and efficiency of safety production anomaly detection in the coal mine production process are improved.

Description

Technical Field

[0001] This invention relates to the field of mine safety technology, and in particular to a video intelligent analysis system for coal mine safety production. Background Technology

[0002] Safety is always a paramount concern in coal mine production operations. Due to the unique nature of coal mine operations, with its numerous complex and dangerous factors, safe production faces significant challenges. From the perspective of the coal mine operation scenario, the underground environment presents a variety of risks. Traditional safety monitoring methods often struggle to comprehensively and promptly detect all potential safety hazards.

[0003] With the continuous development of technology, video surveillance technology has been gradually applied to the field of coal mine safety production. However, most existing coal mine safety production video surveillance systems simply collect and store video images, lacking the ability to intelligently analyze and process video data. When it is necessary to discover potential safety hazards from video data, monitoring personnel need to spend a lot of time and energy to review the video images, which is not only inefficient but also makes it difficult to quickly discover safety hazards from massive amounts of video data, easily leading to the serious consequences of overlooking safety issues. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a coal mine safety production video intelligent analysis system, comprising: The data acquisition module is used to acquire production videos and environmental parameters of the coal mine production area through video acquisition devices and sensors pre-deployed in the coal mine production area. The environmental parameters include multiple types of sub-environmental parameters. The image alignment module is used to convert production video into a production image sequence and align the production image sequence with environmental parameters in time to obtain a standard production image sequence containing environmental parameters. The region of interest extraction module is used to generate a feature enhancement strategy based on the environmental parameters of the target standard production image in the standard production image sequence, and to determine the abnormal region of interest of the target standard production image based on the feature enhancement strategy. The target standard production image is any standard production image in the standard production image sequence. The anomaly analysis module is used to perform feature matching between the abnormal regions of interest in the target standard production image and the preset safety production anomaly types, and to determine the safety production anomalies in the target standard production image based on the matching results. The results generation module is used to generate safety intelligent analysis results for the production video based on the safety production anomalies and their environmental parameters for each standard production image.

[0005] Preferably, the image alignment module includes: An image conversion unit is used to convert production video into a production image sequence using image conversion tools; The parameter alignment unit is used to match the timestamp of each production image in the production image sequence with the timestamp of the environmental parameters. If the timestamp of any production image fails to match, linear interpolation is used to generate interpolated environmental parameters, and the interpolated environmental parameters are matched into the production images to obtain a standard production image sequence consisting of standard production images containing environmental parameters.

[0006] Preferably, the region of interest extraction module includes: The parameter comparison unit is used to compare each type of sub-environmental parameter of the target standard production image with the corresponding preset standard sub-environmental parameter threshold, and calculate the deviation value of each type of sub-environmental parameter of the target standard production image based on the comparison result; The feature enhancement strategy generation unit is used to generate a feature enhancement strategy for the target standard production image based on the deviation value of each type of sub-environmental parameter of the target standard production image. The region of interest segmentation unit is used to perform edge detection on the target standard production image according to the feature enhancement strategy to obtain the abnormal region of interest of the target standard production image.

[0007] Preferably, the feature enhancement strategy generation unit includes: The region identification subunit is used to input the target standard production image into the feature recognition big data model, and the feature recognition big data model outputs several unit regions of the target standard production image and the category label of each unit region. The strategy generation sub-unit is used to generate a sub-feature enhancement strategy for each unit region based on the deviation value of each type of sub-environmental parameter of the target standard production image and a preset strategy lookup table, and all sub-feature enhancement strategies constitute the feature enhancement strategy of the target standard production image.

[0008] Preferably, the region of interest partitioning unit includes: The grayscale conversion sub-unit is used to process each unit region of the target standard production image according to its sub-feature enhancement strategy to obtain a sub-grayscale production image of each unit region; Alignment sub-units are used to align the target standard production image with the preset standard image pixel by pixel, and to find the standard unit region corresponding to each sub-grayscale production image in the preset standard image. The contour extraction subunit is used to calculate the grayscale difference between each sub-grayscale production image and its corresponding standard unit region. Sub-grayscale production images with grayscale differences greater than a preset grayscale difference threshold are taken as difference unit regions. The edge points of each difference unit region are extracted according to the edge recognition algorithm. The edge points of each difference unit region are smoothly connected to obtain all feature contours of the target standard production image. The region of interest extraction subunit is used to determine each standard production feature of the target standard production image based on each feature contour of the target standard production image, and to synthesize all standard production features of the target standard production image to obtain the region of interest.

[0009] Preferably, the anomaly analysis module includes: The coarse matching unit is used to coarsely match each standard production feature in the region of interest with each standard anomaly feature of the preset safety production anomaly type, and obtain multiple coarse matching results for each standard production feature. The fine matching unit is used to construct an anomaly relationship map based on the anomaly region of interest and all coarse matching results of each standard production feature. Based on the anomaly relationship map, it calculates the correlation strength between each coarse matching result with the target standard production image as the main node and each standard production feature, and obtains the production anomaly feature of each standard production feature based on the correlation strength. The integration unit is used to integrate all production anomaly features to form a target standard production image of safety production anomalies.

[0010] Preferably, the coarse matching unit includes: Spatial grid sub-units are used to spatially grid each standard production feature in the region of interest to obtain a simplified feature vector for each standard production feature. Each standard anomaly feature is then vectorized to obtain an anomaly feature vector for each standard anomaly feature. The distance calculation subunit calculates the distance between each simplified feature vector and each abnormal feature vector. If the distance between any simplified feature vector and any abnormal feature vector is less than the coarse matching threshold, it is determined that the standard production feature corresponding to the simplified feature vector and the standard abnormal feature corresponding to the abnormal feature vector have successfully matched coarsely. Multiple coarse matching results are generated for each standard production feature based on the standard abnormal features that have successfully matched coarsely.

[0011] Preferably, the fine matching unit includes: An anomaly relationship graph construction subunit is used to construct an anomaly relationship graph with the target standard production image as the main node, each standard production feature under the target standard production image as a child node, and each coarse matching result under each standard feature as a child node attribute. The association strength calculation subunit is used to process the abnormal relationship graph through a pre-trained graph neural network to obtain the feature association between each standard production feature and its corresponding coarse matching result in the abnormal relationship graph, and calculate the association strength between each standard production feature and each coarse matching result based on the feature association and the preset feature weight of each standard production feature. The matching subunit is used to select the coarse matching result with the strongest correlation from each standard production feature as the production anomaly feature for each standard production feature.

[0012] Preferably, the result generation module includes: Anomaly type analysis unit is used to determine the safety identification anomaly type of each standard production image based on the safety production anomalies of each standard production image. The location determination unit is used to generate the location of the safety identification anomaly type corresponding to each production anomaly feature based on the category label of the unit region where each production anomaly feature is located in each standard production image; The result generation unit is used to combine the timestamp of each security identification anomaly type of each standard production image, each security identification anomaly type and its occurrence location, and the environmental parameters corresponding to the timestamp to form the security intelligent analysis result of the production video.

[0013] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.

[0014] By means of the above solution, the beneficial effects of the present invention are as follows: By generating feature enhancement strategies based on environmental parameters of the target standard production image and determining the region of interest (ROI) of the target standard production image based on the feature enhancement strategies, an adaptive feature enhancement strategy generation mechanism based on environmental parameters is realized. This significantly improves the quality of the standard production image and the accuracy of the abnormal ROI, providing an accurate foundation for subsequent anomaly analysis. By performing feature matching between the abnormal ROI of the target standard production image and preset safety production anomaly types, safety production anomalies in the target standard production image are determined based on the matching results. Based on the safety production anomalies and environmental parameters of each standard production image, the safety intelligent analysis results of the production video are generated. This allows for the rapid discovery of safety production anomalies in coal mines from massive amounts of video data, greatly improving the accuracy and efficiency of safety production anomaly detection during the production process.

[0015] This invention proposes an intelligent analysis system for coal mine safety production videos, which improves the intelligent analysis and processing capabilities of coal mine safety production videos. It can accurately analyze safety situations involving personnel, equipment, and the environment, thereby reducing potential safety hazards in coal mine production.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the composition structure of the intelligent video analysis system for coal mine safety production provided in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the unit region and its category label of the target standard production image in an embodiment of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] like Figure 1 As shown, this embodiment of the invention provides a coal mine safety production video intelligent analysis system, including: The data acquisition module is used to acquire production videos and environmental parameters of the coal mine production area through video acquisition devices and sensors pre-deployed in the coal mine production area. The environmental parameters include multiple types of sub-environmental parameters. The image alignment module is used to convert production video into a production image sequence and align the production image sequence with environmental parameters in time to obtain a standard production image sequence containing environmental parameters. The region of interest extraction module is used to generate a feature enhancement strategy based on the environmental parameters of the target standard production image in the standard production image sequence, and to determine the abnormal region of interest of the target standard production image based on the feature enhancement strategy. The target standard production image is any standard production image in the standard production image sequence. The anomaly analysis module is used to perform feature matching between the abnormal regions of interest in the target standard production image and the preset safety production anomaly types, and to determine the safety production anomalies in the target standard production image based on the matching results. The results generation module is used to generate safety intelligent analysis results for the production video based on the safety production anomalies and their environmental parameters for each standard production image.

[0021] Specifically, the video acquisition device can be a CCTV camera, infrared camera, or panoramic camera, etc.; sensors can include temperature sensors, humidity sensors, dust concentration sensors, and light sensors, etc.; environmental parameters include temperature, humidity, dust concentration, and light intensity, etc.; temperature, humidity, dust concentration, and light intensity are each considered as a type of raw sub-environmental parameter. These raw sub-environmental parameters of multiple types, acquired by multiple sensors, are time-aligned to obtain multiple types of sub-environmental parameters. Production video refers to video data recorded in real-time in the coal mine production area by video acquisition devices, documenting the coal mine production process and working environment.

[0022] In the image alignment module, the production image sequence is a series of consecutive images extracted from the production video. Aligning the production image sequence with environmental parameters in time means aligning production images and environmental parameters acquired at the same timestamp.

[0023] In the region of interest extraction module, the feature enhancement strategy is a method for extracting features from the target standard production image based on its environmental parameters. For example, when environmental parameters of the target standard production image include excessive dust concentration, the feature enhancement strategy should involve adding edge enhancement algorithms (such as Gaussian filtering or cross-gradient algorithms) to ensure that the edges of the target standard production image can be identified. Anomaly regions of interest refer to areas in the target standard production image where security anomalies may exist.

[0024] In the anomaly analysis module, the preset safety production anomaly types are pre-defined types of safety anomalies that may occur during the production process in coal mine production areas based on historical data. Safety production anomalies include personnel, equipment, and environmental safety anomalies, such as personnel not wearing safety helmets, equipment damage, and colored gas leaks in the underground environment.

[0025] In the results generation module, the safety intelligent analysis results are safety status assessment results derived from the analysis of production videos and environmental parameters of the coal mine production area.

[0026] In one specific embodiment, the image alignment module includes: An image conversion unit is used to convert production video into a production image sequence using image conversion tools; The parameter alignment unit is used to match the timestamp of each production image in the production image sequence with the timestamp of the environmental parameters. If the timestamp of any production image fails to match, linear interpolation is used to generate interpolated environmental parameters, and the interpolated environmental parameters are matched into the production images to obtain a standard production image sequence consisting of standard production images containing environmental parameters.

[0027] Specifically, in the image conversion unit, the image conversion tool can be OpenCV, VLC, Python Pillow, etc. When converting the production video into a production image sequence, the image conversion tool extracts production images from the production video according to a set extraction time interval, which can be 3s, 5s, etc.

[0028] In the parameter alignment unit, if the timestamp of a production image successfully matches (is consistent with) the timestamp of an environmental parameter, then the environmental parameter corresponding to that timestamp is matched to that production image. If the timestamp of a production image fails to match, that is, if no environmental parameter's timestamp matches the timestamp of that production image, then when generating interpolated environmental parameters using linear interpolation, the average of each sub-environmental parameter corresponding to the timestamps of the two production images to the left and right of that production image is performed to obtain the sub-environmental interpolation parameter of each type, and each type of sub-environmental interpolation parameter is matched to that production image.

[0029] In one specific embodiment, the region of interest extraction module includes: The parameter comparison unit is used to compare each type of sub-environmental parameter of the target standard production image with the corresponding preset standard sub-environmental parameter threshold, and calculate the deviation value of each type of sub-environmental parameter of the target standard production image based on the comparison result; The feature enhancement strategy generation unit is used to generate a feature enhancement strategy for the target standard production image based on the deviation value of each type of sub-environmental parameter of the target standard production image. The region of interest segmentation unit is used to perform edge detection on the target standard production image according to the feature enhancement strategy to obtain the abnormal region of interest of the target standard production image.

[0030] Specifically, in the parameter comparison unit, a certain preset standard sub-environmental parameter threshold is an empirical value for that type of sub-environmental parameter set according to the underground safety standards for coal mines. When calculating the deviation value of a certain type of sub-environmental parameter based on the comparison results, the difference between the sub-environmental parameter of that type and the corresponding preset standard sub-environmental parameter threshold is calculated to obtain the deviation value of the sub-environmental parameter of that type.

[0031] In the feature enhancement strategy generation unit, under a specific condition, if the illumination intensity in the environmental parameters of the target standard production image is 800 lux, and the deviation from the preset standard sub-environmental parameter threshold (1000 lux) is -200 lux, then it is determined that the illumination in the environmental parameters is insufficient, and the generated feature enhancement strategy is to improve the illumination intensity of the target standard production image by adjusting the exposure.

[0032] In the region of interest (ROI) segmentation unit, edge detection is an image processing technique used to identify regions in a standard production image where pixel values ​​vary significantly, i.e., edges in the standard production image. Edge detection algorithms can employ operators such as Sobel, Prewitt, and Laplacian.

[0033] In one specific embodiment, the feature enhancement strategy generation unit includes: The region identification subunit is used to input the target standard production image into the feature recognition big data model, and the feature recognition big data model outputs several unit regions of the target standard production image and the category label of each unit region. The strategy generation sub-unit is used to generate a sub-feature enhancement strategy for each unit region based on the deviation value of each type of sub-environmental parameter of the target standard production image and a preset strategy lookup table, and all sub-feature enhancement strategies constitute the feature enhancement strategy of the target standard production image.

[0034] Specifically, in the region recognition subunit, a unit region refers to a different region in the target standard production image, including equipment regions, personnel entry / exit regions, work areas, etc. The feature recognition big data model is a model that takes a production image as input and outputs multiple pre-divided unit regions of the production image and their labels. In this embodiment, a convolutional neural network model can be used as the feature recognition big data model.

[0035] It should be noted that before inputting the target standard production image into the feature recognition big data model, the feature recognition big data model needs to be trained first. Specifically, when training the feature recognition big data model, a large number of historical production images with manually labeled unit regions are used as training images. For example... Figure 2 As shown, it is a schematic diagram of several unit regions and their category labels of a target standard production image. Figure 2 The area in the left box represents the cell area of ​​the "device" category label identified from the target standard production image, and the area in the right box represents the cell area of ​​the "channel" category label identified from the target standard production image.

[0036] In the strategy generation sub-unit, the preset strategy lookup table is a mapping table that associates the deviation value of each type of sub-environmental parameter with the corresponding feature enhancement strategy, as shown in Table 1. When constructing the feature enhancement strategy of the target standard production image by combining all sub-feature enhancement strategies, the sub-feature enhancement strategies of all unit regions of the target standard production image are combined to form the feature enhancement strategy of the target standard production image.

[0037]

[0038] In one specific embodiment, the region of interest partitioning unit includes: The grayscale conversion sub-unit is used to process each unit region of the target standard production image according to its corresponding sub-feature enhancement strategy to obtain a sub-grayscale production image of each unit region. Alignment sub-units are used to align the target standard production image with the preset standard image pixel by pixel, and to find the standard unit region corresponding to each sub-grayscale production image in the preset standard image. The contour extraction subunit is used to calculate the grayscale difference between the standard unit region and the corresponding position in each sub-grayscale production image, and to take the sub-grayscale production image with a grayscale difference greater than the preset grayscale difference threshold as the difference unit region. The edge points of each difference unit region are extracted according to the edge recognition algorithm, and the edge points of each difference unit region are smoothly connected to obtain all feature contours of the target standard production image. The region of interest extraction subunit is used to determine each standard production feature of the target standard production image based on each feature contour of the target standard production image, and to synthesize all standard production features of the target standard production image to obtain the region of interest.

[0039] Specifically, in the grayscale conversion sub-unit, each sub-grayscale production image is the image obtained by extracting features and processing grayscale values ​​for each unit region through a sub-feature enhancement strategy.

[0040] In the alignment sub-unit, the preset standard image is an image of a coal mine production scene under normal operating conditions, captured by a video acquisition device. When acquiring production video, the configuration of the video acquisition device is consistent with the configuration when acquiring the preset standard image. Therefore, the pixels of the target standard production image can be directly aligned with the pixels of the preset standard image. Based on this, the standard unit region corresponding to a certain unit region of the target standard production image can be found by matching the pixel range of that unit region in the preset standard image.

[0041] In the contour extraction sub-unit, the grayscale difference D between a certain grayscale production image and its corresponding standard unit region is calculated using formula (1): (1); In formula (1), p represents the position of any pixel in the sub-grayscale production image, M represents the pixel range of the sub-grayscale production image, I(p) represents the grayscale value of pixel position p in the sub-grayscale production image, and I'(p) represents the grayscale value of pixel position p in the standard unit area.

[0042] It should be noted that if no sub-grayscale production image exists with a grayscale difference greater than the preset grayscale difference threshold, the target standard production image is determined to be normal, and no further operations are required. Edge recognition algorithms can employ operators such as Sobel, Prewitt, and Laplacian.

[0043] In the region of interest extraction sub-unit, the standard production feature refers to the feature consisting of the unit region where each feature contour of the target standard production image is located, as well as the pixel feature information (position and pixel value).

[0044] In one specific embodiment, the anomaly analysis module includes: The coarse matching unit is used to coarsely match each standard production feature in the region of interest with each standard anomaly feature of the preset safety production anomaly type, and obtain multiple coarse matching results for each standard production feature. The fine matching unit is used to construct an anomaly relationship map based on the anomaly region of interest and all coarse matching results of each standard production feature. Based on the anomaly relationship map, it calculates the correlation strength between each coarse matching result with the target standard production image as the main node and each standard production feature, and obtains the production anomaly feature of each standard production feature based on the correlation strength. The integration unit is used to integrate all production anomaly features to form a target standard production image of safety production anomalies.

[0045] Specifically, in the coarse matching unit, the standard anomaly features for preset safety production anomaly types include typical features such as personnel not wearing safety helmets, equipment damage, and colored gas leaks in the underground environment. These features are obtained by extensively annotating historical experience images. Coarse matching is a preliminary, rapid, but relatively loose matching process between the standard production features and standard anomaly features in the region of interest. Multiple coarse matching results are obtained for each standard production feature. If a standard production feature does not match a coarse matching result, the unit region containing that standard production feature is reported to notify personnel for inspection.

[0046] In the fine matching unit, the anomaly relationship graph is a graph structure data model used to represent the relationship between standard production features and coarse matching results. Each edge in the anomaly relationship graph is an edge with a subordinate relationship. For example, if an edge in the anomaly relationship graph is: target standard production image as master node A → child node standard production feature B, then the subordinate relationship of this edge is B subordinate to A.

[0047] In the integrated unit, all production anomaly features of the target standard production image are combined to obtain the safety production anomalies of the target standard production image.

[0048] In one specific embodiment, the coarse matching unit includes: Spatial grid sub-units are used to spatially grid each standard production feature in the region of interest to obtain a simplified feature vector for each standard production feature. Each standard anomaly feature is then vectorized to obtain an anomaly feature vector for each standard anomaly feature. The distance calculation subunit calculates the distance between each simplified feature vector and each abnormal feature vector. If the distance between any simplified feature vector and any abnormal feature vector is less than the coarse matching threshold, it is determined that the standard production feature corresponding to the simplified feature vector and the standard abnormal feature corresponding to the abnormal feature vector have successfully matched coarsely. Multiple coarse matching results are generated for each standard production feature based on the standard abnormal features that have successfully matched coarsely.

[0049] Specifically, in the spatial grid division sub-unit, when determining the simplified feature vector of a standard production feature, the standard production feature is spatially gridded to obtain multiple small grid regions. Feature values ​​of various features of the pixel values ​​in each small grid region are calculated, and the average of the feature values ​​of each feature across all small grid regions is calculated. The average of each feature constitutes the simplified feature vector of the standard production feature. When determining the abnormal feature vector of a standard abnormal feature, the standard abnormal feature is numerically converted into the numerical values ​​of each feature, and these values ​​are combined to form the abnormal feature vector. For example, if the temperature of a standard abnormal feature is 50 degrees Celsius and the illumination is 200 lux, then the abnormal feature vector of this standard abnormal feature is [50, 200].

[0050] In the distance calculation subunit, the distance between the simplified feature vector and the outlier feature vector is represented by Euclidean distance. The coarse matching threshold is an empirically determined distance threshold for coarse matching. When generating multiple coarse matching results for each standard production feature based on the standard outlier features that have successfully coarsely matched, all the standard outlier features that have successfully coarsely matched are combined to obtain multiple coarse matching results for each standard production feature.

[0051] In one specific embodiment, the fine matching unit includes: An anomaly relationship graph construction subunit is used to construct an anomaly relationship graph with the target standard production image as the main node, each standard production feature under the target standard production image as a child node, and each coarse matching result under each standard feature as a child node attribute. The association strength calculation subunit is used to process the abnormal relationship graph through a pre-trained graph neural network to obtain the feature association between each standard production feature and its corresponding coarse matching result in the abnormal relationship graph, and calculate the association strength between each standard production feature and each coarse matching result based on the feature association and the preset feature weight of each standard production feature. The matching subunit is used to select the coarse matching result with the strongest correlation from each standard production feature as the production anomaly feature for each standard production feature.

[0052] Specifically, in the correlation strength calculation subunit, the anomaly relationship graph is input into a pre-trained graph neural network, which then outputs the feature correlation between each standard production feature in the anomaly relationship graph and its corresponding coarse matching result. The graph neural network is trained using multiple historical anomaly relationship graphs.

[0053] When calculating the correlation strength S between a standard production feature and any coarse matching result based on feature correlation and a preset feature weight of a certain standard production feature, it is achieved through formula (2): (2); In formula (2), T represents the feature correlation between the standard production feature and the coarse matching result, and w represents the preset weight of the standard production feature. Each standard production feature has a preset weight, which is proportional to the area of ​​the region of interest in which each standard production feature is located.

[0054] In one specific embodiment, the result generation module includes: Anomaly type analysis unit is used to determine the safety identification anomaly type of each standard production image based on the safety production anomalies of each standard production image. The location determination unit is used to generate the location of the safety identification anomaly type corresponding to each production anomaly feature based on the category label of the unit region where each production anomaly feature is located in each standard production image; The result generation unit is used to combine the timestamp of each security identification anomaly type of each standard production image, each security identification anomaly type and its occurrence location, and the environmental parameters corresponding to the timestamp to form the security intelligent analysis result of the production video.

[0055] Specifically, when determining the safety identification anomaly type for each standard production image based on safety production anomalies, the anomaly type analysis unit matches a specific production anomaly feature within the safety production anomaly with preset features for all safety identification anomaly types. If a match is successful, the corresponding safety identification anomaly type is determined. These preset features are determined based on historical experience values. When matching a production anomaly feature with a preset feature for a specific safety identification anomaly type, the Euclidean distance between the two features is calculated. If the Euclidean distance between the two features is greater than 0.5, the match is considered successful.

[0056] In the location determination unit, based on the category label of the unit region containing a certain production anomaly feature in a standard production image, the location of the safety identification anomaly type corresponding to that production anomaly feature is determined as the location within that unit region, and the name of the location is the category label of that unit region. Figure 2 For example, suppose there is a certain production anomaly feature in the red-framed unit area. The location where the production anomaly feature occurs is the location of the red-framed unit area, and the name of the location where the production anomaly feature occurs is "equipment location".

[0057] In the result generation unit, the safety intelligent analysis result is obtained by linguistic logic processing of the timestamp of the safety identification anomaly type, the safety identification anomaly type and its occurrence location, and the environmental parameters corresponding to the timestamp for each standard production image. For example, at the timestamp of 13:50, a safety anomaly occurred in the personnel entry and exit area of ​​the coal mine where personnel were not wearing safety helmets. The environmental parameters at the current time are a temperature of 35 degrees Celsius and a light intensity of 400 lux.

[0058] Based on the above embodiments, the present invention proposes a coal mine safety production video intelligent analysis system, which has the following characteristics: First, production videos and environmental parameters of the coal mine production area are acquired by pre-deploying video acquisition devices and sensors in the coal mine production area, providing a rich data foundation for subsequent feature enhancement strategies based on environmental parameters.

[0059] Next, the production video is converted into a production image sequence through the image alignment module, and the production image sequence is aligned with the environmental parameters in time to obtain a standard production image sequence containing environmental parameters. This realizes the time alignment of the production image sequence with the environmental parameters, providing a time-series data foundation for subsequent environmental parameter analysis of each production image.

[0060] Secondly, by generating feature enhancement strategies based on the environmental parameters of each standard production image in the standard production image sequence, and determining the abnormal regions of interest in each standard production image based on the feature enhancement strategies, a feature enhancement strategy generation mechanism that adapts to environmental parameters is constructed. This can significantly improve the quality of standard production images and the accuracy of abnormal regions of interest localization, providing an accurate foundation for subsequent anomaly analysis.

[0061] In addition, by performing feature matching between the abnormal region of interest of each standard production image and the preset safety production anomaly type, and determining the safety production anomaly of each standard production image based on the matching results, the safety production anomaly of each standard production image can be identified efficiently and accurately, thereby improving the safety of the coal mine production process and the efficiency of anomaly identification.

[0062] Finally, based on the safety anomalies and environmental parameters of each standard production image, the safety intelligent analysis results of the production video are generated, providing inspectors with accurate and convenient safety intelligent analysis results and optimizing the coal mine underground safety intelligent early warning process.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A coal mine safety production video intelligent analysis system, characterized in that, include: The data acquisition module is used to acquire production videos and environmental parameters of the coal mine production area through video acquisition devices and sensors pre-deployed in the coal mine production area. The environmental parameters include multiple types of sub-environmental parameters. The image alignment module is used to convert production video into a production image sequence and align the production image sequence with environmental parameters in time to obtain a standard production image sequence containing environmental parameters. The region of interest extraction module is used to generate a feature enhancement strategy based on the environmental parameters of the target standard production image in the standard production image sequence, and to determine the abnormal region of interest of the target standard production image based on the feature enhancement strategy. The target standard production image is any standard production image in the standard production image sequence. The anomaly analysis module is used to perform feature matching between the abnormal regions of interest in the target standard production image and the preset safety production anomaly types, and to determine the safety production anomalies in the target standard production image based on the matching results. The results generation module is used to generate safety intelligent analysis results for the production video based on the safety production anomalies and their environmental parameters for each standard production image.

2. The intelligent video analysis system for coal mine safety production according to claim 1, characterized in that, The image alignment module includes: An image conversion unit is used to convert production video into a production image sequence using image conversion tools; The parameter alignment unit is used to match the timestamp of each production image in the production image sequence with the timestamp of the environmental parameters. If the timestamp of any production image fails to match, linear interpolation is used to generate interpolated environmental parameters, and the interpolated environmental parameters are matched into the production images to obtain a standard production image sequence consisting of standard production images containing environmental parameters.

3. The intelligent video analysis system for coal mine safety production according to claim 1, characterized in that, The region of interest extraction module includes: The parameter comparison unit is used to compare each type of sub-environmental parameter of the target standard production image with the corresponding preset standard sub-environmental parameter threshold, and calculate the deviation value of each type of sub-environmental parameter of the target standard production image based on the comparison result; The feature enhancement strategy generation unit is used to generate a feature enhancement strategy for the target standard production image based on the deviation value of each type of sub-environmental parameter of the target standard production image. The region of interest segmentation unit is used to perform edge detection on the target standard production image according to the feature enhancement strategy to obtain the abnormal region of interest of the target standard production image.

4. The intelligent video analysis system for coal mine safety production according to claim 3, characterized in that, The feature enhancement strategy generation unit includes: The region identification subunit is used to input the target standard production image into the feature recognition big data model, and the feature recognition big data model outputs several unit regions of the target standard production image and the category label of each unit region. The strategy generation sub-unit is used to generate a sub-feature enhancement strategy for each unit region based on the deviation value of each type of sub-environmental parameter of the target standard production image and a preset strategy lookup table, and all sub-feature enhancement strategies constitute the feature enhancement strategy of the target standard production image.

5. The intelligent video analysis system for coal mine safety production according to claim 4, characterized in that, The region of interest partitioning unit includes: The grayscale conversion sub-unit is used to process each unit region of the target standard production image according to its sub-feature enhancement strategy to obtain a sub-grayscale production image of each unit region; Alignment sub-units are used to align the target standard production image with the preset standard image pixel by pixel, and to find the standard unit region corresponding to each sub-grayscale production image in the preset standard image. The contour extraction subunit is used to calculate the grayscale difference between each sub-grayscale production image and its corresponding standard unit region. Sub-grayscale production images with grayscale differences greater than a preset grayscale difference threshold are taken as difference unit regions. The edge points of each difference unit region are extracted according to the edge recognition algorithm. The edge points of each difference unit region are smoothly connected to obtain all feature contours of the target standard production image. The region of interest extraction subunit is used to determine each standard production feature of the target standard production image based on each feature contour of the target standard production image, and to synthesize all standard production features of the target standard production image to obtain the region of interest.

6. The intelligent video analysis system for coal mine safety production according to claim 5, characterized in that, The anomaly analysis module includes: The coarse matching unit is used to coarsely match each standard production feature in the region of interest with each standard anomaly feature of the preset safety production anomaly type, and obtain multiple coarse matching results for each standard production feature. The fine matching unit is used to construct an anomaly relationship map based on the anomaly region of interest and all coarse matching results of each standard production feature. Based on the anomaly relationship map, it calculates the correlation strength between each coarse matching result with the target standard production image as the main node and each standard production feature, and obtains the production anomaly feature of each standard production feature based on the correlation strength. The integration unit is used to integrate all production anomaly features to form a target standard production image of safety production anomalies.

7. The intelligent video analysis system for coal mine safety production according to claim 6, characterized in that, The coarse matching unit includes: Spatial grid sub-units are used to spatially grid each standard production feature in the region of interest to obtain a simplified feature vector for each standard production feature. Each standard anomaly feature is then vectorized to obtain an anomaly feature vector for each standard anomaly feature. The distance calculation subunit calculates the distance between each simplified feature vector and each abnormal feature vector. If the distance between any simplified feature vector and any abnormal feature vector is less than the coarse matching threshold, it is determined that the standard production feature corresponding to the simplified feature vector and the standard abnormal feature corresponding to the abnormal feature vector have successfully matched coarsely. Multiple coarse matching results are generated for each standard production feature based on the standard abnormal features that have successfully matched coarsely.

8. The intelligent video analysis system for coal mine safety production according to claim 6 or 7, characterized in that, The fine matching unit includes: An anomaly relationship graph construction subunit is used to construct an anomaly relationship graph with the target standard production image as the main node, each standard production feature under the target standard production image as a child node, and each coarse matching result under each standard feature as a child node attribute. The association strength calculation subunit is used to process the abnormal relationship graph through a pre-trained graph neural network to obtain the feature association between each standard production feature and its corresponding coarse matching result in the abnormal relationship graph, and calculate the association strength between each standard production feature and each coarse matching result based on the feature association and the preset feature weight of each standard production feature. The matching subunit is used to select the coarse matching result with the strongest correlation from each standard production feature as the production anomaly feature for each standard production feature.

9. The intelligent video analysis system for coal mine safety production according to claim 6, characterized in that, The result generation module includes: Anomaly type analysis unit is used to determine the safety identification anomaly type of each standard production image based on the safety production anomalies of each standard production image. The location determination unit is used to generate the location of the safety identification anomaly type corresponding to each production anomaly feature based on the category label of the unit region where each production anomaly feature is located in each standard production image; The result generation unit is used to combine the timestamp of each security identification anomaly type of each standard production image, each security identification anomaly type and its occurrence location, and the environmental parameters corresponding to the timestamp to form the security intelligent analysis result of the production video.

Images