Safety determination method for coal mine transportation and safety determination system for coal mine transportation

Abstract

This application provides a method and system for determining the safety of coal mine transportation. The method includes: acquiring video information, which is video information of a target area captured by a video acquisition device; acquiring positioning information, including vehicle location information and pedestrian location information collected by a UWB positioning card; constructing a first model and a second model based on the video information and positioning information respectively, and calculating a weighted average of the output results of the first model and the second model to obtain the target result. This method automates the determination of driving safety and employs multi-source data fusion, thus avoiding the inaccuracies of relying on single data sources. Therefore, the safety detection in this solution has high reliability.

Description

Technical Field

[0001] This application relates to the field of coal mine safety detection technology, and more specifically, to a method, apparatus, computer-readable storage medium, and system for determining the safety of coal mine transportation. Background Technology

[0002] Coal is my country's primary energy source, accounting for over 60% of primary energy consumption. Safe, healthy, and efficient coal mine production is crucial for ensuring my country's energy security and promoting the stable development of the national economy. In recent years, with the widespread application of advanced information technology in coal mines and the government's emphasis on intelligent mine construction, the level of intelligence in my country's mines has been greatly improved. With the continuous implementation and upgrading of mine informatization and automation technologies, applications such as precise positioning, wireless communication, and vehicle-mounted video have laid the information foundation for more comprehensive, accurate, and real-time perception of personnel, vehicles, roads, and materials in underground coal mine auxiliary transportation systems. To avoid safety accidents such as collisions and abrasions between personnel and mine cars, the coal mining industry has established a transportation management system where vehicles are allowed to pass but not personnel. However, in actual coal mine production, there is still a reliance on the driver's initiative and the passive manual monitoring and identification by safety officers, which can lead to uncontrollable safety risks to some extent. Therefore, the reliability of safety monitoring for underground coal mine auxiliary transportation systems in current solutions is relatively low. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, computer-readable storage medium, and system for determining the safety of coal mine transportation, so as to at least solve the problem of low reliability of safety monitoring in underground coal mine auxiliary transportation systems in the prior art.

[0004] To achieve the above objectives, according to one aspect of this application, a method for determining the safety of coal mine transportation is provided, comprising: acquiring video information, wherein the video information is video information of a target area acquired by a video acquisition device, the target area being located in the direction of vehicle travel, and the minimum distance between the target area and the vehicle remaining constant; acquiring positioning information, wherein the positioning information includes information on the location of the vehicle acquired by a UWB positioning card and information on the location of pedestrians; constructing a first model and a second model based on the video information and the positioning information respectively, and calculating a weighted average of the output results of the first model and the output results of the second model to obtain a target result, wherein the first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the positioning information, and the target result is used to determine whether coal mine transportation is safe.

[0005] Optionally, before calculating the weighted average of the output results of the first model and the second model to obtain the target result, the method further includes: constructing an initial first model, wherein the initial first model is trained using multiple sets of training data, each set of training data including: historical image information and the historical area occupied by historical pedestrians in the historical image information acquired within a historical time period, wherein the historical image information is any frame in historical video information; inputting the image information into the initial first model to obtain the initial area corresponding to the image information; constructing a second model, wherein the second model is trained using multiple sets of training data, each set of training data including: historical location information and the historical distance between historical vehicles and historical pedestrians corresponding to the historical location information acquired within a historical time period; inputting the location information into the second model to obtain the distance corresponding to the location information.

[0006] Optionally, the area range of the historical area and the distance range of the historical distance correspond one-to-one. Before calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result, the method further includes: if the area range of the initial area does not correspond to the distance range, obtaining the target area range of the initial area corresponding to the distance range; inputting the target area range into the initial first model, and retraining the initial first model until the area range of the initial area corresponds to the distance range, thereby obtaining the first model.

[0007] Optionally, before constructing the second model, the method further includes: obtaining a first timestamp of the ranging request frame sent by the UWB positioning card of the vehicle; obtaining a second timestamp of the ranging response frame received by the UWB positioning card of the vehicle from the UWB base station; and calculating the distance between the vehicle and the UWB base station based at least on the first timestamp and the second timestamp.

[0008] Optionally, calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result includes: analyzing the video quality of the video information according to the PSNR avg.MSE algorithm to obtain a first evaluation score; analyzing the signal strength of the positioning information according to the RSS algorithm to obtain a second evaluation score; determining the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model according to the first evaluation score and the second evaluation score; and calculating the sum of the first product and the second product to obtain the target result, wherein the first product is the product of the first output result of the first model and the first weight coefficient, and the second product is the product of the second output result of the second model and the second weight coefficient.

[0009] Optionally, determining a first weight coefficient for the first output result of the first model and a second weight coefficient for the second output result of the second model based on the first evaluation score and the second evaluation score respectively includes: determining the first weight coefficient based on the first evaluation score and determining the second weight coefficient based on the second evaluation score, wherein the first evaluation score and the first weight coefficient are positively correlated, the second evaluation score and the second weight coefficient are positively correlated, and the magnitude relationship between the first weight coefficient and the second weight coefficient is related to a first difference and a second difference, wherein the first difference is the difference between the first evaluation score and a first score threshold, and the second difference is the value between the second evaluation score and a second score threshold; when the first difference is greater than the second difference, determining that the first weight coefficient is greater than the second weight coefficient; when the first difference is less than the second difference, determining that the first weight coefficient is less than the second weight coefficient; and when the first difference is equal to the second difference, determining that the first weight coefficient is equal to the second weight coefficient.

[0010] Optionally, after calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result, the method further includes: determining that coal mine transportation is safe if the target result is greater than or equal to a preset threshold; and determining that coal mine transportation is unsafe and generating alarm information if the target result is less than the preset threshold.

[0011] According to another aspect of this application, a safety determination device for coal mine transportation is provided, comprising: a first acquisition unit for acquiring video information, wherein the video information is video information of a target area acquired by a video acquisition device, the target area being located in the direction of vehicle travel, and the minimum distance between the target area and the vehicle remaining constant; a second acquisition unit for acquiring positioning information, wherein the positioning information includes information on the location of the vehicle acquired by a UWB positioning card and information on the location of pedestrians; and a first determination unit for constructing a first model and a second model based on the video information and the positioning information, respectively, and calculating a weighted average of the output results of the first model and the output results of the second model to obtain a target result, wherein the first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the positioning information, and the target result is used to determine whether coal mine transportation is safe.

[0012] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the aforementioned coal mine transportation safety determination methods.

[0013] According to another aspect of this application, a safety determination system for coal mine transportation is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the aforementioned safety determination methods for coal mine transportation.

[0014] The technical solution of this application determines the safety of the driving process through automation, and adopts a multi-source data fusion approach to determine the safety of the driving process. This can also avoid the inaccuracy of judgment based on single data. Therefore, the safety detection in this solution has high reliability. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for performing a safety determination method for coal mine transportation, provided in an embodiment of this application, is shown.

[0017] Figure 2 A flowchart illustrating a method for determining the safety of coal mine transportation according to an embodiment of this application is shown.

[0018] Figure 3 A schematic diagram of the two-stage data fusion process of this scheme is shown;

[0019] Figure 4 A schematic diagram of the training process for a machine learning-based model is shown.

[0020] Figure 5 A flowchart illustrating the adaptive fusion algorithm of this scheme is shown;

[0021] Figure 6 A structural block diagram of a safety determination device for coal mine transportation provided according to an embodiment of this application is shown.

[0022] The above figures include the following reference numerals:

[0023] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0028] Active learning: During the learning process, the learner selects unlabeled samples and requests labeling information from the outside world. Its goal is to achieve good learning performance with as few queries as possible.

[0029] Semi-supervised learning is a learning method that combines supervised and unsupervised learning. It uses a large amount of unlabeled data, along with labeled data, to perform pattern recognition tasks. When using semi-supervised learning, it requires as few people as possible to perform the work, while still achieving relatively high accuracy.

[0030] In today's data-driven information age, traditional methods cannot effectively utilize accumulated data to improve regulatory quality, nor can they achieve real-time monitoring and early warning.

[0031] In recent years, deep learning (DL) has achieved significant research results in various fields such as image recognition, machine translation, and natural language processing (NLP). Consequently, an increasing number of safety production scenarios are adopting deep learning-based computer vision technology for anomaly detection. Compared to the drawbacks of manual inspection, such as high cost, low efficiency, susceptibility to environmental influences, and inconsistent detection standards, as well as the shortcomings of traditional machine vision, such as poor anti-interference capabilities, low accuracy, and difficulty in optimization, deep learning-based intelligent inspection algorithms are more adaptable, more accurate, reusable, and easier to iterate. However, high-precision deep learning models generally require a large amount of training data, necessitating extensive data annotation by a large number of personnel in practical applications. In the complex environment of underground coal mines, the complexity of the environment and the difficulty in data acquisition mean that deep learning-based image recognition safety monitoring models cannot achieve ideal detection accuracy in practical applications, greatly limiting the practical application of this method.

[0032] Meanwhile, the rise of Ultra Wide Band (UWB) positioning technology has elevated positioning accuracy to new heights. This technology boasts advantages such as strong anti-interference performance and high penetration capability, making it more suitable for the complex positioning environment of underground coal mines. Furthermore, numerous scholars and application manufacturers are continuously developing wireless positioning algorithms in practice to meet the high-precision positioning requirements. UWB technology has been implemented in several smart mine projects, providing excellent data conditions for personnel and vehicle positioning. However, in practical use, UWB technology is prone to positioning drift and decreased accuracy due to signal interference, affecting the operational efficiency of safety detection systems.

[0033] In practical applications, neither of these two methods alone can effectively achieve safety monitoring of pedestrians and vehicles in auxiliary transportation systems. Furthermore, the algorithmic principles of deep learning-based intelligent inspection algorithms and UWB positioning-based safety monitoring methods differ significantly, resulting in poor fusion of the two methods and failing to compensate for their respective shortcomings. Therefore, to better meet the safety management requirements of pedestrian and vehicle traffic in conjunction with the development of next-generation information technology, it is necessary to acquire information from multiple dimensions such as positioning, images, and videos, and to better and more fully integrate data from different dimensions through multi-stage intelligent fusion algorithms, thereby improving the robustness of the monitoring algorithm. This endows machines with the ability to learn and fuse signals from these multiple domains, enabling the complementarity of various heterogeneous information and ultimately improving the accuracy of underground safety detection models in coal mines.

[0034] In the traditional field of ground transportation, pedestrian detection, as an important branch of object detection, has achieved significant breakthroughs in both theory and application after years of research by experts and scholars. Current pedestrian detection algorithms can be divided into two categories: those based on traditional manual features and those based on deep learning. These two main categories are based on traditional image processing and deep learning. In recent years, with the significant increase in computer computing speed, deep learning-based methods have achieved increasingly higher detection speeds and accuracy, leading to their wider application in pedestrian detection. Two-stage object detection algorithms, such as Fast R-CNN and Faster R-CNN, have laid the foundation for subsequent derivative networks. Some solutions improve upon the classic Fast R-CNN detection framework by constructing a multi-scale adaptive joint discriminant network. This first distinguishes the pedestrian scale, then builds corresponding detection sub-networks based on different scales, and integrates the large-scale and small-scale pedestrian detection frameworks, optimizing the network's ability to detect pedestrians at multiple scales.

[0035] Traditional road pedestrian detection systems and the rapidly developing intelligent driving systems utilize training images with good lighting conditions and large sample sizes during model building. However, underground coal mines face complex production conditions, weak underground lighting, high humidity in some areas, and a limited number of effective samples. Furthermore, data collection and labeling are difficult, making it impossible to establish large-scale datasets for training reliable models. Therefore, the application of deep learning-based video analysis algorithms in underground coal mine environments is severely limited, and their effectiveness in practical applications is not significant.

[0036] To overcome the interference of the complex underground coal mine environment on commonly used visual models, high-definition infrared cameras can be used to acquire and recognize video images. However, infrared cameras can only be installed in fixed locations, have a small coverage area, and are expensive to purchase. They do not effectively utilize the precise positioning information and vehicle-mounted video that have been rapidly developed and deployed in various underground coal mines in recent years, and they do not use positioning algorithms and deep learning. If a high-definition infrared camera solution is adopted for large-scale application to better meet the needs of vehicle inspection, it will result in significant user costs and construction time.

[0037] Some solutions employ data acquisition and fusion for precise positioning of underground vehicles. However, this method requires extensive annotation of point cloud data, incurring significant manpower costs and presenting considerable annotation challenges. Secondly, this method directly fuses information from multiple sensors, including lidar, cameras, wheel speed odometers, and ultra-wideband positioning sensors, without further considering the properties and application scenarios of different modalities. Consequently, the fusion effect is poor in practical use, failing to truly achieve the complementary and compatible benefits of multimodal data, thus limiting its application in underground production environments.

[0038] As described in the background section, the reliability of safety monitoring in existing underground coal mine auxiliary transportation systems is low. To address the above problems, embodiments of this application provide a method, apparatus, computer-readable storage medium, and system for determining the safety of coal mine transportation.

[0039] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0040] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a method of determining the safety of coal mine transportation according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0041] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the device information display method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] This embodiment provides a method for determining the safety of coal mine transportation that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] Figure 2 This is a flowchart illustrating a method for determining the safety of coal mine transportation according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0044] Step S201: Obtain video information, wherein the video information is the video information of the target area collected by the video acquisition device, the target area is located in the direction of travel of the vehicle, and the minimum distance between the target area and the vehicle remains unchanged.

[0045] Specifically, the video capture device can be installed on the vehicle, allowing it to capture video of the target area and thus obtain video information about the vehicle's direction of travel.

[0046] Specifically, video capture devices can be cameras, video recorders, and so on.

[0047] Step S202: Obtain location information, wherein the location information includes the location information of the vehicle and the location information of the pedestrian collected by the UWB positioning card;

[0048] Specifically, vehicles are equipped with UWB devices, and pedestrians wear UWB identification cards. UWB precise positioning technology can be used to determine the location information of both vehicles and pedestrians. This location information can be determined by sending signals to a base station.

[0049] Step S203: Construct a first model and a second model based on the video information and the location information, respectively, and calculate the weighted average of the output results of the first model and the second model to obtain the target result. The first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the location information, and the target result is used to determine whether coal mine transportation is safe.

[0050] Specifically, a first model can be constructed based on video information, and a second model can be constructed based on location information. Then, the safety of coal mine transportation can be preliminarily determined based on the first model to obtain a first preliminary detection result. The safety of coal mine transportation can be preliminarily determined based on the second model to obtain a second preliminary detection result. Finally, the two preliminary detection results can be combined to determine whether the safety of coal mine transportation is finalized.

[0051] Specifically, if both the first and second preliminary detection results indicate unsafe coal mine transportation, then the target result is also unsafe coal mine transportation. Conversely, if both the first and second preliminary detection results indicate safe coal mine transportation, then the target result is also safe coal mine transportation. Alternatively, coal mine transportation safety can be determined by comparing the target results obtained from the two models with preset thresholds. If both the first and second preliminary detection results indicate safe coal mine transportation, then the target result is also unsafe coal mine transportation.

[0052] This embodiment uses an automated method to determine the safety of the driving process, and it employs a multi-source data fusion approach to determine the safety of the driving process. This avoids the inaccuracy of judging based on single data, so the safety detection in this solution has high reliability.

[0053] In the specific implementation process, before calculating the weighted average of the output results of the first model and the second model to obtain the target result, the method further includes the following steps: constructing an initial first model, wherein the initial first model is trained using multiple sets of training data, each set of training data includes historical image information and the historical area occupied by historical pedestrians in the historical image information acquired within a historical time period, wherein the historical image information is any frame in the historical video information; inputting the image information into the initial first model to obtain the initial area corresponding to the image information; constructing the second model, wherein the second model is trained using multiple sets of training data, each set of training data includes historical location information and the historical distance between historical vehicles and historical pedestrians corresponding to the historical location information acquired within a historical time period; inputting the location information into the second model to obtain the distance corresponding to the location information.

[0054] In this scheme, an initial first model and a second model can be constructed first. The initial first model can determine the area occupied by pedestrians in the target region image. The closer the pedestrian is to the vehicle, the larger the area occupied. However, since the initial first model is affected by the environment in the coal mine, it can be trained again. The distance between the vehicle and the pedestrian can be determined based on the second model. Since the second model is trained based on UWB precise localization, its accuracy is also high. Therefore, the initial first model and the precise second model can be used to determine the distance between the vehicle and the pedestrian. Then, different models can be used to initially determine whether coal mine transportation is safe, so as to ensure that this scheme can be further fused and detected based on different retrieval results, so as to improve the detection accuracy.

[0055] Specifically, a first model constructed using deep learning algorithms can be used to detect personnel targets. After identifying personnel, the model is analyzed in conjunction with the vehicle's real-time speed and the size of the personnel image. When the speed exceeds a preset threshold, an audible and visual alarm can be used to alert the driver (i.e., to initially determine whether coal mine transportation is safe). This first model is constructed using deep learning algorithms such as ResNet-34, which can effectively balance accuracy and speed. Different scales of detection are achieved by adding a Deep Layer Aggregation (DLA) structure to the backbone network. The first model can be an existing artificial intelligence model, such as a neural network model or a convolutional network model.

[0056] For the second model, the distance between the vehicle and the person can be determined based on the vehicle's location information and the person's location information. The vehicle has a UWB device, and the person also carries a UWB identification card. In this way, the distance between the vehicle and the person can be determined based on the location of the UWB device and the location of the UWB identification card.

[0057] Specifically, this solution is divided into two main intelligent fusion stages. These two stages utilize location-aided annotation and reliability-based intelligent fusion, respectively, to fuse video and location information from two different modalities. By analyzing the different properties and applicable environments of visual analysis and precise positioning technologies, conventional fusion techniques cannot effectively fuse information from these two different modalities. The two-stage intelligent fusion technology can achieve more accurate information fusion based on the different properties of the information, meeting the requirements for accuracy and robustness of security monitoring results under practical application conditions, while reducing the cost of related model training and deployment. The two main intelligent fusion stages are described below.

[0058] To improve the accuracy of the first model and thus enhance the accuracy of subsequent security detection, the area range of the historical area and the distance range of the historical distance are in one-to-one correspondence. Before calculating the weighted average of the output results of the first model and the second model to obtain the target result, the method further includes the following steps: when the area range of the initial area does not correspond to the distance range, obtain the target area range of the initial area that corresponds to the distance range; input the target area range into the initial first model, and retrain the initial first model until the area range of the initial area corresponds to the distance range, thus obtaining the first model.

[0059] In this scheme, a second model can be used to adjust the first model. Since the initial first model is affected by the environment in the coal mine during detection, such as exposure or dust, it will lead to inaccurate detection. The second model is based on UWB for positioning detection, which is a real-time detection process. Therefore, the accuracy of the second model is relatively high. The second model can be used to update the initial first model until an accurate first model is obtained. This ensures that the accuracy of the first model is high, and the accuracy of the results output by the first model is also high, thereby ensuring that the accuracy of the subsequent target results is high.

[0060] Specifically, such as Figure 3As shown, the first-stage fusion is mainly a training process for the fusion of vision and localization based on active learning and semi-supervised learning. It involves two systems: an intelligent vision system and a precise localization system. The intelligent vision system includes a first model (person and vehicle detection model) (which also includes video capture equipment for shooting videos), and the precise localization system includes a second model (distance calculation model based on localization data) (which also includes localization base stations). From a software perspective, the first-stage fusion also includes a data feedback module and a model training module.

[0061] In the data feedback module, location information and video information can be fused for selecting training videos. A major challenge in modeling safety detection for underground coal mine cranes lies in the limited amount of abnormal data, while the training process of deep learning algorithms largely depends on data accumulation. Previous methods primarily relied on manual annotation by personnel, which was not only time-consuming and labor-intensive but also couldn't guarantee annotation accuracy. Therefore, this solution addresses this issue by designing a data feedback module based on location data fusion. This module can automatically select and accumulate abnormal data, reducing manual costs, and can also perform preliminary fusion of location and video information for model training. This module employs a dual-threshold strategy, designing two different thresholds for the first and second models (a high threshold is defined as a probability of detecting pedestrians greater than 97%, and a low threshold as a probability of detecting pedestrians greater than 90%). The high threshold ensures a false alarm rate of ≤3% and is used for online operation, minimizing false alarms while maintaining the detection rate. The high threshold is also used for two-stage fusion and adaptive data fusion to obtain security detection results. The low threshold ensures a false alarm rate of ≤10%. When either the first or second model detects video segments exceeding the low threshold, the system collects suspected abnormal data (data acquisition) for subsequent model iterations.

[0062] Data accumulation and re-entry techniques based on fused precise positioning information address the challenge that conventional methods cannot effectively accumulate large amounts of training data due to the low frequency of outlier data. These techniques not only automatically accumulate relevant data but also integrate positioning information into the initial model training, achieving preliminary data fusion.

[0063] Of the data obtained through the data feedback module, only a small portion exhibits obvious anomalies; the majority is difficult to distinguish and requires annotation. However, the requirements for labeled data vary across different scenarios, resulting in high costs. To save on annotation costs and efficiently utilize the feedback data for data iteration, video clips are extracted from a video library and fused with location information for training. Active learning and semi-supervised learning are introduced during model training, and a single-scale dehazing algorithm is used to reduce interference from moisture and improve model robustness. The model training process is as follows: Figure 4 As shown:

[0064] a. Active Learning: To address the labeling problem of massive datasets, the concept of active learning is introduced. Before labeling, a batch of data that is relatively helpful for subsequent model iterations is selected from the massive dataset through active learning. This batch of data is then labeled (manual labeling to obtain labeled samples) to reduce costs and save time. Several commonly used strategies are first compared using a small amount of data, such as: uncertainty sampling, query-by-committee, and expected model change. After confirming the appropriate strategy, it is then used for training.

[0065] b. Semi-supervised learning: This method utilizes a small number of labeled samples selected through active learning, combined with a large number of unlabeled samples, to improve the algorithm's capabilities and continue model training, such as with a soft teacher. After the model's performance improves, active learning will have a higher probability of selecting important data, thus iterating the model more efficiently, forming a closed loop, and finally, the first model can be deployed online.

[0066] By combining active learning and semi-supervised learning, the detection accuracy of the first model can be significantly improved by requiring only a small amount of annotation on the image information of the scene to be distinguished by the safety management personnel of coal mine enterprises, thus reducing the difficulty of deploying related models.

[0067] Based on the fusion of active learning and semi-supervised learning, the first model can be adjusted according to the real-time changes in the underground coal mine environment, and the positioning information of the second model can be initially fused. This provides relevant information for data collection and annotation for deep learning, reduces the cost of deep learning development and deployment, and ensures the widespread application of the model in underground coal mine scenarios.

[0068] This coal mine underground visual monitoring training technology, which combines active learning and semi-supervised learning, addresses the challenges of difficult data collection and annotation in underground coal mine environments. By combining active learning and semi-supervised learning, relevant data can be automatically collected, and a relatively reliable first and second model can be obtained based on a very small amount of manual annotation results.

[0069] The method for determining the distance between the vehicle and the UWB base station described above can also be implemented in other ways. For example, before constructing the second model described above, the method further includes the following steps: obtaining a first timestamp of the ranging request frame sent by the UWB positioning card of the vehicle; obtaining a second timestamp of the ranging response frame received by the UWB positioning card of the vehicle from the UWB base station; and calculating the distance between the vehicle and the UWB base station based at least on the first timestamp and the second timestamp.

[0070] In this scheme, the distance between the vehicle and the base station can be determined based on the one-way flight time of the signal. The flight time of the signal can be determined based on the timestamp of the data sent by the UWB positioning card and the timestamp of the data received by the UWB positioning card. Then, the flight time can be multiplied by the speed of light propagation (or the speed of signal propagation) to obtain the distance between the vehicle and the base station. This allows for a simple and direct determination of the distance between the vehicle and the base station.

[0071] Specifically, the distance between pedestrians and base stations can be determined based on the flight time of the signal between the pedestrian's UWB positioning card and the base station. After obtaining the distance between pedestrians and base stations, as well as the time distance between vehicles and base stations, a triangular relationship is obtained, which can then be used to determine the distance between vehicles and pedestrians.

[0072] Specifically, the distance between the vehicle and the UWB base station can be determined by the phase difference between the first and second timestamps and the speed of light propagation.

[0073] Specifically, the above scheme uses a one-way time-of-flight model for distance calculation. Of course, a two-way time-of-flight model can also be used to calculate the distance from location information uploaded by the UWB precise positioning system. Here, T is first defined... SP The timestamp for sending the ranging request frame to the vehicle identification card (UWB positioning card), T RP T is the timestamp for the ranging request frame received by the UWB base station. SR The timestamp for the ranging response frame sent by the UWB base station, T RR T is the timestamp for the ranging response frame received by the vehicle identification card. SF Send ranging data frames (T) to the vehicle identification card SP T RP T SR The timestamp, T RF For the base station to receive ranging data frames (T SP T RP T SF (timestamp) It is T SR With T RP Time difference, It is T SF and T RR Due to the time difference, after receiving the ranging data frame sent by the vehicle identification card, the UWB base station calculates the distance according to the following process based on the received data:

[0074] (1) Calculate the time difference between the vehicle identification card sending and receiving ranging frames: T TRT =T RR -T SP ;

[0075] (2) Calculate the time difference between the UWB base station receiving and transmitting ranging frames:

[0076] (3) Calculate the time difference between the ranging frame sent and the data frame received by the UWB base station: T ART =T RF -T SR ;

[0077] (4) Calculate the time difference between the data frame sent by the vehicle identification card and the ranging frame received:

[0078] (5) According to and Calculate the flight time separately for each case to determine if they are equal:

[0079]

[0080] (6) Calculate the distance between the vehicle identification card and the UWB base station: L = T TOF gC, where L is the distance and C is the speed of light.

[0081] The first deep learning-based model and the second precise positioning-based model have different reliability in different scenarios in underground coal mines. In the specific implementation process, the weighted average of the output results of the first and second models is calculated to obtain the target result. This can be achieved through the following steps: Analyzing the video quality of the video information using the PSNR avg.MSE algorithm to obtain a first evaluation score; analyzing the signal strength of the positioning information using the RSS algorithm to obtain a second evaluation score; determining the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model based on the first and second evaluation scores; and calculating the sum of the first and second products to obtain the target result. Here, the first product is the product of the first output result of the first model and the first weight coefficient, and the second product is the product of the second output result of the second model and the second weight coefficient.

[0082] This scheme can calculate the reliability of different models in real time, determine the weights of different models by analyzing video quality and signal strength, and then adaptively fuse the outputs of the two models, thereby improving the overall detection accuracy and reliability of the scheme.

[0083] Specifically, the above embodiments are actually a two-stage fusion process. This two-stage fusion is an adaptive intelligent fusion process based on reliability analysis. During the operation of a single vehicle, the second model, based on the real-time positions of the vehicle and personnel in the three-dimensional space of the coal mine obtained by UWB precise positioning technology, calculates the actual distance between the vehicle and personnel in the adjacent area according to the collected roadway topology. An alarm is triggered when the distance is less than a safety threshold. For example... Figure 5 As shown, after obtaining the security detection and analysis results of the first model and the second model, it is necessary to perform a fusion analysis of the two results. Considering that the detection accuracy of the first model drops significantly in situations with low video quality such as dust, water mist, and low light, while the second model based on UWB precise positioning cannot provide accurate results when the signal is poor, this solution uses an adaptive result fusion method to better integrate the security monitoring results obtained from the two models, thereby improving both overall accuracy and robustness.

[0084] Specifically, such as Figure 5 As shown, video quality analysis algorithms can be used to analyze the video quality of video information, and signal strength detection algorithms can be used to analyze the signal strength of positioning information. The video quality analysis algorithm and signal strength detection algorithm are described below:

[0085] Video quality analysis employed the PSNR avg.MSE algorithm (image evaluation algorithm) to analyze the quality of surveillance videos. PSNR (Peak Signal-to-Noise Ratio) is the ratio of the energy of the peak signal to the average energy of the noise; essentially, it compares the pixel values ​​of two images. The unit of PSNR is dB; a higher value indicates less distortion. The formula is:

[0086]

[0087] MSE represents the mean square error of two m×n monochrome images I and K, where I represents the reference image, K represents the image to be evaluated, and x and y represent the coordinates of pixels in the images. For PSNR avg.MSE, when aggregating the frame-by-frame scores of the entire video, the arithmetic mean of the MSEs is first calculated, and then the logarithm is taken to obtain the first evaluation score. The specific formula is as follows:

[0088]

[0089] Where V represents the original video frame, The table shows the reference frame, where n represents the total number of frames in the video, and MAX represents the maximum color value of an image point. If each sample point is represented by 8 bits, then it is 255.

[0090] Signal strength analysis uses the RSS algorithm (signal strength analysis algorithm) to analyze signal strength. When using the RSS algorithm, the distance between the target and the receiver and the signal strength can be estimated by measuring the energy between nodes. Since the strength of the received signal is inversely proportional to the propagation distance, when the positioning signal is far away from the positioning base station, the strength of the positioning signal weakens, and the accuracy and reliability of the positioning data will also decrease significantly.

[0091] To improve the accuracy of the target results, the reliability and accuracy of the output results of the first model and the second model need to be further determined. Based on the first evaluation score and the second evaluation score, the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model are determined respectively. This can be achieved through the following steps: determine the first weight coefficient based on the first evaluation score, and determine the second weight coefficient based on the second evaluation score. The first evaluation score and the first weight coefficient are positively correlated, and the second evaluation score and the second weight coefficient are positively correlated. The relationship between the first weight coefficient and the second weight coefficient is related to the first difference and the second difference. The first difference is the difference between the first evaluation score and the first score threshold, and the second difference is the difference between the second evaluation score and the second score threshold. If the first difference is greater than the second difference, the first weight coefficient is determined to be greater than the second weight coefficient. If the first difference is less than the second difference, the first weight coefficient is determined to be less than the second weight coefficient. If the first difference is equal to the second difference, the first weight coefficient is determined to be equal to the second weight coefficient.

[0092] In this scheme, since adaptive result fusion technology is used, the reliability of the first output result of the first model and the reliability of the second output result of the second model need to be further determined. Different weight coefficients are determined through different output results, and then adaptive result fusion can be performed. The fusion algorithm can be dynamically adjusted according to the real-time environment in the coal mine, ensuring the accuracy and reliability of the entire scheme, making this scheme more suitable for the actual environment in the coal mine.

[0093] Specifically, in the adaptive result fusion method, for the first video-based model, video quality can be evaluated using a video quality detection algorithm. For the second model based on UWB precise positioning, the signal strength of the positioning information from UWB precise positioning can be collected. Subsequently, the video quality parameters and signal strength are normalized to obtain the weight values ​​(first weight coefficient and second weight coefficient) of the monitoring results of different models, such as... Figure 5 As shown, the monitoring results are fused according to the weight values. The fusion formula is:

[0094]

[0095] Where R represents the fused result, W V Let V(x) represent the first weight coefficient of the first model, and W(x) represent the first output of the first model. D Let R represent the second weight coefficient of the second model, and D(x) represent the second output result of the second model. The final target result is obtained after normalization of R. Furthermore, the weight coefficients can be analyzed. When the weight coefficients of both models are below a safety threshold (which can be 0.5), an alarm can be triggered to remind safety monitoring personnel to conduct manual verification, thereby ensuring good overall detection accuracy and precision.

[0096] Specifically, this solution achieves accurate and reliable safety monitoring of pedestrians and vehicles by deeply integrating the existing video intelligent analysis system and precise positioning system in underground coal mines. This solves the problems of the previous construction of underground auxiliary transportation safety monitoring systems in coal mines, which required a lot of manpower and resources, and had low accuracy and poor reliability.

[0097] In addition, the prompt voice can be differentiated in different situations of two-stage fusion. When the reliability of both the first model and the second model is high, the prompt is "There is a pedestrian ahead"; when the reliability of the second model is high and the reliability of the first model is low, the prompt is "The second model has detected a pedestrian"; otherwise, the prompt is "The video has detected a pedestrian"; when the reliability of both models is low, the prompt is "Manual review".

[0098] Specifically, in this solution, in response to the complex scenario of safety monitoring for vehicles and pedestrians in underground coal mines, where a single information source can easily lead to low accuracy in vehicle safety monitoring, a two-stage intelligent fusion technology is used to more comprehensively integrate precise positioning technology and deep learning technology. This allows for the deployment of a high-precision safety monitoring model based on the existing precise positioning system and vehicle video recording system (video intelligent analysis system) in coal mines.

[0099] The two-stage intelligent fusion model fully integrates and complements visual and positioning data from two different modalities through data accumulation, model training, and reliability-based adaptive fusion. By fusing precise positioning technology and deep learning-based visual detection technology for vehicle and pedestrian safety detection, it achieves dual prevention and protection. The combination of these two technologies effectively records and controls driver behavior, reduces safety risks, improves monitoring robustness, and provides intelligent model applications for the construction of the overall business system. Furthermore, this solution does not require specialized visual sensors such as infrared cameras for the complex environment of underground coal mines, and can be deployed on existing systems in coal mines, reducing deployment costs.

[0100] In some embodiments, after calculating the weighted average of the output results of the first model and the second model to obtain the target result, the method further includes the following steps: if the target result is greater than or equal to a preset threshold, determine that the coal mine transportation is safe; if the target result is less than the preset threshold, determine that the coal mine transportation is unsafe and generate an alarm message.

[0101] In this scheme, the target result can be directly compared with the preset threshold. The safety of coal mine transportation can be determined simply and directly by the relationship between the target result and the preset threshold, without the need for complicated calculations. In the event of unsafe coal mine transportation, alarm information can be generated to promptly remind the driver.

[0102] In addition, this solution can use UWB-based precise positioning technology for safety monitoring of vehicles and pedestrians in coal mines. However, due to the complex structure of underground mine roadways, to avoid the significant reduction in positioning accuracy caused by weak signals in UWB precise positioning, and to improve the overall safety monitoring accuracy and operational efficiency of the coal mine, visual safety monitoring technology based on infrared cameras can be used. Infrared cameras are more suitable for the low-light environment and environments with dust and water mist that interfere with the accuracy of visual sensors in coal mines. However, the deployment cost of infrared cameras is high, requiring additional hardware procurement and deployment, increasing the overall deployment cost.

[0103] This application also provides a safety determination device for coal mine transportation. It should be noted that this safety determination device for coal mine transportation can be used to execute the safety determination method for coal mine transportation provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0104] The following describes the safety determination device for coal mine transportation provided in the embodiments of this application.

[0105] Figure 6 This is a structural block diagram of a safety determination device for coal mine transportation according to an embodiment of this application. Figure 6 As shown, the device includes:

[0106] The first acquisition unit 10 is used to acquire video information, wherein the video information is the video information of the target area acquired by the video acquisition device, the target area is located in the direction of travel of the vehicle, and the minimum distance between the target area and the vehicle remains unchanged.

[0107] The second acquisition unit 20 is used to acquire positioning information, wherein the positioning information includes the location information of the vehicle and the location information of the pedestrian collected by the UWB positioning card.

[0108] The first determining unit 30 is used to construct a first model and a second model based on the video information and the positioning information, respectively, and to calculate the weighted average of the output results of the first model and the output results of the second model to obtain the target result. The first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the positioning information, and the target result is used to determine whether coal mine transportation is safe.

[0109] This embodiment uses an automated method to determine the safety of the driving process, and it employs a multi-source data fusion approach to determine the safety of the driving process. This avoids the inaccuracy of judging based on single data, so the safety detection in this solution has high reliability.

[0110] In the specific implementation process, the above-mentioned device further includes a first construction unit, a first processing unit, a second construction unit, and a second processing unit. The first construction unit is used to construct an initial first model before calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result. The initial first model is trained using multiple sets of training data. Each set of training data includes historical image information and the historical area occupied by historical pedestrians in the historical image information, which is any frame in the historical video information. The first processing unit is used to input the image information into the initial first model to obtain the initial area corresponding to the image information. The second construction unit is used to construct the second model. The second model is trained using multiple sets of training data. Each set of training data includes historical location information and the historical distance between historical vehicles and historical pedestrians corresponding to the historical location information, which is acquired within the historical time period. The second processing unit is used to input the location information into the second model to obtain the distance corresponding to the location information.

[0111] In this scheme, an initial first model and a second model can be constructed first. The initial first model can determine the area occupied by pedestrians in the target region image. The closer the pedestrian is to the vehicle, the larger the area occupied. However, since the initial first model is affected by the environment in the coal mine, it can be trained again. The distance between the vehicle and the pedestrian can be determined based on the second model. Since the second model is trained based on UWB precise localization, its accuracy is also high. Therefore, the initial first model and the precise second model can be used to determine the distance between the vehicle and the pedestrian. Then, different models can be used to initially determine whether coal mine transportation is safe, so as to ensure that this scheme can be further fused and detected based on different retrieval results, so as to improve the detection accuracy.

[0112] To improve the accuracy of the first model and thus enhance the accuracy of subsequent security detection, the area range of the historical area and the distance range of the historical distance are in one-to-one correspondence. The device further includes a third acquisition unit and a third processing unit. The third acquisition unit is used to acquire the target area range of the initial area corresponding to the distance range of the distance when the area range of the initial area does not correspond to the distance range of the distance before calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result. The third processing unit is used to input the target area range into the initial first model and retrain the initial first model until the area range of the initial area corresponds to the distance range of the distance, thus obtaining the first model.

[0113] In this scheme, a second model can be used to adjust the first model. Since the initial first model is affected by the environment in the coal mine during detection, such as exposure or dust, it will lead to inaccurate detection. The second model is based on UWB for positioning detection, which is a real-time detection process. Therefore, the accuracy of the second model is relatively high. The second model can be used to update the initial first model until an accurate first model is obtained. This ensures that the accuracy of the first model is high, and the accuracy of the results output by the first model is also high, thereby ensuring that the accuracy of the subsequent target results is high.

[0114] The method for determining the distance between the vehicle and the UWB base station described above can also be implemented in other ways. For example, the device further includes a fourth acquisition unit, a fifth acquisition unit, and a calculation unit. The fourth acquisition unit is used to acquire the first timestamp of the ranging request frame sent by the UWB positioning card of the vehicle before constructing the second model. The fifth acquisition unit is used to acquire the second timestamp of the ranging response frame sent by the UWB base station received by the UWB positioning card of the vehicle. The calculation unit is used to calculate the distance between the vehicle and the UWB base station based at least on the first timestamp and the second timestamp.

[0115] In this scheme, the distance between the vehicle and the base station can be determined based on the one-way flight time of the signal. The flight time of the signal can be determined based on the timestamp of the data sent by the UWB positioning card and the timestamp of the data received by the UWB positioning card. Then, the flight time can be multiplied by the speed of light propagation (or the speed of signal propagation) to obtain the distance between the vehicle and the base station. This allows for a simple and direct determination of the distance between the vehicle and the base station.

[0116] The first deep learning-based model and the second precise positioning-based model have different reliability in different scenarios in underground coal mines. In the specific implementation process, the first determining unit includes a first processing module, a second processing module, a determining module, and a calculation module. The first processing module is used to analyze the video quality of the above video information according to the PSNR avg.MSE algorithm to obtain a first evaluation score. The second processing module is used to analyze the signal strength of the above positioning information according to the RSS algorithm to obtain a second evaluation score. The determining module is used to determine the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model according to the first evaluation score and the second evaluation score, respectively. The calculation module is used to calculate the sum of the first product and the second product to obtain the target result, wherein the first product is the product of the first output result of the first model and the first weight coefficient, and the second product is the product of the second output result of the second model and the second weight coefficient.

[0117] This scheme can calculate the reliability of different models in real time, determine the weights of different models by analyzing video quality and signal strength, and then adaptively fuse the outputs of the two models, thereby improving the overall detection accuracy and reliability of the scheme.

[0118] To improve the accuracy of the target result, the reliability and accuracy of the output results of the first model and the second model need to be further determined. The determination module includes a first determination submodule and a second determination submodule. The first determination submodule is used to determine the first weight coefficient based on the first evaluation score and the second weight coefficient based on the second evaluation score. The first evaluation score and the first weight coefficient are positively correlated, and the second evaluation score and the second weight coefficient are positively correlated. The relationship between the first weight coefficient and the second weight coefficient is related to the first difference and the second difference. The first difference is the difference between the first evaluation score and the first score threshold, and the second difference is the difference between the second evaluation score and the second score threshold. The second determination submodule is used to determine that the first weight coefficient is greater than the second weight coefficient when the first difference is greater than the second difference, that the first weight coefficient is less than the second weight coefficient when the first difference is less than the second difference, and that the first weight coefficient is equal to the second weight coefficient when the first difference is equal to the second difference.

[0119] In this scheme, since adaptive result fusion technology is used, the reliability of the first output result of the first model and the reliability of the second output result of the second model need to be further determined. Different weight coefficients are determined through different output results, and then adaptive result fusion can be performed. The fusion algorithm can be dynamically adjusted according to the real-time environment in the coal mine, ensuring the accuracy and reliability of the entire scheme, making this scheme more suitable for the actual environment in the coal mine.

[0120] In some embodiments, the above-mentioned device further includes a second determining unit and a third determining unit. The second determining unit is used to determine that coal mine transportation is safe when the target result is greater than or equal to a preset threshold after calculating the weighted average of the output result of the first model and the output result of the second model. The third determining unit is used to determine that coal mine transportation is unsafe when the target result is less than the preset threshold and to generate an alarm message.

[0121] In this scheme, the target result can be directly compared with the preset threshold. The safety of coal mine transportation can be determined simply and directly by the relationship between the target result and the preset threshold, without the need for complicated calculations. In the event of unsafe coal mine transportation, alarm information can be generated to promptly remind the driver.

[0122] The aforementioned safety determination device for coal mine transportation includes a processor and a memory. The first acquisition unit, the second acquisition unit, and the first determination unit are all stored as program units in the memory. The processor executes these program units stored in the memory to achieve their respective functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0123] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the low reliability of safety monitoring in existing underground coal mine auxiliary transportation systems.

[0124] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0125] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the coal mine transportation safety determination method.

[0126] This invention provides a processor for running a program, wherein the program executes the aforementioned method for determining the safety of coal mine transportation.

[0127] This application also provides a safety determination system for coal mine transportation, including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any of the above-described safety determination methods for coal mine transportation.

[0128] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements at least the following steps of a method for determining the safety of coal mine transportation. The device described herein may be a server, PC, PAD, mobile phone, etc.

[0129] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes a method for determining the safety of coal mine transportation, including at least the following steps.

[0130] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0131] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0133] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0136] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0137] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0138] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0139] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0140] 1) The safety determination method for coal mine transportation in this application determines the safety of the driving process through automation and adopts a multi-source data fusion approach to determine the safety of the driving process. This can also avoid the inaccuracy of judgment based on single data. Therefore, the safety detection in this solution has high reliability.

[0141] 2) The safety determination device for coal mine transportation in this application determines the safety of the driving process in an automated manner and adopts a multi-source data fusion method to determine the safety of the driving process. This can also avoid the inaccuracy of single data judgment. Therefore, the safety detection in this solution has high reliability.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining the safety of coal mine transportation, characterized in that, include: Acquire video information, wherein the video information is information about a target area captured by a video acquisition device, the target area is located in the direction of travel of the vehicle, and the minimum distance between the target area and the vehicle remains constant; Acquire location information, wherein the location information includes the location information of the vehicle and the location information of the pedestrian collected by the UWB positioning card; A first model and a second model are constructed based on the video information and the location information, respectively. A weighted average of the outputs of the first model and the second model is calculated to obtain the target result. The first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the location information, and the target result is used to determine whether coal mine transportation is safe. Before calculating the weighted average of the outputs of the first model and the second model to obtain the target result, the method further includes: Construct an initial first model, wherein the initial first model is trained using multiple sets of training data, each set of training data includes: historical image information and historical area occupied by historical pedestrians in the historical image information, which is any frame in historical video information. The image information is input into the initial first model to obtain the initial area corresponding to the image information; Construct the second model, wherein the second model is trained using multiple sets of training data, each set of training data including: historical location information and historical distances between historical vehicles and historical pedestrians corresponding to the historical location information, acquired within a historical time period; The location information is input into the second model to obtain the distance corresponding to the location information. The area range of the historical area and the distance range of the historical distance correspond one-to-one. Before calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result, the method further includes: If the area range of the initial area does not correspond to the distance range of the distance, obtain the target area range of the initial area that corresponds to the distance range of the distance. The target area range is input into the initial first model, and the initial first model is trained again until the area range of the initial area corresponds to the distance range of the distance, thus obtaining the first model.

2. The method according to claim 1, characterized in that, Before constructing the second model, the method further includes: Obtain the first timestamp of the ranging request frame sent by the UWB positioning card of the vehicle; Obtain the second timestamp of the ranging response frame sent by the UWB base station received by the UWB positioning card of the vehicle; The distance between the vehicle and the UWB base station is calculated based at least on the first timestamp and the second timestamp.

3. The method according to claim 1, characterized in that, Calculate the weighted average of the outputs of the first model and the second model to obtain the target result, including: The video quality of the video information is analyzed according to the PSNR avg.MSE algorithm to obtain the first evaluation score; The signal strength of the location information is analyzed using the RSS algorithm to obtain a second evaluation score; Based on the first evaluation score and the second evaluation score, determine the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model, respectively. The sum of the first product and the second product is calculated to obtain the target result, wherein the first product is the product of the first output result of the first model and the first weight coefficient, and the second product is the product of the second output result of the second model and the second weight coefficient.

4. The method according to claim 3, characterized in that, Based on the first evaluation score and the second evaluation score, determine the first weight coefficient of the first output result of the first model and the second weight coefficient of the second output result of the second model, including: The first weight coefficient is determined based on the first evaluation score, and the second weight coefficient is determined based on the second evaluation score. The first evaluation score and the first weight coefficient are positively correlated, and the second evaluation score and the second weight coefficient are positively correlated. The magnitude relationship between the first weight coefficient and the second weight coefficient is related to the first difference and the second difference. The first difference is the difference between the first evaluation score and the first score threshold, and the second difference is the value between the second evaluation score and the second score threshold. If the first difference is greater than the second difference, the first weight coefficient is determined to be greater than the second weight coefficient; if the first difference is less than the second difference, the first weight coefficient is determined to be less than the second weight coefficient; if the first difference is equal to the second difference, the first weight coefficient is determined to be equal to the second weight coefficient.

5. The method according to claim 3, characterized in that, After calculating the weighted average of the outputs of the first model and the second model to obtain the target result, the method further includes: If the target result is greater than or equal to a preset threshold, coal mine transportation safety is determined. If the target result is less than the preset threshold, it is determined that coal mine transportation is unsafe, and an alarm message is generated.

6. A safety determination device for coal mine transportation, characterized in that, include: The first acquisition unit is used to acquire video information, wherein the video information is information about a target area captured by a video acquisition device, the target area is located in the direction of travel of the vehicle, and the minimum distance between the target area and the vehicle remains unchanged. The second acquisition unit is used to acquire positioning information, wherein the positioning information includes the location information of the vehicle and the location information of the pedestrian collected by the UWB positioning card; A first determining unit is configured to construct a first model and a second model based on the video information and the location information, respectively, and calculate a weighted average of the output results of the first model and the second model to obtain a target result. The first model is used to preliminarily determine whether coal mine transportation is safe based on the video information, the second model is used to preliminarily determine whether coal mine transportation is safe based on the location information, and the target result is used to determine whether coal mine transportation is safe. The device further includes a first construction unit, a first processing unit, a second construction unit, and a second processing unit. The first construction unit is used to construct an initial first model before calculating the weighted average of the output results of the first model and the second model to obtain the target result. The initial first model is trained using multiple sets of training data. Each set of training data includes historical image information and the historical area occupied by historical pedestrians within the historical image information, acquired within a historical time period. The historical image information is any frame from historical video information. The first processing unit is used to input the image information into the initial first model to obtain the initial area corresponding to the image information. The second construction unit is used to construct the second model, which is trained using multiple sets of training data. Each set of training data includes historical location information and the historical distance between historical vehicles and historical pedestrians corresponding to the historical location information, acquired within a historical time period. The second processing unit is used to input the location information into the second model to obtain the distance corresponding to the location information. The historical area range and the historical distance range are in one-to-one correspondence. The device further includes a third acquisition unit and a third processing unit. The third acquisition unit is used to acquire the target area range of the initial area corresponding to the distance range when the area range of the initial area does not correspond to the distance range before calculating the weighted average of the output results of the first model and the output results of the second model to obtain the target result. The third processing unit is used to input the target area range into the initial first model and retrain the initial first model until the area range of the initial area corresponds to the distance range, thereby obtaining the first model.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the safety determination method for coal mine transportation as described in any one of claims 1 to 5.

8. A safety determination system for coal mine transportation, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing the safety determination method for coal mine transportation as described in any one of claims 1 to 5.

Images