Method and device for detecting articles carried by personnel riding overhead man-riding device

Abstract

The invention provides a method and a device for detecting articles carried by a person in an overhead man-riding device. The method comprises the following steps: acquiring a target video; a video analysis model is constructed, and the video analysis model is obtained by training through a PP-YOLOE algorithm by using multiple groups of training data; each group of training data in the multiple groups of training data comprises a historical target video acquired in a historical time period, a historical personnel bounding box corresponding to the historical target video and a historical article bounding box corresponding to the historical target video; inputting the target video into a video analysis model to obtain a personnel bounding box and an article bounding box corresponding to the target video; and determining the size of the article according to the article bounding box, and determining that the person carries the illegal article under the condition that the size of the article is greater than or equal to a preset size threshold value. The problem that in the prior art, the accuracy rate of manually judging whether a worker takes an overhead man-riding device to carry a super-long and super-large object is low is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and more specifically, to a method for detecting items carried by a person riding a monkey car, a device for detecting items carried by a person riding a monkey car, a computer program product, and a monkey car detection system. Background Art

[0002] In coal mining, it's common for workers to ride on monkey cars (special rail vehicles used to transport personnel). However, current safety monitoring systems are inadequate for identifying workers carrying oversized or long items. Traditional manual monitoring methods rely on subjective judgment, which can be prone to oversight and misjudgment. Summary of the Invention

[0003] The main purpose of this application is to provide a method for detecting whether a person is carrying items on a monkey cart, a device for detecting whether a person is carrying items on a monkey cart, a computer program product and a monkey cart detection system, so as to at least solve the problem in the prior art of low accuracy in manually judging whether a worker is carrying extra-long or extra-large items on a monkey cart.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for detecting people carrying items on a monkey car is provided, comprising: obtaining a target video, wherein the target video is a video of the operating area of ​​the monkey car in a mine; constructing a video analysis model, wherein the video analysis model is trained by the PP-YOLOE algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical target videos obtained within a historical time period, historical person bounding boxes corresponding to the historical target videos, and historical item bounding boxes corresponding to the historical target videos; inputting the target video into the video analysis model to obtain a person bounding box and an item bounding box corresponding to the target video; determining the size of the item based on the item bounding box, and determining that the person is carrying an illegal item if the size of the item is greater than or equal to a preset size threshold, wherein the preset size threshold and the size corresponding to the person bounding box have a one-to-one correspondence.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a device for detecting people carrying items on a monkey car is provided, comprising: a first acquisition unit, for acquiring a target video, wherein the target video is a video of the operating area of ​​the monkey car in a mine; a first construction unit, for constructing a video analysis model, wherein the video analysis model is trained by the PP-YOLOE algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical target videos acquired within a historical time period, historical person bounding boxes corresponding to the historical target videos, and historical item bounding boxes corresponding to the historical target videos; a first processing unit, for inputting the target video into the video analysis model to obtain a person bounding box and an item bounding box corresponding to the target video; a first determination unit, for determining the size of the item based on the item bounding box, and determining that the person is carrying an illegal item if the size of the item is greater than or equal to a preset size threshold, wherein the preset size threshold and the size corresponding to the person bounding box have a one-to-one correspondence.

[0006] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein a processor executes the steps of any one of the above-mentioned methods for detecting items carried by a person riding a monkey car.

[0007] According to another aspect of the present application, a monkey car detection system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the above-mentioned methods for detecting a person carrying items on a monkey car.

[0008] By applying the technical solution of the present application, when workers ride on the monkey car, a video of the monkey car's operating area is obtained, and a model is constructed using the PP-YOLOE algorithm. The PP-YOLOE algorithm is an optimized target detection algorithm. Therefore, this solution can identify the workers riding the monkey car in the video, as well as the items carried by the workers. If the items are larger than the preset size, it can be determined that the workers are carrying extra-long and extra-large items on the monkey car. Due to the high accuracy of the PP-YOLOE algorithm, accurate identification of people and items can be achieved, thereby making the detection accuracy of this solution higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0010] Figure 1The following is a hardware structure block diagram of a mobile terminal for detecting a person carrying items on a monkey car according to an embodiment of the present application;

[0011] Figure 2 A schematic flow chart of a method for detecting items carried by a person riding a monkey car according to an embodiment of the present application is shown;

[0012] Figure 3 A schematic flow chart of an optional method for detecting items carried by a person riding a monkey car provided in accordance with an embodiment of the present application is shown;

[0013] Figure 4 The figure shows a structural block diagram of a device for detecting items carried by a person riding a monkey car according to an embodiment of the present application.

[0014] The above drawings include the following reference numerals:

[0015] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. DETAILED DESCRIPTION

[0016] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] With the continuous advancement of coal mine safety production, computer vision technology is increasingly being used in the security field. In coal mine production, it is common for workers to ride on monkey cars (special rail vehicles used to transport personnel). However, current safety monitoring systems still have some shortcomings in identifying whether workers are carrying oversized or long items:

[0020] 1. Traditional manual monitoring methods rely on subjective judgment by personnel and are prone to negligence and misjudgment.

[0021] 2. Some existing automated monitoring systems mainly focus on people counting and simple abnormal behavior detection, and cannot accurately identify complex situations of carrying items.

[0022] 3. The coal mine environment is complex and the lighting conditions are poor, which increases the difficulty of visual recognition.

[0023] 4. Existing technologies make it difficult to accurately determine in real time whether the size of an object exceeds the standard, especially in long-range monitoring scenarios.

[0024] The above problems may result in workers carrying extra-long and extra-large items not being discovered in time, thereby causing safety hazards such as jams, collisions and other accidents, seriously threatening the personal safety of miners and the normal operation of coal mines.

[0025] As introduced in the background technology, the accuracy of manual judgment of whether workers are carrying extra-long or extra-large items on monkey carts in the existing technology is low. To solve the above problem, the embodiments of the present application provide a method for detecting whether people are carrying items on monkey carts, a device for detecting whether people are carrying items on monkey carts, a computer program product and a monkey cart detection system.

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for detecting a person carrying items on a monkey car according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0028] The memory 104 can be used to store computer programs, such as application software programs and modules, such as the computer program corresponding to the method for detecting items carried by a person riding a monkey car in the embodiments of the present invention. The processor 102 executes the computer programs stored in the memory 104 to execute various functional applications and data processing, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory 104 may further include memory remotely located from the processor 102, and such remote memory may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or transmit data via a network. Specific examples of such networks may include a wireless network provided by the mobile terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] In this embodiment, a method for detecting items carried by a person riding in a monkey car is provided, which runs on a mobile terminal, a computer terminal or a similar computing device. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] Figure 2 FIG. 1 is a flow chart of a method for detecting items carried by a person riding a monkey car according to an embodiment of the present application. Figure 2 As shown, the method includes the following steps:

[0031] Step S201, obtaining a target video, wherein the target video is a video of the operation area of ​​the monkey car in the mine.

[0032] Specifically, obtaining the target video can help the system accurately locate the operating area of ​​the monkey car, thereby more accurately detecting the situation where people are carrying items in the monkey car. By analyzing the information in the target video, the system can promptly discover and warn people whether they are carrying items in the monkey car, so as to ensure safety and order in the mine, improve the efficiency and accuracy of mine safety management, and effectively avoid potential safety hazards.

[0033] Step S202: construct a video analysis model, wherein the video analysis model is trained using the PP-YOLOE algorithm using multiple sets of training data, each set of training data including historical target videos acquired within a historical time period, historical person bounding boxes corresponding to the historical target videos, and historical object bounding boxes corresponding to the historical target videos.

[0034] PP-YOLOE is an object detection algorithm within the PaddlePaddle framework. It's an upgraded version of PP-YOLO, primarily optimizing the model's efficiency and accuracy. The "E" in PP-YOLOE stands for "Efficient," meaning the model prioritizes computational efficiency and inference speed while maintaining high performance, making it more suitable for real-time and resource-constrained scenarios. The video analysis model trained with the PP-YOLOE algorithm can effectively detect the bounding boxes of people, the location of monkey carts, and the location of items, enabling monitoring and detection of people riding monkey carts and carrying items. By training on multiple sets of training data, the video analysis model can more accurately detect people carrying items on monkey carts, improving detection accuracy and efficiency. The training data, including historical target videos, historical person bounding boxes, and historical item bounding boxes, helps the model learn the characteristics of people, monkey carts, and items in different scenarios, improving its generalization and adapting to detection tasks in diverse environments and conditions. The video analysis model can help detect the location and behavior of people, monkey carts, and items.

[0035] Step S203: Input the target video into the video analysis model to obtain the person bounding box and object bounding box corresponding to the target video.

[0036] Specifically, by inputting the target video into the video analysis model, the detection and recognition of the bounding boxes of people and objects can be realized, and the situation of people carrying objects on monkey cars can be accurately identified, thereby realizing the monitoring and analysis of this behavior. Through the detection of the bounding boxes of people and objects, the system can accurately capture the situation of people carrying objects on monkey cars, and then realize real-time monitoring and early warning of such behavior, which helps to improve the efficiency and accuracy of the monitoring system, helps monitoring personnel to promptly discover and deal with potential safety issues, and can effectively help the monitoring system to realize the monitoring and analysis of specific behaviors, and improve the effectiveness and efficiency of security monitoring.

[0037] Step S204 , determining the size of the item based on the item bounding box, and determining that the person is carrying an illegal item if the size of the item is greater than or equal to a preset size threshold, wherein the preset size threshold corresponds to the size corresponding to the person bounding box.

[0038] Specifically, by determining the size of an item based on its bounding box, it is possible to effectively determine whether a person is carrying illegal items. If the size of the item is greater than or equal to the preset size threshold, it is identified as an illegal item, thereby promptly discovering and preventing potential security risks. This helps to quickly and accurately identify people carrying illegal items, improve the efficiency and accuracy of security detection, and through automated bounding box size detection, it can reduce the burden of manual detection and avoid misjudgments and omissions caused by subjective judgment.

[0039] Specifically, the system sets a series of size thresholds, which correspond to the size of the person's bounding box, to determine whether an item is oversized or overlong. For example, for each person, the system can set a threshold for an item to be considered illegal if its length exceeds 1.5 times the person's height.

[0040] In addition, the preset size thresholds in the system can also be the following: the length of the item exceeds 1.5 times the height of the person; the width of the item exceeds 2 times the width of the person; the height of the item exceeds 0.5 times the height of the person.

[0041] Through this embodiment, when workers ride on the monkey car, a video of the monkey car's operating area is obtained, and a model is constructed using the PP-YOLOE algorithm. The PP-YOLOE algorithm is an optimized target detection algorithm. Therefore, this solution can identify the workers riding the monkey car in the video, as well as the items carried by the workers. If the items are larger than the preset size, it can be determined that the workers are carrying extra-long and extra-large items on the monkey car. Due to the high accuracy of the PP-YOLOE algorithm, accurate identification of people and items can be achieved, thereby making the detection accuracy of this solution higher.

[0042] Specifically, the system uses the PP-YOLOE+ (Post-Processing You Only Look Once Enhanced) model as a detection model to detect people and objects. PP-YOLOE+ is an optimized target detection model with efficient detection capabilities, especially suitable for scenarios with high real-time requirements in mining environments. The main features of PP-YOLOE+ include: high precision, efficient reasoning, and robustness.

[0043] High Precision: PP-YOLOE+ can effectively distinguish miners from background objects by accurately identifying people in complex scenes, and maintain high detection accuracy even in noisy and low-light environments.

[0044] Efficient reasoning: The model has an extremely high reasoning speed and can maintain stable operation in real-time monitoring systems, ensuring that every frame of the monitoring screen can be processed quickly and the detection results can be output in a timely manner.

[0045] Robustness: PP-YOLOE+ is highly robust and can cope with complex shooting conditions in mines, such as uneven lighting and personnel wearing safety equipment.

[0046] In actual operation, the system first pre-processes each frame of the image, and then inputs the image into the PP-YOLOE+ model. The model automatically detects the miners in the picture and outputs the location information of the personnel, including the bounding box and confidence level.

[0047] In each frame, the system not only detects the miner and the object they are carrying, but also calculates and analyzes the relative size of the object and the person. The specific process is as follows:

[0048] 1. Bounding boxes of people and objects: The PP-YOLOE+ model generates bounding boxes for miners and objects respectively.

[0049] 2. Relative scale calculation: The system compares the size of the detected object with the size of the corresponding person. Since the changes in human body size are relatively stable, the system can infer the true relative size of the object based on the height and width of the person's bounding box.

[0050] 3. Overlong object determination: If the length of an object exceeds a certain ratio threshold of the human body (for example, more than 1.5 times the length of the human body), the object will be marked as an overlong object (i.e., an illegal item), thereby triggering an early warning.

[0051] During the specific implementation process, after the target video is input into the video analysis model and the person bounding box and object bounding box corresponding to the target video are obtained, the above method also includes the following steps: obtaining the person positions in the images of all frames in the target video; using the Hybrid-SORT algorithm to associate the person positions in the images of all frames in the target video to obtain the target tracking results.

[0052] In this solution, by obtaining the positions of people in the images of all frames in the target video and correlating them using the Hybrid-SORT algorithm, it is possible to track people in the video, and then more accurately monitor the items people carry when riding in the monkey car. Through the target tracking results, potential safety hazards or violations can be discovered in a timely manner, thereby improving the efficiency and accuracy of the monitoring system.

[0053] It should be noted that the above-mentioned HybridSORT algorithm (Hybrid Sorting Algorithm) is an efficient and accurate tracking algorithm that combines quick sort and merge sort, and can perform long-term and stable tracking of multiple targets in complex dynamic scenes.

[0054] To achieve continuous monitoring of miners, the system uses the HybridSORT algorithm to track detected personnel. Its main features include: multi-target tracking, fast association, trajectory prediction and hybrid data input.

[0055] Multi-target tracking: The HybridSORT algorithm can track multiple targets simultaneously, making it suitable for scenarios where multiple miners appear in the same frame. It can distinguish between different individuals and assign a unique identifier to each target, thereby maintaining target consistency throughout the tracking process.

[0056] Fast Association: HybridSORT uses a matching mechanism between target detection results and target trajectories, combined with classic methods such as the Kalman filter and the Hungarian algorithm, to quickly associate the miner's location information in consecutive video frames, ensuring the stability and accuracy of tracking results.

[0057] Trajectory Prediction: Through the Kalman filter, HybridSORT can predict the position of the target in the next frame, and can maintain continuous tracking of the target even if there are brief occlusions or detection errors in individual frames.

[0058] Hybrid data input: HybridSORT can combine the target's appearance features (such as the equipment worn by miners) and spatial motion information for multi-dimensional correlation tracking, improving tracking effects in complex scenarios.

[0059] During the specific implementation process, after the target video is input into the video analysis model and the person bounding box and object bounding box corresponding to the target video are obtained, the above method also includes the following steps: constructing a behavior recognition model, wherein the behavior recognition model is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical target videos obtained within a historical time period and historical behavior recognition results corresponding to the historical target videos, wherein the historical behavior recognition results are used to characterize whether the person has historical abnormal actions, wherein the abnormal actions are actions that characterize dragging and / or lifting; inputting the target video into the behavior recognition model to obtain the behavior recognition results corresponding to the target video.

[0060] In this solution, by constructing a behavior recognition model and using multiple sets of training data for training, it is possible to effectively identify whether a person has abnormal movements, such as dragging and / or lifting movements. When the target video is input into the behavior recognition model, the behavior recognition results corresponding to the target video can be obtained, thereby timely discovering and preventing possible safety hazards, ensuring the safety of people when carrying items on monkey cars, improving the intelligence level of the monitoring system, effectively reducing the risk of accidents, and improving safety and management efficiency.

[0061] Specifically, training an action recognition model requires a large amount of historical data, including historical target videos (i.e., previously monitored video clips) and corresponding historical action recognition results. These historical action recognition results are labeled by security experts or system-preset rules, indicating whether the person in the video performed unusual actions, such as dragging or lifting an oversized object. Using these multiple sets of training data, a deep learning model, such as a convolutional neural network (CNN) or a 3D CNN, is trained to identify the movement patterns of people in videos. By learning the characteristics of normal and unusual actions from the historical data, the model can identify unusual actions in new video clips. The model focuses on features such as the person's body movements, the relative position of objects, and the speed of movement in the video—key clues for identifying unusual actions. For example, a dragging action might appear as a continuous horizontal movement of the person's arm, while a lifting action might appear as a continuous upward or downward movement of the arm accompanied by the corresponding movement of an object. Once the model is trained, the target video (i.e., the currently monitored video clip) can be fed into the model, which then outputs an action recognition result, indicating whether the person in the video performed unusual actions.

[0062] Figure 3 FIG. 1 is a flow chart of an optional method for detecting a person carrying items on a monkey car according to an embodiment of the present application, such as Figure 3 As shown in the figure, by combining the advanced deep learning model PP-YOLOE+ for personnel detection and using the HybridSORT algorithm for target tracking, accurate monitoring of miners riding in the monkey car is achieved. Its core function is to detect miners in the picture and continuously track their movement on the monkey car, thereby ensuring the integrity of the monitoring process. The specific process of personnel and object detection and tracking is as follows:

[0063] 1. Image input: The camera captures the image of miners riding on the monkey car in real time and transmits the image to the processing unit.

[0064] 2. Personnel Detection: The PP-YOLOE+ model analyzes each frame of the image, detects the location of the miners, and generates a bounding box with a confidence level.

[0065] 3. Tracking Update: Through HybridSORT, detection results in consecutive frames are correlated and a trajectory for each target is generated, allowing continuous tracking of miners even if they briefly move or are obscured in the frame.

[0066] At the same time, it has the following performance advantages:

[0067] 1. Real-time detection and tracking: Based on the combination of PP-YOLOE+ and HybridSORT, the system can maintain efficient personnel detection and tracking in high-frequency video streams, ensuring real-time monitoring.

[0068] 2. High stability: Even if miners are obstructed during transportation, partially leave the frame, or experience changes in ambient light, HybridSORT can still reliably track the target, significantly reducing the probability of missed detection and false detection.

[0069] 3. Accurate early warning judgment: The combination of personnel detection and object detection modules ensures that the system can accurately trigger early warnings when miners carry extra-long objects, improving mine safety.

[0070] During the specific implementation process, after the target video is input into the video analysis model and the person boundary box and object boundary box corresponding to the target video are obtained, the above method also includes the following steps: obtaining a collision warning threshold; obtaining an object space distance, wherein the object space distance is the distance between a first object carried by a first person and a second object carried by a second person; when the object space distance is less than or equal to the collision warning threshold, determining that the objects will collide; when it is determined that the objects will collide, controlling the monkey car to slow down.

[0071] In this solution, by obtaining the collision warning threshold and the spatial distance between objects, early warning and avoidance of collision events can be achieved. When the spatial distance between objects is less than or equal to the collision warning threshold, potential collision risks can be detected in a timely manner and control measures can be taken, such as slowing down the monkey car to avoid a collision, thereby ensuring the safety of people and objects, effectively preventing collision accidents, and improving safety and efficiency in the workplace.

[0072] Specifically, based on factors such as the monkey car's operating environment, speed, and object size, a collision warning threshold is preset. This is a safe distance. When the distance between two people carrying objects is less than or equal to this threshold, the system will determine that there is a collision risk. Based on 3D reconstruction technology, the system can calculate the spatial distance between different objects in real time. This distance not only takes into account the position of the object in the image, but also combines depth information to accurately reflect the relative position of the object in real space. The spatial distance between objects is continuously monitored. Once the distance is less than or equal to the collision warning threshold, that is, the two people carrying objects may collide while the monkey car is running, the system will respond immediately. When the risk of collision is confirmed, the system will automatically control the monkey car to slow down to reduce the possibility of a collision and the impact force during a collision, thereby reducing potential risks of personal injury and equipment damage.

[0073] For example, multiple high-definition cameras and depth sensing devices have been installed along the monkey cart's operating path in a coal mine, and a real-time response mechanism has been integrated into the monkey cart's control system. After the camera captures the video, the system first performs image preprocessing to enhance image quality. It then uses a deep learning model to detect people and objects, and then uses 3D reconstruction technology to calculate the precise location and spatial distance of the objects. Based on historical data analysis, the system presets a collision warning threshold, such as 50 centimeters. This is a safe estimate based on the monkey cart's operating speed and the size of the objects, ensuring that at this distance, the monkey cart's deceleration can effectively avoid collisions. The system continuously analyzes the video stream and calculates the spatial distance between people carrying objects. Once the distance between two objects is less than or equal to 50 centimeters (the collision warning threshold), the system immediately determines that there is a collision risk.

[0074] After confirming a collision risk, the system automatically adjusts the monkey car's speed through its control system, causing it to slow down. The degree of deceleration depends on the distance and the monkey car's current speed, ensuring that the collision risk is effectively reduced without affecting normal operation processes.

[0075] In addition to automatically slowing down, the system also issues visual and audible alarms to alert on-site security personnel and workers to potential collision risks. This warning is also transmitted to the monitoring center via the network, allowing management to take further action. Based on the effectiveness of the monkey car's deceleration, the system assesses whether the collision risk is being effectively controlled. If the risk persists, the system will continue to adjust the monkey car's speed or take other measures, such as completely stopping the monkey car, until security personnel confirm that there is no collision risk.

[0076] During the specific implementation process, after determining the size of the object based on the object boundary box and determining that the person is carrying illegal items when the size of the object is greater than or equal to a preset size threshold, the above method also includes the following steps: obtaining target audio, wherein the target audio is the audio of the operating area of ​​the monkey car in the mine; performing audio analysis on the target audio to determine whether there is abnormal sound in the target audio, and obtaining an audio analysis result, wherein the abnormal sound is one or more of dragging sound, collision sound and wind noise; when the audio analysis result indicates that there is abnormal sound in the target audio, it is determined that the person is carrying illegal items.

[0077] In this solution, by analyzing the audio in the monkey car operating area in the mine, it is possible to detect whether there are abnormal sounds, such as dragging sounds, collision sounds and wind noise, so as to determine whether anyone is carrying illegal items. It can monitor and identify the illegal behavior of monkey car passengers, which helps to improve the level of mine safety management, prevent accidents, and effectively ensure the safety of mine workers and improve production efficiency.

[0078] Specifically, microphones are deployed in the area where the monkey cart operates, capturing real-time audio data from that area. This audio data includes various sounds generated by the operation of the monkey cart, the movement of people, and any items that may be carried. Using audio signal processing technology and machine learning models, the system analyzes the target audio and identifies any unusual sounds, such as dragging, collisions, and wind noise. These sounds may be caused by oversized objects moving on the ground or on the monkey cart, colliding with other objects, or air friction caused by the object's shape and size. The system sets a series of characteristic parameters for abnormal sounds, such as frequency, volume, and duration, to determine whether these abnormal sounds are included in the audio analysis results. If they are present, the system further analyzes the source and nature of the sound to determine whether it is related to the illegal carrying of items. If the presence of unusual sounds in the target audio is confirmed and these sounds match the behavioral characteristics of carrying illegal items, the system triggers an early warning mechanism, deeming the person to be carrying an oversized item and requiring further video surveillance and manual confirmation.

[0079] Specifically, in the system, because the images captured by security cameras can cause significant differences in the size of objects in the picture due to changes in viewing angle, it is unreasonable to directly use pixel size to determine whether an object is too long or too large. To solve this problem, the relative size of the object and the human body is used to make alarm judgments to ensure the accuracy of detection. The specific early warning mechanism is as follows:

[0080] 1. Dynamically adjust the threshold:

[0081] The system dynamically adjusts based on the distance of the person. For example, as the camera captures images from a farther distance, the bounding box of the human body becomes smaller. The system will automatically adjust the length judgment threshold of the object based on the actual proportion to ensure that the distance does not affect the accuracy of the alarm.

[0082] 2. Warning trigger conditions:

[0083] The triggering of the early warning mechanism is based on the combination of the following two conditions: personnel detection, the system uses the PP-YOLOE+ model to confirm whether there are miners in the picture and track whether the miners are sitting in the monkey car seat; over-long object detection, based on the relative size relationship between the human body and the object, determines whether the miners are carrying over-long objects (such as objects longer than 1.5 times the human body). Once confirmed, the system immediately triggers an early warning.

[0084] 3. Warning form:

[0085] When the above conditions are detected, the system will issue an early warning in the form of local sound and light alarms and remote notifications: sound and light alarms will be issued on site to alert miners and on-site safety officers, and the system will send the early warning information to the central monitoring system via the network for management personnel to intervene in time.

[0086] During the specific implementation process, after inputting the target video into the video analysis model and obtaining the person bounding box and object bounding box corresponding to the target video, the above method also includes the following steps: obtaining point cloud data, wherein the point cloud data includes the first point cloud data of the mine, the second point cloud data of the monkey car, the third point cloud data of the person and the fourth point cloud data of the object; constructing a 3D model based on the point cloud data, and displaying the 3D model on the display page.

[0087] In this solution, obtaining point cloud data can help monitor the location and dynamic information of people, vehicles and objects in the mine in real time. By constructing a 3D model and displaying it on the display page, the real-time situation in the mine can be intuitively displayed, thereby helping to monitor the situation of people carrying objects. By analyzing point cloud data and 3D models, potential safety hazards can be discovered in a timely manner, ensuring the safety of people and objects in the mine, and effectively improving the safety and efficiency of mine operations.

[0088] Specifically, point cloud data is a collection of points obtained through 3D laser scanning or depth image capture of a space, each containing 3D coordinate information. The system deploys LiDAR (Light Detection and Ranging) or other depth sensing equipment in the mine to capture point cloud data of the mine, vehicles, people, and objects in real time.

[0089] The acquired point cloud data is divided into four categories: the first point cloud data of the mine, which is used to construct the three-dimensional environmental model of the mine; the second point cloud data of the monkey car, which is used to construct the three-dimensional model of the monkey car; the third point cloud data of the personnel, which is used to construct the three-dimensional model of the personnel; and the fourth point cloud data of the objects, which is used to construct the three-dimensional model of the objects. The system uses a point cloud processing algorithm to convert the above four types of point cloud data into a three-dimensional model. This process may include pre-processing of the point cloud data (such as denoising and smoothing), registration of the point cloud data (aligning point cloud data at different times or perspectives), and construction of the three-dimensional model (such as surface reconstruction and texture mapping).

[0090] The completed 3D model is displayed on a monitor in the monitoring center or presented to on-site operators through virtual reality (VR) or augmented reality (AR) devices. The display page or device provides detailed viewing angle adjustment tools so that users can view and analyze the 3D model from all angles.

[0091] Optionally, the real-time situation data in the above-mentioned mine may include but is not limited to: personnel location, monkey car location, object location, etc.

[0092] During the specific implementation process, before obtaining the target video, the above method also includes the following steps: obtaining the environmental parameters of the mine, wherein the environmental parameters include light intensity and dust concentration; when the light intensity is less than or equal to a preset light intensity threshold, adjusting the control parameters of the video acquisition device, wherein the control parameters include one or more of exposure time, gain, white balance, resolution, and frame rate; when the dust concentration is greater than or equal to a preset concentration threshold, controlling the cleaning device to clean the video acquisition device.

[0093] In this solution, the safety of miners is guaranteed by monitoring the environmental parameters of the mine. By real-time monitoring of light intensity and dust concentration, abnormal conditions in the mine environment can be discovered in a timely manner, thereby adjusting the control parameters of the video acquisition equipment to ensure that the video acquisition equipment can work normally and obtain clear image information. At the same time, when the dust concentration exceeds the preset threshold, the cleaning equipment can be automatically triggered to clean the video acquisition equipment, ensuring the normal operation of the video acquisition equipment and improving the safety level of miners.

[0094] Specifically, light intensity sensors are deployed throughout the mine to monitor ambient light intensity in real time. If the light intensity falls below a preset threshold (e.g., the system's minimum light intensity standard), the system automatically adjusts the control parameters of the video capture device, such as increasing exposure time, increasing gain, and adjusting white balance, to improve video clarity in low-light environments.

[0095] The system is also equipped with a dust concentration sensor to monitor dust concentration in the mine environment. When dust concentration reaches or exceeds a preset threshold, the system triggers a cleaning device to clean the video capture device's lens to prevent dust accumulation from affecting video quality.

[0096] For example, in the mine's monkey car operating area, the system not only deploys video surveillance equipment, but also installs environmental monitoring sensors, including light intensity sensors and dust concentration sensors, to understand environmental conditions in real time. When the light intensity of the mine environment falls below a preset threshold (for example, 50 lux), the system automatically adjusts the control parameters of the video acquisition equipment. The specific adjustments are as follows: the system increases the exposure time to collect more light signals to ensure that the video remains clear in low-light environments. The gain of the image signal is increased to enhance the brightness and contrast of the video. The white balance setting is adjusted to ensure the true reproduction of colors under different lighting conditions. Although resolution adjustment has limited effect on improving video clarity in low-light environments, in some cases the system may reduce the resolution to reduce processing latency and ensure real-time monitoring. In low-light environments, the system may reduce the frame rate to optimize the power consumption and performance of the device, but it will generally not fall below the minimum frame rate standard required for real-time monitoring.

[0097] If the dust concentration in the mine environment exceeds the preset concentration threshold (for example, 10 mg / m 3 ), the system automatically activates cleaning equipment, such as a high-pressure spray washer, to clean the video capture device's lens, ensuring it remains dust-free and maintaining video clarity. The cleaning process can be scheduled to run automatically at regular intervals or initiated immediately when dust concentrations rise abnormally.

[0098] The system continuously monitors environmental parameters and automatically adjusts the control parameters of the video capture equipment based on changes. At the same time, the system has a feedback mechanism that re-evaluates the video quality after the cleaning equipment has been cleaned, ensuring that all adjustments have effectively improved the quality of video capture.

[0099] Operators at the monitoring center can remotely monitor environmental parameters and the status of video capture equipment. In special circumstances, such as equipment failure or environmental parameters exceeding normal ranges, operators can remotely adjust equipment parameters or dispatch maintenance personnel for on-site inspection and resolution.

[0100] To ensure that the system can effectively monitor whether miners are carrying extra-long objects when riding the monkey car, the installation of security cameras is crucial. This system deploys multiple cameras at key locations on the aerial ropeway to ensure full monitoring of personnel and their belongings. The specific installation plan is as follows:

[0101] 1. Number and distribution of cameras

[0102] According to the length and specific running route of the aerial ropeway (monkey car), the system installs cameras in multiple key areas. Typically, 1 to 2 cameras are arranged in each monitoring area, and the coverage overlap between the cameras is kept reasonably to ensure that there are no blind spots in the field of view.

[0103] Key areas: may include but are not limited to monkey car entrances and exits, tunnel bends, areas where miners get on and off monkey cars, etc., especially narrow passages where accidents may occur, which require key control.

[0104] Regional distribution: Cameras are evenly distributed throughout the entire route of the monkey car, especially in narrow places or places where scratches are likely to occur to strengthen monitoring.

[0105] 2. Camera installation height and angle

[0106] The camera's installation height and angle have been strictly calculated to ensure that the camera can capture both the miner's full-body image and the objects he is carrying. Usually, the camera is installed 2-3 meters above the ground and tilted slightly downward to obtain the best viewing angle.

[0107] Mounting height: The camera can be mounted on the tunnel wall or support structure at a height suitable for capturing the entire seating area, including the miner’s head, torso, and any objects the miner is carrying around.

[0108] Shooting angle: The camera is usually installed at an angle of about 30 degrees downward, which can cover the head of the person and surrounding objects, and effectively capture the area under the seat to prevent objects from being blocked.

[0109] Wide-angle or zoom lens: Depending on the width of the specific monitoring area, choose a wide-angle lens or a camera with zoom function to ensure that the monitoring field of view covers the entire monkey car area, especially for wider channel sections.

[0110] 3. Technical requirements for cameras

[0111] The cameras used in the system must have high performance and adaptability to complex mining environments, including but not limited to:

[0112] High-definition resolution: To accurately identify miners and the items they carry, the camera resolution must be at least 1080P to ensure clear image details even in dark environments.

[0113] Night vision capability: Underground mines are usually poorly lit, so the camera should be equipped with infrared night vision to ensure image quality even in low-light environments.

[0114] Dustproof and waterproof: Due to the complex environment and high dust content in mines, cameras should be dustproof and waterproof. Industrial cameras that meet IP65 or higher standards are usually used to ensure stable operation under harsh conditions.

[0115] Wide Dynamic Range (WDR): Faced with the drastic changes in light conditions in mines, cameras should have a wide dynamic range (WDR) function, which can capture clear images in both bright and dark areas at the same time, avoiding image distortion caused by conditions such as backlighting.

[0116] 4. Network and power support

[0117] The camera is connected to the system's central processing unit via a wired or wireless network to ensure real-time transmission of image data. In underground mining environments, wired transmission is usually preferred to reduce signal interference.

[0118] Wired connection: The camera is connected to the system server through a high-voltage and tensile-resistant optical fiber cable to ensure the stability of data transmission.

[0119] Power supply: The camera can be powered by a dedicated cable or use PoE (Power over Ethernet) technology to solve network connection and power supply problems at the same time and reduce wiring complexity.

[0120] 5. Safety protection and maintenance

[0121] The camera must have a shock-proof structure to cope with the vibration and impact in the mining environment. At the same time, the camera must be maintained and cleaned regularly to ensure that the camera lens surface is free of dust, thereby ensuring its normal operation and image quality.

[0122] The technical solutions provided by the above embodiments of the present application solve the following problems: 1) How to accurately identify and determine whether workers are carrying extra-long and extra-large items in a complex coal mine environment, especially in scenarios where workers are riding in monkey carts; 2) How to improve the real-time and accuracy of identification so as to promptly detect potential safety hazards; 3) How to improve the recognition capability of computer vision systems in environments with poor lighting conditions; 4) How to effectively distinguish between normal items and extra-long and extra-large items without affecting the normal operation of workers; 5) How to realize the automation and intelligence of the system, reduce manual intervention, and improve the overall efficiency of safety monitoring.

[0123] At the same time, it has the following beneficial effects:

[0124] 1. Improve efficiency and accuracy: 1) Automated monitoring. Computer vision-based solutions enable 24 / 7 automatic monitoring without manual intervention, avoiding omissions or fatigue that may occur during manual monitoring. 2) High alarm accuracy. The use of an early warning mechanism based on the relative size of objects and the human body can greatly improve the accuracy of alarms, avoiding false alarms or missed alarms caused by the distance of objects. This ensures that no matter what distance the miners are at, the system can effectively detect potential safety hazards and issue timely warnings.

[0125] 2. Real-time response and alarm: 1) Instant response. The system can process the images captured by the camera in real time. When a person carrying an oversized item rides on a monkey car, the system will immediately issue an alarm to ensure that timely measures are taken; 2) Rapid decision support. Compared with manual monitoring, which requires observation, analysis and decision-making, the computer vision system can provide instant decision support, shorten response time and reduce the risk of accidents.

[0126] 3. Reduce labor costs: 1) Reduce dependence on manpower. Traditional manual monitoring requires a large number of people to monitor in shifts. Especially when working for long hours or in harsh environments, the management and maintenance costs of personnel are high. The automated system can significantly reduce the consumption of human resources; 2) Low long-term operating costs. Although the initial deployment of the computer vision system may require a certain amount of cost investment, in the long run, the operation and maintenance costs of this solution are low, which can reduce the economic losses caused by human misjudgment and misoperation.

[0127] 4. Strong environmental adaptability: 1) Adaptability to complex environments. Computer vision systems can adapt to different lighting, weather conditions, and other complex environments, while the performance of manual monitoring under these conditions may be affected; 2) Self-learning ability. The system can continuously adapt and optimize its performance in different environments and conditions through self-learning mechanisms, thereby improving the robustness and accuracy of the overall system.

[0128] From the above, it can be seen that in the embodiment of the present application, an intelligent detection system based on image processing and deep learning technology is proposed to monitor whether miners carry extra-long objects when riding an aerial cableway (monkey car) and trigger an early warning when specific conditions are met. The purpose is to significantly improve the level of coal mine safety monitoring, especially in preventing safety accidents that may be caused by carrying extra-long and extra-large objects, thereby providing more reliable technical guarantees for safe production in coal mines.

[0129] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for detecting items carried by a person riding a monkey car of the present application will be described in detail below with reference to specific embodiments.

[0130] This embodiment relates to a specific method for detecting items carried by a person riding a monkey car, comprising the following steps:

[0131] 1. Data Collection and Preprocessing

[0132] Data collection: Install high-definition cameras in the monkey car operation area in the coal mine to collect video data in real time.

[0133] Data preprocessing: Frame extraction is performed on the collected video data, which is converted into an image sequence. The image is preprocessed using an image enhancement algorithm to improve image quality and facilitate subsequent feature extraction.

[0134] 2. Deep Learning Model Selection

[0135] Model selection: Select a core detection model to ensure that the model has the characteristics of fast speed and high accuracy, and is suitable for real-time detection scenarios.

[0136] 3. Model Training

[0137] Dataset construction: Collect and annotate image data of people riding monkey cars and carrying objects of different sizes (including extra-long and extra-large objects) to construct a training dataset.

[0138] Training process: Input the preprocessed image data into the model for model training, and optimize the model performance by adjusting the model parameters.

[0139] 4. Feature extraction and detection

[0140] Feature extraction: Use the convolutional neural network part of the model to extract features from the image and generate a feature map.

[0141] Target detection: Detect people and items in the image on the feature map, remove redundant detection frames through the non-maximum suppression algorithm, and obtain the final detection result.

[0142] 5. Item size determination

[0143] Size calculation: Calculate the actual size of the object based on the detected object bounding box size (needs to be corrected in combination with camera viewing angle, focal length and other parameters).

[0144] Threshold judgment: Set the size threshold for oversized items, compare the calculated item size with the threshold, and determine whether it is an oversized item.

[0145] The embodiments of the present application also provide a device for detecting items carried by a person riding a monkey car. It should be noted that the device for detecting items carried by a person riding a monkey car according to the embodiments of the present application can be used to execute the detection method for detecting items carried by a person riding a monkey car provided by the embodiments of the present application. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0146] The following introduces the detection device for people carrying items on monkey cars provided in the embodiments of the present application.

[0147] Figure 4 This is a structural block diagram of a device for detecting items carried by a person riding a monkey car according to an embodiment of the present application. Figure 4 As shown, the device includes: a first acquiring unit 41 , a first constructing unit 42 , a first processing unit 43 and a first determining unit 44 .

[0148] The first acquisition unit 41 is used to acquire a target video, wherein the target video is a video of the operation area of ​​the monkey car in the mine.

[0149] The first construction unit 42 is used to construct a video analysis model, wherein the video analysis model is trained by the PP-YOLOE algorithm using multiple sets of training data, and each set of training data includes historical target videos obtained within a historical time period, historical person bounding boxes corresponding to the historical target videos, and historical object bounding boxes corresponding to the historical target videos.

[0150] The first processing unit 43 is configured to input the target video into the video analysis model to obtain a person bounding box and an object bounding box corresponding to the target video.

[0151] The first determining unit 44 is configured to determine the size of the item based on the item bounding box, and determine that the person is carrying an illegal item if the size of the item is greater than or equal to a preset size threshold, wherein the preset size threshold corresponds to the size of the person bounding box.

[0152] Through this embodiment, when workers ride on the monkey car, a video of the monkey car's operating area is obtained, and a model is constructed using the PP-YOLOE algorithm. The PP-YOLOE algorithm is an optimized target detection algorithm. Therefore, this solution can identify the workers riding the monkey car in the video, as well as the items carried by the workers. If the items are larger than the preset size, it can be determined that the workers are carrying extra-long and extra-large items on the monkey car. Due to the high accuracy of the PP-YOLOE algorithm, accurate identification of people and items can be achieved, thereby making the detection accuracy of this solution higher.

[0153] During the specific implementation process, the above-mentioned device also includes a second acquisition unit and a second processing unit. The second acquisition unit is used to obtain the positions of people in the images of all frames in the target video after inputting the target video into the video analysis model to obtain the person boundary box and object boundary box corresponding to the target video; the second processing unit is used to use the Hybrid-SORT algorithm to associate the positions of people in the images of all frames in the target video to obtain the target tracking results.

[0154] In this solution, by obtaining the positions of people in the images of all frames in the target video and correlating them using the Hybrid-SORT algorithm, it is possible to track people in the video, and then more accurately monitor the items people carry when riding in the monkey car. Through the target tracking results, potential safety hazards or violations can be discovered in a timely manner, thereby improving the efficiency and accuracy of the monitoring system.

[0155] During the specific implementation process, the above-mentioned device also includes a second construction unit and a third processing unit. The second construction unit is used to construct a behavior recognition model after inputting the target video into the video analysis model to obtain the person boundary box and object boundary box corresponding to the target video, wherein the behavior recognition model is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical target videos obtained within a historical time period and historical behavior recognition results corresponding to the historical target videos, wherein the historical behavior recognition results are used to characterize whether the person has historical abnormal actions, wherein the abnormal actions are actions that characterize dragging and / or lifting; the third processing unit is used to input the target video into the behavior recognition model to obtain the behavior recognition results corresponding to the target video.

[0156] In this solution, by constructing a behavior recognition model and using multiple sets of training data for training, it is possible to effectively identify whether a person has abnormal movements, such as dragging and / or lifting movements. When the target video is input into the behavior recognition model, the behavior recognition results corresponding to the target video can be obtained, thereby timely discovering and preventing possible safety hazards, ensuring the safety of people when carrying items on monkey cars, improving the intelligence level of the monitoring system, effectively reducing the risk of accidents, and improving safety and management efficiency.

[0157] During the specific implementation process, the above-mentioned device also includes a third acquisition unit, a fourth acquisition unit, a second determination unit and a first control unit. The third acquisition unit is used to obtain the collision warning threshold after inputting the target video into the video analysis model to obtain the person boundary box and the object boundary box corresponding to the target video; the fourth acquisition unit is used to obtain the object spatial distance, wherein the object spatial distance is the distance between the first object carried by the first person and the second object carried by the second person; the second determination unit is used to determine that the objects will collide when the object spatial distance is less than or equal to the collision warning threshold; the first control unit is used to control the monkey car to decelerate when it is determined that the objects will collide.

[0158] In this solution, by obtaining the collision warning threshold and the spatial distance between objects, early warning and avoidance of collision events can be achieved. When the spatial distance between objects is less than or equal to the collision warning threshold, potential collision risks can be detected in a timely manner and control measures can be taken, such as slowing down the monkey car to avoid a collision, thereby ensuring the safety of people and objects, effectively preventing collision accidents, and improving safety and efficiency in the workplace.

[0159] During the specific implementation process, the above-mentioned device also includes a fifth acquisition unit, an analysis unit and a third determination unit. The fifth acquisition unit is used to determine the size of the object according to the object boundary box, and when the size of the object is greater than or equal to the preset size threshold, determine that the person is carrying illegal items and then obtain the target audio, wherein the target audio is the audio of the operating area of ​​the monkey car in the mine; the analysis unit is used to perform audio analysis on the target audio, determine whether there is abnormal sound in the target audio, and obtain an audio analysis result, wherein the abnormal sound is one or more of dragging sound, collision sound and wind noise; the third determination unit is used to determine that the person is carrying illegal items when the audio analysis result indicates that there is abnormal sound in the target audio.

[0160] In this solution, by analyzing the audio in the monkey car operating area in the mine, it is possible to detect whether there are abnormal sounds, such as dragging sounds, collision sounds and wind noise, so as to determine whether anyone is carrying illegal items. It can monitor and identify the illegal behavior of monkey car passengers, which helps to improve the level of mine safety management, prevent accidents, and effectively ensure the safety of mine workers and improve production efficiency.

[0161] During the specific implementation process, the above-mentioned device also includes a sixth acquisition unit and a third construction unit. The sixth acquisition unit is used to obtain point cloud data after inputting the target video into the video analysis model and obtaining the person boundary box and object boundary box corresponding to the target video, wherein the point cloud data includes the first point cloud data of the mine, the second point cloud data of the monkey car, the third point cloud data of the person and the fourth point cloud data of the object; the third construction unit is used to construct a 3D model based on the point cloud data and display the 3D model on the display page.

[0162] In this solution, obtaining point cloud data can help monitor the location and dynamic information of people, vehicles and objects in the mine in real time. By constructing a 3D model and displaying it on the display page, the real-time situation in the mine can be intuitively displayed, thereby helping to monitor the situation of people carrying objects. By analyzing point cloud data and 3D models, potential safety hazards can be discovered in a timely manner, ensuring the safety of people and objects in the mine, and effectively improving the safety and efficiency of mine operations.

[0163] During the specific implementation process, the above-mentioned device also includes a seventh acquisition unit, an adjustment unit and a second control unit. The seventh acquisition unit is used to obtain the environmental parameters of the mine before obtaining the target video, wherein the environmental parameters include light intensity and dust concentration; the adjustment unit is used to adjust the control parameters of the video acquisition device when the light intensity is less than or equal to the preset light intensity threshold, wherein the control parameters include one or more of exposure time, gain, white balance, resolution, and frame rate; the second control unit is used to control the cleaning device to clean the video acquisition device when the dust concentration is greater than or equal to the preset concentration threshold.

[0164] In this solution, the safety of miners is guaranteed by monitoring the environmental parameters of the mine. By real-time monitoring of light intensity and dust concentration, abnormal conditions in the mine environment can be discovered in a timely manner, thereby adjusting the control parameters of the video acquisition equipment to ensure that the video acquisition equipment can work normally and obtain clear image information. At the same time, when the dust concentration exceeds the preset threshold, the cleaning equipment can be automatically triggered to clean the video acquisition equipment, ensuring the normal operation of the video acquisition equipment and improving the safety level of miners.

[0165] The device for detecting items carried by a person riding a monkey car includes a processor and a memory. The first acquisition unit, the first construction unit, the first processing unit, and the first determination unit are all stored as program units in the memory. The processor executes the program units stored in the memory to implement the corresponding functions. The modules are all located in the same processor; alternatively, the modules can be located in different processors in any combination.

[0166] The processor includes a core, which retrieves the corresponding program unit from the memory. One or more cores can be configured, and by adjusting the core parameters, the problem of low accuracy in manually determining whether a worker riding a monkey car is carrying an oversized or long item in the existing technology can be solved.

[0167] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0168] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is running, the device where the computer-readable storage medium is located is controlled to execute a method for detecting items carried by a person riding a monkey car.

[0169] An embodiment of the present invention provides a processor, which is used to run a program, wherein when the program is run, a method for detecting items carried by a person riding a monkey car is executed.

[0170] A computer program product includes a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method for detecting objects carried by people riding monkey cars in various embodiments of the present application: obtaining a target video, wherein the target video is a video of the operating area of ​​the monkey car in a mine; constructing a video analysis model, wherein the video analysis model is trained using a PP-YOLOE algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical target videos obtained within a historical time period, historical person bounding boxes corresponding to the historical target videos, and historical object bounding boxes corresponding to the historical target videos; inputting the target video into the video analysis model to obtain a person bounding box and an object bounding box corresponding to the target video; determining the size of the object based on the object bounding box, and determining that the person is carrying an illegal object if the size of the object is greater than or equal to a preset size threshold, wherein the preset size threshold and the size corresponding to the person bounding box have a one-to-one correspondence.

[0171] The present application also provides a monkey car detection system, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include methods for executing any of the above-mentioned methods for detecting a person carrying items on a monkey car.

[0172] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0173] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0177] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0178] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0179] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0181] The above are merely preferred embodiments of the present application and are not intended to limit the present application. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting items carried by a person riding a monkey car, characterized in that: include: Acquire a target video, wherein the target video is a video of an operation area of ​​a monkey car in a mine; Constructing a video analysis model, wherein the video analysis model is trained by a PP-YOLOE algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes a historical target video acquired within a historical time period, a historical person bounding box corresponding to the historical target video, and a historical object bounding box corresponding to the historical target video; Inputting the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video; The size of the object is determined according to the object boundary box, and when the size of the object is greater than or equal to a preset size threshold, it is determined that the person is carrying an illegal item, wherein the preset size threshold and the size corresponding to the person boundary box have a one-to-one correspondence.

2. The method according to claim 1, characterized in that After inputting the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video, the method further includes: Obtaining the positions of people in images of all frames in the target video; The Hybrid-SORT algorithm is used to associate the positions of people in the images of all frames in the target video to obtain the target tracking result.

3. The method according to claim 1, characterized in that After inputting the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video, the method further includes: Constructing a behavior recognition model, wherein the behavior recognition model is trained using multiple sets of training data, each set of training data in the multiple sets of training data includes historical target videos acquired within a historical time period and historical behavior recognition results corresponding to the historical target videos, wherein the historical behavior recognition results are used to characterize whether a person has historical abnormal actions, wherein the abnormal actions are actions that characterize dragging and / or resisting lifting; The target video is input into the behavior recognition model to obtain a behavior recognition result corresponding to the target video.

4. The method according to claim 1, characterized in that After inputting the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video, the method further includes: Get the collision warning threshold; Acquire an object space distance, wherein the object space distance is a distance between a first object carried by a first person and a second object carried by a second person; When the spatial distance between the objects is less than or equal to the collision warning threshold, determining that the objects will collide; When it is determined that the objects will collide, the monkey car is controlled to slow down.

5. The method according to claim 1, characterized in that: After determining the size of the object according to the object boundary box and determining that the person is carrying illegal objects when the size of the object is greater than or equal to a preset size threshold, the method further includes: Acquire target audio, wherein the target audio is the audio of the operation area of ​​the monkey car in the mine; Performing audio analysis on the target audio to determine whether there is an abnormal sound in the target audio, and obtaining an audio analysis result, wherein the abnormal sound is one or more of a dragging sound, a collision sound, and a wind noise; When the audio analysis result indicates that abnormal sound exists in the target audio, it is determined that the person is carrying illegal items.

6. The method according to any one of claims 1 to 5, characterized in that After inputting the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video, the method further includes: Acquire point cloud data, wherein the point cloud data includes first point cloud data of the mine, second point cloud data of the monkey car, third point cloud data of the person, and fourth point cloud data of the object; A 3D model is constructed according to the point cloud data, and the 3D model is displayed on a display page.

7. The method according to any one of claims 1 to 5, characterized in that Before acquiring the target video, the method further includes: Acquiring environmental parameters of the mine, wherein the environmental parameters include light intensity and dust concentration; When the light intensity is less than or equal to a preset light intensity threshold, adjusting control parameters of the video acquisition device, wherein the control parameters include one or more of exposure time, gain, white balance, resolution, and frame rate; When the dust concentration is greater than or equal to a preset concentration threshold, a cleaning device is controlled to clean the video acquisition device.

8. A device for detecting items carried by people riding on a monkey car, characterized in that: include: A first acquisition unit is used to acquire a target video, wherein the target video is a video of an operation area of ​​a monkey car in a mine; A first construction unit is used to construct a video analysis model, wherein the video analysis model is trained by a PP-YOLOE algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes a historical target video acquired within a historical time period, a historical person bounding box corresponding to the historical target video, and a historical object bounding box corresponding to the historical target video; A first processing unit, configured to input the target video into the video analysis model to obtain a person boundary box and an object boundary box corresponding to the target video; The first determination unit is used to determine the size of the item according to the item boundary box, and determine that the person is carrying illegal items when the size of the item is greater than or equal to a preset size threshold, wherein the preset size threshold and the size corresponding to the person boundary box have a one-to-one correspondence.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for detecting items carried by a person riding a monkey car as described in any one of claims 1 to 7 are implemented.

10. A monkey car detection system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the method for detecting items carried by a person riding in a monkey car as described in any one of claims 1 to 7.

Images