Hydraulic support pose recognition method and device for fully mechanized coal mining face

Abstract

The invention relates to a hydraulic support pose recognition method and device for a fully mechanized coal mining face, and the method comprises the steps: obtaining hydraulic support monitoring data of the fully mechanized coal mining face, and the hydraulic support monitoring data comprises the working state of a hydraulic support; inputting the monitoring data of the hydraulic support to a trained key point identification model, and obtaining current pose information of the hydraulic support in the fully mechanized coal mining face; wherein the trained key point identification model is used for acquiring position information of key points in the hydraulic support, and determining pose information of the hydraulic support based on the position information of the key points. Through detection and pose recognition of the key points of the hydraulic support, the problems of large measurement error, high cost and the like caused by adoption of related sensors in related technologies can be avoided while the recognition precision is ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a method and device for recognizing the posture of a hydraulic support in a fully mechanized mining face. Background Art

[0002] During coordinated coal mining in fully mechanized face mining, the primary function of hydraulic supports is to support and protect the coal seam roof, preventing it from collapsing and ensuring safe and continuous mining. As mining depths increase and geological conditions become more complex, the operating environment for hydraulic supports becomes increasingly harsh, leading to abnormal changes in their position and posture, directly impacting mining operations. Therefore, accurate identification of hydraulic support position and posture has become a crucial prerequisite for the development of intelligent and unmanned coal mining.

[0003] In related technologies, hydraulic support posture recognition methods primarily rely on sensors for monitoring, such as inclination sensors, pressure sensors, and displacement sensors. While these methods can achieve hydraulic support posture monitoring to a certain extent, the sensors are susceptible to environmental interference, resulting in large measurement errors and high sensor maintenance costs. This results in low hydraulic support posture recognition efficiency in related technologies. Summary of the Invention

[0004] The present invention provides a method and device for recognizing the posture of a hydraulic support in a fully mechanized mining working face.

[0005] According to a first aspect of the present disclosure, a method for recognizing the position and posture of a hydraulic support in a fully mechanized mining working face is provided, the method comprising:

[0006] Acquire monitoring data of a hydraulic support of a fully mechanized mining working face, wherein the monitoring data of the hydraulic support includes a working status of the hydraulic support;

[0007] Input the hydraulic support monitoring data into the trained key point recognition model to obtain the current posture information of the hydraulic support in the comprehensive mining working face; wherein, the trained key point recognition model is used to obtain the key point position information in the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

[0008] According to a second aspect of the present disclosure, a device for recognizing the position and posture of a hydraulic support in a fully mechanized mining working face is provided, the device comprising:

[0009] A monitoring data acquisition module is used to acquire the monitoring data of the hydraulic support of the fully mechanized mining working face, wherein the monitoring data of the hydraulic support includes the working status of the hydraulic support;

[0010] A posture recognition module is used to input the hydraulic support monitoring data into the trained key point recognition model to obtain the current posture information of the hydraulic support in the comprehensive mining working face; wherein, the trained key point recognition model is used to obtain the key point position information in the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

[0011] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the program.

[0012] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above method of the present disclosure is implemented.

[0013] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method of the present disclosure when executed by a processor.

[0014] The disclosed embodiments provide a method and device for recognizing the posture of a hydraulic support in a fully mechanized mining face. This method involves inputting monitoring data of the hydraulic supports in the fully mechanized mining face into a trained key point recognition model. The trained key point recognition model detects key points of the hydraulic supports in the monitoring data, obtains position information of the key points of the hydraulic supports, and determines the posture information of the hydraulic supports based on the key point position information. By detecting and recognizing the posture of the key points of the hydraulic supports, the embodiments can avoid the problems of large measurement errors and high costs associated with the use of related sensors in related technologies while ensuring recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0016] Figure 1 A schematic diagram of a key point recognition model construction process provided by an exemplary embodiment of the present disclosure;

[0017] Figure 2 A flow chart of a method for recognizing the position and posture of a hydraulic support in a fully mechanized mining face provided by an exemplary embodiment of the present disclosure;

[0018] Figure 3 A schematic block diagram of the functional modules of a hydraulic support posture recognition device for a fully mechanized mining working face provided by an exemplary embodiment of the present disclosure;

[0019] Figure 4 A structural block diagram of an electronic device provided as an exemplary embodiment of the present disclosure;

[0020] Figure 5 A structural block diagram of a computer system provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0023] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0026] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0028] As an optional but non-limiting implementation method, in response to receiving the user's active request, the method of sending a prompt message to the user can be, for example, a pop-up window, and the prompt message can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understandable that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation method of the present disclosure. Other methods that meet relevant laws and regulations can also be applied to the implementation method of the present disclosure.

[0029] Related art methods for detecting the posture of hydraulic supports primarily rely on sensors, such as inclination sensors, pressure sensors, and displacement sensors. While these methods can achieve a certain degree of posture monitoring, the sensors' susceptibility to environmental interference leads to significant measurement errors and high sensor maintenance costs. Furthermore, these technologies struggle to achieve full posture monitoring of hydraulic supports, particularly in the complex and ever-changing underground environment of coal mines, where lighting is dim and large amounts of floating powder and coal dust are present in tunnels and working surfaces. This makes it difficult to meet the requirements for high precision, real-time performance, and automation.

[0030] Specifically, the related technologies for monitoring and identifying the hydraulic support posture mainly include the following:

[0031] 1) Posture monitoring based on a single sensor.

[0032] (1) Inclination / displacement sensors: Earlier domestic research mainly relied on inclination and displacement sensors for posture detection. For example, by installing inclination sensors at key locations on a hydraulic support, the inclination angles of the top beam and base of the hydraulic support are collected. After the sensors collect data, the posture is calculated using a mathematical model of the rigid body motion of the mechanical equipment. This method has a certain degree of reliability, but is limited by the accuracy of the sensor and the limitations of a single sensor.

[0033] (2) Inertial Measurement Unit (IMU): In recent years, IMUs have been gradually applied underground. For example, some studies use IMUs (including accelerometers, gyroscopes, etc.) to monitor the attitude information of hydraulic supports in real time and combine them with data fusion algorithms to perform high-precision attitude calculations. IMUs have high integration and good dynamic responsiveness, but they may still be affected by electromagnetic interference in the complex environment of underground mines.

[0034] 2) Multi-sensor fusion technology.

[0035] (1) Multi-sensor fusion: In order to improve the reliability of hydraulic support posture detection, multiple sensor technologies are combined for data fusion to overcome the limitations of single sensor methods. Foreign scholars have also conducted in-depth research in the field of multi-sensor fusion. Some domestic studies combine multi-sensor fusion technology with intelligent algorithms (such as deep learning and machine learning) to improve the intelligence and automation level of hydraulic support posture detection. For example, using classic fusion algorithms such as Kalman filtering and particle filtering, the data of multiple sensors are fused to improve the robustness and accuracy of hydraulic support posture detection. These studies provide theoretical support for the reliability of hydraulic support posture monitoring systems.

[0036] (2) Fusion of multi-sensor data and intelligent algorithms: Some domestic studies have combined multi-sensor fusion technology with intelligent algorithms (such as deep learning and machine learning) to improve the intelligence and automation level of hydraulic support posture detection. By fusing multiple sensor data and combining neural networks or other intelligent algorithms, the posture detection and prediction of hydraulic supports can be achieved. This method strives to improve the system's adaptability and accuracy. This method combines the advantages of multiple technologies and can adapt to complex environments. It has high accuracy and real-time performance, but the system complexity also increases accordingly, and the requirements for hardware and computing resources are high.

[0037] 3) Non-contact monitoring technology.

[0038] (1) Posture detection based on point cloud SLAM technology: Some advanced research teams abroad have applied Simultaneous Localization and Mapping (SLAM) technology to the posture detection of hydraulic supports, especially in dynamic and unknown environments. SLAM technology obtains environmental information through cameras or lidar, and simultaneously performs environmental modeling and posture calculations. Combined with the motion characteristics of the hydraulic support, its position and posture are calculated in real time. SLAM technology can provide high-precision posture detection and is suitable for dynamic and unknown environments, but it is computationally intensive and requires high hardware performance.

[0039] (2) Image processing and pose estimation based on deep learning: With the development of deep learning technology, researchers have begun to apply convolutional neural networks (CNN) to hydraulic support pose detection, achieving high-precision pose estimation through image processing. By training deep learning models and using large-scale data sets, key points of hydraulic supports in videos are identified and extracted to perform three-dimensional pose estimation. Fusion processing is performed with three-dimensional point cloud data or other sensor data to improve detection accuracy and robustness. Deep learning methods can automatically learn complex features and have strong adaptability, but they have high requirements for data sets and computing resources, and the model training and inference process are complex. Real-time performance remains a challenge.

[0040] Currently, significant progress has been made in the research of hydraulic support posture detection methods. Domestically, greater emphasis is placed on the combination of traditional sensor technology and intelligent algorithms to improve system accuracy and reliability. Extensive application research has been conducted in multi-sensor fusion technology and three-dimensional point cloud technology, and the technology is gradually developing towards intelligence and automation. Foreign countries, on the other hand, are more focused on algorithm optimization and the application of inertial navigation systems, SLAM technology, and deep learning in complex dynamic environments. Machine learning and artificial intelligence technologies are also being introduced into posture monitoring. With technological advancements, hydraulic support posture detection will develop in the direction of intelligence, high precision, and high reliability in the future, focusing on the integrated application of multiple technologies and their adaptability in complex underground environments.

[0041] The technical problems existing in the relevant technologies are as follows:

[0042] (1) Single sensor technology: The accuracy of the inclination sensor is a problem. The inclination sensor is easily affected by vibration and noise in complex environments, resulting in insufficient detection accuracy. In coal mines, the vibration and electromagnetic interference of mechanical equipment can cause deviations in sensor data, reducing the accuracy of posture monitoring. At the same time, factors such as dust, humidity, and temperature changes caused by the underground environment can affect the performance of the sensor, leading to data drift, sensor failure and other problems.

[0043] (2) Inertial Measurement Unit: IMUs are prone to drift after long periods of operation, especially when stationary. Drift errors accumulate, leading to large deviations in attitude calculations. Furthermore, the IMU is affected by the complex geomagnetic environment downhole, which can lead to inaccurate attitude calculations.

[0044] (3) Multi-sensor fusion: Multi-sensor systems usually require sensors to be placed at key locations on the hydraulic support. However, the sensor installation location and method may affect the data collection effect. In addition, improper initial calibration of sensors or the accumulation of calibration errors after long-term use can also lead to deviations in attitude detection results.

[0045] (4) Fusion of multi-sensor data and intelligent algorithms: Multi-sensor fusion methods, such as Kalman filtering, particle filtering, and neural networks, can improve the accuracy of attitude monitoring. However, these algorithms are highly complex and require high computing resources. Especially in underground environments, where computing resources are limited, the real-time and stability of these algorithms are restricted. Although fusion algorithms can improve accuracy to a certain extent, they are still not robust enough in the face of sensor failures or extreme environments, which may lead to uncertainty in attitude solution results. In particular, when there is conflicting sensor data or a large amount of redundant data, the fusion algorithm may not be able to accurately distinguish between valid data and noise.

[0046] (5) Posture detection based on point cloud SLAM technology: Although three-dimensional point cloud technology can provide accurate posture detection, the amount of point cloud data is large and the processing and storage requirements are high. Especially in real-time monitoring, the processing speed of point cloud data cannot meet the actual needs, resulting in system response delays. In addition, the accuracy of point cloud data is easily affected by environmental factors such as underground dust and insufficient light. When facing large-scale point cloud data, the point cloud registration algorithm has a heavy computational burden and is prone to falling into local optimal solutions in high-noise environments, resulting in insufficient registration accuracy. This makes the application of point cloud technology in underground environments face great challenges.

[0047] (6) Image processing and pose estimation based on deep learning: The application of intelligent algorithms based on deep learning (such as neural networks and genetic algorithms) in posture detection requires a large amount of training data to ensure the accuracy and stability of the model. However, in practical applications, it is often difficult to obtain sufficient and representative training data. In addition, the insufficient generalization ability of the model under different downhole environments also limits the popularization and application of intelligent algorithms. Intelligent algorithms usually require large computing resources and storage space, which is a bottleneck problem in downhole monitoring systems. The computing resources in the downhole environment are limited, and it is difficult to support complex intelligent algorithms for real-time calculations, which limits the application of intelligent algorithms.

[0048] (7) System integration and maintenance: The hydraulic support posture detection system needs to integrate multiple sensors, data fusion algorithms, communication modules and host computer systems, and the system integration level is high. However, due to the complexity of the underground environment and the diversity of sensor types, the stability and reliability of the system are difficult to guarantee, and the integrated system often faces the problem of insufficient coordination between modules. At the same time, the maintenance cost of underground equipment is high, and the troubleshooting, calibration and maintenance of sensors and communication equipment are difficult. Once the sensor deviates or fails, the overall performance of the system will be affected, and regular calibration and maintenance are required, which is often difficult to achieve in a production environment.

[0049] Despite significant progress in hydraulic support posture detection technology, there are still some shortcomings and drawbacks that need to be addressed in practical applications. Future research should focus on improving the robustness of sensors and fusion algorithms, optimizing the efficiency of point cloud data processing, enhancing system integration, and exploring more efficient and stable deep learning intelligent algorithms to meet the needs of posture detection in complex underground environments.

[0050] With the rapid development of artificial intelligence (AI) and computer vision technologies in recent years, AI-based visual keypoint detection algorithms have provided a new solution for hydraulic support posture monitoring. By analyzing real-time video images captured by surveillance cameras, AI-based visual keypoint detection algorithms automatically identify and extract key feature points of the hydraulic support, and then calculate its position and posture within the image. This method not only offers the advantages of being non-contact, highly real-time, and highly adaptable, but also maintains high detection accuracy even in complex environmental conditions.

[0051] Therefore, in order to solve the technical problems existing in the related technologies, the embodiments of the present disclosure propose a method for hydraulic support posture recognition of a comprehensive mining working face based on AI deep learning, which is targeted and highly applicable. Based on the working face and machine camera video, by constructing a hydraulic support rigid body key point annotation data set and realizing the migration application of the algorithm, it fills the current gap in rigid body mechanical production equipment in the coal mine field based on AI visual key point monitoring and recognition. Furthermore, by introducing an algorithm module to update the key point recognition deep learning network structure, the hydraulic support posture key point detection and recognition link is optimized for the special environment of coal mines underground, so as to achieve simultaneous improvement in recognition accuracy and algorithm performance, and provide a new idea for hydraulic support posture monitoring. The embodiments of the present disclosure not only have the advantages of non-contact, strong real-time performance and high adaptability, but also can maintain a high detection accuracy under complex environmental conditions, can solve many problems existing in the related technologies, and lay a foundation for the intelligent and unmanned automatic follow-up control of hydraulic supports on comprehensive mining working faces.

[0052] The disclosed embodiments are aimed at the problem of hydraulic support posture recognition in actual industrial applications underground in coal mines. They can overcome the many difficulties and shortcomings of sensor recognition, multi-sensor fusion, point cloud SLAM technology in data acquisition, equipment maintenance, system integration and other aspects in related technologies. They aim at the defects of current research on hydraulic support posture detection and recognition in underground field applications, and optimize each link of the recognition process. The disclosed embodiments are different from previous related studies. By introducing an algorithm module to update the key point recognition deep learning network structure and using an optimized key point detection algorithm, the accuracy and stability of key point recognition of hydraulic support equipment can still be guaranteed in the special environment of underground coal mines. In the coal mine industry application scenario, this method is completely based on the existing video surveillance equipment of the underground working face. It has low cost, strong applicability, high robustness, good performance and is easy to implement. It has a wider range of application scenarios and has a higher recognition accuracy and recognition stability. It can provide monitoring information for AI visual monitoring of the position of underground hydraulic supports and automatic following and pulling of hydraulic supports, and make up for the problems of complex installation, high cost and high failure rate of hydraulic support straightness monitoring based on sensors, inertial navigation and other technologies, and provide a new technical solution for the straightness of hydraulic supports and the identification and monitoring of guard plate retraction.

[0053] In the embodiment provided by the present disclosure, a key point recognition model can be constructed to identify the current posture information of the hydraulic support in the fully mechanized mining working face through the key point recognition model, thereby realizing accurate monitoring of the posture of the hydraulic support.

[0054] Specifically, such as Figure 1 As shown, Figure 1 This is a schematic diagram of the process of building a key point recognition model provided by an embodiment of the present disclosure. In the process of building a key point recognition model, the following steps may be specifically included:

[0055] Step S110: construct a data training data set.

[0056] In this embodiment, cameras can be installed at various locations within the mine. These cameras can capture video footage of the fully-mechanized mining face from different angles, thereby obtaining monitoring videos of the hydraulic supports within the fully-mechanized mining face. Because this embodiment requires identifying the hydraulic supports within the fully-mechanized mining face and obtaining their positional information, the current operating status of the hydraulic supports can be determined. Therefore, it is necessary to extract video images containing the operating status of the hydraulic supports from the monitoring videos of the hydraulic supports.

[0057] In one embodiment, the hydraulic support monitoring video can be segmented to extract video images of different working states of the hydraulic support. The working states of the hydraulic support can include: retracting / extending the side guards, raising / lowering the uprights, moving the base legs, or pushing the chute with the cylinder.

[0058] In an embodiment, the video image may also be enhanced.

[0059] Specifically, in the process of extracting target features, visual AI recognition in coal mines often ignores the impact of the special working environment underground on the original video or image material, resulting in inaccurate recognition of the algorithm model and affecting the model recognition effect. In view of the low-light, high-dust and high-water-mist monitoring video environment characteristics unique to the fully mechanized mining working face in coal mines, in order to improve the recognition accuracy of the hydraulic support key point recognition model, the collected video data is enhanced, which mainly includes low-light video filtering and mixed brightness enhancement, and low-light video image defogging. By processing the original video with a video image enhancement algorithm, the contrast of the video can be significantly improved, and complex interference factors such as noise, stray light, and water mist and dust in the environment can be suppressed, so that the enhanced video image presents clearer and richer detail information, thereby improving the recognition accuracy of the support posture key point detection.

[0060] In the embodiment, in the process of constructing the data training set, data annotation strategy design and data annotation implementation may also be performed on the key points of the hydraulic support in the above video image.

[0061] Specifically, the real-time status of the hydraulic support posture in the surveillance video can be classified. Based on the key structural parts of the support and the actual visual range of the working surface camera, the embodiment can select only the parts that the camera may see as key points for annotation. At the same time, the distance relationship between the key parts of the support and the camera must also be considered for classification and annotation.

[0062] In this embodiment, the hydraulic support's position under the camera is primarily influenced by three factors: the distance from the lens, the displacement of the support's components during operation, and the lens's field of view. Taking these factors into consideration, the embodiment identifies key support positions, primarily including the position of the support's side guard and top beam, and the position of the support's base legs. Within these two areas, the support's field of view in different camera images is further considered to determine detailed position classifications.

[0063] In the embodiment provided in the present disclosure, since the data training set constructed based on the surveillance video is limited, the training set can also be expanded based on the obtained training set so that the model has sufficient training samples for training to ensure the accuracy of model recognition.

[0064] Specifically, the data training set can be expanded through image enhancement or inverse enhancement algorithms. Image enhancement or inverse enhancement algorithms can increase the diversity of the training data set by generating images under different conditions, thereby improving the robustness and generalization ability of the model, because the model needs to accurately identify targets in the complex environment of underground working faces in coal mines.

[0065] For example, when performing image inverse enhancement, embodiments can help generate low-quality images to simulate real-world data, improving the model's performance in practical applications. Expanding the dataset through these methods not only alleviates data insufficiency but also enhances the model's adaptability to different scenarios and conditions, thereby improving ultimate detection accuracy.

[0066] In addition, image enhancement or inverse enhancement methods may include: image channel, mask or filter processing, image histogram equalization or image Fourier transform, etc., but the embodiment is not limited thereto.

[0067] Step S120: optimizing the key point recognition model architecture.

[0068] In this embodiment, the key point recognition model is illustrated using the Yolov8-Pose key point recognition algorithm model as an example. Based on the official Yolov8-Pose human key point recognition algorithm model, this embodiment improves the Yolov8-Pose model to enhance target detection accuracy and efficiency. Using the MobileViT architecture within the standard Yolov8-Pose model improves the model's ability to recognize the poses of key points on targets such as hydraulic support guards, top beams, and base legs in low-light, misty, and dusty underground environments, effectively addressing interference from densely packed targets.

[0069] Specifically, the embodiment can use the MobileViT architecture in the standard YOLOv8-Pose model. By introducing a lightweight Transformer module to optimize the backbone network, global feature capture can be achieved, and the key point pose recognition accuracy of targets such as hydraulic support guard plates, top beams, and base legs in weak light, water mist, and dust environments underground can be improved.

[0070] MobileViT uses CNN as its basic architecture and embeds Transformer layers in key modules to form a local-global feature fusion mechanism. Its core module, MobileViT Block, consists of a three-layer structure:

[0071] (1) Local feature extraction layer: extracts spatial local features through standard convolution kernel (such as 3×3).

[0072] (2) Global modeling layer: Use Transformer to perform self-attention calculation on the expanded image block sequence to capture global semantic relationships.

[0073] (3) Feature folding layer: folds the processed block sequence back into image format output.

[0074] The embodiment can adopt a lightweight optimization strategy to reduce the model calculation amount by controlling the number of channels and layers of multi-head self-attention, and reduce the design complexity by repeatedly stacking MobileViT Blocks with the same structure.

[0075] In this embodiment, to enhance the feature extraction capabilities of the YOLOv8 network, the MobileViT module is introduced into the YOLOv8 backbone network, replacing some of the original convolutional blocks to improve the model's ability to capture both global and local features. The MobileViT module first uses convolution operations to extract local spatial information and then encodes global information through the Transformer. This ensures that the network can process both local and global features. Given an input feature map, the MobileViT module first encodes the local spatial information of the input feature map through a convolutional layer. It then expands these features into multiple pixel blocks, which are then encoded using the Transformer to encode global information within these blocks. Finally, the encoded information is folded back to its original form and fused with the input features through a convolution operation to generate enhanced output features. Compared to traditional Transformer blocks, the MobileViT block has fewer parameters and is therefore more computationally efficient, helping to reduce the number of parameters and computational complexity of the overall model. This optimization not only improves the network's feature extraction capabilities but also improves the overall accuracy of personnel recognition in underground environments with low light, water mist, and dust, while maintaining high detection speed.

[0076] In the embodiment, further, to address the problems of obstruction by interfering objects underground, mutual obstruction by the large feet of the support base, and feature misidentification, a PSA (Pyramid Split Attention) self-attention mechanism can be introduced to improve the model's ability to extract target key points and enhance the accuracy of target detection for personnel. The anchor box screening method of the Soft Non-Maximum Suppression algorithm (Soft-NMS) is used to reduce the problem of missed detection of identified targets. Model training is performed based on the optimized model architecture.

[0077] Specifically, PSA uses a channel splitting strategy to divide the input feature map into two parts: a local branch and a global branch. The local branch can preserve spatial details through 3×3 convolution, while the global branch can use multi-head self-attention to model long-range dependencies. The outputs of the two branches can then be fused through 1×1 convolution, and residual connections are added to maintain gradient stability.

[0078] In the deep layers of the backbone network, the Bottleneck structure in the original CSP module can be replaced to enhance the extraction of high-level semantic features. In the neck of the feature pyramid, PSA is inserted at the cross-scale connection of PANet to optimize multi-scale feature fusion.

[0079] In an embodiment, PSA can enhance the model's ability to extract keypoint information at specific locations by calculating the relationships between input features and adjusting feature weights. Its main principles include: The self-attention mechanism adjusts feature weights by calculating the relationships between input features. The model calculates the similarity between each feature and other features and generates a weighted feature representation based on this. This mechanism enables the model to focus on the most relevant information in the input data, thereby improving feature extraction. Position sensitivity: The PSA mechanism introduces position sensitivity based on traditional self-attention, namely, considering the relative spatial position of features when calculating attention. This means that the model not only focuses on the content of the features, but also their specific location in the input image. This is particularly important for keypoint detection, as features at different locations have different impacts on target recognition. Information fusion: The PSA mechanism can generate a richer feature representation by weighting and aggregating information from different locations in the feature map. This process can effectively suppress background noise and interference, improving keypoint recognition accuracy.

[0080] By introducing SPA, the effects of the embodiment may include:

[0081] (1) Enhanced local feature extraction: The PSA mechanism is particularly suitable for dense target detection tasks because it can effectively focus on local features around key points, enhancing the detection ability of small or occluded targets. This capability is particularly important in the complex environment of coal mines, where support structures may be obscured or affected by dust and water mist.

[0082] (2) Improved detection accuracy: By incorporating spatial location information, the PSA mechanism can more accurately identify key points of an object, especially in complex backgrounds. When determining the location of a key point, the model takes into account its importance in the overall structure, thereby reducing misidentification and missed identification.

[0083] (3) Reduced computational complexity: The PSA mechanism reduces the computation of irrelevant features by focusing on specific locations, thereby improving the computational efficiency of the model. This allows the model to run efficiently even when resources are limited, meeting the needs of real-time monitoring.

[0084] (4) Improved model robustness: Since the PSA mechanism takes positional relationships into account during feature extraction, the model is more adaptable to environmental changes (such as illumination changes, occlusion, etc.). This robustness makes the model more stable in practical applications, especially in the ever-changing coal mine environment.

[0085] (5) Improved model performance: After introducing the PSA self-attention mechanism, the YOLOv8 model can more accurately extract the key point information of the hydraulic support, thereby significantly improving the recognition accuracy of the key points. This improvement not only improves the detection performance, but also provides more reliable support for subsequent real-time monitoring and safety management.

[0086] In the embodiments provided in the present disclosure, the missed detection rate can be reduced by optimizing the Yolov8-Pose key point recognition model architecture - the Soft-NMS anchor box screening method. Soft-NMS is an improved anchor box screening method used in the post-processing step of target detection. Compared with the traditional non-maximum suppression (NMS) method, Soft-NMS has significant advantages in reducing the missed detection rate. In traditional NMS, the model first sorts the anchor boxes according to the prediction score and selects the box with the highest score as the retained box. Then, for boxes whose overlap (Intersection over Union, IoU) with the box exceeds a set threshold, they are directly removed. This hard suppression method may lead to missed detections, especially in the case of dense objects, because if a box is suppressed, it may cause the related boxes below it to be discarded. Soft-NMS replaces hard suppression by assigning a weight to each anchor box. When the IoU between anchor boxes exceeds the set threshold, Soft-NMS does not directly delete the overlapping boxes, but adjusts their scores based on their IoU values ​​with the retained boxes. Specifically, the score of the overlapping box will be attenuated according to the IoU value with the retained box, so that some boxes with relatively high scores but some overlap can be retained. Ultimately, Soft-NMS will select the box with a score higher than a certain threshold as the final detection result.

[0087] By combining the anchor box screening method with Soft-NMS, the following effects can be achieved:

[0088] (1) Reduce missed detection rate: Since Soft-NMS allows overlapping boxes to retain their scores to a certain extent, it reduces the loss of targets in dense areas. This is especially important for key point detection of hydraulic supports, because support parts may appear in multiple overlapping anchor boxes in the same image, and traditional NMS can easily cause important parts to be missed.

[0089] (2) Improved detection accuracy: By retaining overlapping boxes with relatively high scores, Soft-NMS can more comprehensively reflect the presence of the target. This approach not only improves the overall detection accuracy, but also enhances the model's performance in complex environments, such as in coal mines with low illumination and high dust conditions, where key points may be obscured due to environmental factors.

[0090] (3) Strong adaptability: Soft-NMS has better adaptability and can handle scenes with different target densities. Its score decay mechanism enables the model to more flexibly select appropriate anchor boxes to retain in target-dense areas, which can effectively cope with the challenges brought by environmental changes and target changes.

[0091] (4) High computational efficiency: Although Soft-NMS increases the complexity of post-processing to a certain extent, compared with traditional NMS, it can improve the final detection results under the same computational conditions, especially in tasks requiring high precision, thereby improving the overall processing efficiency.

[0092] (5) Enhanced model robustness: Soft-NMS retains multiple high-scoring anchor boxes, allowing the model to maintain high recognition capabilities in complex situations such as occlusion and similar targets. This robustness is very important in practical applications, especially in dynamic environments such as coal mines.

[0093] In an embodiment, the optimized model can be trained using the training set constructed above.

[0094] Step S130: Accuracy evaluation and iterative update of the key point recognition model.

[0095] In an embodiment, when constructing a training data set for a hydraulic support, the training data set and the test data set can be divided into a certain ratio (such as 8:2). Based on the difference between the real data annotation of the test data set and the model inference result, the model is evaluated for the false alarm and missed alarm of each key point annotation under the classification of each support guard plate and top beam posture, and the support base big foot posture. Based on this, the recognition effect of the model on the posture of each key point of the hydraulic support is judged, and for the key point categories with higher false alarm and missed alarm rates, the relevant category data annotation is further added, and the model is iteratively trained. The model accuracy evaluation and update iterative process are repeated until the false alarm rate and missed alarm rate of the posture key point recognition of each category of the model meet the requirements, and the final version of the model is output.

[0096] Step S140: performing position recognition on the hydraulic support of the fully mechanized mining working face based on the key point recognition model.

[0097] In this embodiment, the current position of a hydraulic support can be identified based on the trained key point recognition model. Inputs to the key point recognition model include real-time video streams from monitoring cameras on the fully-mechanized working face in an underground coal mine, and videos or images of the working face. Real-time key point position information on the hydraulic support's side guards and base legs can be obtained to provide a reference for real-time position monitoring, such as identifying the hydraulic support's side guards and monitoring the straightness of the support.

[0098] Based on the above embodiments, the embodiments of the present disclosure have the following beneficial effects:

[0099] (1) Innovative introduction of AI visual key point detection technology and algorithm: Innovatively apply AI visual key point detection and recognition technology and algorithm used for human body to hydraulic support mechanical equipment in underground fully-mechanized mining face of coal mines, which can provide monitoring information for AI visual monitoring of underground hydraulic support posture and automatic following and pulling of hydraulic support, and make up for the problems of complex installation, high cost and high failure rate of hydraulic support straightness monitoring based on sensors, inertial navigation and other technologies.

[0100] (2) Innovative data labeling strategy for key point detection of hydraulic supports: Based on the working characteristics of hydraulic support equipment and the video image acquisition of underground working surface cameras, and referring to the human visual AI key point detection data set labeling method, an innovative data labeling strategy for key point detection of hydraulic supports suitable for the visual AI key point detection and recognition algorithm model is designed to determine the detailed posture classification of key points of hydraulic supports, laying the data foundation for applying human key point technology to underground hydraulic support equipment.

[0101] (3) Richness and diversity of training data: The appropriate use of video image enhancement / de-enhancement algorithms in the dataset construction further expands the diversity of the dataset, enabling the model to be trained under different conditions. This diversity helps improve the robustness of the model in complex environments, enabling it to better adapt to the actual working conditions of underground hydraulic support equipment.

[0102] (4) Innovative algorithm model architecture optimization: The introduction of new algorithm modules such as MobileViT, PSA self-attention mechanism, and anchor box screening of the soft non-maximum suppression algorithm (Soft-NMS) optimizes the model architecture of the Yolov8-Pose human key point recognition and detection algorithm, achieving the recognition accuracy and recognition stability of hydraulic support key point detection in complex environments. The lightweight characteristics of the MobileViT architecture enable the model to run efficiently even with limited computing resources. At the same time, the PSA mechanism enhances the model's ability to extract target key points, especially in the presence of occlusion or complex backgrounds, and can effectively distinguish and identify the key parts of the support, thereby improving the overall detection efficiency and accuracy.

[0103] (5) Video image enhancement improves recognition accuracy: Video image enhancement / inverse enhancement algorithms are introduced into the key point detection data collection, data set construction, and model inference process. In the low-light, high-dust, and high-water mist environment unique to coal mines, it is often difficult to clearly identify targets in original videos or images. By implementing low-light video filtering and mixed brightness enhancement, defogging, and other image enhancement algorithms during training data collection and model inference, the contrast of the video is significantly improved, noise and stray light are suppressed, and the key points of the hydraulic support are visually clearer, which improves the performance of the model in key point detection, reduces the risk of false positives and false negatives, and improves the overall detection accuracy.

[0104] (6) Continuous improvement mechanism and practical application capability of the model: By dividing the training set into the test set, the model is dynamically adjusted based on the difference between the real data annotation and the inference results. For key point categories with poor recognition results, relevant data is continuously added and iterative training is performed to ensure that the final model's recognition false alarm rate and false negative rate for each key point meet the expected requirements. This continuous improvement process makes the model more reliable and applicable in practical applications.

[0105] (7) High practicality and low cost: In the application scenario of coal mine industry, the embodiment of the present disclosure is based on the existing video surveillance equipment of the underground working face, which has low cost, strong applicability, high robustness, good performance and is easy to implement. It has loose restrictions on data input, a wider range of application scenarios, high recognition accuracy and recognition stability, and can provide reliable information for the real-time recognition of the posture of the hydraulic support guard plate, base foot, push-pull frame, etc. of the underground fully mechanized mining face of the coal mine.

[0106] Based on the above embodiments, the present disclosure also provides a method for recognizing the position of a hydraulic support in a fully mechanized mining working face. Figure 2 As shown, the method may include the following steps:

[0107] In step S210, the monitoring data of the hydraulic supports of the fully mechanized mining face is obtained, wherein the monitoring data of the hydraulic supports includes the working status of the hydraulic supports.

[0108] In step S220, the hydraulic support monitoring data is input into the trained key point recognition model to obtain the current posture information of the hydraulic support in the fully mechanized mining working face.

[0109] Among them, the trained key point recognition model is used to obtain the key point position information of the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

[0110] In this embodiment, the current position information of the hydraulic support in the fully mechanized mining face can be determined based on the position information of key points in the hydraulic support in the fully mechanized mining face. The trained key point recognition model can determine the current position information of the hydraulic support based on the relative relationship between the position information of each key point in the historical data and the corresponding position information of the hydraulic support, and based on the position information of the key points in the current hydraulic support.

[0111] In an embodiment, the hydraulic support monitoring data may include monitoring videos or images of the hydraulic support in the fully mechanized mining working face, but the embodiment is not limited thereto.

[0112] By inputting the monitoring data of hydraulic supports in a fully mechanized mining face into a trained key point recognition model, the trained key point recognition model detects the key points of the hydraulic supports in the monitoring data, obtains the position information of the key points of the hydraulic supports, and determines the position information of the hydraulic supports based on the key point position information. By detecting and recognizing the key points of the hydraulic supports, this embodiment avoids the large measurement errors and high costs associated with the use of related sensors in related technologies while ensuring recognition accuracy.

[0113] Based on the above embodiment, in another embodiment provided by the present disclosure, the above step S210 may further include:

[0114] Step S211, obtaining the monitoring video of the hydraulic support of the fully mechanized mining working face.

[0115] Step S212: pre-process the hydraulic support monitoring video to obtain hydraulic support monitoring data, wherein the pre-processing includes at least one of the following: low-light video filtering and mixed brightness enhancement processing and low-light video image defogging processing.

[0116] In the embodiment, the recognition accuracy can be improved by obtaining the hydraulic support monitoring video of the fully mechanized mining working face and preprocessing the hydraulic support monitoring video.

[0117] Specifically, to improve recognition accuracy, the collected video data can be enhanced to address the unique low-light, high-dust, and high-water-mist surveillance video environment characteristics found in underground coal mines. The algorithms used primarily include low-light video filtering and mixed brightness enhancement, as well as low-light video image defogging. The original video is processed using a video image enhancement algorithm to improve contrast, suppress noise, stray light, and water-mist / dust interference, and provide the enhanced video with richer detail information. Currently, existing gait recognition systems or algorithms often neglect targeted image enhancement and quality improvement of the initial video or image material during the gait extraction process, resulting in inaccurate gait contour extraction and impacting gait recognition results.

[0118] The embodiment targets the special application scenarios of low illumination visible light, high fog and high dust underground. Through certain image enhancement algorithms, low illumination video filtering and hybrid brightness enhancement algorithm, and low illumination video fast defogging algorithm, video image enhancement processing is performed on the surveillance camera video to improve the recognition accuracy of posture.

[0119] In an embodiment, a training sample may be obtained and a preset model may be trained using the training sample to obtain a trained key point recognition model, wherein the training sample may include working videos of hydraulic supports in different working states.

[0120] In one embodiment, the training sample may include the training dataset described in the above embodiment. During the acquisition of the training sample, historical camera surveillance videos of hydraulic supports in a fully mechanized mining face may be obtained. The historical surveillance videos are segmented to generate multiple sub-videos, which are then used as training samples. Each sub-video carries annotation information about the hydraulic support, which indicates the hydraulic support's operating status.

[0121] In an embodiment, the training samples may include a first training sample, and the first training sample may be expanded by an image enhancement or inverse enhancement algorithm to obtain a second training sample, so as to solve the problem of insufficient training samples.

[0122] Specifically, a first training sample can be obtained, processed using an image inverse enhancement algorithm to obtain a second training sample, and the first and second training samples are used as training samples; wherein the image inverse enhancement algorithm is used to reduce the image quality of the first training sample. In this embodiment, by processing the first training sample using an inverse enhancement algorithm to obtain a second training sample, the embodiment can more closely simulate the actual environment in a mine, enabling the trained model to more accurately identify the position information of the hydraulic support. This position information can include at least one of the following: the position of the support guard plate and the top beam, and the position of the support base leg.

[0123] In the case of dividing each functional module according to each function, an embodiment of the present disclosure provides a hydraulic support posture recognition device for a fully mechanized mining working face. The hydraulic support posture recognition device for a fully mechanized mining working face can be a server, a terminal, or a chip applied to a server. Figure 3 This is a schematic block diagram of the functional modules of a hydraulic support posture recognition device for a fully mechanized mining working face provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the hydraulic support posture recognition device of the fully mechanized mining working face includes:

[0124] A monitoring data acquisition module 31 is used to acquire monitoring data of the hydraulic support of the fully mechanized mining working face, wherein the monitoring data of the hydraulic support includes the working status of the hydraulic support;

[0125] The posture recognition module 32 is used to input the hydraulic support monitoring data into the trained key point recognition model to obtain the current posture information of the hydraulic support in the comprehensive mining working face; wherein, the trained key point recognition model is used to obtain the key point position information in the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

[0126] The obtaining of the monitoring data of the hydraulic support of the fully mechanized mining working face includes:

[0127] Obtaining a monitoring video of the hydraulic support of the fully mechanized mining working face;

[0128] The hydraulic support monitoring video is preprocessed to obtain the hydraulic support monitoring data; wherein the preprocessing includes at least one of the following: low-light video filtering and mixed brightness enhancement processing and low-light video image defogging processing.

[0129] In another embodiment provided by the present disclosure, the apparatus further includes:

[0130] A training sample acquisition module is used to acquire training samples; wherein the training samples include working videos of hydraulic supports in different working states;

[0131] The training module is used to train the preset model through training samples to obtain the trained key point recognition model.

[0132] In another embodiment provided by the present disclosure, the training sample acquisition module is specifically configured to:

[0133] Obtaining historical monitoring videos of the hydraulic supports in the fully mechanized mining face by cameras;

[0134] The historical monitoring video is segmented to obtain a plurality of sub-videos, and the plurality of sub-videos are used as training samples; wherein each of the sub-videos carries the annotation information of the hydraulic support, and the annotation information represents the working status of the hydraulic support.

[0135] In another embodiment provided by the present disclosure, the apparatus further includes a training sample expansion module, specifically configured to:

[0136] Obtain a first training sample;

[0137] The first training sample is processed by an image inverse enhancement algorithm to obtain a second training sample, and the first training sample and the second training sample are used as the training samples; wherein the image inverse enhancement algorithm is used to reduce the quality of the image in the first training sample.

[0138] In another embodiment provided by the present disclosure, the posture information includes at least one of the following: the posture of the support guard plate and the top beam, and the posture of the support base large foot.

[0139] The disclosed embodiments provide a device for recognizing the position and posture of a hydraulic support in a fully mechanized mining face. The device inputs monitoring data of the hydraulic supports in the fully mechanized mining face into a trained key point recognition model, detects key points of the hydraulic supports in the monitoring data using the trained key point recognition model, obtains position information of the key points of the hydraulic supports, and determines the position and posture information of the hydraulic supports based on the key point position information. By detecting and recognizing the position and posture of the key points of the hydraulic supports, the embodiments avoid the problems of large measurement errors and high costs associated with the use of related sensors in related technologies while ensuring recognition accuracy.

[0140] An embodiment of the present disclosure further provides an electronic device, comprising: at least one processor; a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the above method disclosed in the embodiment of the present disclosure.

[0141] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 4 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.

[0142] The processor 1801 can also be referred to as a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in the embodiments of the present disclosure can be completed by hardware integrated logic circuits in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present disclosure can be directly implemented as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in the memory 1802, such as a storage medium mature in the art, such as a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The processor 1801 reads the information in the memory 1802 and, in conjunction with its hardware, completes the steps of the method.

[0143] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Figure 5 The computer system 1900 shown is installed with the programs constituting the software. When the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 5 A structural block diagram of a computer system provided by an exemplary embodiment of the present disclosure.

[0144] Computer system 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0145] like Figure 5As shown, computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. Various programs and data required for the operation of computer system 1900 may also be stored in RAM 1903. Computing unit 1901, ROM 1902, and RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to bus 1904.

[0146] Several components within computer system 1900 are connected to I / O interface 1905, including an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input numeric or character information and generate key input signals related to user settings and / or function control of an electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 1908 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices over a network, such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0147] The computing unit 1901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned method disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above-mentioned method disclosed in the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0148] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.

[0149] The computer-readable storage medium in the embodiments of the present disclosure can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. The above-mentioned computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specifically, the above-mentioned computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0150] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0151] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method disclosed in the embodiments of the present disclosure is implemented.

[0152] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The modules, components, or units described in the embodiments of the present disclosure may be implemented in software or hardware. The names of the modules, components, or units do not necessarily limit the modules, components, or units themselves.

[0155] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0156] The above descriptions are merely some embodiments of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present disclosure.

[0157] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for recognizing the position and posture of a hydraulic support in a fully mechanized mining working face, characterized in that: The method comprises: Acquire monitoring data of a hydraulic support of a fully mechanized mining working face, wherein the monitoring data of the hydraulic support includes a working status of the hydraulic support; Input the hydraulic support monitoring data into the trained key point recognition model to obtain the current posture information of the hydraulic support in the comprehensive mining working face; wherein, the trained key point recognition model is used to obtain the key point position information in the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

2. The method according to claim 1, characterized in that The obtaining of the monitoring data of the hydraulic support of the fully mechanized mining working face includes: Obtaining a monitoring video of the hydraulic support of the fully mechanized mining working face; The hydraulic support monitoring video is preprocessed to obtain the hydraulic support monitoring data; wherein the preprocessing includes at least one of the following: low-light video filtering and mixed brightness enhancement processing and low-light video image defogging processing.

3. The method according to claim 1, characterized in that The method further comprises: Acquire training samples; wherein the training samples include working videos of hydraulic supports in different working states; The preset model is trained using training samples to obtain the trained key point recognition model.

4. The method according to claim 3, characterized in that The obtaining of training samples includes: Obtaining historical monitoring videos of the hydraulic supports in the fully mechanized mining face by cameras; The historical monitoring video is segmented to obtain a plurality of sub-videos, and the plurality of sub-videos are used as training samples; wherein each of the sub-videos carries the annotation information of the hydraulic support, and the annotation information represents the working status of the hydraulic support.

5. The method according to claim 3, characterized in that The method further comprises: Obtain a first training sample; The first training sample is processed by an image inverse enhancement algorithm to obtain a second training sample, and the first training sample and the second training sample are used as the training samples; wherein the image inverse enhancement algorithm is used to reduce the quality of the image in the first training sample.

6. The method according to claim 1, characterized in that The posture information includes at least one of the following: the posture of the support guard plate and the top beam, and the posture of the support base large foot.

7. A hydraulic support posture recognition device for a fully mechanized mining working face, characterized in that: The device comprises: A monitoring data acquisition module is used to acquire the monitoring data of the hydraulic support of the fully mechanized mining working face, wherein the monitoring data of the hydraulic support includes the working status of the hydraulic support; A posture recognition module is used to input the hydraulic support monitoring data into the trained key point recognition model to obtain the current posture information of the hydraulic support in the comprehensive mining working face; wherein, the trained key point recognition model is used to obtain the key point position information in the hydraulic support, and determine the posture information of the hydraulic support based on the key point position information.

8. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images