A deep learning-based method and system for measuring the visual state of a fender

By employing a deep learning-based visual state measurement method, which utilizes dual-camera cross-shooting and weighted processing, the accuracy and real-time performance issues of side panel state detection in existing technologies are resolved, achieving efficient and low-cost side panel state detection.

CN119737183BActive Publication Date: 2025-12-05SHANDONG UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411801542.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-09
Publication Date
2025-12-05
Estimated Expiration
2044-12-09

AI Technical Summary

Technical Problem

Existing technologies for sidewall condition detection suffer from problems such as complex layout, high installation and maintenance costs, difficulty in signal transmission, low measurement accuracy, and poor real-time performance. In particular, in the downhole environment, the consistency of multi-sensor data fusion is poor, and it is greatly affected by equipment vibration, making it difficult to achieve synchronous measurement of multiple targets in all time and space.

Method used

A deep learning-based visual state measurement method is adopted, which uses dual cameras to capture images of the side panels by cross-shooting from the left and right adjacent sides. The deep learning model is then used for feature extraction and weighted processing to achieve accurate detection of the side panel state.

Benefits of technology

It improves the accuracy and real-time performance of side panel condition detection, reduces sensor deployment and maintenance costs, enhances the robustness and synergy of detection, and realizes end-to-end, all-time, multi-target synchronous measurement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119737183B_ABST
    Figure CN119737183B_ABST
Patent Text Reader

Abstract

The application provides a kind of based on deep learning's guard board visual state measurement method and system, comprising: image acquisition unit and visual detection system, image acquisition unit includes: the first image acquisition unit being arranged below the first hydraulic support, the light source being arranged below the second hydraulic support and the second image acquisition unit being arranged below the third hydraulic support;Second hydraulic support is provided with guard board;Light source is directly opposite the guard board, for providing the light intensity and light irradiation angle of measuring guard board;First image acquisition unit and second image acquisition unit are obtained by the way of left and right adjacent side intersection shooting guard board and obtain first acquisition image and second acquisition image;Visual detection system is configured to: receive first acquisition image and second acquisition image, and transmission to deep learning model carries out feature extraction, based on the extracted feature guard board state monitoring, judge whether its working state is consistent with preset state, if not consistent, then early warning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of visual state measurement technology for side panels, and particularly relates to a method and system for visual state measurement of side panels based on deep learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] As a key component of the hydraulic support system for supporting the coal face, the side guard plate's main function is to prevent coal face spalling. During the coal cutting process of the coal shearing machine's drum, the side guard plate on the coal face should retract promptly as the drum passes to avoid cutting interference. Therefore, there are strict requirements for monitoring the working condition of the hydraulic support side guard plate.

[0004] Currently, the main method for detecting the condition of the sidewalls in underground fully mechanized mining faces is through contact sensors. For example, patent application number 201310087316.4, entitled "Hydraulic Support Sidewall Control Device and Control Method Thereof," uses a series connection of sidewall sensors, installing an angle sensor and a pressure sensor on each sidewall to detect and control the condition of each sidewall.

[0005] The aforementioned contact sensors suffer from complex layout and high installation and maintenance costs, along with difficulties in signal transmission and poor consistency in multi-sensor data fusion. Furthermore, the vibrations and weak signals from underground coal mining machinery lead to data transmission difficulties, low measurement accuracy, and challenges in achieving simultaneous measurement of multiple targets across all times and spaces.

[0006] In addition, existing technologies utilize neural network models to measure the extension angle of the side guard plate. For example, patent 202111596613.2, entitled "Method and Device for Measuring the Extension Angle of Side Guard Plate Based on Image Sequence," can achieve real-time measurement and visualization of the extension angle during the movement of the side guard plate. This has significant practical implications for achieving intelligent control of hydraulic supports, preventing coal face spalling and collision interference between the side guard plate and the coal machine drum, and reducing the labor intensity of personnel.

[0007] Existing technologies utilize algorithms to measure the sidewall support. For example, patent CN111173510A employs an inertial navigation device, tilt and displacement sensors, and image analysis methods to construct a detection system for truncated interference. This system suffers from multiple data coupling errors, and the measurement error is large due to the influence of equipment vibration on the contact sensor measurement. Patent CN111810155B uses a visual measurement scheme, which suffers from high false alarm and false negative rates due to the influence of downhole dust and light, and the decision-making and early warning of a single camera sensor. Patent CN114511799B uses two deep learning network models to measure the sidewall support angle, but the model data is large, the training is complex, and the real-time performance is poor. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, the present invention provides a deep learning-based visual state measurement method for side panels, which achieves high accuracy in detecting the working state of side panels.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] In a first aspect, a deep learning-based visual state measurement system for a protective panel is disclosed, comprising: an image acquisition unit and a visual detection system, wherein the image acquisition unit transmits the acquired image to the visual detection system;

[0011] The image acquisition unit includes: a first image acquisition unit disposed below the first hydraulic support, a light source disposed below the second hydraulic support, and a second image acquisition unit disposed below the third hydraulic support;

[0012] The second hydraulic support is equipped with a side guard plate;

[0013] The light source is positioned directly opposite the protective panel to provide light intensity and light illumination angle for measuring the protective panel.

[0014] The first image acquisition unit and the second image acquisition unit capture images of the side panel by taking cross-shooting images from the left and right adjacent sides to obtain the first and second captured images;

[0015] The visual detection system is configured to: receive a first acquired image and a second acquired image, transmit them to a deep learning model for feature extraction, monitor the status of the protective board based on the extracted features, determine whether its working status matches the preset status, and issue an early warning if they do not match.

[0016] As a further technical solution, the first hydraulic support, the second hydraulic support and the third hydraulic support are arranged in sequence and have the same structure. Each of them includes a bottom plate and a top plate, and a connecting column is provided between the bottom plate and the top plate. A side guard plate is provided at the suspended end of the top plate of the second hydraulic support.

[0017] As a further technical solution, the visual inspection system includes a big data management and control platform, a computer, and a cloud server;

[0018] The big data management and control platform stores the first and second captured images of the protective board and the status information of the protective board.

[0019] Both the computer and the cloud server communicate with the big data management and control platform.

[0020] Secondly, a deep learning-based method for measuring the visual state of a protective panel is disclosed, including:

[0021] The first and second images of the protective panel were obtained by taking cross-shot images from the left and right adjacent sides.

[0022] The first and second acquired images are transmitted to a deep learning model for processing to obtain the working status of the protective board.

[0023] The obtained working status of the side guard plate is weighted to determine the final working status of the side guard plate and to determine whether there is any abnormality in the working status of the side guard plate.

[0024] As a further technical solution, the first and second acquired images are transmitted to a deep learning model for processing, including:

[0025] The first and second acquired images are processed to obtain grayscale images of the first and second protective panels respectively.

[0026] Based on the grayscale image of the first side protection plate, the gap between the right side of the side protection plate and the coal wall and the gap between the top beam of the hydraulic support are identified and judged to obtain the working status of the side protection plate.

[0027] Based on the grayscale image of the second side protection plate, the gap between the left side of the side protection plate and the coal wall and the gap between the top beam of the hydraulic support are identified and judged to obtain the working status of the side protection plate.

[0028] As a further technical solution, the six types of data information identified—the left and right working states of the side guard plate, the gap between the side guard plate and the coal wall, and the gap between the side guard plate and the hydraulic support top beam—are integrated and processed, and the actual working state of the side guard plate is finally determined according to the weighting formula.

[0029] As a further technical solution, the specific calculation formula is as follows:

[0030] ;

[0031] in, -1 indicates no, 1 indicates yes, and 0 indicates no target detected. The weighting formula... , , , respectively represent the detection results of the gap between the side guard plate and the coal wall, the detection results of the side guard plate and the top beam of the hydraulic support, and the detection results of the working characteristics of the side guard plate; R represents the final working state of the side guard plate; w i The denominator represents the weight of the impact of three different categories of detection results on the final working state R.

[0032] As a further technical solution, the training process of a deep learning model is as follows:

[0033] A protective board state detection dataset is established based on the first and second acquired images. The protective board state detection dataset is then augmented to obtain a deep learning dataset.

[0034] A deep learning model for detection is trained using a deep learning dataset to obtain a well-trained deep learning model.

[0035] The above one or more technical solutions have the following beneficial effects:

[0036] The technical solution of this invention is to detect and distinguish three different working states of the side guard plate (including support state, retraction state, and running state). It uses a positive light source to illuminate the side guard plate to improve the quality of the captured images, and uses cameras at adjacent workstations to capture images to detect the working state of the side guard plate. By weighted processing of the detection results from the dual workstations, the detection accuracy of the side guard plate working state is high.

[0037] The technical solution of this invention adopts an end-to-end full-vision detection scheme. It inputs the same frame image of the side guard plate captured by dual-position cameras, that is, the image of the same side guard plate captured by two cameras at the same moment, and outputs the working status information of the side guard plate. It has high detection efficiency, strong real-time performance, and strong collaborative perception of the working status of the side guard plate of the entire fully mechanized mining face at the same time. It is less affected by vibration of the working environment and has good robustness.

[0038] The technical solution of this invention uses a visual sensor, which is simple to arrange, has a small total number of sensor points, is easy to install and maintain, and has a low detection cost.

[0039] The technical solution of this invention saves the detection results and image / video information to the cloud, and realizes remote monitoring and alarm through a deep learning model platform. It can conveniently and timely classify the status of the side panel, and facilitate data storage, detection result statistics and hydraulic support control.

[0040] The technical solution of this invention adopts a multi-view status detection scheme for a side panel using dual cameras. It comprehensively decides the current operating status of the side panel by using multiple positions and multiple evaluation criteria, and adopts an end-to-end side panel status detection algorithm, which has high real-time detection performance.

[0041] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0043] Figure 1 This is a schematic diagram of the structure of an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of cloud control according to an embodiment of the present invention;

[0045] Figure 3 This is a flowchart of the detection method according to an embodiment of the present invention;

[0046] Figure 4 This is a flowchart illustrating the image processing of an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the image acquisition structure according to an embodiment of the present invention;

[0048] Figure 6 This is a flowchart of the deep learning algorithm according to an embodiment of the present invention;

[0049] Figure 7 This is a schematic diagram of the gap in an embodiment of the present invention;

[0050] In the diagram, 1. First hydraulic support; 2. Second hydraulic support; 3. Third hydraulic support; 4. Side guard plate; 5. Left-facing workstation camera; 6. Right-facing workstation camera; 7. Positive light source; 8. First gap; 9. Top beam gap. Detailed Implementation

[0051] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0052] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0053] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0054] Existing technologies using contact sensors to measure the sidewall status have several drawbacks: detection accuracy is easily affected by the environment of the fully mechanized mining face, leading to inaccurate data and poor robustness; data transmission in the underground working environment is difficult and lacks real-time performance; multi-sensor data makes it difficult to achieve full-time, multi-target sidewall status detection; sensor installation and maintenance are difficult and costly; and data synchronization and processing of multiple types of sensors are cumbersome, making it impossible to achieve end-to-end status detection.

[0055] In addition, the vibration and equipment displacement caused by the operation of underground fully mechanized mining equipment result in high noise, large measurement errors, and easy damage to contact sensors, making sensor installation and maintenance difficult. Since the fully mechanized mining face is a narrow and long roadway, the number of hydraulic supports varies from 50 to 200, with multiple contact sensors on the same support, making data acquisition, data transmission, and fusion difficult and resulting in cumulative errors. Furthermore, the process of further judging the operating status of the hydraulic support sidewall plate through sensor detection data is complex and has limited real-time performance.

[0056] Example 1

[0057] See appendix Figure 1 , Figure 5 As shown, this embodiment discloses a deep learning-based visual state measurement system for a protective panel, including a first hydraulic support 1, a second hydraulic support 2, a third hydraulic support 3, a protective panel 4, a left-facing workstation camera 5, a right-facing workstation camera 6, and a positive light source 7. The protective panel 4 is the target to be detected. The positive light source 7 illuminates the protective panel 4, and its main function is to improve the clarity of the captured image.

[0058] In this embodiment, the left-facing workstation camera is positioned directly to the lower left of the hydraulic support; the right-facing workstation camera is positioned directly to the lower right of the hydraulic support; and a positive light source is positioned directly below the hydraulic support to provide illumination for the cameras. The side workstation cameras all target the adjacent hydraulic support side panels, while the workstation cameras positioned on both ends target the side panels of their respective supports.

[0059] The left-facing workstation camera 5 on the hydraulic support 3 has a field of view of the side guard plate 4 on the hydraulic support 2, and the right-facing workstation camera 6 on the hydraulic support 1 has a field of view of the side guard plate 4 on the hydraulic support 2. The side guard plate 4 is photographed by shooting the left and right adjacent sides in a cross-shooting manner.

[0060] See appendix Figure 2 As shown, the visual inspection system also includes a big data management and control platform, a computer, and a cloud server;

[0061] The data management platform transmits and stores downhole visual measurement results in a unified manner, while model and data processing are carried out on a computer.

[0062] The big data management and control platform stores the first and second images of the protective panel and the status information of the protective panel;

[0063] Both the computer and the cloud server communicate with the big data management platform.

[0064] Among them, the computer will issue an alarm and stop the machine in time if the side workstation camera captures abnormal working status of the side support plate.

[0065] When the above system is working, please refer to the appendix. Figure 3 As shown, by adjusting the light source and camera field of view, the corresponding first and second acquisition images are obtained. The computer stores a trained deep learning model, which is used to detect the working status of the protective board and determine whether its working status matches the preset status. If it matches, the information is uploaded to the cloud platform; otherwise, an alarm is issued.

[0066] Example 2

[0067] This embodiment discloses a visual detection method for side panel protection based on a deep learning model, the steps of which are as follows:

[0068] Step 1: Under the influence of a light source, the side panel to be tested is photographed by the workstation camera on the adjacent side panels to obtain an image of the current working status of the side panel.

[0069] Step 2: Process the images collected in Step 1 to check the working status of the side panel;

[0070] Step 3: Weight the detection results of the workstation cameras on both sides of the side guardrails to determine the final working state of the side guardrails;

[0071] Step 4: Based on the weighted test results from Step 3, determine whether there are any abnormalities in the working status of the side guard plate;

[0072] Step 5: Store the collected images and obtained status information in the big data management and control platform, and remotely monitor, maintain and adjust parameters in real time through the cloud server.

[0073] In this implementation example, the image processing procedure for the acquired image in step two is as follows:

[0074] The image obtained in step one is processed in grayscale, and feature filtering and redundant information removal are performed to obtain the grayscale image of the side panel.

[0075] After obtaining the grayscale image of the protective board, the ResNet deep learning model is used to extract the feature information in the image.

[0076] The feature maps obtained in step two are subjected to feature analysis, feature fusion, and feature detection. Specifically, feature extraction is performed through image convolutional layers and semantic compression pooling layers. The output feature maps are decoded and compressed through upsampling operations. Feature maps of different scales are linked to achieve contextual feature fusion. CNN and sliding anchor box methods are used to detect targets. The NMS operator is used to remove duplicate detected targets.

[0077] The final detection results obtained in step three are used for state classification and result evaluation using a weighted algorithm.

[0078] For more details, see Figure 4 , 6 The visual inspection system of this embodiment is used to detect the state of the hydraulic support side plate. The training process of the deep learning model and the detection process of the hydraulic support side plate state are as follows:

[0079] S1. Adjust the light intensity and angle of illumination;

[0080] Adjust the light intensity and illumination angle of the positive light source 7 to ensure that the captured image does not have problems such as reflection, overexposure, glare, and halo, and ensure that the captured image is clear and the condition characteristics of the protective plate 4 are obvious.

[0081] S2. Adjust the camera's shooting field of view;

[0082] See appendix Figure 7 As shown, adjust the right-facing workstation camera 6 on the hydraulic support 1 so that its field of view includes all parts of the side guard plate 4, the first gap 8 between the left side of the side guard plate 4 and the coal wall 7, and the gap 9 between the left side of the side guard plate 4 and the top beam of the hydraulic support 2.

[0083] Adjust the left-facing camera 5 on the hydraulic support 3 so that its field of view includes all parts of the side guard plate 4, the gap between the right side of the side guard plate 4 and the coal wall, and the gap between the right side of the side guard plate 4 and the top beam of the hydraulic support 2.

[0084] After adjusting the light intensity, illumination angle, and camera field of view, images are acquired. Data sets are built based on the acquired images, and the model is trained based on the built dataset to obtain the trained model.

[0085] S3. Check the working status of the side guard plate.

[0086] The images with detection are obtained again and input into the trained model. The trained model uses a deep learning state detection algorithm to extract features and detect the gaps and working status of the side support plate in the image. The purpose of detecting gaps is to comprehensively judge the working status of the side support plate. The left-facing workstation camera 5 judges the working characteristics of the side support plate 4, and makes a judgment by comprehensively identifying the gap between the right side of the side support plate 4 and the coal wall and the gap between the top beam of the hydraulic support 2. The right-facing workstation camera 6 judges the working characteristics of the side support plate 4, and makes a judgment by comprehensively identifying the gap between the left side of the side support plate 4 and the coal wall and the gap between the top beam of the hydraulic support 2.

[0087] S4. Integrate and process the six types of data collected in Step 3, including the left and right working status, the gap with the coal wall, and the gap with the hydraulic support top beam, and finally determine the actual working status of the side guard plate according to the weight formula. In the weight formula, x1, x2, and x3 represent the detection results of the gap between the side guard plate 4 and the coal wall, the detection results of the gap between the side guard plate 4 and the hydraulic support top beam, and the detection results of the working characteristics of the side guard plate, respectively.

[0088] .

[0089] in, Represents the test results and In the three test results, -1 represents no gap, no contact with the top beam, and is in a supported state, respectively; 1 represents gap, contact with the top beam, and is in a non-supported state, respectively. The weights represent the influence of three different categories of detection results on the final working state R. The detection results of the side panel working characteristics are the primary detection results, while the detection results of the gap between the side panel and the coal wall and the gap between the side panel and the hydraulic support top beam are auxiliary detection results. Therefore, the weights are set as follows: It is 0.3. R represents the final working state of the side guard plate, where R In a supported state, R It is in an unsupported state.

[0090] S5. Store the acquired images and work status information in the cloud, analyze whether truncation interference will occur, and facilitate subsequent status statistical analysis, hazard cause analysis, and historical data query.

[0091] See Figure 4 The specific process of image processing for the collected working images of the protective board during the creation of the dataset in this embodiment is as follows:

[0092] The camera captures images of the underground mine, and preprocesses these images, including random cropping, color processing, and random rotation at set angles. This completes the first round of image acquisition and processing, establishing a sidewall condition detection dataset. The dataset includes the sidewall's support status, retraction status, operating status, roof beam gap, and coal wall gap. Its main purpose is to train a deep learning model for detection. Then, the first round of acquired data is processed through image enhancement. The main operations include: random cropping to a 640*640 resolution, random selection of RGB values, and arbitrary 15° rotation of the image angle. After image enhancement, manual image annotation is performed to establish the deep learning dataset.

[0093] The established deep learning dataset was fed into the object detection model YOLOv5s for model training, with the key training parameters set as follows: learning rate 0.012, epochs 500, and batch size 64.

[0094] The newly acquired images of the working status of the side panel are fed into the already trained deep learning model.

[0095] Specifically, the dataset is loaded into a deep learning model for forward propagation and backward gradient descent to train the model. Feature extraction is performed through image convolutional layers and semantic compression pooling layers. The output feature maps are then decoded and compressed through upsampling operations. Feature maps of different scales are linked to achieve contextual feature fusion.

[0096] The deep learning model uses the YOLOv5s model, and the YOLOv5s model detection process is as follows:

[0097] The working state image of the side panel is fed into the feature extraction part of the pre-trained deep learning model. The image is entered into the feature extraction network in RGB three-channel format. The final output feature map is 20*20*1024 through the image feature extraction method of convolution and pooling, where 1024 is the number of feature maps extracted. The working state features of the side panel image are fully extracted through the feature extraction network part, and these image features are saved for the subsequent state detection part.

[0098] The output of the feature extraction part is fused with feature maps of different scales (80*80, 40*40, and 20*20) and the feature map to be detected through convolution and upsampling, respectively. The result is then fed into different model detection parts for target detection, and duplicate detection results are removed by the NMS (Non-Maximum Suppression) algorithm.

[0099] By fusing feature maps at different scales, the contextual feature information extracted from the image can be integrated, and object detection at different scales can improve detection accuracy. The model directly outputs an image, on which the state at each location is marked.

[0100] Analyze the test results from different angles and use the weighting formula of the test results to determine the final working state of the side panel.

[0101] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A deep learning-based visual state measurement system for side panel protection boards, characterized in that, The utility model relates to a kind of coal face guard plate state monitoring method and device, including: Image acquisition unit and visual inspection system, the image acquisition unit is transmitted to the visual inspection system by the image collected; The image acquisition unit includes: the first image acquisition unit being arranged below the first hydraulic support, the light source being arranged below the second hydraulic support and the second image acquisition unit being arranged below the third hydraulic support; The second hydraulic support is provided with a guard plate on the second hydraulic support; The light source is directly opposite the guard plate, for providing the light intensity and light irradiation angle of measurement guard plate; The first image acquisition unit and the second image acquisition unit obtain first acquisition image and second acquisition image by the way of left and right adjacent side cross shooting; The visual inspection system is configured to: receive first acquisition image and second acquisition image, and transmit to deep learning model for feature extraction, based on the extracted features, the guard plate state monitoring is carried out, to judge whether its working state is consistent with preset state, if not, it is carried out early warning.

2. The deep learning-based fairing visual condition measurement system of claim 1, wherein The first hydraulic support, the second hydraulic support and the third hydraulic support are sequentially arranged and have the same structure, and each include a bottom plate and a top plate, a connecting column is arranged between the bottom plate and the top plate, and a guard plate is arranged on the suspended end of the top plate of the second hydraulic support.

3. The deep learning-based fairing visual condition measurement system of claim 1, wherein The visual inspection system includes a big data management and control platform, a computer and a cloud server; The big data management and control platform stores the first acquisition image and the second acquisition image of the guard plate, and the state information of the guard plate; The computer stores a trained deep learning model; The computer and the cloud server communicate with the big data management and control platform.

4. A method for measuring the visual state of a fender based on deep learning, characterized by, Including: Obtain first acquisition image and second acquisition image by the way of left and right adjacent side cross shooting; First acquisition image and second acquisition image are transmitted to deep learning model for processing to obtain the working state of guard plate; The obtained working state of guard plate is weighted and processed to determine the final working state of guard plate and judge whether the working state of guard plate is abnormal.

5. The method of claim 4, wherein the method further comprises: determining a first distance between the first and second cameras; determining a second distance between the first and second cameras; and determining a third distance between the first and second cameras based on the first and second distances. Transmitting first acquisition image and second acquisition image to deep learning model for processing includes: First acquisition image and second acquisition image are respectively processed in grayscale to obtain first guard plate grayscale image and second guard plate grayscale image; Based on the first guard plate grayscale image, the gap between the right side of the guard plate and the coal wall and the gap between the top beam of the hydraulic support are identified and judged to obtain the working state of the guard plate; Based on the second guard plate grayscale image, the gap between the left side of the guard plate and the coal wall and the gap between the top beam of the hydraulic support are identified and judged to obtain the working state of the guard plate.

6. The method of claim 4, wherein the method is characterized by, The six kinds of data information of the identified left and right working states of the guard plate, the gap with the coal wall and the gap with the top beam of the hydraulic support are integrated and processed according to the weight formula to finally determine the actual working state of the guard plate.

7. The method of claim 4, wherein the method is based on deep learning. The specific calculation formula is: ; wherein, -1 is no, 1 is yes, 0 is no target detected, x1, x2, x3 in the weight formula respectively represent the detection results of the gap between the guard side plate and the coal wall, the detection results of the guard side plate and the hydraulic support top beam, and the working characteristic detection results of the guard side plate, R represents the final working state of the guard side plate, and w i represents the influence weight of the three different categories of detection results on the final working state R.

8. The method of claim 4, wherein the method is characterized by, The training process of deep learning model is: Based on the first acquisition image and the second acquisition image, a guard plate state detection data set is established, and the guard plate state detection data set is enhanced to obtain a deep learning data set; A deep learning model for detection is trained using the deep learning data set to obtain a trained deep learning model.

9. The deep learning-based visual state measurement method for side panel as described in claim 4, characterized in that, The first and second collected images include all parts of the shield plate, a first gap between the left side of the shield plate and the coal wall, and a gap between the left side of the shield plate and the top beam of the hydraulic support.

10. The method of claim 4, wherein the method is based on deep learning. The first collected image is used to judge the working characteristics of the shield plate, and the gap between the right side of the shield plate and the coal wall and the gap between the top beam of the hydraulic support are comprehensively identified for judgment. The second collected image is used to judge the working characteristics of the shield plate, and the gap between the left side of the shield plate and the coal wall and the gap between the top beam of the hydraulic support are comprehensively identified for judgment.

Citation Information

Patent Citations

  • Hydraulic support protective side control device and control method thereof

    CN103161488A

  • Intelligent control method and system for fully mechanized mining device used for complex condition working face

    CN111173510A

  • Interference identification and monitoring system between hydraulic support side guard plate and coal mining machine drum cutting

    CN111810155B

  • Method and device for measuring extension angle of side guard plate based on image sequence

    CN114511799B

  • Image processing technology-based coal mining face real-time video splicing system

    CN105554447A