Fully mechanized coal mining face hydraulic support orientation monitoring method and system

Abstract

The present invention provides a fully mechanized coal mining face hydraulic support orientation monitoring method and system. The method comprises: once a hydraulic support completes a support moving action, obtaining a monitoring video frame image used for monitoring a fully mechanized coal mining face; on the basis of the monitoring video frame image, extracting a target image, wherein the target image is a local image in the monitoring video frame image comprising a hydraulic support base image and a push rod image; and inputting the target image to a convolutional neural network model obtained by pre-training to obtain a monitoring result, output by the convolutional neural network model, of whether a support loss anomaly has occurred in the hydraulic support in the monitoring video frame image, the convolutional neural network model being used for performing feature extraction on the target image and outputting the monitoring result corresponding to the feature on the basis of the extracted feature. Whether a support loss anomaly has occurred in the hydraulic support can be automatically and accurately recognized, and the problem of system hydraulic support orientation automatic adjustment is solved, thereby reducing the labor cost, and improving the system intelligence level.

Description

Method and system for monitoring the posture of hydraulic supports in fully mechanized mining working faces Technical Field

[0001] The present invention relates to the technical field of underground coal mines, and in particular to a method and system for monitoring the posture of a hydraulic support on a fully mechanized mining working face. Background Art

[0002] During automated production on a fully mechanized mining face, once the shearer completes its current coal-cutting process, the hydraulic supports near the shearer automatically move and push the coal. During this process, the system follows pre-set control logic to control the hydraulic supports' movement and pushing. Ideally, when the hydraulic supports complete their movement and pushing, the working face will be essentially straight.

[0003] However, due to complex geological conditions, equipment conditions, and control system accuracy, hydraulic supports often experience abnormal frame loss after the frame movement is completed. Currently, this abnormal state of hydraulic support loss is often detected by manual visual perception by underground workers or monitoring video systems, which increases labor costs and lacks intelligent perception and automated adjustment and control capabilities.

[0004] Therefore, in the actual production process of the working face, a technical method is needed that can intelligently and accurately identify the abnormal phenomenon of hydraulic support frame loss, and the system can automatically adjust and control the posture of the hydraulic support according to the frame loss situation. Summary of the Invention

[0005] The present invention provides a method and system for monitoring the posture of hydraulic supports in a fully mechanized mining working face, which can automatically and accurately identify whether a hydraulic support has lost its support abnormality, solve the problem of automatic adjustment of the posture of the system's hydraulic supports, thereby reducing labor costs and improving the intelligence level of the system.

[0006] The present invention provides a method for monitoring the posture of a hydraulic support in a fully mechanized mining working face, the method comprising: obtaining a monitoring video frame image for monitoring the fully mechanized mining working face after the hydraulic support completes a frame moving action; extracting a target image based on the monitoring video frame image, wherein the target image is a local image of the monitoring video frame image containing a hydraulic support base image and a push rod image; inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support in the monitoring video frame image has a frame loss abnormality output by the convolutional neural network model, wherein the convolutional neural network model is used to extract features from the target image, and output the monitoring result corresponding to the feature based on the extracted feature; and performing posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

[0007] According to a method for monitoring the posture of a hydraulic support in a fully mechanized mining working face provided by the present invention, extracting a target image based on the monitoring video frame image specifically includes: performing image instance segmentation processing on the monitoring video frame image to extract the target image.

[0008] According to a method for monitoring the posture of a hydraulic support in a comprehensive mining working face provided by the present invention, before inputting the target image into a pre-trained convolutional neural network model, the method further includes: performing image distortion correction processing on the target image to obtain a corrected target image; determining the first position information of the hydraulic support base image and the second position information of the push rod image in the corrected target image based on the corrected target image; obtaining a reprocessed target image based on the first position information and the second position information, wherein the reprocessed target image is a partial image of the hydraulic support base with the first position information and the push rod with the second position information; inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, specifically including: inputting the reprocessed target image into the pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model.

[0009] According to a method for monitoring the posture of hydraulic supports in a fully-mechanized mining working face provided by the present invention, after obtaining the monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, the method further includes: when the monitoring result is that the hydraulic support has no frame loss abnormality in the monitoring video frame image, outputting the monitoring video frame image.

[0010] According to a method for monitoring the posture of a hydraulic support in a fully-mechanized mining working face provided by the present invention, after obtaining the monitoring result of whether the hydraulic support has an abnormal frame loss in the monitoring video frame image output by the convolutional neural network model, the method further includes: when the monitoring result is that the hydraulic support has an abnormal frame loss in the monitoring video frame image, issuing an alarm reminder, and controlling the hydraulic support to adjust its posture until the hydraulic support posture is adjusted to a preset required state.

[0011] According to a method for monitoring the posture of hydraulic supports in a fully-mechanized mining working face provided by the present invention, after obtaining the monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, the method further includes: when the monitoring result is that the hydraulic support has a frame loss abnormality in the monitoring video frame image, extracting the target image and adding a frame loss abnormality label to the monitoring video frame image where the target image is located; and outputting the monitoring video frame image where the target image with the frame loss abnormality label is located.

[0012] The present invention also provides a hydraulic support posture monitoring system for a comprehensive mining working face, the system comprising: an acquisition module, for acquiring a monitoring video frame image for monitoring the comprehensive mining working face after the hydraulic support completes the frame moving action; an extraction module, for extracting a target image based on the monitoring video frame image, wherein the target image is a local image of the monitoring video frame image containing a hydraulic support base image and a push rod image; a detection module, for inputting the target image into a pre-trained convolutional neural network model, and obtaining a monitoring result of whether the hydraulic support in the monitoring video frame image output by the convolutional neural network model has a frame loss abnormality, wherein the convolutional neural network model is used to perform feature extraction on the target image, and output the monitoring result corresponding to the feature based on the extracted feature; a control and adjustment module, for performing posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

[0013] According to a hydraulic support posture monitoring system for a fully mechanized mining working face provided by the present invention, the extraction module extracts a target image based on the monitoring video frame image in the following manner: performing image instance segmentation processing on the monitoring video frame image to extract the target image.

[0014] The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and runnable on the processor. When the processor executes the program, a method for monitoring the posture of a hydraulic support in a comprehensive mining working face as described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for monitoring the posture of a hydraulic support in a fully mechanized mining working face as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for monitoring the posture of a hydraulic support in a fully mechanized mining working face as described in any one of the above.

[0017] The method and system for monitoring the posture of hydraulic supports in a fully mechanized mining face, provided by the present invention, obtains a monitoring video frame image used to monitor the fully mechanized mining face after the hydraulic supports complete the frame shifting action. A target image is then extracted based on the monitoring video frame image, and the target image is input into a pre-trained convolutional neural network model. The monitoring result output by the convolutional neural network model indicates whether the hydraulic supports in the monitoring video frame image have experienced any abnormal frame loss. The system then adjusts and controls the posture of the hydraulic supports in the monitoring video frame image based on the monitoring result. This system automatically and accurately identifies whether a hydraulic support has experienced any abnormal frame loss, solves the problem of automatic adjustment of the system's hydraulic support posture, thereby reducing labor costs and improving the system's intelligence level. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] FIG1 is a flow chart of a method for monitoring the posture of a hydraulic support in a fully mechanized mining face provided by the present invention;

[0020] FIG2 is a schematic diagram of a hydraulic support in an abnormal state without frame loss;

[0021] FIG3 is a schematic diagram of an abnormal state of a hydraulic support frame being lost;

[0022] FIG4 is a second flow chart of the method for monitoring the posture of a hydraulic support in a fully mechanized mining working face provided by the present invention;

[0023] FIG5 is a third flow chart of the method for monitoring the posture of a hydraulic support in a fully mechanized mining working face provided by the present invention;

[0024] FIG6 is a schematic structural diagram of a hydraulic support posture monitoring system for a fully mechanized mining working face provided by the present invention;

[0025] FIG7 is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] During the automated production process of the working face, when the coal mining machine completes the current coal cutting process, the hydraulic supports near the coal mining machine will automatically complete the frame shifting and pushing action. During the hydraulic support frame shifting and pushing control process, the system will control the hydraulic support frame shifting and pushing according to the pre-set control logic. Ideally, when the hydraulic supports complete the frame shifting and pushing action, the working face will be in a basically straight line state. When the hydraulic supports complete the frame shifting action, as shown in Figure 2, the edge lines of the adjacent hydraulic support bases are basically in a straight line state, which is called the abnormal state of the hydraulic supports on the working face without frame loss.

[0028] However, due to complex geological conditions, equipment conditions, and control system accuracy, hydraulic supports often experience an abnormal state of lost support after the shift is completed, as shown in Figure 3 below. This occurs when the edges of adjacent hydraulic support bases are not aligned, which is known as an abnormal state of lost support on the working face. When a hydraulic support experiences an abnormal state of lost support after a shift, prompt system or human intervention is required to restore the hydraulic support to its normal state and maintain a straight working face, ensuring the next coal cutting operation proceeds smoothly.

[0029] The method for monitoring the posture of hydraulic supports in a fully mechanized mining working face provided by the present invention can intelligently and accurately identify abnormal phenomena of hydraulic support loss, and the system automatically adjusts and controls the posture of the hydraulic supports according to the loss situation, thereby reducing labor costs and improving the intelligence level of the system.

[0030] FIG1 is a flow chart of a method for monitoring the posture of a hydraulic support in a fully mechanized mining face provided by the present invention.

[0031] In an exemplary embodiment of the present invention, as can be seen from FIG. 1 , a method for monitoring the posture of a hydraulic support in a fully mechanized mining working face may include steps 110 to 140 , each of which will be described below.

[0032] In step 110, after the hydraulic support completes the frame moving action, a monitoring video frame image for monitoring the fully mechanized mining working face is obtained.

[0033] In step 120 , a target image is extracted based on the monitoring video frame image, wherein the target image is a partial image of the monitoring video frame image containing the hydraulic support base image and the push rod image.

[0034] In step 130, the target image is input into the pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model.

[0035] In another embodiment, the hydraulic support number to be moved and its corresponding surveillance camera IP address can be calculated based on the shearer's position information during mining. After the hydraulic support completes its movement, the surveillance video image is read based on the surveillance camera IP address. In other words, after the hydraulic support completes its movement, a surveillance video frame image for monitoring the fully mechanized mining face can be obtained.

[0036] In another embodiment, a target image can be extracted based on the monitoring video frame image, wherein the target image is a partial image of the monitoring video frame image containing the hydraulic support base image and the push rod image.

[0037] In another exemplary embodiment of the present invention, the target image is extracted based on the monitoring video frame image, which can be achieved in the following manner:

[0038] Perform image instance segmentation on the surveillance video frame image and extract the target image.

[0039] In one embodiment, an image instance segmentation method may be used to segment the video image frame into various regions, thereby completing visual structuring of the work surface scene and acquiring a target image.

[0040] Furthermore, the target image is input into a pre-trained convolutional neural network model, which then outputs a monitoring result indicating whether the hydraulic support has experienced an abnormal loss of support in the surveillance video frame. This enables automatic detection of abnormal loss of support. The convolutional neural network model is used to extract features from the target image and output the corresponding monitoring result based on the extracted features.

[0041] In step 140, the posture of the hydraulic support in the monitoring video frame image is adjusted and controlled according to the monitoring result.

[0042] In one embodiment, the posture adjustment control of the hydraulic support in the monitoring video frame image can be performed according to the monitoring results, thereby solving the problem of automatic posture adjustment of the system hydraulic support, thereby reducing labor costs and improving the intelligence level of the system.

[0043] The method for monitoring the posture of hydraulic supports in a fully mechanized mining face, provided by the present invention, obtains a monitoring video frame image used to monitor the fully mechanized mining face after the hydraulic supports complete their shifting action. A target image is then extracted based on the monitoring video frame image, and the target image is input into a pre-trained convolutional neural network model. The convolutional neural network model outputs a monitoring result indicating whether the hydraulic supports in the monitoring video frame image have experienced any abnormal loss of support, and the posture of the hydraulic supports in the monitoring video frame image is adjusted and controlled based on the monitoring result. This method automatically and accurately identifies whether a hydraulic support has experienced any abnormal loss of support, solves the problem of automatic adjustment of the system's hydraulic support posture, thereby reducing labor costs and improving the system's intelligence level.

[0044] FIG4 is a second flow chart of the method for monitoring the posture of the hydraulic support in the fully mechanized mining working face provided by the present invention.

[0045] The process of another method for monitoring the posture of a hydraulic support in a fully mechanized mining face will be described below with reference to FIG4 .

[0046] In another exemplary embodiment of the present invention, the method for monitoring the posture of the hydraulic support of the comprehensive mining working face may include steps 410 to 470, wherein steps 410 to 420 are respectively the same as or similar to steps 110 to 120, and step 470 is the same as or similar to step 140. Please refer to the above description for its specific implementation and beneficial effects. No specific limitation is made in this embodiment. Steps 430 to 460 will be introduced in detail below.

[0047] In step 430, image distortion correction processing is performed on the target image to obtain a corrected target image.

[0048] In step 440 , first position information of the hydraulic support base image and second position information of the push rod image in the corrected target image are determined based on the corrected target image.

[0049] In one embodiment, the extracted target image can be subjected to image distortion correction to obtain a corrected target image. Since the corrected target image is a distortion-corrected target image, image defects in the target image can be effectively overcome, thereby avoiding the problem of inaccurate lost frame recognition caused by image defects.

[0050] In another embodiment, first position information of the hydraulic support base image and second position information of the push rod image in the corrected target image can be determined based on the corrected target image. The first position information can be used to represent the position information of the hydraulic support base image in the corrected target image, thereby inferring the position information of the hydraulic support base in a world coordinate system. The second position information can be used to represent the position information of the push rod image in the corrected target image, thereby inferring the position information of the push rod in a world coordinate system.

[0051] In step 450 , a reprocessed target image is obtained based on the first position information and the second position information, wherein the reprocessed target image is a partial image of the hydraulic support base having the first position information and the push rod having the second position information.

[0052] In step 460, the reprocessed target image is input into the pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model.

[0053] In another embodiment, a reprocessed target image can be obtained based on the first position information and the second position information, wherein the reprocessed target image is a partial image of the hydraulic support base having the first position information and the push rod having the second position information. Because the reprocessed target image has undergone image distortion correction and contains both the first position information of the hydraulic support base and the partial image of the push rod having the second position information, it can better lay the foundation for identifying whether the hydraulic support has experienced a frame loss anomaly.

[0054] In another embodiment, the reprocessed target image can be input into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss anomaly in the monitoring video frame image output by the convolutional neural network model. It can be understood that in this scenario, the reprocessed target image replaces the target image and is input into the pre-trained convolutional neural network model. Since the reprocessed target image has undergone image distortion correction relative to the target image and accurately contains the first position information of the hydraulic support base and the second position information of the push rod, based on the reprocessed target image, the monitoring result of whether the hydraulic support has a frame loss anomaly in the monitoring video frame image output by the convolutional neural network model will be more accurate.

[0055] FIG5 is a third flow chart of the method for monitoring the posture of the hydraulic support in the fully mechanized mining working face provided by the present invention.

[0056] The process of another method for monitoring the posture of a hydraulic support in a fully mechanized mining face will be described below with reference to FIG5 .

[0057] In an exemplary embodiment of the present invention, in combination with Figure 5, it can be seen that the method for monitoring the posture of the hydraulic support of the comprehensive mining working face may include steps 510 to 570, wherein steps 510 to 530 are respectively the same as or similar to steps 110 to 130 described above. Please refer to the above description for its specific implementation methods and beneficial effects. No specific limitation is made in this embodiment. Steps 540 to 570 will be introduced below.

[0058] In step 540, when the monitoring result shows that the hydraulic support does not have any frame loss abnormality in the monitoring video frame image, the monitoring video frame image is output.

[0059] In step 550, if the monitoring result indicates that the hydraulic support has lost its support frame, an alarm is issued and the hydraulic support is controlled to adjust its posture until the hydraulic support posture is adjusted to a preset desired state. The preset desired state can be adjusted according to actual conditions and is not specifically limited in this embodiment.

[0060] In step 560, when the monitoring result shows that the hydraulic support has a frame loss anomaly in the monitoring video frame image, the target image is extracted and a frame loss anomaly label is added to the monitoring video frame image where the target image is located.

[0061] In step 570, the surveillance video frame image where the target image with the frame loss anomaly label is located is output.

[0062] In one embodiment, when the monitoring result shows that the hydraulic support does not show any abnormality of losing the frame in the monitoring video frame image, the monitoring video frame image, that is, the original image frame is output and pushed to the video management server for user use.

[0063] In another embodiment, if the monitoring result indicates that the hydraulic support is abnormally lost in the monitoring video frame image, an early warning signal can be automatically sent to the control module, and the hydraulic support can be controlled to adjust its posture until the hydraulic support posture is adjusted to a preset required state. In addition, the abnormal state can be stored. During application, the early warning signal can be analyzed and a command can be sent to the downhole controller to implement the hydraulic support movement adjustment.

[0064] In another embodiment, when the monitoring result is that the hydraulic support has a frame loss abnormality in the monitoring video frame image, the target image is extracted and a frame loss abnormality label is added to the monitoring video frame image where the target image is located; and the monitoring video frame image where the target image with the frame loss abnormality label is located is output.

[0065] In another embodiment, if there is a frame loss anomaly, the hydraulic support with the abnormal frame loss can be detected and identified based on the positioning data of the hydraulic support base, and an abnormal label can be added to extract the target image (the target image in this scenario is the image with the frame loss anomaly), and a frame loss anomaly label can be added to the target image; then the monitoring video frame image where the target image with the frame loss anomaly label is added is output and pushed to the video management server.

[0066] According to the above description, the method for monitoring the posture of the hydraulic support of the fully-mechanized mining working face provided by the present invention obtains a monitoring video frame image for monitoring the fully-mechanized mining working face after the hydraulic support completes the frame moving action; then extracts the target image based on the monitoring video frame image, and inputs the target image into the pre-trained convolutional neural network model to obtain the monitoring result of whether the hydraulic support in the monitoring video frame image output by the convolutional neural network model has a frame loss anomaly, and adjusts the posture of the hydraulic support in the monitoring video frame image according to the monitoring result. This method can automatically and accurately identify whether the hydraulic support has a frame loss anomaly, solves the problem of automatic adjustment of the system's hydraulic support posture, thereby reducing labor costs and improving the intelligence level of the system.

[0067] Based on the same concept, the present invention also provides a hydraulic support posture monitoring system for a fully mechanized mining working face.

[0068] The hydraulic support posture monitoring system for a fully mechanized mining working face provided by the present invention is described below. The hydraulic support posture monitoring system for a fully mechanized mining working face described below and the hydraulic support posture monitoring method for a fully mechanized mining working face described above can be referred to each other.

[0069] FIG6 is a schematic structural diagram of a hydraulic support posture monitoring system for a fully mechanized mining working face provided by the present invention.

[0070] In an exemplary embodiment of the present invention, as shown in FIG6 , the hydraulic support posture monitoring system for a fully mechanized mining working face may include an acquisition module 610 , an extraction module 620 , a detection module 630 , and a control and adjustment module 640 , each of which will be described below.

[0071] The acquisition module 610 may be configured to acquire a monitoring video frame image for monitoring the fully mechanized mining working face after the hydraulic support completes the frame moving action;

[0072] The extraction module 620 may be configured to extract a target image based on the monitoring video frame image, wherein the target image is a partial image of the monitoring video frame image containing the hydraulic support base image and the push rod image;

[0073] The detection module 630 may be configured to input the target image into a pre-trained convolutional neural network model to obtain a monitoring result output by the convolutional neural network model as to whether the hydraulic support has a frame loss anomaly in the monitoring video frame image, wherein the convolutional neural network model is used to extract features from the target image and output the monitoring result corresponding to the feature based on the extracted features;

[0074] The control adjustment module 640 may be configured to perform posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

[0075] In an exemplary embodiment of the present invention, the extraction module 620 may extract the target image based on the surveillance video frame image in the following manner:

[0076] Perform image instance segmentation processing on the surveillance video frame image to extract the target image.

[0077] In an exemplary embodiment of the present invention, the detection module 630 may also be configured to:

[0078] Performing image distortion correction processing on the target image to obtain a corrected target image;

[0079] Determining, according to the corrected target image, first position information of the hydraulic support base image and second position information of the push rod image in the corrected target image;

[0080] Based on the first position information and the second position information, a reprocessed target image is obtained, wherein the reprocessed target image is a partial image of the hydraulic support base having the first position information and the push rod having the second position information;

[0081] The detection module 630 may also input the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has lost its support in the monitoring video frame image output by the convolutional neural network model in the following manner:

[0082] The reprocessed target image is input into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model.

[0083] In an exemplary embodiment of the present invention, the detection module 630 may also be configured to:

[0084] When the monitoring result shows that the hydraulic support does not have any frame loss abnormality in the monitoring video frame image, the monitoring video frame image is output.

[0085] In an exemplary embodiment of the present invention, the control adjustment module 640 may also be configured to:

[0086] When the monitoring result shows that the hydraulic support in the monitoring video frame image is abnormally lost, an alarm reminder is issued, and the hydraulic support is controlled to adjust its posture until the posture of the hydraulic support is adjusted to a preset required state.

[0087] In an exemplary embodiment of the present invention, the detection module 630 may also be configured to:

[0088] When the monitoring result is that the hydraulic support in the monitoring video frame image has a frame loss anomaly, extracting a target image and adding a frame loss anomaly label to the monitoring video frame image where the target image is located;

[0089] And the monitoring video frame image where the target image with the lost frame abnormality label is added is output.

[0090] Figure 7 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 7, the electronic device may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the posture monitoring method of the hydraulic support of the comprehensive mining working face, which includes: after the hydraulic support completes the frame moving action, obtaining a monitoring video frame image for monitoring the comprehensive mining working face; based on the monitoring video frame image, extracting the target image, wherein the target image is a local image of the monitoring video frame image containing the hydraulic support base image and the push rod image; inputting the target image into a pre-trained convolutional neural network model to obtain the monitoring result of whether the hydraulic support in the monitoring video frame image has a frame loss abnormality output by the convolutional neural network model, wherein the convolutional neural network model is used to extract features of the target image, and output the monitoring result corresponding to the feature based on the extracted feature; and performing posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

[0091] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the hydraulic support posture monitoring method for the comprehensive mining working face provided by the above-mentioned methods, and the method includes: after the hydraulic support completes the frame moving action, obtaining a monitoring video frame image for monitoring the comprehensive mining working face; based on the monitoring video frame image, extracting a target image, wherein the target image is a local image of the monitoring video frame image containing a hydraulic support base image and a push rod image; inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support in the monitoring video frame image has a frame loss abnormality output by the convolutional neural network model, wherein the convolutional neural network model is used to extract features from the target image, and output the monitoring result corresponding to the feature based on the extracted feature; and adjusting the posture of the hydraulic support in the monitoring video frame image according to the monitoring result.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for monitoring the posture of the hydraulic support of the comprehensive mining working face provided by the above-mentioned methods, the method comprising: after the hydraulic support completes the frame moving action, obtaining a monitoring video frame image for monitoring the comprehensive mining working face; based on the monitoring video frame image, extracting a target image, wherein the target image is a local image of the monitoring video frame image containing the hydraulic support base image and the push rod image; inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support in the monitoring video frame image has a frame loss abnormality output by the convolutional neural network model, wherein the convolutional neural network model is used to extract features from the target image, and output the monitoring result corresponding to the feature based on the extracted feature; and performing posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0096] It should be further understood that, although operations are described in a particular order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring that the operations be performed in the particular order shown or in a serial order, or that all of the operations shown be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for monitoring the posture of a hydraulic support in a fully mechanized mining working face, characterized in that: The method comprises: After the hydraulic support completes the frame moving action, a monitoring video frame image for monitoring the fully mechanized mining working face is obtained; Extracting a target image based on the monitoring video frame image, wherein the target image is a partial image of the monitoring video frame image containing a hydraulic support base image and a push rod image; Inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, wherein the convolutional neural network model is used to extract features from the target image and output the monitoring result corresponding to the feature based on the extracted features; The posture of the hydraulic support in the monitoring video frame image is adjusted and controlled according to the monitoring result.

2. The method for monitoring the posture of a hydraulic support in a fully mechanized mining working face according to claim 1, characterized in that: The extracting of the target image based on the monitoring video frame image specifically includes: Perform image instance segmentation processing on the surveillance video frame image to extract the target image.

3. The method for monitoring the posture of a hydraulic support in a fully mechanized mining face according to claim 1 or 2, characterized in that: Before inputting the target image into the pre-trained convolutional neural network model, the method further includes: Performing image distortion correction processing on the target image to obtain a corrected target image; Determining, according to the corrected target image, first position information of the hydraulic support base image and second position information of the push rod image in the corrected target image; Based on the first position information and the second position information, a reprocessed target image is obtained, wherein the reprocessed target image is a partial image of the hydraulic support base having the first position information and the push rod having the second position information; Inputting the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model specifically includes: The reprocessed target image is input into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model.

4. The method for monitoring the posture of a hydraulic support in a fully mechanized mining working face according to claim 1, characterized in that: After obtaining the monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, the method further includes: When the monitoring result shows that the hydraulic support does not have any frame loss abnormality in the monitoring video frame image, the monitoring video frame image is output.

5. The method for monitoring the posture of a hydraulic support in a fully mechanized mining working face according to claim 1 is characterized in that: After obtaining the monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, the method further includes: When the monitoring result shows that the hydraulic support in the monitoring video frame image is abnormally lost, an alarm reminder is issued, and the hydraulic support is controlled to adjust its posture until the posture of the hydraulic support is adjusted to a preset required state.

6. The method for monitoring the posture of a hydraulic support in a fully mechanized mining working face according to claim 1, characterized in that: After obtaining the monitoring result of whether the hydraulic support has a frame loss abnormality in the monitoring video frame image output by the convolutional neural network model, the method further includes: When the monitoring result is that the hydraulic support in the monitoring video frame image has a frame loss anomaly, extracting a target image and adding a frame loss anomaly label to the monitoring video frame image where the target image is located; And the monitoring video frame image where the target image with the lost frame abnormality label is added is output.

7. A hydraulic support posture monitoring system for a fully mechanized mining working face, characterized in that: The system comprises: An acquisition module is used to acquire a monitoring video frame image for monitoring the fully mechanized mining working face after the hydraulic support completes the frame moving action; An extraction module is used to extract a target image based on the monitoring video frame image, wherein the target image is a partial image of the monitoring video frame image containing the hydraulic support base image and the push rod image; A detection module is configured to input the target image into a pre-trained convolutional neural network model to obtain a monitoring result of whether the hydraulic support has lost its support in the monitoring video frame image output by the convolutional neural network model, wherein the convolutional neural network model is configured to extract features from the target image and output the monitoring result corresponding to the feature based on the extracted features; A control and adjustment module is used to perform posture adjustment control on the hydraulic support in the monitoring video frame image according to the monitoring result.

8. The hydraulic support posture monitoring system for fully mechanized mining working face according to claim 7 is characterized in that: The extraction module extracts the target image based on the monitoring video frame image in the following manner: Perform image instance segmentation processing on the surveillance video frame image to extract the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for monitoring the posture of the hydraulic support of the fully mechanized mining working face as described in any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring the posture of the hydraulic support of a fully mechanized mining working face as described in any one of claims 1 to 6 is implemented.

Images