Methods and devices for monitoring the motion status of hydraulic cylinders, storage media, and terminals

Abstract

This invention discloses a method, device, storage medium, and terminal for monitoring the motion state of a hydraulic cylinder, relating to the field of equipment monitoring technology. Its main objective is to solve the problem of low accuracy in monitoring the motion state of hydraulic cylinders. The method primarily includes acquiring continuous frame images of a monitoring target, which includes a target hydraulic cylinder and a first component and a second component connected to both ends of the target hydraulic cylinder; extracting and tracking the target from the continuous frame images to obtain a first tracking result for the first component and a second tracking result for the second component; and determining the motion state of the target hydraulic cylinder based on the first and second tracking results. This method is mainly used for monitoring the motion state of hydraulic cylinders.

Description

Technical Field

[0001] This invention relates to the field of equipment monitoring technology, and in particular to a method and device for monitoring the motion status of a hydraulic cylinder, a storage medium, and a terminal. Background Technology

[0002] With the advancement of technology, the mining industry is becoming increasingly automated, with more and more mining processes achieving automated control. However, automated control still has a certain degree of instability. To ensure the accuracy of automated control and the normal operation of equipment, it is necessary to monitor the production process. Hydraulic supports, as crucial moving equipment in the mining process, directly determine the continuous and reliable operation of the coal mining machine and scraper conveyor through the movement of their target cylinders. Therefore, monitoring the movement of the hydraulic support's target cylinders is an important aspect of mining process monitoring.

[0003] Existing methods for monitoring the movement of target cylinders in coal mining machines rely on hardware such as photoelectric rangefinders, infrared transceivers, and magnetic rings to measure the displacement of the target cylinders, thereby monitoring their movement. However, the accuracy of these methods is low because the hardware is susceptible to temperature and humidity fluctuations and suffers from frictional losses. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for monitoring the motion state of a hydraulic cylinder, a storage medium, and a terminal, the main purpose of which is to solve the problem of low accuracy in existing hydraulic cylinder motion state monitoring.

[0005] According to one aspect of the present invention, a method for monitoring the motion state of a hydraulic cylinder is provided, comprising:

[0006] Acquire continuous frame images of the monitored target, the monitored target including a target hydraulic cylinder, and a first component and a second component connected to both ends of the target hydraulic cylinder;

[0007] By performing target extraction and target tracking on the continuous frame images, a first tracking result of the first component and a second tracking result of the second component are obtained;

[0008] The motion state of the target cylinder is determined based on the first tracking result and the second tracking result.

[0009] Further, the step of obtaining the first tracking result of the first component and the second tracking result of the second component by extracting and tracking the target from the consecutive frame images includes:

[0010] The target detection network model that has been trained is used to extract targets from the consecutive frame images to obtain a first target detection box and a second target detection box.

[0011] Target tracking is performed based on the first target detection box and the second target detection box to obtain the first tracking result of the first component and the second tracking result of the second component.

[0012] Furthermore, before extracting targets from the consecutive frames of images using the trained target detection network model to obtain the first target detection box and the second target detection box, the method further includes:

[0013] Construct a training sample set and an initial target detection network model. The sample images in the training sample set include a first part image that has been labeled and a second part image that has been labeled.

[0014] Based on the anchor box size and anchor box position in the initial target detection network model, target detection boxes are predicted for the sample image to complete the training of the initial target detection network model. The anchor box size and anchor box position are set based on the image size and image position of the first component and the second component in historical images.

[0015] Further, the step of tracking the target based on the first target detection box and the second target detection box to obtain the first tracking result of the first component and the second tracking result of the second component includes:

[0016] Feature extraction is performed on the targets in the first target detection box and the second target detection box respectively to obtain the first attribute feature corresponding to the first target detection box and the second attribute feature corresponding to the second target detection box. The first similarity of the first attribute feature and the second similarity of the second attribute feature in two adjacent frames are calculated.

[0017] Image frames with a first similarity or a second similarity less than the preset confidence level are deleted according to a preset deletion quantity threshold;

[0018] Motion feature extraction and attribute association are performed on the image frames after deletion to obtain the first tracking result of the first component and the second tracking result of the second component.

[0019] The preset reliability and the preset deletion quantity threshold are set based on the analysis results of historical images of the monitored target.

[0020] Furthermore, the motion state includes a first state and a second state. After determining the motion state of the target cylinder based on the tracking result, the method further includes:

[0021] If the motion state is the first state, then the pixel distance of the first target detection box in every two adjacent frame images is calculated, and the pixel distance is accumulated to obtain the first displacement;

[0022] If the motion state is the second state, then the pixel distance between the second target detection box in every two adjacent frame images is calculated, and the pixel distances are accumulated to obtain the second displacement;

[0023] If the first displacement or the second displacement is less than a preset displacement threshold, a displacement warning message indicating that the displacement of the target cylinder is abnormal is generated, so that the staff can inspect the target cylinder.

[0024] Further, the first component is a scraper conveyor, the second component is a pilot valve, the motion state includes a first state and a second state, and determining the motion state of the target cylinder based on the first tracking result and the second tracking result includes:

[0025] If the first tracking result is greater than a preset threshold, the motion state is determined to be the first state, which is used to characterize that the target cylinder is in an extended state.

[0026] If the second tracking result is greater than the preset threshold, the motion state is determined to be the second state, which is used to characterize that the target cylinder is in a retracted state.

[0027] Furthermore, after determining the motion state of the target cylinder based on the first tracking result and the second tracking result, the method further includes:

[0028] Obtain control instructions for the target cylinder corresponding to the time of the consecutive frame images, wherein the control instructions carry control information for controlling the motion state of the target cylinder;

[0029] The motion state is matched and verified with the control information to obtain the verification result;

[0030] If the verification result is a failure, a status warning message is generated to indicate that the target cylinder is in an abnormal operating state, so that the staff can inspect and repair the target cylinder.

[0031] According to another aspect of the present invention, a hydraulic cylinder motion state monitoring device is provided, comprising:

[0032] The acquisition module is used to acquire continuous frame images of the monitored target, the monitored target including a target cylinder and a first component and a second component connected to both ends of the target cylinder;

[0033] The target tracking module is used to extract and track targets from the continuous frame images to obtain a first tracking result of the first component and a second tracking result of the second component.

[0034] The determination module is used to determine the motion state of the target cylinder based on the first tracking result and the second tracking result.

[0035] Furthermore, the target tracking module includes:

[0036] The target extraction unit is used to extract targets from the consecutive frame images using a trained target detection network model to obtain a first target detection box and a second target detection box.

[0037] The target tracking unit is used to perform target tracking based on the first target detection box and the second target detection box to obtain a first tracking result of the first component and a second tracking result of the second component.

[0038] Furthermore, the device also includes:

[0039] The construction module is used to construct a training sample set and an initial object detection network model. The sample images in the training sample set include a first part image that has been labeled and a second part image that has been labeled.

[0040] The training module is used to predict target detection boxes on the sample image based on the anchor box size and anchor box position in the initial target detection network model, so as to complete the training of the initial target detection network model. The anchor box size and anchor box position are set based on the image size and image position of the first component and the second component in the historical image.

[0041] Furthermore, in specific application scenarios, the target tracking unit is used to extract features from the targets in the first target detection box and the second target detection box respectively, to obtain the first attribute feature corresponding to the first target detection box and the second attribute feature corresponding to the second target detection box, and to calculate the first similarity of the first attribute feature and the second similarity of the second attribute feature in two adjacent frames of images;

[0042] Image frames with a first similarity or a second similarity less than the preset confidence level are deleted according to a preset deletion quantity threshold;

[0043] Motion feature extraction and attribute association are performed on the image frames after deletion to obtain the first tracking result of the first component and the second tracking result of the second component.

[0044] The preset reliability and the preset deletion quantity threshold are set based on the analysis results of historical images of the monitored target.

[0045] Furthermore, the device also includes:

[0046] The first calculation module is used to calculate the pixel distance between the first target detection box in every two adjacent frame images if the motion state is the first state, and to accumulate the pixel distance to obtain the first displacement.

[0047] The second calculation module is used to calculate the pixel distance between the second target detection box in every two adjacent frame images if the motion state is the second state, and to accumulate the pixel distance to obtain the second displacement.

[0048] The generation module is used to generate a displacement warning message indicating that the displacement of the target cylinder is abnormal if the first displacement or the second displacement is less than a preset displacement threshold, so that the staff can inspect the target cylinder.

[0049] Further, the modules are identified, including:

[0050] The first determining unit is configured to determine the motion state as a first state if the first tracking result is greater than a preset threshold, wherein the first state is used to characterize the target cylinder as being in an extended state.

[0051] The second determining unit is used to determine the motion state as a second state if the second tracking result is greater than the preset threshold. The second state is used to characterize that the target cylinder is in a retracted state.

[0052] Furthermore, the device also includes:

[0053] The acquisition module is further configured to acquire a control command for the target cylinder corresponding to the time of the consecutive frame images, the control command carrying control information for controlling the motion state of the target cylinder;

[0054] The verification module is used to match and verify the motion state with the control information to obtain the verification result;

[0055] The generation module is also used to generate a status warning message indicating that the target cylinder is in an abnormal operating state if the verification result is a failure, so that the staff can inspect and repair the target cylinder.

[0056] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform an operation corresponding to the above-described hydraulic cylinder motion state monitoring method.

[0057] According to another aspect of the present invention, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0058] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described hydraulic cylinder motion state monitoring method.

[0059] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:

[0060] This invention provides a method, device, storage medium, and terminal for monitoring the motion state of a hydraulic cylinder. In this embodiment, continuous frame images of a monitored target are acquired. The monitored target includes a target hydraulic cylinder and a first component and a second component connected to both ends of the target hydraulic cylinder. Target extraction and tracking are performed on the continuous frame images to obtain a first tracking result for the first component and a second tracking result for the second component. The motion state of the target hydraulic cylinder is determined based on the first and second tracking results. This significantly reduces the interference of the mining face environment on the hydraulic cylinder motion data acquisition process, overcomes the difficulties of indistinct hydraulic cylinder features and the difficulty in capturing key points, while ensuring the accuracy of hydraulic cylinder motion data acquisition, thereby greatly improving the accuracy of hydraulic cylinder motion state monitoring.

[0061] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0062] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0063] Figure 1 A flowchart of a method for monitoring the motion state of a hydraulic cylinder provided by an embodiment of the present invention is shown;

[0064] Figure 2 A flowchart of another method for monitoring the motion state of a hydraulic cylinder provided by an embodiment of the present invention is shown;

[0065] Figure 3 This diagram illustrates a block diagram of a hydraulic cylinder motion state monitoring device according to an embodiment of the present invention.

[0066] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present invention is shown. Detailed Implementation

[0067] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0068] Existing methods for monitoring the movement state of target hydraulic cylinders in coal mining machines rely on hardware such as photoelectric rangefinders, infrared transceivers, and magnetic rings to measure the displacement of the target hydraulic cylinder, thereby monitoring its motion state. However, the accuracy of this monitoring is low due to the susceptibility of the hardware to temperature and humidity fluctuations, as well as frictional losses. This invention provides a method for monitoring the motion state of hydraulic cylinders, such as... Figure 1 As shown, the method includes:

[0069] 101. Acquire continuous frame images of the monitored target.

[0070] In this embodiment of the invention, an image acquisition device installed in the environment of a fully mechanized mining face is used to acquire video images of the target cylinder of the hydraulic support and the device connected to the target cylinder. The acquired video images are then processed to extract continuous frame images. Because the environment of a fully mechanized mining face is relatively confined and subject to electromagnetic interference, it is inconvenient to install sensing devices, and such installations would interfere with the operation of existing sensing devices. Using an image acquisition device to obtain the motion data of the target cylinder of the hydraulic support to replace sensing devices avoids the need for a large number of sensors, thereby reducing the difficulty and cost of data acquisition.

[0071] It should be noted that acquiring continuous frame images of the monitoring target includes the target hydraulic cylinder and a first component and a second component connected to both ends of the target hydraulic cylinder. Due to the relatively dark environment of the fully mechanized mining face and the intermittent obstruction caused by the activities of mining personnel, extracting key points of the target hydraulic cylinder is difficult in practical applications. Furthermore, since the first and second components are connected to both ends of the target hydraulic cylinder and share the same direction and amplitude of extension and retraction movement as the target hydraulic cylinder, they can characterize the extension and retraction movement of the target hydraulic cylinder. Therefore, images of the first and second components connected to the target hydraulic cylinder are acquired, and these first and second components are used as targets for image analysis. The first component can be a scraper conveyor connected to the piston rod of the target hydraulic cylinder of the coal mining machine's hydraulic support; the second component can be a pilot valve connected to the cylinder body of the target hydraulic cylinder of the coal mining machine's hydraulic support, or other devices connected to the cylinder body and piston rod of the target hydraulic cylinder of the coal mining machine's hydraulic support. This embodiment of the invention does not impose specific limitations. By transforming the target of image analysis from the target hydraulic cylinder into the first and second components with salient features, the salientity of the analysis target is improved, the difficulty of image capture and feature extraction is reduced, thereby effectively improving the accuracy of hydraulic cylinder motion state monitoring.

[0072] 102. By extracting and tracking the target from the continuous frame images, the first tracking result of the first component and the second tracking result of the second component are obtained.

[0073] In this embodiment of the invention, to identify the first component and the second component from an image, target extraction needs to be performed on each frame of a series of images to determine the positions of the first and second components in the image. For example, a target extraction network model is used to extract the detection boxes of the first component and the second component in each frame of the image. Further, based on the target extraction results, target tracking is performed to obtain the motion features of the first component (i.e., the first tracking result) and the motion features of the second component (i.e., the second tracking result). The tracking results include at least the motion speed.

[0074] It should be noted that by extracting targets from consecutive frames of images, the pixel regions of the first and second components in the images can be identified. Furthermore, by using the first or second component in the identified pixel region as the tracking target, the motion features of the first and second components can be extracted, thereby enabling the identification of the target cylinder's motion state based on the motion features of the first and second components.

[0075] 103. Determine the motion state of the target cylinder based on the first tracking result and the second tracking result.

[0076] In this embodiment of the invention, one complete extension / retraction process is included in one movement cycle of the hydraulic cylinder. The extension process involves the cylinder body remaining stationary while the piston rod moves forward, resulting in the overall extension of the hydraulic cylinder. The retraction process involves the cylinder body moving forward while the piston rod remains stationary, resulting in the overall shortening of the hydraulic cylinder. Since the first component and the second component are respectively connected to the piston rod end and cylinder body end of the target hydraulic cylinder, the movement state of the hydraulic cylinder can be determined through the movement correlation between the first component, the second component, and the target hydraulic cylinder. For example, if the first component is connected to the piston rod end of the target hydraulic cylinder, and the second component is connected to the cylinder body end of the target hydraulic cylinder, if the first component is moving and the second component is stationary, it indicates that the target hydraulic cylinder is currently pushing the first component forward, in an extension state; if the second component is moving and the first component is stationary, it indicates that the target hydraulic cylinder is currently driving the second component forward, in a retraction state. By tracking the first and second components, accurate identification of their movement characteristics can be achieved, thereby enabling accurate monitoring of the movement state of the target hydraulic cylinder.

[0077] In one embodiment of the present invention, for further explanation and limitation, step 102, which involves extracting and tracking the target from the consecutive frame images to obtain the first tracking result of the first component and the second tracking result of the second component, includes:

[0078] 201. Using the trained target detection network model, target extraction is performed on the consecutive frame images to obtain a first target detection box and a second target detection box;

[0079] 202. Target tracking is performed based on the first target detection box and the second target detection box to obtain the first tracking result of the first component and the second tracking result of the second component.

[0080] In this embodiment of the invention, in order to identify the target to be tracked, namely the first component and the second component, from each frame of image, a pre-trained target detection network model is used to determine the detection box of the first component in each frame of image, namely the first target detection box, and the detection box of the second component, namely the second target detection box. The base model of the pre-trained target detection network model can be the YOLOv5 model, or other target detection models; this embodiment of the invention does not impose specific limitations. Further, after obtaining the first target detection box and the second target detection box, the device image within the target detection box is used as the target for tracking to extract features, obtaining a first tracking result and a second tracking result.

[0081] In one embodiment of the present invention, for further explanation and limitation, before extracting targets from the consecutive frame images using a trained target detection network model to obtain a first target detection box and a second target detection box, the method further includes:

[0082] Construct a training sample set and an initial target detection network model. The sample images in the training sample set include a first part image that has been labeled and a second part image that has been labeled.

[0083] Based on the anchor box size and anchor box position in the initial target detection network model, target detection boxes are predicted for the sample image to complete the training of the initial target detection network model. The anchor box size and anchor box position are set based on the image size and image position of the first component and the second component in historical images.

[0084] In this embodiment of the invention, to accurately extract the first and second components from consecutive frame images, a training sample set needs to be pre-constructed to train the initial object detection network model. Each frame in the training sample set includes a labeled first component and a labeled second component. Specifically, each frame in the training sample set is compressed to a size of 608*608, and the compressed images are stitched together using random scaling, random cropping, and random arrangement to achieve image mosaic enhancement. Further, based on the pre-set anchor box positions and sizes of the initial object detection network model, the target detection boxes in each frame are predicted. The parameters of the initial object detection network model are then updated and iterated in reverse based on the residual between the predicted detection boxes and the labeled ground truth boxes of the first or second component.

[0085] In this embodiment of the invention, based on the performance of the server deployed on the fully mechanized mining surface, the target detection network model selected is the yolov5m model from the yolov5 model. Since yolov5 includes three sizes—yolov5s, yolov5m, and yolov5l—and a larger model size results in higher detection accuracy but longer detection time, selecting the yolov5m model can accelerate detection speed while maintaining accuracy. To further accelerate target detection, the BN layers of the Conv, Focus, C3, and SPP modules in the yolov5m model structure can be pruned to reduce the number of model parameters. Furthermore, the TensorRT framework can be used to accelerate the prediction speed of the yolov5m model, thereby meeting the real-time requirements for target detection in the fully mechanized mining environment.

[0086] It should be noted that the anchor frame size and position are set based on the image size and position of the first and second components in historical images. By statistically analyzing the image size and position of the first and second components in historical images, anchor frame sizes and positions that better match the image size and position of the first and second components are determined. Based on this, the default anchor frame size and position in the YOLOv5 network model are modified, making the target detection network model more suitable for target detection of the first and second components in a fully mechanized mining environment. This accelerates the convergence of the initial target detection network model and improves the accuracy and recognition rate of target detection.

[0087] In one embodiment of the present invention, for further explanation and limitation, the motion state in step 103 includes a first state and a second state. After determining the motion state of the target cylinder based on the tracking result, the method further includes:

[0088] If the motion state is the first state, then the pixel distance between the first target detection box in every two adjacent frame images is calculated, and the pixel distance is accumulated to obtain the first displacement.

[0089] If the motion state is the second state, then the pixel distance between the second target detection box in every two adjacent frame images is calculated, and the pixel distance is accumulated to obtain the second displacement.

[0090] If the first displacement or the second displacement is less than a preset displacement threshold, a displacement warning message indicating that the displacement of the target cylinder is abnormal is generated, so that the staff can inspect the target cylinder.

[0091] In this embodiment of the invention, when the hydraulic cylinder is in the first state, it indicates that the first component is in motion. Using the horizontal center point of the target detection box of the first component in each frame as a reference, the pixel position difference of the first component in two adjacent frames is calculated until the first component stops moving or a new target appears. The pixel position differences of the first component during one motion process are accumulated to obtain the displacement of the first component, i.e., the first displacement. The calculation process for the second displacement is the same as that for the first displacement, and will not be repeated here. After obtaining the displacements of the first and second components in one motion, to identify whether the extension and retraction of the hydraulic cylinder are in place, the first and second displacements are compared with preset displacement thresholds. If the first and second displacements are greater than or equal to the preset displacement thresholds, it indicates that the target hydraulic cylinder has extended and retracted in place and is operating normally; if the first and second displacements are less than the preset displacement thresholds, it indicates that the target hydraulic cylinder has not extended and retracted in place and is operating abnormally, requiring notification to personnel for maintenance. The preset displacement threshold is determined based on the accumulated value of the pixel position differences generated by the same target in one motion process in consecutive frames of images acquired under normal hydraulic cylinder operation. By calculating and comparing the displacements of the first and second components, it is possible to determine whether the target cylinder has extended or retracted in place, and to provide timely warnings in cases where the target cylinder has not extended or retracted in place, thereby ensuring the normal operation of the target cylinder.

[0092] In one embodiment of the present invention, for further explanation and limitation, the step of performing target tracking based on the first target detection box and the second target detection box to obtain the first tracking result of the first component and the second tracking result of the second component includes:

[0093] Feature extraction is performed on the targets in the first target detection box and the second target detection box respectively to obtain at least two attribute features, and the matching degree of the same attribute feature in two adjacent frames is calculated;

[0094] Attribute features with a matching degree less than the preset confidence level are deleted until the number of deletions of the same attribute feature is greater than or equal to the preset deletion quantity threshold. Then, the attribute feature is determined as the attribute feature of the new target.

[0095] Motion features are extracted for targets with the same attribute characteristics, and attribute association is performed to obtain the first tracking result of the first component and the second tracking result of the second component.

[0096] In this embodiment of the invention, the attribute feature is the target ID. By calculating the similarity between the first target ID and the second target ID in two adjacent frames, it is determined whether the targets in the first target detection box and the second target detection box in the two adjacent frames are the same target. If they are the same target, the target in the next frame is checked. If they are not the same target, the target is determined as a miss and deleted until the number of deletions reaches a preset deletion threshold, at which point the target is set as a new target. Motion features are extracted for the same target ID to obtain the motion features corresponding to the target ID. Based on the attribute association between the target ID and the first and second target detection boxes, the tracking results of the target in the first target detection box are obtained, i.e., the first tracking result, and the tracking results of the target in the second target detection box are obtained, i.e., the second tracking result. The above calculation process can be completed based on a target tracking network model. The target tracking network model can be DeepSort or other target tracking models that can perform multi-target tracking. This embodiment of the invention does not make specific limitations.

[0097] It should be noted that the preset confidence level and preset deletion threshold are set based on the analysis results of historical images of the monitored targets. Since the positions of the first and second components are relatively fixed, new targets usually do not appear in the target detection box. Therefore, based on the experimental results of historical images, the default confidence level in the target tracking network model is lowered, changing the default confidence level from 0.5 to 0.3. Furthermore, the preset deletion threshold corresponds to the maximum number of misses (max_age) before deletion in the target tracking network model. If the number of unidentified instances for the same target reaches the preset deletion threshold, then that target is considered a target with new attribute characteristics. Since the inability to identify a target is generally caused by personnel occlusion rather than the appearance of a new target, based on the experimental results of historical images, the default max_age in the target tracking network model is increased, changing the default max_age50 to 100. The above confidence level and preset deletion threshold are obtained based on experiments with historical images of the fully mechanized mining face. They can also be customized according to specific application scenarios, and this embodiment of the invention does not impose specific limitations. By lowering the confidence level and raising the preset deletion threshold, we can prevent the target tracking process from filtering out detected targets due to excessively high confidence, and prevent occluded targets from being treated as new targets. This makes the target extraction and target tracking process suitable for application scenarios where the targets in the fully mechanized mining face are relatively fixed and there are many occlusions, thereby effectively improving the target tracking recognition rate.

[0098] In one embodiment of the present invention, for further explanation and limitation, step 103, which involves determining the motion state of the target cylinder based on the first tracking result and the second tracking result, includes:

[0099] If the first tracking result is greater than a preset threshold, then the motion state is determined to be the first state.

[0100] If the second tracking result is greater than the preset threshold, then the motion state is determined to be the second state.

[0101] In this embodiment of the invention, the motion state includes a first state and a second state. The first state indicates that the target cylinder is in an extended state; the second state indicates that the target cylinder is in a retracted state. The first component is a scraper conveyor, and the second component is a pilot valve. The target cylinder is located between the pilot valve and the scraper conveyor, responsible for pushing the scraper conveyor towards the coal face. Specifically, the scraper conveyor is connected to the piston rod end of the target cylinder, and the pilot valve is connected to the cylinder body end of the target cylinder. The preset threshold can be zero, or it can be a value close to zero set by the user according to specific application needs; this embodiment of the invention does not impose a specific limitation. If the speed of the scraper conveyor is greater than the preset threshold, it indicates that the target cylinder is currently pushing the scraper conveyor towards the coal face, and the motion state of the target cylinder is determined to be extended. If the speed of the pilot valve is greater than the preset threshold, it indicates that the target cylinder is currently driving the pilot valve to move towards the coal face, and the motion state of the target cylinder is determined to be retracted. As can be seen from the working principle of the hydraulic cylinder, the scraper conveyor and the pilot valve will not move simultaneously. Therefore, the movement state of the target hydraulic cylinder can be determined by comparing the first tracking result or the second tracking result with the preset threshold.

[0102] In one embodiment of the present invention, for further explanation and limitation, after determining the motion state of the target cylinder based on the first tracking result and the second tracking result, the method further includes:

[0103] Obtain the control command for the target cylinder corresponding to the time of the consecutive frame images.

[0104] The motion state is matched and verified with the control information to obtain the verification result.

[0105] If the verification result is a failure, a status warning message is generated to indicate that the target cylinder is in an abnormal operating state, so that the staff can inspect and repair the target cylinder.

[0106] In this embodiment of the invention, the control command carries control information to control the motion state of the target cylinder. The motion process of the target cylinder is based on the control command. However, during the control process, problems such as equipment failure may occur, causing the actual motion process of the target cylinder to deviate from the control command. For example, during the equipment debugging phase, the control information carried by the equipment control command instructs the cylinder to extend, while the actual cylinder is in a retracting motion state. By matching and verifying the motion state with the control information in the control command, the mismatch between the target cylinder and the control command can be detected in a timely manner, thereby ensuring that the target cylinder works according to the control expectation.

[0107] This invention provides a method for monitoring the motion state of a hydraulic cylinder. In this embodiment, continuous frame images of a target are acquired. The target includes a target hydraulic cylinder and a first component and a second component connected to both ends of the target hydraulic cylinder. Target extraction and tracking are performed on the continuous frame images to obtain a first tracking result for the first component and a second tracking result for the second component. The motion state of the target hydraulic cylinder is determined based on the first and second tracking results. This method significantly reduces the interference of the mining face environment on the hydraulic cylinder motion data acquisition process, overcomes the difficulties of indistinct hydraulic cylinder features and the difficulty in capturing key points, while ensuring the accuracy of hydraulic cylinder motion data acquisition, thereby greatly improving the accuracy of hydraulic cylinder motion state monitoring.

[0108] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a hydraulic cylinder motion state monitoring device, such as... Figure 3 As shown, the device includes:

[0109] The acquisition module 31 is used to acquire continuous frame images of the monitored target, the monitored target including a target cylinder and a first component and a second component connected to both ends of the target cylinder;

[0110] The target tracking module 32 is used to extract and track targets from the continuous frame images to obtain a first tracking result of the first component and a second tracking result of the second component.

[0111] The determination module 33 is used to determine the motion state of the target cylinder based on the first tracking result and the second tracking result.

[0112] Furthermore, the target tracking module includes:

[0113] The target extraction unit is used to extract targets from the consecutive frame images using a trained target detection network model to obtain a first target detection box and a second target detection box.

[0114] The target tracking unit is used to perform target tracking based on the first target detection box and the second target detection box to obtain a first tracking result of the first component and a second tracking result of the second component.

[0115] Furthermore, the device also includes:

[0116] The construction module is used to construct a training sample set and an initial object detection network model. The sample images in the training sample set include a first part image that has been labeled and a second part image that has been labeled.

[0117] The training module is used to predict target detection boxes on the sample image based on the anchor box size and anchor box position in the initial target detection network model, so as to complete the training of the initial target detection network model. The anchor box size and anchor box position are set based on the image size and image position of the first component and the second component in the historical image.

[0118] Furthermore, in specific application scenarios, the target tracking unit is used to extract features from the targets in the first target detection box and the second target detection box respectively, to obtain the first attribute feature corresponding to the first target detection box and the second attribute feature corresponding to the second target detection box, and to calculate the first similarity of the first attribute feature and the second similarity of the second attribute feature in two adjacent frames of images;

[0119] Image frames with a first similarity or a second similarity less than the preset confidence level are deleted according to a preset deletion quantity threshold;

[0120] Motion feature extraction and attribute association are performed on the image frames after deletion to obtain the first tracking result of the first component and the second tracking result of the second component.

[0121] The preset reliability and the preset deletion quantity threshold are set based on the analysis results of historical images of the monitored target.

[0122] Furthermore, the device also includes:

[0123] The first calculation module is used to calculate the pixel distance between the first target detection box in every two adjacent frame images if the motion state is the first state, and to accumulate the pixel distance to obtain the first displacement.

[0124] The second calculation module is used to calculate the pixel distance between the second target detection box in every two adjacent frame images if the motion state is the second state, and to accumulate the pixel distance to obtain the second displacement.

[0125] The generation module is used to generate a displacement warning message indicating that the displacement of the target cylinder is abnormal if the first displacement or the second displacement is less than a preset displacement threshold, so that the staff can inspect the target cylinder.

[0126] Further, module 33 is defined as including:

[0127] The first determining unit is configured to determine the motion state as a first state if the first tracking result is greater than a preset threshold, wherein the first state is used to characterize the target cylinder as being in an extended state.

[0128] The second determining unit is used to determine the motion state as a second state if the second tracking result is greater than the preset threshold. The second state is used to characterize that the target cylinder is in a retracted state.

[0129] Furthermore, the device also includes:

[0130] The acquisition module 31 is further configured to acquire the control command of the target cylinder corresponding to the time of the continuous frame image, the control command carrying control information for controlling the motion state of the target cylinder;

[0131] The verification module is used to match and verify the motion state with the control information to obtain the verification result;

[0132] The generation module is also used to generate a status warning message indicating that the target cylinder is in an abnormal operating state if the verification result is a failure, so that the staff can inspect and repair the target cylinder.

[0133] This invention provides a hydraulic cylinder motion state monitoring device. In this embodiment, continuous frame images of a monitored target are acquired. The monitored target includes a target hydraulic cylinder and a first component and a second component connected to both ends of the target hydraulic cylinder. Target extraction and tracking are performed on the continuous frame images to obtain a first tracking result for the first component and a second tracking result for the second component. The motion state of the target hydraulic cylinder is determined based on the first and second tracking results. This significantly reduces the interference of the mining face environment on the hydraulic cylinder motion data acquisition process, overcomes the difficulties of indistinct hydraulic cylinder features and the difficulty in capturing key points, while ensuring the accuracy of hydraulic cylinder motion data acquisition, thereby greatly improving the accuracy of hydraulic cylinder motion state monitoring.

[0134] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the cylinder motion state monitoring method in any of the above method embodiments.

[0135] Figure 4The diagram shows a structural schematic of a terminal according to an embodiment of the present invention. The specific implementation of the terminal is not limited by the specific embodiment of the present invention.

[0136] like Figure 4 As shown, the terminal may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0137] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0138] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0139] The processor 402 is used to execute program 410, which can specifically execute the relevant steps in the above embodiment of the hydraulic cylinder motion state monitoring method.

[0140] Specifically, program 410 may include program code that includes computer operation instructions.

[0141] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The terminal may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0142] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0143] Specifically, program 410 can be used to cause processor 402 to perform the following operations:

[0144] Acquire continuous frame images of the monitored target, the monitored target including a target hydraulic cylinder, and a first component and a second component connected to both ends of the target hydraulic cylinder;

[0145] By performing target extraction and target tracking on the continuous frame images, a first tracking result of the first component and a second tracking result of the second component are obtained;

[0146] The motion state of the target cylinder is determined based on the first tracking result and the second tracking result.

[0147] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of monitoring the state of motion of a cylinder, characterized in that The method comprises the following steps: obtaining continuous frame images of a monitoring target, the monitoring target comprising a target oil cylinder and a first component and a second component connected to two ends of the target oil cylinder, wherein the first component is a scraper and the second component is a pilot valve; synchronously extracting motion characteristics of the first component and the second component by target extraction and target tracking on the continuous frame images to obtain a first tracking result of the first component and a second tracking result of the second component, wherein the tracking result is used to represent the motion characteristics of the corresponding component, the target extraction is realized based on a target detection network model that has completed training, the anchor box size and anchor box position of the target detection network model are pre-set based on the image size and position of the first component and the second component in historical images, the target tracking process comprises: respectively extracting features of targets in a first target detection box of the first component and a second target detection box of the second component to obtain first attribute features corresponding to the first target detection box and second attribute features corresponding to the second target detection box, and calculating a first similarity of the first attribute features and a second similarity of the second attribute features in adjacent two frame images; deleting image frames with a preset confidence level that is less than a preset number threshold according to the first similarity or the second similarity; and extracting motion characteristics and attribute correlation of the image frames after the deletion processing to obtain the first tracking result of the first component and the second tracking result of the second component; wherein the preset confidence level and the preset number threshold are set based on an analysis result of historical images of the monitoring target; determining a motion state of the target oil cylinder according to the first tracking result and the second tracking result, wherein if the first component is in motion and the second component is in a static state, it indicates that the target oil cylinder is in an elongation state; if the second component is in motion and the first component is in a static state, it indicates that the target oil cylinder is in a retraction state.

2. The method of claim 1, wherein, obtaining the first tracking result of the first component and the second tracking result of the second component by target extraction and target tracking on the continuous frame images, comprising: performing target extraction on the continuous frame images by using a target detection network model that has completed training to obtain a first target detection box and a second target detection box; performing target tracking according to the first target detection box and the second target detection box to obtain the first tracking result of the first component and the second tracking result of the second component.

3. The method of claim 2, wherein, Before the step of performing target extraction on the continuous frame images by using a target detection network model that has completed training to obtain a first target detection box and a second target detection box, the method further comprises: constructing a training sample set and an initial target detection network model, wherein sample images in the training sample set comprise a first component image that has completed labeling and a second component image that has completed labeling; The sample image is subjected to target detection frame prediction based on anchor frame size and anchor frame position in the initial target detection network model, so as to complete training of the initial target detection network model, wherein the anchor frame size and the anchor frame position are set based on image size and image position of the first component and the second component in a historical image.

4. The method of claim 2, wherein, The motion state includes a first state and a second state, and after determining the motion state of the target oil cylinder according to the tracking result, the method further includes: If the motion state is the first state, the pixel distance of the first target detection frame in each two adjacent frame images is calculated, and the pixel distance is accumulated to obtain a first displacement; If the motion state is the second state, the pixel distance of the second target detection frame in each two adjacent frame images is calculated, and the pixel distance is accumulated to obtain a second displacement; If the first displacement or the second displacement is less than a preset displacement threshold, displacement warning information indicating that the displacement amount of the target oil cylinder is abnormal is generated, so that the staff can maintain the target oil cylinder.

5. The method of claim 1, wherein, After determining the motion state of the target oil cylinder according to the first tracking result and the second tracking result, the method further includes: obtaining a control instruction of the target oil cylinder corresponding to the time of the continuous frame images, the control instruction carrying control information for controlling the motion state of the target oil cylinder; matching and verifying the motion state with the control information to obtain a verification result; If the verification result fails the verification, state warning information indicating that the running state of the target oil cylinder is abnormal is generated, so that the staff can maintain the target oil cylinder.

6. An oil cylinder motion state monitoring device characterized by comprising: The device is used to perform operations corresponding to the oil cylinder motion state monitoring method in any one of claims 1-5, and includes: an acquisition module, configured to acquire continuous frame images of a monitoring target, the monitoring target including a target oil cylinder, and a first component and a second component connected to two ends of the target oil cylinder, wherein the first component is a scraper, and the second component is a pilot valve; a target tracking module, configured to obtain a first tracking result of the first component and a second tracking result of the second component by performing target extraction and target tracking on the continuous frame images, wherein the tracking result is used to represent the motion characteristics of the corresponding component; a determination module, configured to determine the motion state of the target oil cylinder according to the first tracking result and the second tracking result, wherein if the first component is in motion and the second component is in a stationary state, it indicates that the target oil cylinder is in an elongated state; and if the second component is in motion and the first component is in a stationary state, it indicates that the target oil cylinder is in a retracted state.

7. A storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the oil cylinder motion state monitoring method in any one of claims 1-5.

8. A terminal comprising: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface completing communication with each other through the communication bus; The memory is configured to store at least one executable instruction, and the executable instruction is configured to enable the processor to perform operations corresponding to the cylinder motion state monitoring method according to any one of claims 1-5.

Images