Automatic machine vision inspection system for shearer drum, shearer system and method

Abstract

The present invention relates to an automatic machine vision inspection system for shearer drums, a shearer system, and a method. The automatic machine vision inspection system for shearer drums includes: a camera device for capturing a side view of the shearer drum; and a drum machine vision intelligent recognition system. The camera device is communicatively connected to the drum machine vision intelligent recognition system and outputs a real-time side view to the drum machine vision intelligent recognition system. The drum machine vision intelligent recognition system is configured to perform automatic machine vision inspection of the shearer drum based on the side view. The drum machine vision intelligent recognition system includes a shearer drum position tracking and measurement mode. In this mode, the drum machine vision intelligent recognition system determines the positional deviation of the shearer drum relative to the camera device based on the side view and adjusts the shearer drum to a preset observation position based on the positional deviation. The present invention improves inspection efficiency and accuracy through automated drum pick inspection.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mine mechanical equipment detection, and in particular to an automatic machine vision inspection system for a shearer drum, a shearer system and an inspection method. Background Art

[0002] The shearer drum and its mounted picks perform all the functions of rotary coal and rock crushing, as well as loading. They are subject to extreme impacts that can cause loss and severe wear, requiring regular inspections to ensure proper operation. Traditional manual inspections pose significant safety risks and are inefficient.

[0003] Currently, underground machine vision technology has been used to monitor coal mining equipment. While existing technologies can identify the position of the shearer drum, they primarily focus on monitoring and early warning functions, and still face key technical challenges in detecting the status of drum cutters.

[0004] For example, existing technologies usually require operators to manually adjust the position of the coal mining machine or the angle of the camera equipment, which has problems such as complex and time-consuming adjustment process, limited adjustment accuracy, and safety hazards. Summary of the Invention

[0005] In view of this, the present invention provides an automatic machine vision inspection system for a coal mining machine drum, a coal mining machine system and a method, thereby solving or at least alleviating one or more of the above-mentioned problems and other problems existing in the prior art.

[0006] In order to achieve the aforementioned objectives, a first aspect of the present invention provides an automatic machine vision inspection system for a shearer drum, wherein the automatic machine vision inspection system for a shearer drum comprises:

[0007] A camera device, the camera device is used to capture a side view of the shearer drum;

[0008] The drum machine vision intelligent recognition system, wherein the camera device is communicatively connected to the drum machine vision intelligent recognition system and outputs a real-time side view to the drum machine vision intelligent recognition system, and the drum machine vision intelligent recognition system is used to perform automatic machine vision inspection of the coal mining machine drum based on the side view.

[0009] in,

[0010] The drum machine vision intelligent recognition system includes a coal mining machine drum position tracking and measurement mode. In the coal mining machine drum position tracking and measurement mode, the drum machine vision intelligent recognition system determines the position deviation of the coal mining machine drum relative to the camera device based on the side view, and adjusts the coal mining machine drum to a preset observation position based on the position deviation.

[0011] In the aforementioned automatic machine vision inspection system for the shearer drum, optionally, the camera device is installed below the top beam of the hydraulic support of the coal mining face and faces the goaf side, and can move forward and backward with the hydraulic support.

[0012] In the aforementioned automatic machine vision inspection system for the shearer drum, optionally, the shearer is a double-drum shearer, the camera device includes at least two cameras capable of simultaneously photographing the drum side surfaces of the two drums of the double-drum shearer, and the spacing between the cameras is equal to the spacing between the two drums when the left and right rocker arms of the double-drum shearer are simultaneously in the horizontal position;

[0013] or

[0014] The coal shearer is a single-drum coal shearer, and the camera device includes at least one camera capable of photographing the side of the drum of the single-drum coal shearer.

[0015] In the aforementioned automatic machine vision inspection system for the shearer drum, optionally, in the position tracking and measurement mode of the shearer drum, the drum machine vision intelligent recognition system includes:

[0016] an intelligent search module configured to search for a shearer drum in the side view; and

[0017] A positioning measurement module is used to measure and output the relative values ​​of the X-axis and Y-axis deviations of the coal mining machine drum from the camera device and the preset observation position through machine vision after the intelligent search module searches for the coal mining machine drum.

[0018] In the automatic machine vision inspection system for the coal shearer drum as described above, optionally, the automatic machine vision inspection system for the coal shearer drum is communicatively connected to the coal shearer travel control system and the coal shearer height adjustment control system of the coal shearer, and is used to send a travel instruction to the coal shearer travel control system according to the X-axis deviation of the relative deviation value, so that the coal shearer travels to a preset X-axis position, and at the same time send a height adjustment instruction to the coal shearer height adjustment control system according to the Y-axis deviation of the relative deviation value, so as to raise or lower the coal shearer drum to a preset Y-axis position.

[0019] In the automatic machine vision inspection system for the coal mining machine drum as described above, optionally, the walking speed is 1 m / min, the maximum rising or falling speed is 0.5 m / s, and the adjustment hysteresis between the position to which the coal mining machine walks and the X-axis position is 5%, and the adjustment hysteresis between the position to which the coal mining machine drum rises or falls and the Y-axis position is 5%.

[0020] In the automatic machine vision inspection system for the shearer drum as described above, optionally, the automatic machine vision inspection system for the shearer drum includes an embedded edge computing device, in which a drum contour detection model is deployed, the drum contour detection model is trained via a data set including side images under different lighting conditions and occlusion conditions, the drum contour detection model receives the real-time video stream of the camera device and outputs a bounding box and confidence information of the contour of the shearer drum, normalizes the coordinates of the center point of the bounding box, and obtains the position of the shearer drum in the image coordinate system; and,

[0021] While the camera device continuously captures images of the drum, the automatic machine vision inspection system for the shearer drum uses a tracking algorithm to track the contour of the rotating shearer drum, calculates the motion trajectory of the shearer drum, and predicts the next frame position of the shearer drum. If the drum target is lost for a predetermined number of consecutive frames, the shearer drum position tracking mode is triggered to restart, and the automatic machine vision inspection process of the shearer drum is reinitialized.

[0022] In the coal mining machine drum automatic machine vision inspection system as described above, optionally, the drum machine vision intelligent recognition system includes a drum pick detection mode. When the coal mining machine drum is adjusted to the preset observation position, the coal mining machine drum automatic machine vision inspection system sends an instruction to the coal mining machine to stop the drum rotation. At the same time, the drum machine vision intelligent recognition system switches to the drum pick detection mode. Moreover, in the drum pick detection mode, the driving motor of the coal mining machine drum is powered off so that the coal mining machine drum continuously decelerates and rotates according to inertia. During the deceleration rotation process, the drum machine vision intelligent recognition system performs 360-degree visual inspection coverage on the coal mining machine drum to complete automatic detection output.

[0023] In the automatic machine vision inspection system for the coal mining machine drum as described above, optionally, when the speed of the decelerated rotation is reduced to a predetermined speed, the video sequence of the inertial rotation stage is used to detect the missing and wear of the drum teeth, and the detection results are verified in combination with the timing information, and the confidence threshold is a confidence threshold that is dynamically adjusted between 0.5 and 0.7.

[0024] In the automatic machine vision inspection system for the coal mining machine drum as described above, optionally, the automatic machine vision inspection system for the coal mining machine drum has an alarm unit. When it is detected that the drum pick of the coal mining machine drum is missing or the degree of wear exceeds a preset threshold, the automatic machine vision inspection system for the coal mining machine drum automatically generates a detection result, and the alarm unit sends an alarm message to the remote monitoring terminal or maintenance personnel, prompting the replacement or maintenance of the drum pick.

[0025] In the aforementioned automatic machine vision inspection system for the shearer drum, optionally, the visual image is a video or an image.

[0026] To achieve the aforementioned objectives, a second aspect of the present invention provides a coal shearer system, wherein the coal shearer system has an automatic machine vision inspection system for coal shearer drums as described in any one of the first aspects.

[0027] To achieve the aforementioned objectives, a third aspect of the present invention provides a method for automatic machine vision inspection of a shearer drum using the automatic machine vision inspection system for a shearer drum according to any one of the first aspects, wherein the method comprises:

[0028] Step 1: starting the automatic machine vision inspection system for the shearer drum;

[0029] Step 2: The drum machine vision intelligent recognition system enters the coal mining drum position tracking mode, determines the position deviation of the coal mining drum relative to the camera device based on the side view, and adjusts the coal mining drum to a preset observation position based on the position deviation;

[0030] Step 3: The drum machine vision intelligent recognition system enters the drum pick detection mode to perform drum pick missing and wear detection.

[0031] In the method as described above, optionally, in step 2:

[0032] First, the drum contour detection model of the shearer drum automatic machine vision inspection system detects the contour of the shearer drum. If the shearer drum cannot be identified in the video, a position adjustment command is sent to the shearer to guide it to move into the monitoring range of the camera device.

[0033] Secondly, after the shearer drum automatic machine vision inspection system detects the contour of the shearer drum, the coordinate position of the shearer drum is recorded, and a tracking algorithm is started to track and update the X / Y axis deviation of the drum in real time;

[0034] Then, the coal shearer drum automatic machine vision inspection system determines whether it is in the preset observation position and the optimal detection field of the drum pick based on the X / Y axis deviation of the coal shearer drum. If the coal shearer drum is not in the preset observation position, the coal shearer drum automatic machine vision inspection system automatically sends the coal shearer travel adjustment instruction and the coal shearer height control instruction to adjust the coal shearer travel position and the drum height so that the coal shearer drum is in the optimal detection field of the drum pick.

[0035] In the above method, optionally, in step 3:

[0036] First, in the drum pick detection mode, a drum power-off instruction is sent to the coal mining machine;

[0037] Secondly, after the drum loses power, it rotates slowly due to inertia. The shearer drum automatic machine vision inspection system monitors the drum's rotation speed in real time. When the rotation speed drops to a set threshold, the drum pick detection model automatically activates to analyze the wear and tear of the drum picks on the drum.

[0038] Then, after the inspection is completed, the coal mining machine drum automatic machine vision inspection system automatically generates the inspection results and synchronizes the inspection results to the remote monitoring terminal to remind the maintenance personnel to replace or maintain the drum teeth.

[0039] The automatic machine vision inspection system for the coal mining machine drum of the present invention obtains the side view of the coal mining machine drum in real time through the camera device. Combined with the coal mining machine drum position tracking and measurement mode of the drum machine vision intelligent recognition system, it can automatically determine the position deviation of the coal mining machine drum relative to the camera device, and automatically adjust the coal mining machine drum to the preset observation position based on the position deviation, thereby realizing a fully automated inspection process and completely getting rid of the dependence on traditional manual inspection.

[0040] According to an optional embodiment of the present invention, the system can use machine vision technology to accurately measure and automatically adjust position deviations, avoiding the subjectivity and instability of manual operation, ensuring that each inspection can be performed at the same optimal observation position, and significantly improving inspection accuracy and consistency.

[0041] In an optional embodiment, further, through remote automated operation, operators do not need to approach dangerous areas, which greatly reduces the safety risks of underground operations and improves operational safety.

[0042] In an optional embodiment, further, the automatic machine vision inspection system for the coal mining machine drum has automatic position tracking and measurement functions, can intelligently determine the drum position and automatically adjust to the optimal inspection position, realize intelligent management of the inspection process, and reduce the need for manual intervention. At the same time, the system performs inspections through side vision, can adapt to complex lighting and environmental conditions underground, and enhance the adaptability and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The disclosure of the present invention will become more apparent with reference to the accompanying drawings. It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings:

[0044] Figure 1 is a schematic block diagram of an embodiment of an automatic machine vision inspection system for a coal mining machine drum according to the present invention;

[0045] Figure 2 A schematic diagram of the arrangement of components of an embodiment of an automatic machine vision inspection system for a shearer drum according to the present invention; and

[0046] Figure 3 The figure is a schematic flow chart of an embodiment of the method for automatic machine vision inspection of a coal mining machine drum according to the present invention.

[0047] Figure numerals: 1 - automatic machine vision inspection system for shearer drum; 2 - camera device; 3 - shearer drum; 4 - drum machine vision intelligent recognition system; 5 - hydraulic support; 6 - drum pick; 7 - rocker arm; 8 - controller. DETAILED DESCRIPTION

[0048] With reference to the accompanying drawings and specific embodiments, the structure, composition, characteristics and advantages of the automatic machine vision inspection system for coal mining machine drums, coal mining machine systems and methods of the present invention will be described below in an exemplary manner. However, all descriptions should not be used to form any limitations on the present invention.

[0049] In addition, for any single technical feature described or implied in the embodiments mentioned in this document, or any single technical feature shown or implied in the drawings, the present invention still allows for continued arbitrary combination or deletion between these technical features (or their equivalents) without any technical obstacles, and thus it should be considered that these more embodiments according to the present invention are also within the scope of the description in this document.

[0050] Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features being referred to. Thus, a feature identified as "first," "second," or "third" may explicitly or implicitly include at least one of these features.

[0051] The shearer drum is the core working component of the shearer, directly responsible for cutting and crushing the coal seam and loading it into the scraper conveyor. It is typically mounted at the end of the shearer's rocker arm. Shearers can be single-drum or dual-drum designs. In the dual-drum design, a drum is mounted at each end of the machine body. The drum is mounted on the output shaft of the shearer's rocker arm via large bearings. It is driven by a reduction gear mechanism and electric motor within the rocker arm to rotate at high speed. The drum pick is the "cutting tool" that directly participates in crushing the coal and rock. The drum pick is secured to the tooth adapter by a retaining ring or circlip. These picks are susceptible to loss or wear, requiring regular maintenance or replacement.

[0052] Figure 1 FIG. 1 is a schematic block diagram of an embodiment of an automatic machine vision inspection system for a coal mining machine drum according to the present invention.

[0053] As shown in the figure, the shearer drum automatic machine vision inspection system includes a camera device, a drum machine vision intelligent recognition system, a shearer control system, and a drum pick detection module. In this embodiment, the camera device is a camera array, and the shearer control system includes a shearer travel control system and a shearer height adjustment control system.

[0054] As can be seen from the figure, the camera device is able to communicate with the drum machine vision intelligent recognition system, allowing the camera device to transmit the data of the captured side view of the shearer drum to the drum machine vision intelligent recognition system. The drum machine vision intelligent recognition system communicates with the shearer control system and the drum pick detection module. The drum machine vision intelligent recognition system is able to detect and determine the deviation of the side view of the shearer drum relative to the camera device. The shearer control system adjusts the position of the shearer drum in the side view based on this deviation, and the drum pick detection module can be used to detect the status of the drum picks, such as missing or worn. The shearer control system can communicate with the drum pick detection module and, after the shearer control system has adjusted the side view position, initiate the drum pick detection mode.

[0055] Specifically, in this embodiment, a camera device is used to capture a side view of the shearer drum. The camera device is communicatively connected to the drum machine vision intelligent recognition system and outputs real-time side views to it. The drum machine vision intelligent recognition system is used to automatically perform machine vision inspections of the shearer drum based on the side views. Furthermore, the drum machine vision intelligent recognition system can include a shearer drum position tracking and measurement mode. In this mode, the shearer drum machine vision intelligent recognition system determines the positional deviation of the shearer drum relative to the camera device based on the side views and adjusts the shearer drum to a preset observation position based on the positional deviation.

[0056] With this arrangement, if the shearer drum is not located at the preset observation position for the side view captured by the camera, the shearer control system can adjust the shearer's travel position and the height of its rocker arm to bring the shearer drum to the preset observation position relative to the camera. For example, at this preset observation position, the shearer drum can be located at the center of the side view, ensuring that the entire shearer drum is completely within the view. In an optional embodiment, while ensuring that the shearer drum is completely within the side view, the preset observation position can also be another position in the side view that is offset from its center. In this context, "view" can be understood as a video or image.

[0057] Figure 2This figure is a schematic diagram illustrating the component layout of an embodiment of an automatic machine vision inspection system for a shearer drum according to the present invention. The figure shows the automatic machine vision inspection system 1 for a shearer drum, along with a camera 2, a shearer drum 3, an intelligent machine vision recognition system 4, a hydraulic support 5, a drum pick 6, a shearer drum rocker arm 7, and a controller 8.

[0058] As shown in the figure, the camera device 2 can be installed below the top beam of the hydraulic support 5 of the coal mining face and facing the goaf side, and can move forward and backward with the hydraulic support 5.

[0059] exist Figure 2 Only a single shearer drum is schematically shown. That is, in this embodiment, the shearer may be a single-drum shearer, and the camera device 2 includes at least one camera capable of capturing images of the side of the drum of the single-drum shearer. In an optional embodiment, the shearer may be a double-drum shearer, in which case the camera device 2 may include at least two cameras capable of simultaneously capturing images of the sides of both drums of the double-drum shearer, with the spacing between the cameras being equal to the spacing between the two drums when the left and right rocker arms 7 of the double-drum shearer are simultaneously in a horizontal position. The real-time video data output by the camera device 2 is connected to the drum machine vision intelligent recognition system 4.

[0060] According to the illustrated embodiment of the present invention, in the shearer drum position tracking and measurement mode, the drum machine vision intelligent recognition system 4 may include an intelligent search module (not shown) and a positioning measurement module (not shown). The intelligent search module is configured to search for the shearer drum 3 in the side view image. Once the intelligent search module locates the shearer drum 3, the positioning measurement module uses machine vision to measure and output the relative deviations of the shearer drum 3 from the camera device 2 along the X and Y axes from the preset observation position.

[0061] In the machine vision intelligent recognition system for the drum of a coal mining machine, in order to improve the efficiency of the drum search and the accuracy of the initial positioning, this embodiment can introduce a predictive guided search mechanism. This mechanism is based on the visual prior knowledge and historical drum position distribution data already available at the coal mining operation site, and probabilistically models the possible locations of the drum to form a heat map or position confidence interval. When the system starts searching, the camera no longer blindly rotates across the entire range, but instead prioritizes scanning these high-probability areas using target detection algorithms such as YOLO, thereby significantly shortening the target search time. At the same time, the system has a built-in adaptive adjustment strategy that dynamically corrects the search path based on the detection results in several consecutive frames of images. For example, if the drum is not detected in the continuous image, the search angle area is automatically expanded or offset based on the current image feature feedback to achieve rapid locking and tracking of the drum position.

[0062] Intelligent search is an advanced information retrieval method enabled by artificial intelligence and big data technologies. It understands user intent and returns precise results, and has become a crucial tool for improving efficiency in many fields. It differs from traditional search in that it deeply integrates capabilities such as semantic analysis and multimodal processing, providing a proactive and personalized service experience. This article will not be further elaborated here.

[0063] Furthermore, as mentioned above, the automatic machine vision inspection system for the coal shearer drum is communicatively connected to the controller 8 of the coal shearer, including a coal shearer travel control system and a coal shearer height adjustment control system, which is used to send travel instructions to the coal shearer travel control system according to the X-axis deviation of the relative deviation value, so that the coal shearer travels to the preset X-axis position, and at the same time send height adjustment instructions to the coal shearer height adjustment control system according to the Y-axis deviation of the relative deviation value, so as to raise or lower the coal shearer drum 3 to the preset Y-axis position.

[0064] In an optional embodiment, the travel speed can be 1 meter per minute, and the maximum speed of ascent or descent can be 0.5 meters per second. The adjustment hysteresis between the position to which the shearer travels and the X-axis position can be 5%, and the adjustment hysteresis between the position to which the shearer drum 3 rises or descends and the Y-axis position can be 5%. The adjustment hysteresis is an error, and in this example, a 5% error range is acceptable.

[0065] In a further embodiment, the automatic machine vision inspection system for the shearer drum may include an embedded edge computing device. A drum contour detection model is deployed within the embedded edge computing device. The drum contour detection model is trained using a dataset of side images under various lighting and occlusion conditions. The drum contour detection model receives a real-time video stream from camera 2 and outputs a bounding box and confidence information for the shearer drum's contour. The coordinates of the center point of the bounding box are normalized to determine the position of the shearer drum 3 in the image coordinate system. While camera 2 continuously captures drum images, the automatic machine vision inspection system for the shearer drum utilizes a tracking algorithm to track the contour of the rotating shearer drum 3, calculates its motion trajectory, and predicts its next frame position. If the drum target is lost for a predetermined number of consecutive frames, the drum position tracking mode is triggered to restart, reinitializing the automatic machine vision inspection process for the shearer drum 3.

[0066] In a further embodiment, in addition to the shearer drum position tracking and measurement mode, the drum machine vision intelligent recognition system 4 may also include a drum pick detection mode. When the shearer drum 3 is adjusted to a preset observation position, the shearer drum automatic machine vision inspection system issues a command to the shearer to stop drum rotation. Simultaneously, the drum machine vision intelligent recognition system 4 switches to the drum pick detection mode. Furthermore, in the drum pick detection mode, the drive motor of the shearer drum 3 is powered off, causing the shearer drum 3 to continuously decelerate due to inertia. During this deceleration, the drum machine vision intelligent recognition system 4 performs a 360-degree visual inspection of the shearer drum 3, completing the automatic detection output.

[0067] When the deceleration speed drops to a predetermined speed, the video sequence of the inertial rotation phase is used to detect missing and worn drum pick 6 teeth using a trained model, such as the YOLO model. The detection results are verified in combination with time series information, with a confidence threshold that is dynamically adjusted between 0.5 and 0.7. The confidence threshold is a critical value used in machine learning to filter prediction results, typically used to determine the reliability of the model output. For example, it can be set so that only predictions with a confidence level above the threshold are retained.

[0068] The automatic machine vision inspection system for the coal mining machine drum can also have an alarm unit. When it detects that the drum pick 6 of the coal mining machine drum 3 is missing or the degree of wear exceeds a preset threshold, the automatic machine vision inspection system for the coal mining machine drum automatically generates a detection result, and the alarm unit sends an alarm message to the remote monitoring terminal or maintenance personnel, prompting the replacement or maintenance of the drum pick.

[0069] In one aspect of the present invention, a coal shearer system may be provided. The coal shearer system may include the coal shearer drum automatic machine vision inspection system 1 according to any one of the aforementioned embodiments.

[0070] Figure 3 The figure is a schematic flow chart of an embodiment of the method for automatic machine vision inspection of a coal mining machine drum according to the present invention.

[0071] According to the present invention, the automatic machine vision inspection method for the coal mining machine drum using the automatic machine vision inspection system 1 for the coal mining machine drum as in any one of the aforementioned embodiments may include the following steps: Step 1: Start the automatic machine vision inspection system for the coal mining machine drum; Step 2: The drum machine vision intelligent recognition system 4 enters the coal mining machine drum position tracking mode, determines the position deviation of the coal mining machine drum 3 relative to the camera device 2 based on the side view, and adjusts the coal mining machine drum 3 to a preset observation position based on the position deviation; Step 3: The drum machine vision intelligent recognition system 4 enters the drum cutter detection mode, and performs drum cutter missing and wear detection.

[0072] Specifically, in step 2: first, the drum contour detection model of the automatic machine vision inspection system of the coal shearer drum detects the contour of the coal shearer drum 3. If the coal shearer drum cannot be identified in the video, a position adjustment instruction is sent to the coal shearer to guide the coal shearer drum to move within the monitoring range of the camera device 2; secondly, after the automatic machine vision inspection system of the coal shearer drum detects the contour of the coal shearer drum 3, the coordinate position of the coal shearer drum 3 is recorded, and the tracking algorithm is started to track and update the X / Y axis deviation of the drum in real time; then, the automatic machine vision inspection system of the coal shearer drum determines whether it is in the preset observation position and the optimal detection field of view of the drum pick based on the X / Y axis deviation of the coal shearer drum 3. If the coal shearer drum 3 is not in the preset observation position, the automatic machine vision inspection system of the coal shearer drum automatically sends the coal shearer travel adjustment instruction and the coal shearer height control instruction to adjust the travel position and drum height of the coal shearer so that the coal shearer drum 3 is in the optimal detection field of view of the drum pick. In one embodiment of the present invention, the tracking algorithm may be the DeepSORT tracking algorithm; in other embodiments, different tracking algorithms may be selected according to specific circumstances.

[0073] Specifically, in step 3: first, a drum power-off command is sent to the coal mining machine in the drum pick detection mode; second, after the drum is powered off, the drum relies on inertia to decelerate its rotation, and the coal mining machine drum automatic machine vision inspection system monitors the rotation speed of the drum in real time. When the rotation speed drops to the set threshold, the drum pick detection model is automatically started to analyze the wear and missing of the drum picks on the drum; then, after the detection is completed, the coal mining machine drum automatic machine vision inspection system automatically generates the detection results, and synchronizes the detection results to the remote monitoring terminal to remind maintenance personnel to replace or maintain the drum picks.

[0074] According to various embodiments of the present invention, automatic machine vision inspection can be performed based on the hardness and cutting volume of the coal seam and gangue on the working face. Compared with manual inspection by rotating the drum, the condition of the shearer drum can be detected in a timely, accurate and efficient manner, so that the problem of drum cutting teeth falling off or severe wear can be solved through maintenance or replacement, which is conducive to maintaining high cutting efficiency, avoiding excessive vibration of the coal shearer during operation, and improving cutting reliability.

[0075] An embodiment of the present invention provides an automated machine vision inspection system and implementation method for the shearer drum, which supports the effective implementation of machine vision automatic inspection of the shearer drum installed with the drum pick. In some cases, due to the limitations of the working face conditions and the shearer structure, the installation and wiring of the explosion-proof camera that can effectively observe the working face of the shearer drum can only be done below the top beam of the hydraulic support on the working face. Considering that the rocker arm structure supported by the drum will always block about a quarter of the drum relative to the goaf side of the shearer, and moves back and forth with the hydraulic support, when the shearer is shut down for maintenance, the stop position is generally controlled manually by workers, and the position repeatability is poor. The rocker arm swing angle is not fixed when stopped, and the relative position change after the hydraulic support is pulled causes a large difference in the position of the drum relative to the camera. Although The camera can be adjusted via the pan / tilt system, but it is still difficult to ensure that the drum and drum pick can be observed in the captured image as a complete side projection of the drum cutting cylinder, and it is often not located near the center of the optimal recognition field of view. Moreover, due to the limitations of the workers' equipment and control mode, the operator next to the coal mining machine cannot see the specific status of the drum in the visual recognition camera image, and the operator at the remote control console cannot be immersed in the scene. Manually adjusting the position of the heavy equipment such as the coal mining machine and the height of the drum to the optimal observation position is time-consuming and laborious. These problems seriously restrict the practicality of using machine vision to monitor the shearer pick shedding and wear. The automated coal mining machine drum automatic machine vision inspection system and implementation method of the embodiments of the present invention advantageously solve these problems.

[0076] In combination with the above description, the drum machine vision intelligent identification system of the embodiment of the present invention is configured with two basic working modes.

[0077] One is the coal mining machine drum position tracking and measurement mode. Through the images provided by the camera installed under the top beam of the bracket, the position of the running coal mining machine and its drum is continuously searched and measured by machine vision intelligence. When the coal mining machine enters the vicinity of the observation area, machine vision measurement is performed to output the relative deviation values ​​of the coal mining machine drum from the X-axis (travel position) and Y-axis (drum height) of the best position for effective observation of the drum pick by this camera.

[0078] The other is a drum pick detection mode, which performs machine vision intelligent recognition and detection of drum pick status, such as missing or worn. For example, drum pick detection in this mode can refer to the coal mining machine drum pick detection method disclosed in Chinese patent application CN117876787A, published on April 12, 2024.

[0079] When the user selects the automatic machine vision inspection function, the machine vision intelligent recognition system first operates the shearer drum position tracking measurement mode. Based on the shearer image captured by the camera, it outputs the relative X and Y positions of the shearer drums relative to the current camera. It then sends commands to the shearer travel control system, instructing the shearer to slowly move at a speed of approximately 1 meter / minute to the optimal X-axis position for visual inspection. Simultaneously, based on the Y-axis deviation, it sends commands to the shearer height adjustment control system to raise or lower the shearer drum to near the optimal height for machine vision inspection. To prevent control oscillation, the control system sets a hysteresis of approximately 5% for the shearer travel X-axis position control and Y-axis drum height control. Once the optimal observation position is stabilized, the visual inspection control system issues a command to stop drum rotation. Simultaneously, the machine vision software switches to the drum cutter tooth loss and wear detection mode. After the cutter drum drive motor is powered off, the drum continues to decelerate due to inertia. During this slow rotation and deceleration process, the machine vision inspection software performs a 360-degree visual inspection of the entire drum, automatically outputting the inspection results without manual intervention.

[0080] To implement the drum position tracking and measurement mode of the drum machine vision intelligent recognition system, a coal mining machine drum image dataset containing drum images under various lighting and occlusion conditions was collected. A drum contour detection model was then trained and deployed on an embedded edge computing device (such as the NVIDIA Jetson AGX Xavier). The model inputs a real-time camera video stream, outputs a bounding box and confidence information for the drum contour, and normalizes the coordinates of the bounding box center point to obtain the drum position (x_img, y_img) in the image coordinate system. The camera continuously captures drum images, and a tracking algorithm is used to stably track the rotating drum contour, calculate its motion trajectory, and predict its position in the next frame. If the drum target is lost for five consecutive frames, the position tracking mode is restarted, reinitializing the detection process.

[0081] To implement the drum pick cutter missing and wear detection mode in the drum machine vision intelligent recognition system, the drum position (x_img, y_img) in the image coordinate system is mapped to the physical coordinate system (X / Y axes) through perspective transformation. The optimal detection position range (X-axis and Y-axis thresholds) for the drum is set. If the drum exceeds the set range, the position adjustment mechanism is triggered, calculating the deviation (∆X, ∆Y) of the drum relative to the camera's optimal observation position. ∆X (travel position deviation) and ∆Y (drum height deviation) are output. Travel control commands are generated based on the ∆X value. Upon receiving the travel control command, the shearer moves along the guide rail at a speed of 1 m / min until ∆X falls within the ±5% hysteresis range. The height adjustment system is controlled based on the ∆Y value, driving the drum to the target height. The maximum lift speed is set to 0.5 m / s to prevent mechanical shock until ∆Y falls within the ±5% hysteresis range. When both ∆X and ∆Y meet the threshold conditions, the system sends a power-off command to the shearer, causing the drum to decelerate due to inertia. When the drum's rotation speed slows to a certain level, the trained model detects missing and worn drum picks in the image sequence during the inertial rotation phase. The detection results are then verified using time series information to reduce false positives. The confidence threshold (0.5-0.7) can be dynamically adjusted to balance missed detections with false positives. If a missing drum pick is detected or the wear exceeds a preset threshold, the system automatically generates a detection result and sends an alarm to the remote monitoring terminal or maintenance personnel, prompting them to replace or maintain the drum pick.

[0082] Specifically, in conjunction with the flowchart of the automatic machine vision inspection system for the shearer drum of the present invention, upon system startup, the shearer drum automatic inspection program immediately enters position tracking mode. First, the drum contour detection model detects the drum's contour. If the drum cannot be identified in the video, the system sends a position adjustment command to the shearer operator, directing the drum to move within the camera's monitoring range. Once the system successfully detects the drum's contour, it immediately records its coordinate position and initiates a tracking algorithm to track and update the drum's X / Y axis deviation in real time. Next, based on the drum's X / Y axis deviation, the system determines whether it is within the optimal detection field of view for the drum pick. If the drum is not within the optimal detection range, the system automatically sends travel adjustment commands and height control commands to precisely adjust the shearer's travel position and drum height, ensuring the drum is within the ideal visual inspection area. Once the drum is adjusted to the optimal detection position, the system switches to drum pick detection mode and sends a drum power-off command to the shearer. After the shearer is powered off, the drum slowly decelerates due to inertia, and the system monitors the drum's rotational speed in real time. When the rotational speed drops to a set threshold, the drum pick detection model automatically activates, accurately analyzing the wear and tear of the drum picks on the drum. During the detection process, the system optimizes the recognition results by combining multi-frame timing information to reduce false positives and missed detections. Upon completion, the system automatically generates test results and synchronizes them to the remote monitoring terminal, alerting maintenance personnel to perform necessary drum pick replacement or maintenance, thereby ensuring the proper operation of the shearer. This completes the single drum inspection process, and the system enters standby mode, ready for the next round of inspections.

[0083] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical concept of the present invention, and these deformations and modifications should all fall within the scope of the present invention.

Claims

1. An automatic machine vision inspection system for a coal mining machine drum (1), characterized in that: The automatic machine vision inspection system (1) for the shearer drum comprises: A camera device (2), the camera device (2) being used to capture a side view of the shearer drum (3); A drum machine vision intelligent recognition system (4), wherein the camera device (2) is communicatively connected to the drum machine vision intelligent recognition system (4) and outputs a real-time side view image to the drum machine vision intelligent recognition system (4), and the drum machine vision intelligent recognition system (4) is used to perform automatic machine vision inspection of the coal mining machine drum (3) based on the side view image. The drum machine vision intelligent recognition system (4) includes a coal mining drum position tracking and measurement mode. In the coal mining drum position tracking and measurement mode, the drum machine vision intelligent recognition system (4) determines the position deviation of the coal mining drum (3) relative to the camera device (2) based on the side view, and adjusts the coal mining drum (3) to a preset observation position based on the position deviation. Furthermore, in the coal machine drum position tracking and measurement mode, the drum machine vision intelligent recognition system (4) includes: an intelligent search module, the intelligent search module being used to search for a shearer drum (3) in the side view; and A positioning measurement module is used to measure and output relative values ​​of the deviation of the coal mining machine drum (3) from the camera device (2) and the preset observation position on the X axis and Y axis through machine vision after the intelligent search module searches for the coal mining machine drum (3).

2. The automatic machine vision inspection system (1) for the shearer drum according to claim 1, characterized in that: The camera device (2) is installed below the top beam of the hydraulic support (5) on the coal mining face and faces the goaf side, and is capable of moving forward and backward along with the hydraulic support (5).

3. The automatic machine vision inspection system (1) for the shearer drum according to claim 2, characterized in that: The coal mining machine is a double-drum coal mining machine, and the camera device (2) includes at least two cameras capable of simultaneously photographing the sides of the two drums of the double-drum coal mining machine, and the distance between the cameras is equal to the distance between the two drums when the left and right rocker arms (7) of the double-drum coal mining machine are simultaneously in the horizontal position; or The coal shearer is a single-drum coal shearer, and the camera device (2) includes at least one camera capable of photographing the side of the drum of the single-drum coal shearer.

4. The automatic machine vision inspection system (1) for a coal mining machine drum according to claim 1, characterized in that: The automatic machine vision inspection system for the shearer drum is communicatively connected to the shearer travel control system and the shearer height adjustment control system of the shearer, and is used to send a travel instruction to the shearer travel control system according to the X-axis deviation of the relative deviation value, so that the shearer travels to a preset X-axis position, and at the same time send a height adjustment instruction to the shearer height adjustment control system according to the Y-axis deviation of the relative deviation value, so that the shearer drum (3) is raised or lowered to a preset Y-axis position.

5. The automatic machine vision inspection system (1) for the shearer drum according to claim 4, characterized in that: The walking speed is 1 m / min, the maximum speed of the rising or falling is 0.5 m / s, and the adjustment return difference between the position to which the coal mining machine walks and the X-axis position is 5%, and the adjustment return difference between the position to which the coal mining machine drum (3) rises or falls and the Y-axis position is 5%.

6. The automatic machine vision inspection system (1) for the shearer drum according to claim 1, characterized in that: The automatic machine vision inspection system for the shearer drum includes an embedded edge computing device, in which a drum contour detection model is deployed. The drum contour detection model is trained using a data set including side images under different lighting conditions and occlusion conditions. The drum contour detection model receives a real-time video stream from the camera device (2) and outputs a bounding box and confidence information of the contour of the shearer drum, normalizes the coordinates of the center point of the bounding box, and obtains the position of the shearer drum (3) in the image coordinate system; and When the camera device (2) continuously captures the drum image, the automatic machine vision inspection system for the shearer drum uses a tracking algorithm to track the contour of the rotating shearer drum (3), calculates the motion trajectory of the shearer drum (3), and predicts the next frame position of the shearer drum (3). If the drum target is lost for a predetermined number of consecutive frames, the shearer drum position tracking mode is triggered to restart, and the automatic machine vision inspection process of the shearer drum (3) is reinitialized.

7. The automatic machine vision inspection system (1) for the shearer drum according to claim 1, characterized in that: The drum machine vision intelligent recognition system (4) includes a drum pick detection mode. When the coal mining machine drum (3) is adjusted to the preset observation position, the coal mining machine drum automatic machine vision inspection system sends a command to the coal mining machine to stop the drum rotation. At the same time, the drum machine vision intelligent recognition system (4) switches to the drum pick detection mode. In addition, in the drum pick detection mode, the driving motor of the coal mining machine drum (3) is powered off so that the coal mining machine drum (3) continuously decelerates and rotates according to inertia. During the deceleration rotation process, the drum machine vision intelligent recognition system (4) performs a 360-degree visual inspection coverage on the coal mining machine drum (3) to complete the automatic detection output.

8. The automatic machine vision inspection system (1) for the shearer drum according to claim 7, characterized in that: When the speed of the decelerated rotation is reduced to a predetermined speed, the missing and worn drum pick (6) are detected based on the trained YOLO model for the video sequence of the inertial rotation stage, and the detection result is verified in combination with the time series information, and the confidence threshold is a confidence threshold dynamically adjusted between 0.5 and 0.

7.

9. The automatic machine vision inspection system (1) for a coal mining machine drum according to claim 1, characterized in that: The automatic machine vision inspection system for the shearer drum has an alarm unit. When it is detected that the drum pick (6) of the shearer drum (3) is missing or the degree of wear exceeds a preset threshold, the automatic machine vision inspection system for the shearer drum automatically generates a detection result, and the alarm unit sends an alarm message to a remote monitoring terminal or maintenance personnel, prompting them to replace or maintain the drum pick.

10. The automatic machine vision inspection system (1) for a coal mining machine drum according to claim 1, characterized in that: The visual image is a video or an image.

11. A coal mining machine system, characterized in that: The coal shearer system has the coal shearer drum automatic machine vision inspection system (1) according to any one of claims 1 to 10.

12. An automatic machine vision inspection method for a shearer drum using the automatic machine vision inspection system (1) for a shearer drum according to any one of claims 1 to 10, characterized in that: The method comprises: Step 1: starting the automatic machine vision inspection system for the shearer drum; Step 2: The drum machine vision intelligent recognition system (4) enters the coal mining drum position tracking mode, determines the position deviation of the coal mining drum (3) relative to the camera device (2) based on the side view, and adjusts the coal mining drum (3) to a preset observation position based on the position deviation; Step 3: The drum machine vision intelligent recognition system (4) enters the drum pick detection mode to perform drum pick missing and wear detection.

13. The method according to claim 12, wherein: In step 2: First, the drum contour detection model of the automatic machine vision inspection system for the shearer drum detects the contour of the shearer drum (3), and if the shearer drum cannot be identified in the video, a position adjustment instruction is sent to the shearer to guide the shearer drum to move into the monitoring range of the camera device (2); Secondly, after the shearer drum automatic machine vision inspection system detects the contour of the shearer drum (3), the coordinate position of the shearer drum (3) is recorded, and a tracking algorithm is started to track and update the X / Y axis deviation of the drum in real time; Then, the shearer drum automatic machine vision inspection system determines whether the shearer drum (3) is in a preset observation position and an optimal detection field of view of the drum pick according to the X / Y axis deviation of the shearer drum (3). If the shearer drum (3) is not in the preset observation position, the shearer drum automatic machine vision inspection system automatically sends a shearer travel adjustment instruction and a shearer height control instruction to adjust the shearer travel position and the drum height so that the shearer drum (3) is in the optimal detection field of view of the drum pick.

14. The method according to claim 12, wherein: In step 3: First, in the drum pick detection mode, a drum power-off instruction is sent to the coal mining machine; Secondly, after the drum loses power, it rotates slowly due to inertia. The shearer drum automatic machine vision inspection system monitors the drum's rotation speed in real time. When the rotation speed drops to a set threshold, the drum pick detection model automatically activates to analyze the wear and tear of the drum picks on the drum. Then, after the inspection is completed, the coal mining machine drum automatic machine vision inspection system automatically generates the inspection results and synchronizes the inspection results to the remote monitoring terminal to remind the maintenance personnel to replace or maintain the drum teeth.

Images