Bucket tooth state detection method and device and remote mining excavator monitoring method and system
By applying the target detection model to be automatically detected on the remote mining excavator, the problem of untimely detection of the bucket teeth status in the prior art is solved, and the timely automatic detection and alarm of the bucket teeth status is realized, and safety and equipment reliability are improved.
Patent Information
- Application Number
- CN202510165819.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-12
- Publication Date
- 2025-06-06
AI Technical Summary
In the prior art, the status detection of the bucket teeth of the remote mining excavators relies on manual labor, resulting in untimely detection and the inability to take corresponding measures in time to prevent mechanical accidents caused by the bucket teeth breaking or falling off.
The target detection model is used to detect the images to be detected by the remote mining excavator, and the state of the bucket teeth is automatically determined, including the normal state, the shed state, the wear state, the occlusion state and the intrusion state. The model is obtained through training multiple training samples and labeled data, and the impact of fog is taken into account during the detection process, and the image clarity is improved through dark channel defog technology.
The timely detection of the bucket teeth status of the remote mining excavator is achieved. The bucket teeth status can be automatically determined without shutting down, and alarms and measures are promptly reported to avoid mechanical accidents caused by the bucket teeth breaking or falling off.
Smart Images
Figure CN120107546A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of detection technology, and specifically relates to a bucket tooth state detection method and device, and a remote mining excavator monitoring method and system. Background Art
[0002] Remote mining excavators are intelligent heavy equipment designed for mining operations. They are unmanned through remote control technology. The main working parts are bucket teeth. Bucket teeth are generally made of high-manganese steel or alloy steel with high hardness. Compared with materials such as ore, they are harder. When remote mining excavators work in a complex stress environment for a long time and at high intensity, they are prone to breakage or falling off. If not discovered in time, the broken and fallen bucket teeth will be loaded into the mining truck along with the ore and transported to the crushing station. The flow of lump coal mixed with bucket teeth entering the crushing production line will cause serious mechanical accidents and endanger the personal safety of workers. Therefore, it is extremely important to detect the status of bucket teeth of remote mining excavators.
[0003] In the prior art, the state of the bucket teeth of a remote mining excavator is mainly detected manually. However, when manually detecting the state of the bucket teeth of a remote mining excavator, the operation needs to be performed when the remote mining excavator is stopped, so that the state of the bucket teeth of the remote mining excavator is not determined in time, and thus corresponding measures cannot be taken in time when the bucket teeth are abnormal. Summary of the invention
[0004] The technical problem to be solved by the present application is to provide a bucket tooth state detection method and device, a remote mining excavator monitoring method and system in view of the above-mentioned deficiencies in the prior art. By using the bucket tooth state detection method, the bucket tooth state of the remote mining excavator can be detected in time, thereby providing the possibility of taking corresponding measures in time when the bucket tooth state is abnormal.
[0005] In a first aspect, an embodiment of the present application provides a bucket tooth state detection method, which is applied to a remote mining excavator, and the method includes:
[0006] Acquire an image to be detected, where the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator;
[0007] The target detection model is used to detect the image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained;
[0008] Among them, the target detection model is trained by multiple training samples and the labeled data corresponding to each training sample in the multiple training samples. The target bucket tooth state includes one of the normal state, the falling state, the wear state, the occlusion state and the intrusion state. The wear state includes the normal wear state and the abnormal wear state. The intrusion state includes the vehicle intrusion state and the pedestrian intrusion state.
[0009] In some implementations of the first aspect, the target detection model includes a fog detection model and a bucket tooth state detection model.
[0010] The target detection model is used to detect the image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained, including:
[0011] The fog detection model is used to detect the image to be detected to obtain the fog detection result;
[0012] A target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined according to the fog detection result, the bucket tooth state detection model and the image to be detected.
[0013] In some embodiments of the first aspect, determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator according to a fog detection result, a bucket tooth state detection model, and an image to be detected includes:
[0014] When the fog detection result is that there is no fog, a bucket tooth state detection model is used to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator;
[0015] or,
[0016] When the fog detection result is foggy, the image to be detected is defogged through a dark channel to obtain a defogged image to be detected;
[0017] The bucket tooth state detection model is used to detect the defogging image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained.
[0018] In some implementations of the first aspect, before using the target detection model to detect the image to be detected and obtaining a target bucket tooth state of at least one bucket tooth of the remote mining excavator, the method further includes:
[0019] Create an object detection model;
[0020] Create an object detection model, including:
[0021] Obtaining a training sample set, the training sample set including multiple training samples and labeled data corresponding to each training sample in the multiple training samples;
[0022] According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
[0023] In some embodiments of the first aspect, the target detection model includes a fog detection model and a bucket tooth state detection model, the training samples include a first type of training samples and a second type of training samples, the labeled data corresponding to the first type of training samples include fog and no fog, and the labeled data corresponding to the second type of training samples include a normal state, a falling state, a wear state, an occlusion state, and an invasion state; the preset model includes a first preset model and a second preset model;
[0024] According to the multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain the target detection model, including:
[0025] According to the first type of training samples and their corresponding labeled data, the first preset model is trained to obtain a fog detection model;
[0026] The second preset model is trained according to the second type of training samples and their corresponding labeled data to obtain a bucket tooth state detection model.
[0027] In some implementations of the first aspect, acquiring an image to be detected includes:
[0028] The image to be detected is acquired by a first acquisition device or a second acquisition device, wherein the first acquisition device is installed on the bucket tooth boom of the remote mining excavator, and the second acquisition device is installed on the rotary platform of the remote mining excavator.
[0029] Based on the same inventive concept, in a second aspect, the embodiment of the present application further provides a remote mining excavator monitoring method, comprising:
[0030] Determining a target tooth state of at least one tooth of a remote mining excavator according to the tooth state detection method of any one of the first aspects;
[0031] When the target bucket tooth is in a dropped state, outputting first alarm information, the first alarm information is used to indicate that the target bucket tooth is in a dropped state;
[0032] or,
[0033] When the target bucket tooth state is an invasion state, outputting second alarm information, the second alarm information is used to indicate that the target bucket tooth state is an invasion state;
[0034] or,
[0035] When the target bucket tooth state is in an abnormal wear state, a third alarm message is output, and the third alarm message is used to indicate that the target bucket tooth state is in an abnormal wear state.
[0036] Based on the same inventive concept, in a third aspect, an embodiment of the present application further provides a bucket tooth state detection device, which is applied to a remote mining excavator, and the device includes:
[0037] A first acquisition module is used to acquire an image to be detected, where the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator;
[0038] A first detection module, connected to the first acquisition module, is used to detect the image to be detected using a target detection model to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator;
[0039] Among them, the target detection model is trained by multiple training samples and the labeled data corresponding to each training sample in the multiple training samples. The target bucket tooth state includes one of the normal state, the falling state, the wear state, the occlusion state and the intrusion state. The wear state includes the normal wear state and the abnormal wear state. The intrusion state includes the vehicle intrusion state and the pedestrian intrusion state.
[0040] In some implementations of the third aspect, the target detection model includes a fog detection model and a bucket tooth state detection model.
[0041] The first detection module is specifically used for:
[0042] The fog detection model is used to detect the image to be detected to obtain the fog detection result;
[0043] A target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined according to the fog detection result, the bucket tooth state detection model and the image to be detected.
[0044] In some embodiments of the third aspect, the apparatus further comprises:
[0045] Create a module for creating a target detection model;
[0046] Create modules specifically for:
[0047] Obtaining a training sample set, the training sample set including multiple training samples and labeled data corresponding to each training sample in the multiple training samples;
[0048] According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
[0049] In a fourth aspect, the embodiment of the present application further provides a remote mining excavator monitoring system, including:
[0050] The bucket tooth state detection device of any one of the third aspects is used to determine a target bucket tooth state of at least one bucket tooth of a remote mining excavator;
[0051] A first output module is connected to the bucket tooth state detection device, and is used to output a first alarm message when the target bucket tooth state detected by the bucket tooth state detection device is in a fallen-off state, wherein the first alarm message is used to indicate that the target bucket tooth state is in a fallen-off state;
[0052] or,
[0053] A second output module is connected to the bucket tooth state detection device and is used to output second alarm information when the target bucket tooth state detected by the bucket tooth state detection device is an intrusion state, wherein the second alarm information is used to indicate that the target bucket tooth state is an intrusion state;
[0054] or,
[0055] The third output module is connected to the bucket tooth state detection device, and is used to output a third alarm message when the target bucket tooth state detected by the bucket tooth state detection device is in an abnormal wear state, wherein the third alarm message is used to indicate that the target bucket tooth state is in an abnormal wear state.
[0056] According to the bucket tooth state detection method and device, and the remote mining excavator monitoring method and system provided in the embodiments of the present application, the target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained by adopting a target detection model to detect the image to be detected. The target bucket tooth state of at least one bucket tooth of the remote mining excavator can be automatically determined without stopping the remote mining excavator, and the bucket tooth state of the remote mining excavator can be detected in time, thereby providing the possibility of taking corresponding measures in time when the bucket tooth state is abnormal. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 A schematic diagram showing a flow chart of a bucket tooth state detection method provided in an embodiment of the present application;
[0058] Figure 2 Another schematic flow chart of a bucket tooth state detection method provided in an embodiment of the present application is shown;
[0059] Figure 3 A schematic diagram showing a structure of a bucket tooth state detection device provided in an embodiment of the present application is shown;
[0060] Figure 4 A schematic diagram of the hardware structure of the bucket tooth state detection system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0061] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is further described in detail below with reference to the accompanying drawings and embodiments.
[0062] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0063] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0064] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0065] Example 1
[0066] The bucket tooth state detection method provided in the embodiment of the present application can be applied to a remote mining excavator, and the bucket tooth state detection method can be executed by a bucket tooth state detection device and an electronic device including a bucket tooth state detection system, etc. The following takes the bucket tooth state detection method executed by an electronic device as an example for description.
[0067] The bucket tooth state detection method provided in the embodiment of the present application can be used in the remote mining excavator monitoring process.
[0068] like Figure 1 As shown, the bucket tooth state detection method provided in the embodiment of the present application may include steps S110 to S120.
[0069] S110, acquiring an image to be detected, where the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator.
[0070] S120: Use the target detection model to detect the image to be detected, and obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator.
[0071] Among them, the target detection model is trained by multiple training samples and the labeled data corresponding to each training sample in the multiple training samples. The target bucket tooth state includes one of the normal state, the falling state, the wear state, the occlusion state and the intrusion state. The wear state includes the normal wear state and the abnormal wear state. The intrusion state includes the vehicle intrusion state and the pedestrian intrusion state.
[0072] According to the bucket tooth state detection method provided in the embodiment of the present application, the target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained by adopting a target detection model to detect the image to be detected. The target bucket tooth state of at least one bucket tooth of the remote mining excavator can be automatically determined without stopping the remote mining excavator, and the bucket tooth state of the remote mining excavator can be detected in time, thereby providing the possibility to take corresponding measures in time when the bucket tooth state is abnormal.
[0073] The specific implementation methods of the above steps are introduced below.
[0074] In step S110, in some implementations, obtaining an image to be detected includes:
[0075] The image to be detected is acquired by a first acquisition device or a second acquisition device, wherein the first acquisition device is installed on the bucket tooth boom of the remote mining excavator, and the second acquisition device is installed on the rotary platform of the remote mining excavator.
[0076] Exemplarily, both the first acquisition device and the second acquisition device can be cameras. That is, a camera is installed on the bucket arm of the remote mining excavator and on the rotary platform of the remote mining excavator to obtain the image to be detected.
[0077] Exemplarily, the image to be detected may be an image of the position of at least one bucket tooth of a remote mining excavator collected in real time by the first acquisition device or the second acquisition device, so as to detect the bucket tooth status of the remote mining excavator more timely, thereby providing the possibility of taking corresponding measures in time when the bucket tooth status is abnormal.
[0078] Exemplarily, each frame of an image in a video captured in real time by the first capturing device or the second capturing device may be used as an image to be detected in turn.
[0079] For example, the current time period can be determined according to the acquisition time. When the current time period is daytime, the image to be detected can be directly acquired in real time by the first acquisition device or the second acquisition device. When the current time period is night, the first acquisition device or the second acquisition device can be set to a mixed fill light mode, so that the first acquisition device or the second acquisition device acquires the image to be detected in real time in the mixed fill light mode, so as to improve the quality and imaging stability of the image to be detected. Among them, the mixed fill light mode can be used to achieve the best exposure effect and color restoration in a weak light or uneven light environment.
[0080] In step S120, after acquiring the image to be detected, the electronic device may also use the target detection model to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator.
[0081] For example, the normal wear state may be a state in which the device is worn but can be used normally; and the abnormal wear state may be a state in which the device is worn and cannot be used normally.
[0082] For example, the target detection model may be a model capable of detecting an image to be detected to determine a target bucket tooth state of at least one bucket tooth of a remote mining excavator.
[0083] In some embodiments, the target detection model includes a fog detection model and a bucket tooth state detection model.
[0084] The target detection model is used to detect the image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained, including:
[0085] The fog detection model is used to detect the image to be detected to obtain the fog detection result;
[0086] A target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined according to the fog detection result, the bucket tooth state detection model and the image to be detected.
[0087] In this embodiment, taking into account that the bucket teeth of the remote mining excavator may be obscured by fog during actual operation, resulting in low clarity of the collected image to be detected, thereby affecting the accuracy of determining the target tooth state of the bucket teeth, by determining the target tooth state of at least one bucket tooth of the remote mining excavator based on the fog detection result, the bucket tooth state detection model and the image to be detected, the clarity of the collected image to be detected can be improved, thereby improving the accuracy of determining the target tooth state of the bucket teeth.
[0088] Exemplarily, the fog detection model may be a model that can detect whether there is fog in the image to be detected. For example, the fog detection model can determine the fog detection result by comparing the transmittance of the image to be detected with the transmittance threshold; specifically, when the transmittance of the image to be detected is greater than or equal to the transmittance threshold, the fog detection result is determined to be foggy; when the transmittance is less than the transmittance threshold, the fog detection result is determined to be fog-free. Among them, the transmittance threshold can be set according to actual conditions and is not limited here. For example, the transmittance threshold can be 0.6, 0.65, 0.7, 0.8, etc.
[0089] Exemplarily, the bucket tooth state detection model may be a model capable of detecting a target bucket tooth state of at least one bucket tooth of a remote mining excavator.
[0090] Exemplarily, the fog detection result may include the presence of fog and the absence of fog.
[0091] Exemplarily, the transmittance of the image to be detected can be obtained through the dark channel prior method. When the transmittance is greater than or equal to the transmittance threshold, the fog detection result is determined to be foggy; when the transmittance is less than the transmittance threshold, the fog detection result is determined to be fog-free.
[0092] In some examples, determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator according to a fog detection result, a bucket tooth state detection model, and an image to be detected includes:
[0093] When the fog detection result is that there is no fog, a bucket tooth state detection model is used to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator;
[0094] or,
[0095] When the fog detection result is foggy, the image to be detected is defogged through a dark channel to obtain a defogged image to be detected;
[0096] The bucket tooth state detection model is used to detect the defogging image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained.
[0097] In this embodiment, when the fog detection result is foggy, the image to be detected is defogged through a dark channel, and the defogged image to be detected is able to improve the clarity of the acquired image to be detected; then, a bucket tooth state detection model is used to detect the defogged image to be detected, and a target bucket tooth state of at least one bucket tooth of a remote mining excavator is obtained, which can improve the accuracy of determining the target bucket tooth state of the bucket tooth.
[0098] It should be noted that the implementation process of dark channel defogging is an existing technology and will not be described in detail here.
[0099] Exemplarily, the bucket tooth state detection model can determine whether the target bucket tooth state is a normal wear state or an abnormal wear state according to a first ratio and a preset ratio threshold, wherein the first ratio is the ratio of the pixel area of a bucket tooth in the image to be detected to the standard area of the bucket tooth. Specifically, when the first ratio is less than or equal to the preset ratio threshold, the target bucket tooth state is determined to be an abnormal wear state; when the first ratio is greater than the preset ratio threshold, the target bucket tooth state is determined to be a normal wear state. The preset ratio threshold can be set according to actual conditions and is not limited here. For example, the preset ratio threshold can be 0.89, 0.90, etc.
[0100] In some embodiments, before using the target detection model to detect the image to be detected and obtaining the target bucket tooth state of at least one bucket tooth of the remote mining excavator, the method further includes:
[0101] Create an object detection model;
[0102] Create an object detection model, including:
[0103] Obtaining a training sample set, the training sample set including multiple training samples and labeled data corresponding to each training sample in the multiple training samples;
[0104] According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
[0105] In this embodiment, the preset model is trained using multiple training samples and the labeled data corresponding to each of the multiple training samples to obtain a target detection model, thereby providing a basis for subsequently determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator through the target detection model.
[0106] In some examples, the target detection model includes a fog detection model and a bucket tooth state detection model, the training samples include a first type of training samples and a second type of training samples, the labeled data corresponding to the first type of training samples include fog and no fog, and the labeled data corresponding to the second type of training samples include a normal state, a falling state, a wear state, an occlusion state, and an invasion state; the preset model includes a first preset model and a second preset model;
[0107] According to the multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain the target detection model, including:
[0108] According to the first type of training samples and their corresponding labeled data, the first preset model is trained to obtain a fog detection model;
[0109] The second preset model is trained according to the second type of training samples and their corresponding labeled data to obtain a bucket tooth state detection model.
[0110] Exemplarily, the first type of training samples may be training samples of a fog detection model, and the labeled data corresponding to the first type of training samples include fog and no fog. The labeled data corresponding to each first type of training sample is fog or no fog.
[0111] Exemplarily, the second type of training samples may be training samples of a bucket tooth state detection model. The labeling data corresponding to the second type of training samples include normal state, falling off state, wear state, occlusion state, and intrusion state, wherein the wear state includes normal wear state and abnormal wear state, and the intrusion state includes vehicle intrusion state and pedestrian intrusion state. The labeling data corresponding to each second type of training sample is one of the normal state, falling off state, normal wear state, abnormal wear state, occlusion state, vehicle intrusion state, and pedestrian intrusion state.
[0112] Exemplarily, the first preset model and the second preset model may be the same or different, which is not limited here. For example, the first preset model and the second preset model are both YOLOv11 models. Of course, the first preset model and the second preset model may also be other models, which is not limited here.
[0113] Exemplarily, after the preset model is trained according to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples to obtain the target detection model, the target detection model can also be verified through the verification set.
[0114] For example, the hardware and software of the bucket tooth status detection system are built on site, and verification tests are carried out to test the bucket tooth shedding detection performance and bucket tooth wear detection performance. The trained bucket tooth target detection model (i.e., target detection model) is used to detect and identify the bucket tooth working pictures (i.e., images to be detected) collected in real time. When the electric shovel is operating normally, the bucket teeth are in a normal state. The on-site statistical detection results show that the dual-path bucket tooth target detection speed can reach 15 frames / s, and the average detection accuracy is above 5%. Due to the special situation of bucket tooth missing or broken, bucket tooth occlusion test, missing test and wear test are carried out to verify the detection system's ability to handle abnormal working conditions.
[0115] (1) Occlusion experiment
[0116] During the excavation process of a remote mining excavator, the bucket teeth are likely to be blocked by coal blocks, and the number of bucket teeth displayed is small and the exposed area is incomplete. The occlusion experiment includes two methods: artificial occlusion and random occlusion by coal blocks. Artificial occlusion is to cover a certain bucket tooth with an obstruction, making the feature information of the bucket tooth incomplete, simulating the falling of the bucket tooth and interfering with the judgment of the detection system; random occlusion by coal blocks is to randomly select some bucket teeth to expose incomplete images for testing when the bucket teeth are operating.
[0117] (2) Deletion experiment
[0118] In the missing experiment, the shovel repair team master needs to remove a bucket tooth, and the electric shovel driver cooperates to rotate the bucket to perform bucket tooth fall-off detection. Compared with bucket tooth obstruction, the bucket tooth fall-off experiment is closer to the actual fall-off state, and the test results under this state are more representative and convincing.
[0119] (3) Wear test
[0120] The trained bucket tooth segmentation model (i.e., target detection model) is used to segment and identify the bucket teeth of the fixed-point collected bucket tooth working picture (i.e., the image to be detected), and the wear state of the bucket teeth is calculated by the wear judgment algorithm. In the experiment, the upward monitoring camera needs to be calibrated first, the connection between the camera coordinates and the real coordinates is established, and the ratio K value between the image pixel length and the real length is found. In the embodiment of the present application, the camera pixel is 1.2 million, the bucket tooth image resolution is 1280×960, and the focal length parameters are known. Since the fixed-point acquisition mode is used when collecting bucket tooth wear images, the camera and bucket tooth positions are fixed, and the camera is photographed in front of the bucket tooth. Therefore, the real length of the bucket tooth and the pixel length in the image can be measured to obtain the K value. In this experiment, the No. 3 bucket tooth is replaced with a new bucket tooth that is not worn. The actual length of the new bucket tooth is measured to be 914mm. The bucket tooth of the remote mining excavator is lifted to the key point of wear detection. The bucket tooth pixel length is detected to be 279 and the pixel area is 11796 pixels. The K value is 0.305, which is set as the initial reference value of bucket tooth wear.
[0121] For example, Labelmg software can be used to annotate the training samples, and the image coordinates (x min ,y min ), the upper right corner image coordinates (x max ,y max ), and the labeled data corresponding to the training samples.
[0122] Exemplarily, the labeling procedure may be as follows:
[0123] <object>
[0124] <name> complete< / name>
[0125] <pose> Unspecified< / pose>
[0126] <truncated> 0< / truncated>
[0127] <difficult> 0< / difficult>
[0128] <bndbox>
[0129] <xmin> 184< / xmin>
[0130] <ymin> 164< / ymin>
[0131] <xmax> 202< / xmax>
[0132] <ymax> 206< / ymax>
[0133] < / bndbox>
[0134] < / object>
[0135]
[0136] In some examples, the bucket teeth at different positions can be removed to detect the bucket teeth falling off, and then the detected bucket teeth falling off situation is compared with the actual removal situation to verify whether the target detection model can normally identify the bucket teeth falling off state. For example, the bucket includes five bucket teeth from left to right, and the first bucket tooth on the left, the fourth bucket tooth on the left, the first bucket tooth on the left, and a part of the fifth bucket tooth on the left are removed in turn, and the bucket teeth falling off situation is detected, and the drop is obtained as 42.36%, 63.17%, 75.37%, and 84.80% respectively. Among them, drop represents the percentage of the remaining bucket teeth to all bucket teeth.
[0137] Example 2
[0138] The remote mining excavator monitoring method provided in the embodiment of the present application can be applied to a remote mining excavator, and the remote mining excavator monitoring method can be executed by a remote mining excavator monitoring system, etc.
[0139] The remote mining excavator monitoring method provided in the embodiment of the present application may include steps S210 to S220.
[0140] S210. According to the bucket tooth state detection method of Example 1, determine a target bucket tooth state of at least one bucket tooth of a remote mining excavator.
[0141] S220. When the target bucket tooth is in a detached state, output a first alarm message, the first alarm message is used to indicate that the target bucket tooth is in a detached state; or, when the target bucket tooth is in an invaded state, output a second alarm message, the second alarm message is used to indicate that the target bucket tooth is in an invaded state; or, when the target bucket tooth is in an abnormally worn state, output a third alarm message, the third alarm message is used to indicate that the target bucket tooth is in an abnormally worn state.
[0142] In the embodiment of the present application, by outputting an alarm message when the target bucket tooth is in an abnormal state, the staff can be reminded in time, thereby providing the possibility for taking corresponding measures in time when the bucket tooth is in an abnormal state.
[0143] It should be noted that the output modes of the first alarm information, the second alarm information and the third alarm information can be text display and voice broadcast, etc. For example, the electronic device is connected to the industrial touch all-in-one machine through communication, and the alarm information is played through the built-in speaker of the industrial touch all-in-one machine. The specific contents of the first alarm information, the second alarm information and the third alarm information can be set according to the actual situation and are not limited here.
[0144] In order to better understand the remote mining excavator monitoring method provided in the embodiment of the present application, a specific implementation method is described below.
[0145] like Figure 2As shown, the remote mining excavator monitoring method includes: judging whether the time (i.e., the current time) is day (i.e., daytime) or night (night); when the time is night, collecting input pictures (i.e., images to be detected) in the mixed fill light mode; when the time is day, collecting input pictures, and judging whether the transmittance of the input pictures is greater than or equal to Tfog (i.e., the transmittance threshold). When the transmittance is less than Tfog, dark channel defogging is performed, and then image analysis is performed; when the transmittance is greater than or equal to Tfog, image analysis is performed. The detection result is output by the bucket tooth fall-off detection system (i.e., the target detection model), and the bucket tooth fall-off algorithm (i.e., the target detection model) is used to determine whether the bucket tooth has fallen off. If the bucket tooth has not fallen off (i.e., full), the process jumps to the step of "determining whether the time is day or night". If the bucket tooth has fallen off (i.e., miss), a fall-off alarm is issued; or, it is determined whether it is a vehicle or a pedestrian. If it is a vehicle or a pedestrian (i.e., yes), an intrusion alarm is issued. If it is neither a vehicle nor a pedestrian, the process jumps to the step of "determining whether the time is day or night". Or, the bucket tooth wear detection system (i.e., the target detection model) is used to output the wear result, and the size of the first ratio (i.e., Wi) in the wear result and the preset ratio threshold (i.e., Wt) is determined. If the judgment result is that the first ratio is less than or equal to the preset ratio threshold (i.e., yes), a wear alarm is issued. If the judgment result is that the first ratio is greater than the preset ratio threshold (i.e., no), the process jumps to the step of "determining whether the time is day or night".
[0146] Example 3
[0147] The present application embodiment provides a bucket tooth state detection device, which is applied to a remote mining excavator, such as Figure 3 As shown, the bucket tooth state detection device includes:
[0148] A first acquisition module 510 is used to acquire an image to be detected, where the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator;
[0149] A first detection module 520, connected to the first acquisition module 510, is used to detect the image to be detected using a target detection model to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator;
[0150] Among them, the target detection model is trained by multiple training samples and the labeled data corresponding to each training sample in the multiple training samples. The target bucket tooth state includes one of the normal state, the falling state, the wear state, the occlusion state and the intrusion state. The wear state includes the normal wear state and the abnormal wear state. The intrusion state includes the vehicle intrusion state and the pedestrian intrusion state.
[0151] In some embodiments, the target detection model includes a fog detection model and a bucket tooth state detection model.
[0152] The first detection module 520 is specifically used for:
[0153] The fog detection model is used to detect the image to be detected to obtain the fog detection result;
[0154] A target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined according to the fog detection result, the bucket tooth state detection model and the image to be detected.
[0155] In some implementations, the first detection module 520 is specifically configured to:
[0156] When the fog detection result is that there is no fog, a bucket tooth state detection model is used to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator;
[0157] or,
[0158] When the fog detection result is foggy, the image to be detected is defogged through a dark channel to obtain a defogged image to be detected;
[0159] The bucket tooth state detection model is used to detect the defogging image to be detected, and a target bucket tooth state of at least one bucket tooth of the remote mining excavator is obtained.
[0160] In some embodiments, the device further comprises:
[0161] Create a module for creating a target detection model;
[0162] Create modules specifically for:
[0163] Obtaining a training sample set, the training sample set including multiple training samples and labeled data corresponding to each training sample in the multiple training samples;
[0164] According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
[0165] In some embodiments, the target detection model includes a fog detection model and a bucket tooth state detection model, the training samples include a first type of training samples and a second type of training samples, the labeled data corresponding to the first type of training samples include fog and no fog, and the labeled data corresponding to the second type of training samples include normal state, shedding state, wear state, occlusion state and invasion state; the preset model includes a first preset model and a second preset model;
[0166] Create modules specifically for:
[0167] According to the first type of training samples and their corresponding labeled data, the first preset model is trained to obtain a fog detection model;
[0168] The second preset model is trained according to the second type of training samples and their corresponding labeled data to obtain a bucket tooth state detection model.
[0169] In some implementations, the first acquisition module 510 is specifically configured to:
[0170] The image to be detected is acquired by a first acquisition device or a second acquisition device, wherein the first acquisition device is installed on the bucket tooth boom of the remote mining excavator, and the second acquisition device is installed on the rotary platform of the remote mining excavator.
[0171] The bucket tooth state detection device provided in the embodiment of the present application can be used to execute the bucket tooth state detection method, that is, it has the beneficial effects and implementation methods of the bucket tooth state detection method provided in Example 1 of the present application. For details, please refer to the specific description of the bucket tooth state detection method in the above Example 1, which will not be repeated in this embodiment.
[0172] Example 4
[0173] The present application also provides a remote mining excavator monitoring system, including:
[0174] The bucket tooth state detection device of embodiment 3 is used to determine a target bucket tooth state of at least one bucket tooth of a remote mining excavator;
[0175] A first output module is connected to the bucket tooth state detection device, and is used to output a first alarm message when the target bucket tooth state detected by the bucket tooth state detection device is in a fallen-off state, wherein the first alarm message is used to indicate that the target bucket tooth state is in a fallen-off state;
[0176] or,
[0177] A second output module is connected to the bucket tooth state detection device and is used to output second alarm information when the target bucket tooth state detected by the bucket tooth state detection device is an intrusion state, wherein the second alarm information is used to indicate that the target bucket tooth state is an intrusion state;
[0178] or,
[0179] The third output module is connected to the bucket tooth state detection device, and is used to output a third alarm message when the target bucket tooth state detected by the bucket tooth state detection device is in an abnormal wear state, wherein the third alarm message is used to indicate that the target bucket tooth state is in an abnormal wear state.
[0180] In some examples, such as Figure 4As shown, the remote mining excavator monitoring system may include an industrial camera, a gigabit switch, an embedded processor AGX xavier, an industrial touch all-in-one machine and a programmable logic controller (PLC, i.e., a remote mining excavator control center). Among them, the industrial camera and the gigabit switch, the gigabit switch and the embedded processor AGX xavier, and the embedded processor AGX xavier and the industrial touch all-in-one machine are all connected by data cables; the industrial touch all-in-one machine is connected to the programmable logic controller via Ethernet and a data cable. Exemplarily, the programmable logic controller may be an S7-300 programmable logic controller.
[0181] The remote mining excavator monitoring system is installed on the bucket tooth boom and the slewing platform respectively. The bucket tooth image data (i.e., training samples) is captured through dual-channel cameras (one camera is installed on the boom and one camera is installed on the slewing platform), and transmitted to the industrial touch all-in-one machine through Ethernet, and stored in video format for target detection training, that is, the captured video is frame-by-frame extracted, and the key frame images (i.e., images including the position of the bucket teeth) are selected according to different features (i.e., different states of the bucket teeth), and the key frame images (i.e., images including the position of the bucket teeth) are selected and imported into the training program of YOLOv11 for training. The PLC matching encoder and PLC are responsible for locating the key position of the bucket (the camera identifies the bucket teeth with the boom during the lifting and lowering of the bucket teeth, and the detected image set is transmitted to the control component for identification). Once the predetermined point is reached (i.e., the position where the image including the position of the bucket teeth can be collected), the trigger signal will be transmitted from PCL to the PCL matching encoder through the Python-snap7 protocol, and then sent from the PCL matching encoder to the all-in-one machine, and the current bucket tooth image (i.e., training sample) will be saved in image format for instance segmentation training. During the detection phase, bucket tooth image data (i.e. training samples) are sent to the core hardware embedded development kit for shedding and wear detection. As a display device, the industrial touch all-in-one computer is not only used for displaying the human-computer interaction interface, but also responsible for reading and writing PLC data, and has an alarm function that can display intelligent detection results and send alarm signals through the built-in speaker.
[0182] The embodiments of the present application have at least the following beneficial effects:
[0183] It can realize real-time monitoring of the open-pit mining excavator system 24 hours a day, and can monitor the bucket status in real time, including but not limited to: whether the bucket teeth fall off, the degree of bucket tooth wear (whether it meets the work intensity), whether there are pedestrians and vehicles in the excavator working area, etc. According to the real-time monitoring of the operating status, on the one hand, it can effectively improve the safety factor near the remote mining excavator, ensure the safety of the working surface, and prevent broken teeth from mixing into the coal flow and causing safety accidents; on the other hand, the monitoring system (i.e. the remote mining excavator monitoring system) can cooperate with the remote remote control excavator to realize unmanned operation on site and ensure the safety of unmanned operation.
[0184] The remote mining excavator monitoring system provided in the embodiment of the present application can be used to execute the remote mining excavator monitoring method, that is, it has the beneficial effects and implementation methods of the remote mining excavator monitoring method provided in Example 2 of the present application. For details, please refer to the specific description of the remote mining excavator monitoring method in the above Example 2, which will not be repeated in this embodiment.
[0185] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.
Claims
1. A bucket tooth state detection method, characterized in that: Applied to a remote mining excavator, the method comprises: Acquire an image to be detected, wherein the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator; Using a target detection model to detect the image to be detected, and obtaining a target bucket tooth state of at least one bucket tooth of a remote mining excavator; The target detection model is trained by using a plurality of training samples and labeled data corresponding to each of the plurality of training samples. The target bucket tooth state includes one of a normal state, a falling-off state, a wear state, an occlusion state and an intrusion state. The wear state includes a normal wear state and an abnormal wear state. The intrusion state includes a vehicle intrusion state and a pedestrian intrusion state.
2. The method according to claim 1, characterized in that The target detection model includes the fog detection model and the bucket tooth state detection model. The target detection model is used to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator, including: Using the fog detection model to detect the image to be detected to obtain a fog detection result; A target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined according to the fog detection result, the bucket tooth state detection model and the image to be detected.
3. The method according to claim 2, characterized in that Determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator according to the fog detection result, the bucket tooth state detection model and the image to be detected includes: When the fog detection result is that there is no fog, the bucket tooth state detection model is used to detect the image to be detected to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator; or, When the fog detection result indicates that there is fog, performing dark channel defogging on the image to be detected to obtain a defogged image to be detected; The bucket tooth state detection model is used to detect the defogged image to be detected, so as to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator.
4. The method according to claim 1, characterized in that: Before detecting the image to be detected by using the target detection model to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator, the method further includes: Create an object detection model; The creating of the target detection model comprises: Acquire a training sample set, wherein the training sample set includes a plurality of training samples and labeled data corresponding to each of the plurality of training samples; According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
5. The method according to claim 4, characterized in that The target detection model includes a fog detection model and a bucket tooth state detection model, the training samples include a first type of training samples and a second type of training samples, the labeled data corresponding to the first type of training samples include fog and no fog, and the labeled data corresponding to the second type of training samples include normal state, falling state, wear state, occlusion state and invasion state; the preset model includes a first preset model and a second preset model; According to the multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain the target detection model, including: Training the first preset model according to the first type of training samples and their corresponding labeled data to obtain the fog detection model; The second preset model is trained according to the second type of training samples and their corresponding labeled data to obtain the bucket tooth state detection model.
6. The method according to claim 1, characterized in that Get the image to be detected, including: The image to be detected is acquired by a first acquisition device or a second acquisition device, wherein the first acquisition device is installed on the bucket tooth boom of the remote mining excavator, and the second acquisition device is installed on the rotary platform of the remote mining excavator.
7. A remote mining excavator monitoring method, characterized in that: include: According to the bucket tooth state detection method according to any one of claims 1 to 6, a target bucket tooth state of at least one bucket tooth of a remote mining excavator is determined; When the target bucket tooth state is a dropped state, outputting first alarm information, wherein the first alarm information is used to indicate that the target bucket tooth state is a dropped state; or, When the target bucket tooth state is an invasion state, outputting second alarm information, wherein the second alarm information is used to indicate that the target bucket tooth state is an invasion state; or, When the target bucket tooth state is in an abnormal wear state, a third alarm information is output, where the third alarm information is used to indicate that the target bucket tooth state is in an abnormal wear state.
8. A bucket tooth state detection device, characterized in that: Applied to a remote mining excavator, the device comprises: A first acquisition module is used to acquire an image to be detected, wherein the image to be detected is an image including a position of at least one bucket tooth of a remote mining excavator; a first detection module, connected to the first acquisition module, configured to detect the image to be detected using a target detection model to obtain a target bucket tooth state of at least one bucket tooth of the remote mining excavator; The target detection model is trained by using a plurality of training samples and labeled data corresponding to each of the plurality of training samples. The target bucket tooth state includes one of a normal state, a falling-off state, a wear state, an occlusion state and an intrusion state. The wear state includes a normal wear state and an abnormal wear state. The intrusion state includes a vehicle intrusion state and a pedestrian intrusion state.
9. The device according to claim 8, characterized in that The target detection model includes the fog detection model and the bucket tooth state detection model. The first detection module is specifically used for: Using the fog detection model to detect the image to be detected to obtain a fog detection result; determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator according to the fog detection result, the bucket tooth state detection model and the image to be detected; and / or, The device also includes: Create a module for creating a target detection model; The creation module is specifically used for: Acquire a training sample set, wherein the training sample set includes a plurality of training samples and labeled data corresponding to each of the plurality of training samples; According to multiple training samples and the labeled data corresponding to each training sample in the multiple training samples, the preset model is trained to obtain a target detection model.
10. A remote mining excavator monitoring system, characterized in that: include: A bucket tooth state detection device according to any one of claims 8 to 9, for determining a target bucket tooth state of at least one bucket tooth of a remote mining excavator; a first output module, connected to the bucket tooth state detection device, and configured to output first alarm information when the target bucket tooth state detected by the bucket tooth state detection device is a falling-off state, wherein the first alarm information is used to indicate that the target bucket tooth state is the falling-off state; or, a second output module, connected to the bucket tooth state detection device, and configured to output second alarm information when the target bucket tooth state detected by the bucket tooth state detection device is an intrusion state, wherein the second alarm information is used to indicate that the target bucket tooth state is the intrusion state; or, The third output module is connected to the bucket tooth state detection device and is used to output a third alarm message when the target bucket tooth state detected by the bucket tooth state detection device is an abnormal wear state, wherein the third alarm message is used to indicate that the target bucket tooth state is the abnormal wear state.