Method and device for detecting components of a mine trackless rubber-tyred vehicle

Abstract

The present disclosure relates to a mine trackless rubber-tyred vehicle component detection method and device. The method comprises: using a pre-trained DINOV3 model to extract features from image data to obtain general semantic features; using a pre-trained YOLOv8 model to extract features from the image data to obtain initial features; training the YOLOv8 model based on the loss value between the semantic features and the initial features to obtain a trained YOLOv8 model and target features output by the trained YOLOv8 model; using the YOLOv8 model to identify the target features to obtain an initial detection result of a mine trackless rubber-tyred vehicle component, and evaluating the accuracy of the initial detection result to obtain an evaluation result; and determining the initial detection result as a target component detection result if the evaluation result meets a preset condition. The present scheme can shorten the detection model training period in a new scenario.

Description

Technical Field

[0001] This disclosure relates to the field of coal mining technology, and in particular to a method and apparatus for testing components of a trackless rubber-tired mining vehicle. Background Technology

[0002] In coal mining operations, trackless rubber-tired vehicles are key transportation equipment, and the working condition of their components, such as reflectors, high beams, and turn signals, directly affects the safety of underground transportation.

[0003] Currently, target detection systems built on YOLOv8 are widely used for real-time identification of such components and have achieved good results in specific scenarios. However, when the detection system is deployed to a new coal mine site or the working environment changes, the detection accuracy of the original YOLOv8 model drops sharply due to significant differences in light intensity, dust concentration, and background scenes between different mines. Furthermore, different models of rubber-tired vehicles also differ in appearance features such as the position of reflectors and the shape of headlights. Traditional solutions rely on technicians to re-collect a large number of labeled samples for new scenes and fully train the model, resulting in an adaptation cycle of several weeks. This incurs high manpower and time costs, severely hindering the rapid deployment and widespread application of the detection system. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method and apparatus for testing components of trackless rubber-tired mining vehicles.

[0005] According to a first aspect of the present disclosure, a method for testing components of a trackless rubber-tired mining vehicle is provided, comprising:

[0006] Acquire image data of trackless rubber-tired mining vehicles within the target area; The image data is used to extract features using a pre-trained DINOV3 model to obtain general semantic features; The image data is used to extract features using a pre-trained YOLOv8 model to obtain initial features; The YOLOv8 model is trained based on the loss value between the general semantic features and the initial features. The process then returns to the step of extracting features from the image data using the pre-trained YOLOv8 model until the loss value is less than the preset loss value, thus obtaining the trained YOLOv8 model and its output target features. The target features are identified using the YOLOv8 model to obtain the initial detection results of the trackless rubber-tired vehicle components for mining, and the accuracy of the initial detection results is evaluated to obtain the evaluation results. If the evaluation results meet the preset conditions, the initial detection results will be determined as the detection results of the target component.

[0007] According to a second aspect of the present disclosure, a testing device for components of a trackless rubber-tired mining vehicle is provided, comprising: The acquisition unit is used to acquire image data of trackless rubber-tired mining vehicles within the target area; The first extraction unit is used to extract features from the image data using a pre-trained DINOV3 model to obtain general semantic features; The second extraction unit is used to extract features from the image data using a pre-trained YOLOv8 model to obtain initial features; The training unit is used to train the YOLOv8 model based on the loss value between the general semantic features and the initial features, and then return to execute the step of extracting features from the image data using the pre-trained YOLOv8 model until the loss value is less than the preset loss value, thereby obtaining the trained YOLOv8 model and its output target features. The evaluation unit is used to identify the target features using the YOLOv8 model to obtain the initial detection results of the trackless rubber-tired vehicle components for mining, and to evaluate the accuracy of the initial detection results to obtain the evaluation results. The detection unit is used to determine the initial detection result as the target component detection result when the evaluation result meets the preset conditions.

[0008] According to a third aspect of the present disclosure, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of the first aspects.

[0009] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0010] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0011] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: By acquiring image data of a trackless rubber-tired mining vehicle, extracting general semantic features using a pre-trained DINOV3 model, and extracting initial features using a YOLOv8 model already trained in the original scene, training the YOLOv8 model based on the loss value between the general semantic features and the initial features until the loss value is less than a preset loss value, obtaining the trained YOLOv8 model and its output target features, using the YOLOv8 model to identify the target features to obtain initial detection results and evaluating their accuracy, and confirming the initial detection results when the evaluation results meet preset conditions. The target component detection results are used as the basis for feature distillation training of the YOLOv8 model by leveraging the general semantic features of the DINOV3 model. This allows the target features output by the YOLOv8 model to combine the detection capabilities of the original scene with the scene generalization capabilities of the DINOV3 model. As a result, in new scenes, detection results that meet the accuracy requirements can be obtained directly without recollecting a large number of labeled samples. This significantly reduces the model adaptation cost and greatly shortens the model training cycle in new scenes. In turn, it solves the industry pain points of long adaptation cycles and high labor costs when deploying detection systems in new coal mine sites, and provides efficient and reliable technical support for underground transportation safety in coal mines.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0014] Figure 1 This is a flowchart illustrating a method for inspecting components of a trackless rubber-tired mining vehicle according to an exemplary embodiment.

[0015] Figure 2 This is a block diagram illustrating a component testing device for a trackless rubber-tired mining vehicle according to an exemplary embodiment.

[0016] Figure 3 This is a block diagram illustrating an apparatus for a method of inspecting components of a trackless rubber-tired mining vehicle, according to an exemplary embodiment. Detailed Implementation

[0017] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0018] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0019] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the words “if” and “suppose” as used herein may be interpreted as “when”, “when”, or “in response to a determination”.

[0020] Furthermore, various forms of processes shown in the embodiments of this disclosure can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0021] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 This is a flowchart illustrating a method for inspecting components of a trackless rubber-tired mining vehicle according to an exemplary embodiment, such as... Figure 1 As shown, it should be noted that the method for detecting components of trackless rubber-tired mining vehicles in this embodiment is applied to a device for detecting components of trackless rubber-tired mining vehicles. For example... Figure 1 As shown, the method may include the following steps: Step 101: Obtain image data of trackless rubber-tired mining vehicles within the target area.

[0023] In some embodiments of this disclosure, image data can be acquired using a high-definition industrial camera. The high-definition industrial camera has dustproof, waterproof, and anti-interference functions to adapt to the complex environmental conditions in underground coal mines and ensure the clarity and stability of image acquisition.

[0024] Step 102: Use the pre-trained DINOV3 model to extract features from the image data to obtain general semantic features.

[0025] It should be noted that the DINOV3 model is a self-supervised pre-trained model based on the visual Transformer architecture. It has been pre-trained on large-scale general image datasets such as ImageNet and has powerful general visual representation learning capabilities. The DINOV3 model can extract high-dimensional, dense general semantic feature vectors from images by performing block processing and multi-head self-attention computation. These features do not depend on specific tasks or specific scenes, but encode rich content such as global contextual information, object component structure, spatial layout relationships, and scene semantic categories in the image.

[0026] In this embodiment of the disclosure, general semantic features serve as a stable and robust semantic representation of image data in the new coal mine scenario. They provide a scene-independent feature benchmark for subsequent fusion with the initial target features extracted by the YOLOv8 model, effectively compensating for the feature drift problem caused by scene migration in the YOLOv8 model.

[0027] In one example, general semantic features can refer to the high-level, task-independent semantic information of scenes and objects that the DINOV3 model understands and encodes from images based on self-supervised learning. Specifically, general semantic features can be human-like visual cognition results such as the DINOV3 model recognizing "this is a trackless rubber-tired mining vehicle", "the front of the vehicle is equipped with a lighting device", "the side of the vehicle has a highly reflective safety sign", "the current ambient light is dim and there is dust interference", and "the vehicle is located in a transport tunnel". At the same time, this feature can also perceive the spatial relationship between components, such as "the reflective strip is located at the bottom of the carriage and adjacent to the tires", as well as the quantitative shift of scene attributes, such as "the light intensity is significantly reduced compared to the original scene" and "the position of the reflective strip has moved down compared to the original scene". General semantic features provide a stable and comparable semantic benchmark for subsequent analysis of environmental differences and vehicle differences between new scenes and original scenes.

[0028] Step 103: Use the pre-trained YOLOv8 model to extract features from the image data to obtain initial features.

[0029] In one embodiment, the same image data can be input into a YOLOv8 model that has been trained under supervision in the original coal mine scene for reflective strips, high beams, and turn signals and has reached the required detection accuracy. Using the feature extractor in the YOLOv8 model, the image is downsampled at multiple scales and fused with hierarchical features to output several initial feature maps with different spatial resolutions (e.g., 80×80, 40×40, 20×20). Each grid cell of these initial feature maps encodes task-related information such as the confidence level of whether a mine trackless rubber-tired vehicle component exists at that location, the bounding box offset, and the class probability. Different filters in the channel dimension selectively activate the elongated strip texture of the reflective strips, the circular or rectangular outline of the headlights, and the strong response features of the high-reflectivity area.

[0030] It should be noted that since the YOLOv8 model has accumulated a large amount of knowledge about the specific lighting conditions, dust concentration, background environment and vehicle appearance of the original scene, the initial features it extracts are essentially a direct mapping of the original scene knowledge onto the new scene image. It has the ability to accurately locate the position of the parts. However, due to the scene migration, there is feature distribution drift. Therefore, problems such as poor adaptability to the position of reflector strips of new vehicle models and weakened headlight feature response in low light environment will occur. It is necessary to fuse and optimize with the scene robust general semantic features extracted by the DINOV3 model.

[0031] Step 104: Train the YOLOv8 model based on the loss value between the semantic features and the initial features. Return to execute steps 102 and 103 until the loss value is less than the preset loss value, and obtain the trained YOLOv8 model and its output target features.

[0032] In some embodiments of this disclosure, step 104, training the YOLOv8 model based on the loss value between semantic features and initial features, may specifically include the following steps: The semantic features and initial features are input into a preset loss function to calculate the loss value; Adjust the parameters of the YOLOv8 model based on the loss value.

[0033] In this embodiment, the general semantic features extracted by the DINOV3 model can be used as soft-label supervision signals. The initial features and semantic features extracted by the YOLOv8 model in the current iteration are input into a preset loss function to calculate the distribution difference loss value between the two in the high-dimensional feature space. The backbone network parameters of the YOLOv8 model are updated based on the loss value through backpropagation, and the parameters of the DINOV3 model are frozen. Then, the process returns to the step of extracting features from the image data using the pre-trained YOLOv8 model, re-extracting the initial features and calculating the loss with the same semantic features. It should be noted that through the above iterative training, the initial features output by the YOLOv8 model are continuously aligned with the scene robust representation space represented by the semantic features in terms of feature distribution until the loss value is less than the preset loss threshold. At this point, the YOLOv8 model that has been trained is obtained. This model not only retains the component detection capabilities of reflectors, high beams, and turn signals in the original scene, but also integrates the scene generalization capability given by the DINOV3 model. Its output target features are high discriminative feature maps optimized by feature distillation, which can still stably represent the semantic information of key components of trackless rubber-tired mining vehicles under new scene lighting, dust, vehicle type changes, and other conditions.

[0034] As an example, a preset loss function is used to measure the distributional difference between the general semantic features output by the DINOV3 model and the initial features output by the YOLOv8 model. Specifically, Euclidean distance loss can be used, which treats the general semantic feature vector set output by the DINOV3 model and the initial feature vector set output by the YOLOv8 model as points in two high-dimensional spaces. The elements at each corresponding position in the two vector sets are subtracted, the differences in each dimension are squared and summed, and the square root of the sum is taken to obtain the Euclidean distance between the two feature vectors as the loss value. Mean squared error loss can also be used, which measures the numerical approximation by calculating the sum of the squares of the differences between each corresponding element in the two feature vector sets. Cosine similarity loss can also be used, which calculates the cosine of the angle between the two feature vectors and takes its negative value or the difference from 1, so that the model makes the directions of the two feature vectors tend to be consistent during training. KL divergence loss can also be used, which treats the two feature distributions as probability distributions and measures the distributional difference by calculating the relative entropy. These loss functions can all be used as preset loss functions to drive the YOLOv8 model to align its initial features to the general semantic feature space of DINOV3 during training, thereby achieving feature distillation and scene adaptability improvement.

[0035] Step 105: Use the YOLOv8 model to identify target features, obtain initial detection results of the trackless rubber-tired vehicle components for mining, and evaluate the accuracy of the initial detection results to obtain evaluation results.

[0036] In one embodiment, the target features output by the trained YOLOv8 model can be input into the detection head of the YOLOv8 model. The detection head consists of multiple convolutional layers and is responsible for dense prediction on the target feature map. The detection head traverses each grid cell in the target feature map, predicts whether a mining trackless rubber-tired vehicle component exists at that location based on the depth information encoded in each cell, and regresses the component's bounding box relative to the center coordinates, width, and height of the cell. It also outputs the probability distribution of whether the component belongs to a reflector strip, high beam headlight, or turn signal. By performing confidence threshold filtering and non-maximum suppression to remove duplicates from all prediction results, the bounding box position, category label, and confidence score of each detected mining trackless rubber-tired vehicle component in the image are finally obtained. This information together constitutes the initial detection result for subsequent accuracy evaluation.

[0037] In some embodiments of this disclosure, step 105, which involves evaluating the accuracy of the initial detection results to obtain an evaluation result, may specifically include the following steps: Step a1: Obtain the real annotation information corresponding to the image data.

[0038] The accurate labeling information includes at least one of the following: the location of the boundary frames of the reflective strips, high beams, and turn signals of the trackless rubber-tired mining vehicle, and the category label.

[0039] In this embodiment of the disclosure, in order to evaluate the accuracy of the initial detection results, it is necessary to obtain the real annotation information corresponding to the input image data as an evaluation benchmark. The real annotation information can be pre-annotated by technicians during the model pre-training stage or the sample supplementation stage. Its content can include the real bounding box position coordinates of each reflector, high beam, and turn signal in the image of the trackless rubber-tired mining vehicle (as well as the real category label corresponding to each bounding box (such as "reflector", "high beam", "turn signal"). This real annotation information constitutes the objective factual standard of the components to be detected in the image. In the subsequent evaluation steps, the predicted bounding boxes in the initial detection results are aligned and compared with these real annotation information one by one. The accuracy and completeness of the model detection are quantified by calculating the intersection-union ratio, counting the number of true positives and false positives, etc.

[0040] Step a2: For each real annotation information, align and compare the real annotation information with its corresponding initial detection result to obtain the comparison result.

[0041] In this embodiment, for each ground truth bounding box corresponding to a ground truth annotation, it is compared one by one with all predicted bounding boxes in the initial detection results to calculate the intersection-union ratio (IU) between each predicted bounding box and the ground truth bounding box. Predicted bounding boxes with IU greater than a preset IU threshold are determined to be matching predictions of the ground truth annotation and are recorded as true positives. If a ground truth annotation fails to reach the IU threshold with any predicted bounding box, the ground truth target is determined to be missed and recorded as a false negative. At the same time, all predicted bounding boxes that do not successfully match any ground truth annotation are recorded as false positives. Through the above alignment and comparison process, the matching relationship between each ground truth annotation and the prediction result, as well as the statistical results of true positives, false positives, and false negatives, are obtained as the comparison results for subsequent calculation of evaluation indicators such as accuracy and recall.

[0042] Step a3: Based on the comparison results, calculate at least one evaluation metric among precision, recall, and F1 score.

[0043] In this embodiment of the disclosure, based on the comparison results obtained in step a2, the matching status of all real annotation information and predicted bounding boxes is statistically analyzed to obtain the number of true positives, false positives, and false negatives. Based on these statistical values, the accuracy is calculated as the proportion of correctly detected objects in the predicted bounding boxes, i.e., the number of true positives divided by the sum of the number of true positives and false positives. The recall is calculated as the proportion of real targets successfully detected, i.e., the number of true positives divided by the sum of the number of true positives and false negatives. The F1 score is further calculated as the harmonic mean of accuracy and recall to comprehensively evaluate the model's detection performance. At least one of the calculated accuracy, recall, and F1 score can be used as an evaluation index to determine whether the detection accuracy of the current model in a new scenario reaches a preset threshold, thereby deciding whether to directly output the detection results or trigger the model adaptation and optimization process.

[0044] Step a4: If at least one evaluation indicator is greater than or equal to its corresponding preset threshold, the evaluation result is determined to meet the preset conditions.

[0045] In this embodiment, at least one of the calculated evaluation metrics, namely accuracy, recall, and F1 score, is compared with its corresponding preset threshold. When all the evaluated metrics are greater than or equal to their corresponding preset thresholds, the detection performance of the current model is determined to meet the preset conditions. At this time, there is no need to trigger the adaptation optimization process, and the initial detection result can be directly output as the final target component detection result. The preset conditions reflect the minimum accuracy requirements of the detection system at the coal mine operation site, such as an accuracy of not less than 95% and a recall of not less than 90%. The model is ensured to meet safety standards in terms of both false detection rate and false negative rate through the joint constraint of multiple metrics. If any metric fails to reach the corresponding threshold, the evaluation result is determined not to meet the preset conditions, and the adaptation optimization process based on the DINOV3 model to analyze scenario differences and generate sample labeling suggestions will be entered.

[0046] Step 106: If the evaluation results meet the preset conditions, the initial test results are determined as the test results of the target component.

[0047] In one embodiment, if the evaluation result meets the preset conditions, the initial detection result is determined as the final output target component detection result. The target component detection result may include the bounding box position coordinates, category label (reflective strip, high beam or turn signal) of each detected trackless rubber-tired vehicle component in the image data, as well as the corresponding confidence score. It is a high-quality detection output after feature distillation optimization and accuracy verification.

[0048] Understandably, since this result was determined under the condition that all evaluation indicators reached the preset thresholds (such as accuracy ≥ 95% and recall ≥ 90%), it indicates that the YOLOv8 model that has completed training has successfully adapted to the changes in lighting, dust, and vehicle type in the new coal mine scenario, and its output detection information has the accuracy and reliability required for field application. Finally, the detection results of the target component are presented to coal mine workers in the form of image annotation, data reports, etc. through the result output module, which is used to monitor the status of vehicle components in real time, warn of potential safety hazards, and provide direct technical support for underground transportation safety.

[0049] In some embodiments of this disclosure, the method may further include the following steps: Step b1: If the evaluation results do not meet the preset conditions, determine the differences between the new scene and the original scene corresponding to the target area based on the DINOV3 model, and generate sample annotation suggestions based on the differences.

[0050] The original scene refers to the scene corresponding to the labeled samples used for pre-training the YOLOv8 model.

[0051] In this embodiment of the disclosure, if the evaluation result does not meet the preset conditions, the model adaptation optimization process is triggered. At this time, the general semantic features of the current new scene image are extracted based on the DINOV3 model, and the extracted general semantic features are compared and analyzed with the pre-stored original scene feature distribution model to identify the environmental differences and / or vehicle model differences of the new scene relative to the original scene. Based on the identified difference type and degree, targeted sample annotation suggestions are generated to guide technicians to supplement the collection and annotation of new scene image samples that can cover the current difference features.

[0052] The original scene can refer to the coal mine operation scene corresponding to the labeled samples used for pre-training the YOLOv8 model. It is the knowledge source for the initial detection capability of the model and the reference benchmark for scene transfer.

[0053] Step b1 establishes a closed loop from "insufficient detection accuracy" to "difference analysis" and then to "sample collection guidance," laying the foundation for subsequent model fine-tuning that only a small number of targeted samples are needed.

[0054] In some embodiments of this disclosure, step b1 may specifically include the following steps: Obtain the types and degrees of differences obtained from the DINOV3 model analysis; the types of differences include at least one of the following: differences in light intensity, differences in dust concentration, differences in background scene, differences in reflector position, differences in headlight shape, and differences in vehicle appearance; The degree of difference is compared with a preset degree of difference threshold. When the degree of difference is greater than or equal to the degree of difference threshold, sample labeling suggestions matching the difference type are generated.

[0055] In this embodiment of the disclosure, the difference type and difference degree obtained by the DINOV3 model analysis are obtained. The difference type covers at least one of the following: difference in light intensity, difference in dust concentration, difference in background scene, difference in reflective strip position, difference in headlight shape, and difference in vehicle appearance. The difference degree quantifies the change range of each difference type. Then, the difference degree is compared with a preset difference degree threshold. When the difference degree of a certain difference type is greater than or equal to the corresponding difference degree threshold, it is determined that the difference has significantly affected the model detection performance. At this time, the generation of sample annotation suggestions that match the difference type is triggered.

[0056] For example, when the analysis shows that the difference in "light intensity" is greater than a preset threshold, the sample annotation suggestion is "Please collect 100 to 200 images of mining trackless rubber-tired vehicles in new scenes with dim lighting or local shadows, and highlight the reflective strips, high beams, and turn signals." When the analysis shows that the difference in "dust concentration" is significant, the sample annotation suggestion is "Please collect about 150 blurry images in high-dust environments, ensuring coverage of different dust concentration levels, and annotate all visible components." When the analysis shows that the difference in "reflective strip position" is large, the sample annotation suggestion is "Please collect 80 to 120 images from different angles for new vehicle models with lower reflective strip positions, and highlight the precise bounding boxes of the reflective strips." If multiple differences are detected simultaneously, a composite suggestion is generated, such as "Please collect 200 images of new vehicle models that include low-light, high-dust environments and lower reflective strip positions, and prioritize annotating the reflective strips and headlights." These recommendations provide technicians with clear data collection scenarios, target quantities, and annotation priorities, ensuring that supplementary samples can efficiently cover key discrepancies that lead to degraded model performance.

[0057] Through the above steps, a precise mapping is achieved from qualitative identification of scene differences to quantitative judgment, and then to targeted sample collection guidance. This ensures that the subsequently supplemented labeled samples can efficiently cover the key difference factors that cause the model performance to decline, and minimizes the blindness and redundancy of sample collection.

[0058] Step b2: Obtain new scene-annotated samples supplemented based on the sample annotation suggestions.

[0059] Users can upload new scene annotation samples based on the above sample annotation suggestions.

[0060] Step b3: Adjust the parameters of the trained YOLOv8 model using the new scene labeled samples, and return to execute the step of using the YOLOv8 model to identify target features until the evaluation result meets the preset conditions.

[0061] In this embodiment, after obtaining new scene annotation samples supplemented according to sample annotation suggestions, these annotation samples are input into the YOLOv8 model that has completed feature distillation training. The model parameters are fine-tuned and optimized using the mini-batch gradient descent algorithm. During the fine-tuning process, the model updates its weights through backpropagation based on the real annotation information of reflective strips, high beams, and turn signals in the new scene samples, further enhancing its feature extraction and detection capabilities in the new scene. After completing one round of parameter adjustment, the process returns to the step of using the YOLOv8 model to identify target features, re-detecting the current new scene image and evaluating its accuracy. If the evaluation result still does not meet the preset conditions, the above fine-tuning and detection evaluation process is repeated to iteratively optimize the model parameters until the evaluation indicators such as detection accuracy, recall, or F1 score all meet the preset threshold requirements. Through this closed-loop iterative mechanism, only a small number of targeted supplementary samples are needed to drive the YOLOv8 model to quickly adapt to the new coal mine scene, ultimately achieving the detection accuracy required for field applications.

[0062] In some embodiments of this disclosure, the differences between the new scene and the original scene include environmental differences and / or vehicle model differences; environmental differences include at least one of light intensity differences, dust concentration differences, and background scene differences; vehicle model differences include at least one of reflector strip position differences, headlight shape differences, and vehicle appearance differences.

[0063] In this embodiment, the differences between the new scene and the original scene are divided into two main categories: environmental differences and vehicle model differences. Environmental differences refer to image feature shifts caused by changes in the physical conditions of the coal mine operation site, specifically including differences in light intensity (such as changes in underground lighting brightness and shadow distribution), differences in dust concentration (such as changes in image blur caused by airborne particulate matter), and differences in background scene (such as changes in background elements such as tunnel structure, support forms, and pipeline layout). Vehicle model differences refer to component feature shifts caused by differences in the appearance design of different batches or manufacturers of trackless rubber-tired mining vehicles, specifically including differences in reflector strip position (such as the reflector strip being moved from the middle of the vehicle to the lower or rear), differences in headlight shape (such as round headlights becoming rectangular headlights or LED light strips), and differences in vehicle appearance (such as changes in vehicle size, color, and overall outline). By subdividing and identifying these two categories of differences, the DINOV3 model can accurately locate the specific factors causing the YOLOv8 model's detection performance degradation, providing precise basis for the type of difference in generating targeted sample annotation suggestions.

[0064] In some embodiments of this disclosure, a data storage module can be used to store the collected image data, general semantic features and initial features, target component detection results, and optimization parameters and adaptation records of the model adaptation optimization module, which facilitates subsequent data traceability and model iterative optimization.

[0065] In some embodiments, the DINOV3 model can use officially released pre-trained weights and be pre-trained on the ImageNet dataset to give it good general image feature extraction capabilities. The YOLOv8 model was trained in a coal mine setting by collecting a large number of image samples containing reflectors, high beams, and turn signals from different models of trackless rubber-tired mining vehicles. After labeling the samples, the YOLOv8 model was trained using the labeled sample data until the model achieved a detection accuracy of over 98% in the original scene, thus completing the initialization of the YOLOv8 model.

[0066] According to the component detection method for trackless rubber-tired mining vehicles proposed in this disclosure, image data of the trackless rubber-tired mining vehicle is acquired. A pre-trained DINOV3 model is used to extract general semantic features, and a YOLOv8 model trained in the original scene is used to extract initial features. The YOLOv8 model is trained based on the loss value between the general semantic features and the initial features until the loss value is less than a preset loss value, resulting in a trained YOLOv8 model and its output target features. The YOLOv8 model is used to identify the target features to obtain initial detection results, and their accuracy is evaluated. When the evaluation results meet preset conditions, the initial detection results are confirmed. The target component detection results are used as the basis for feature distillation training of the YOLOv8 model by leveraging the general semantic features of the DINOV3 model. This allows the target features output by the YOLOv8 model to combine the detection capabilities of the original scene with the scene generalization capabilities of the DINOV3 model. As a result, in new scenes, detection results that meet the accuracy requirements can be obtained directly without recollecting a large number of labeled samples. This significantly reduces the model adaptation cost and greatly shortens the model training cycle in new scenes. In turn, it solves the industry pain points of long adaptation cycles and high labor costs when deploying detection systems in new coal mine sites, and provides efficient and reliable technical support for underground transportation safety in coal mines.

[0067] Figure 2 This is a block diagram illustrating a component testing device for a trackless rubber-tired mining vehicle according to an exemplary embodiment. (Refer to...) Figure 2 The device includes an acquisition unit 201, a first extraction unit 202, a second extraction unit 203, a training unit 204, an evaluation unit 205, and a detection unit 206.

[0068] Among them, the acquisition unit 201 is used to acquire image data of trackless rubber-tired mining vehicles within the target area; The first extraction unit 202 is used to extract features from image data using a pre-trained DINOV3 model to obtain general semantic features; The second extraction unit 203 is used to extract features from image data using a pre-trained YOLOv8 model to obtain initial features; Training unit 204 is used to train the YOLOv8 model based on the loss value between semantic features and initial features. It returns to perform the step of extracting features from image data using the pre-trained YOLOv8 model until the loss value is less than the preset loss value, thus obtaining the trained YOLOv8 model and its output target features. Evaluation unit 205 is used to identify target features using the YOLOv8 model, obtain initial detection results of the trackless rubber-tired vehicle components for mining, and evaluate the accuracy of the initial detection results to obtain evaluation results; The detection unit 206 is used to determine the initial detection result as the target component detection result when the evaluation result meets the preset conditions.

[0069] In some embodiments of this disclosure, the detection unit 206 is further configured to: If the evaluation results do not meet the preset conditions, the difference between the new scene and the original scene corresponding to the target region is determined based on the DINOV3 model, and sample annotation suggestions are generated based on the difference; the original scene is the scene corresponding to the annotation samples used to pre-train the YOLOv8 model. Obtain new scene-annotated samples supplemented based on sample annotation suggestions; The parameters of the trained YOLOv8 model are adjusted using labeled samples from the new scene. The process then returns to the step of using the YOLOv8 model to identify target features until the evaluation results meet the preset conditions.

[0070] In some embodiments of this disclosure, the differences between the new scenario and the original scenario include environmental differences and / or vehicle model differences; Environmental differences include at least one of the following: differences in light intensity, differences in dust concentration, and differences in background scene. Vehicle model differences include at least one of the following: differences in reflector position, headlight shape, and vehicle appearance.

[0071] In some embodiments of this disclosure, the training unit 204 may specifically be used for: The semantic features and initial features are input into a preset loss function to calculate the loss value; Adjust the parameters of the YOLOv8 model based on the loss value.

[0072] In some embodiments of this disclosure, the evaluation unit 205 may specifically be used for: Obtain the real annotation information corresponding to the image data; the real annotation information includes at least one of the following: the position of the bounding box of the reflector strip, high beam headlight, and turn signal of the trackless rubber-tired vehicle in the mine; For each real annotation, the real annotation is aligned and compared with its corresponding initial detection result to obtain the comparison result; Based on the comparison results, calculate at least one of the following evaluation metrics: precision, recall, and F1 score; If at least one evaluation indicator is greater than or equal to its corresponding preset threshold, the evaluation result is determined to meet the preset conditions.

[0073] In some embodiments of this disclosure, the detection unit 206 is further configured to: Obtain the types and degrees of differences obtained from the DINOV3 model analysis; the types of differences include at least one of the following: differences in light intensity, differences in dust concentration, differences in background scene, differences in reflector position, differences in headlight shape, and differences in vehicle appearance; The degree of difference is compared with a preset degree of difference threshold. When the degree of difference is greater than or equal to the degree of difference threshold, sample labeling suggestions matching the difference type are generated.

[0074] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0075] According to the mining trackless rubber-tired vehicle component detection device proposed in this embodiment, image data of the mining trackless rubber-tired vehicle is acquired, general semantic features are extracted using a pre-trained DINOV3 model, and initial features are extracted using a YOLOv8 model already trained in the original scene. The YOLOv8 model is trained based on the loss value between the general semantic features and the initial features until the loss value is less than a preset loss value, resulting in a trained YOLOv8 model and its output target features. The YOLOv8 model is used to identify the target features to obtain initial detection results, and their accuracy is evaluated. When the evaluation results meet preset conditions, the initial detection results are confirmed. The target component detection results are used as the basis for feature distillation training of the YOLOv8 model by leveraging the general semantic features of the DINOV3 model. This allows the target features output by the YOLOv8 model to combine the detection capabilities of the original scene with the scene generalization capabilities of the DINOV3 model. As a result, in new scenes, detection results that meet the accuracy requirements can be obtained directly without recollecting a large number of labeled samples. This significantly reduces the model adaptation cost and greatly shortens the model training cycle in new scenes. In turn, it solves the industry pain points of long adaptation cycles and high labor costs when deploying detection systems in new coal mine sites, and provides efficient and reliable technical support for underground transportation safety in coal mines.

[0076] Figure 3 This is a block diagram illustrating an apparatus for a method of inspecting components of a trackless rubber-tired mining vehicle, according to an exemplary embodiment. For example, apparatus 300 may be an electronic device, such as a mobile phone, computer, digital broadcasting terminal, messaging device, tablet device, personal digital assistant, etc.

[0077] Reference Figure 3 The device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0078] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0079] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0080] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0081] Multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0082] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0083] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0084] Sensor assembly 314 includes one or more sensors for providing status assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0085] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0086] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0087] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0088] In an exemplary embodiment, a computer program product is also provided, including a computer program that implements the above-described method when executed by the processor 320 of the device 300.

[0089] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0090] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for detecting a component of a mine trackless rubber-tyred vehicle, characterized in that, include: Acquire image data of trackless rubber-tired mining vehicles within the target area; The image data is used to extract features using a pre-trained DINOV3 model to obtain general semantic features; The image data is used to extract features using a pre-trained YOLOv8 model to obtain initial features; The YOLOv8 model is trained based on the loss value between the general semantic features and the initial features. The process then returns to the step of extracting features from the image data using the pre-trained YOLOv8 model until the loss value is less than the preset loss value, thus obtaining the trained YOLOv8 model and its output target features. The target features are identified using the YOLOv8 model to obtain the initial detection results of the trackless rubber-tired vehicle components for mining, and the accuracy of the initial detection results is evaluated to obtain the evaluation results. If the evaluation results meet the preset conditions, the initial detection results will be determined as the detection results of the target component.

2. The method of claim 1, wherein, The method further includes: If the evaluation result does not meet the preset conditions, the difference between the new scene and the original scene corresponding to the target region is determined based on the DINOV3 model, and sample annotation suggestions are generated based on the difference; the original scene is the scene corresponding to the annotation samples used to pre-train the YOLOv8 model. Obtain new scene-annotated samples supplemented based on the sample annotation suggestions; The parameters of the trained YOLOv8 model are adjusted using the new scene-labeled samples, and the process of recognizing the target features using the YOLOv8 model is repeated until the evaluation result meets the preset conditions.

3. The method of claim 2, wherein, The differences between the new scenario and the original scenario include environmental differences and / or vehicle model differences; The environmental differences include at least one of the following: differences in light intensity, differences in dust concentration, and differences in background scene. The differences in vehicle models include at least one of the following: differences in reflector position, differences in headlight shape, and differences in vehicle appearance.

4. The method of claim 1, wherein, Training the YOLOv8 model based on the loss value between the general semantic features and the initial features includes: The general semantic features and the initial features are input into a preset loss function to calculate the loss value; The parameters of the YOLOv8 model are adjusted based on the loss value.

5. The method of claim 1, wherein, The accuracy evaluation of the initial detection results, to obtain the evaluation results, includes: Obtain the real annotation information corresponding to the image data; the real annotation information includes at least one of the following: the position of the bounding box of the reflector strip, high beam headlight, and turn signal of the trackless rubber-tired vehicle in the mine; For each real annotation, the real annotation is aligned and compared with its corresponding initial detection result to obtain the comparison result; Based on the comparison results, at least one of the following evaluation metrics is calculated: accuracy, recall, and F1 score. If all at least one evaluation index is greater than or equal to its corresponding preset threshold, the evaluation result is determined to meet the preset conditions.

6. The method of claim 2, wherein, The step of generating sample labeling suggestions based on the differences includes: Obtain the types and degrees of differences obtained from the analysis using the DINOV3 model; the types of differences include at least one of the following: differences in light intensity, differences in dust concentration, differences in background scene, differences in reflector position, differences in headlight shape, and differences in vehicle appearance; The degree of difference is compared with a preset degree of difference threshold. When the degree of difference is greater than or equal to the degree of difference threshold, sample labeling suggestions matching the difference type are generated.

7. A mine trackless rubber-tyred vehicle component detection device characterised by, include: The acquisition unit is used to acquire image data of trackless rubber-tired mining vehicles within the target area; The first extraction unit is used to extract features from the image data using a pre-trained DINOV3 model to obtain general semantic features; The second extraction unit is used to extract features from the image data using a pre-trained YOLOv8 model to obtain initial features; The training unit is used to train the YOLOv8 model based on the loss value between the general semantic features and the initial features, and then return to execute the step of extracting features from the image data using the pre-trained YOLOv8 model until the loss value is less than the preset loss value, thereby obtaining the trained YOLOv8 model and its output target features. The evaluation unit is used to identify the target features using the YOLOv8 model to obtain the initial detection results of the trackless rubber-tired vehicle components for mining, and to evaluate the accuracy of the initial detection results to obtain the evaluation results. The detection unit is used to determine the initial detection result as the target component detection result when the evaluation result meets the preset conditions.

8. An electronic device, comprising: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images