Mining trackless rubber-tyred vehicle overspeed identification method based on machine vision
Through machine vision technology, the identification and tracking of trackless rubber wheel trucks is used by YOLOv8-GS and DeepSort algorithms, which solves the problem of difficult-to-control speeding behavior of vehicles in coal mines, improves safety and detection accuracy, and reduces the risk of accidents.
Patent Information
- Application Number
- CN202411145996.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
AI Technical Summary
Trackless rubber wheel trucks underground in coal mines are difficult to control in complex environments, resulting in speeding behavior increasing the risk of collision and overturning accidents and threatening workers' lives and safety.
Using machine vision-based recognition method, video streams are obtained through cameras, data preprocessing and model training are performed, and YOLOv8-GS improved algorithm and DeepSort tracking algorithm are used to realize the identification and speeding behavior detection of trackless rubber wheelbarrows.
It improves detection accuracy and tracking accuracy, reduces calculation complexity and reasoning time, can effectively identify vehicle speeding behavior in complex environments, reduces accident risk, and has practical application value.
Smart Images

Figure CN119006514A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision, and in particular relates to a method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision. Background Art
[0002] In the context of the increasing nationalization of my country's coal industry, coal mine production safety and transportation safety are related to the stability and development of the national economy. In the modern coal mine production process, coal auxiliary transportation refers to the transportation of personnel, materials, equipment, etc. in addition to coal transportation. Traditional auxiliary transportation uses relay transportation methods such as rail mine cars, endless ropes, and small winches, but due to the existence of many operating links, slow speed, many people and machines, low efficiency, poor mobility and adaptability, etc., it restricts the production and development of mines, especially super-large mines. With the continuous development of high-yield and high-efficiency mines in my country, trackless transportation with trackless rubber-wheeled vehicles as transportation tools has been widely used in coal mine production due to its large load capacity, flexible use, large climbing ability and one machine with multiple uses. Its safe driving has an increasingly significant impact on the overall efficient and stable operation of coal mines.
[0003] At present, the safe driving of rubber-tyred trackless vehicles in coal mines faces many challenges. First, the road conditions for rubber-tyred trackless vehicles are complex. Second, there are factors such as insufficient lighting, excessive dust, and poor driver visibility in the tunnels. Speeding has become the biggest safety hazard for the safe driving of rubber-tyred trackless vehicles in coal mines, and it must be given enough attention. Speeding of rubber-tyred trackless vehicles in mines makes it difficult to control the vehicles in the complex environment of coal mines, increases the probability of accidents such as collisions and rollovers, and seriously threatens the lives of workers. Summary of the invention
[0004] In order to solve the technical problems existing in the prior art, the present invention provides a method for identifying overspeeding of a rubber-tyred trackless mining vehicle based on machine vision. The method obtains the vehicle data set of the monitoring points of the transportation scene of the rubber-tyred trackless mining vehicle, performs data preprocessing, model training and tracking, and determines the speeding behavior of the vehicle based on the tracking results.
[0005] To achieve the above purpose, the technical solution adopted by the present invention is: a method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision, and the specific steps are as follows:
[0006] Step 1: Image source data acquisition: Use a camera installed in the material collection area to obtain the real-time video stream of the monitoring point of the trackless rubber-tyred vehicle transportation scene, intercept the video of the trackless rubber-tyred vehicle driving in multiple time periods, and then filter the intercepted video, and intercept the video frame by frame to obtain the data set for subsequent processing;
[0007] Step 2: Data enhancement preprocessing: transform and expand the data collected in step 1;
[0008] Step 3: Image perspective transformation and pixel ratio calculation: Perform perspective transformation on the acquired data and calculate the ratio r of pixel distance to actual distance;
[0009] Step 4: Preparation of training data set: Label and divide the data set processed in step 3, use LabelImg software to label the rubber-tyred trackless vehicles on the images after data preprocessing, and generate a txt file with location coordinates and category names after labeling. The labeled data set is divided into training set, test set and validation set in a ratio of 7:2:1;
[0010] Step 5: Model training: Use the improved YOLOv8-GS algorithm to realize the recognition of trackless rubber-wheeled vehicles;
[0011] Step 6: Realize multi-target tracking based on DeepSort tracking algorithm;
[0012] Step 7: Calculate the speed of the target to be tracked. Based on the YOLOv8-GS improved algorithm and the DeepSort tracking algorithm, complete the acquisition of vehicle speed and speeding behavior detection.
[0013] In step three, the specific steps are as follows:
[0014] Step S31: The specific perspective transformation formula is:
[0015]
[0016] Among them, (x, y, z) is the homogeneous coordinates of the original image pixel, (X, Y, Z) is the homogeneous coordinates of the transformed image pixel, and after normalizing Z, the normalized homogeneous coordinates (X′, Y′, 1) are obtained, that is, the two-dimensional plane coordinates of the pixel corresponding to the original image after the transformation are:
[0017]
[0018] Step S32, take an object of known size and place it under the monitoring point for obtaining source image data, and convert the image into a top view using the perspective transformation in step S31, calculate the pixel distance using pixel points, and calculate the pixel ratio r based on the actual distance of the reference target and the calculated pixel distance.
[0019] In step five, the specific training steps are:
[0020] Step S51, using the slim-neck design paradigm to transform the neck of the baseline model;
[0021] Step S52, introducing an attention mechanism into the neck network;
[0022] Step S53, the present invention uses accuracy, recall rate, mAP@0.5, model parameter scale, model calculation amount, model training time and detection speed as evaluation indicators.
[0023] In step six, the specific steps are as follows:
[0024] Step S61: When the softmax loss function is used for classification, the softmax loss function is:
[0025]
[0026] Among them, m is the number of samples, n is the number of categories, Is the discriminant X i The weight vector of the true label, Indicates the discrimination X i The weight vector of the class, X i Represents the feature vector of the i-th image, y i Represents X i The true label of , b is the bias term;
[0027] The center loss function is used to train the vehicle data. The calculation formula of the center loss function is as follows:
[0028]
[0029] Where m is the number of samples, x i Represents the feature value of the i-th image, y i is the output of the network features, Represents the central value of the feature, the central value of the feature Varies with the characteristics of depth.
[0030] Therefore, the softmax loss function combined with the center loss function defines the final loss function L veh L , the calculation formula is as follows:
[0031]
[0032] Among them, λ represents the fusion relationship between the two, L S represents the softmax loss function, L C represents the center loss function;
[0033] Step S62, the calculation formula of multi-target tracking accuracy MOTA is as follows:
[0034]
[0035] Among them, Fn t is the number of missed targets in the tth frame, Fpt is the number of error detections in frame t, ID st is the number of ID Switches in frame t, GT t Indicates the number of correctly labeled training data samples in the tth frame;
[0036] The multi-target tracking accuracy MOTP is the ratio of the correctly predicted targets to the total number of predicted targets under the detection frame overlap threshold. The specific calculation formula is as follows:
[0037]
[0038] Among them, d t,i represents the average metric distance between target i and the annotation box at time t, i.e., the matching error, c t Indicates the number of successful matches at time t.
[0039] In step seven, the specific steps are:
[0040] Step S71, real-time position acquisition of the vehicle: based on the tracking result, the center point position of the object in each frame is obtained, and the calculation formula is as follows:
[0041]
[0042] Where u represents the horizontal coordinate of the center of the vehicle detection frame; v represents the vertical coordinate of the center of the vehicle detection frame; w represents the aspect ratio of the vehicle detection frame; h represents the height of the vehicle detection frame; (x k ,y k ) represents the final calculated vehicle center of mass;
[0043] It can be obtained that the vehicle center of mass trajectory set is as follows:
[0044]
[0045] in, represents the coordinates of the center of mass of the i-th car in the k-th frame, and frames is the total number of frames;
[0046] S72, determination of vehicle speed: the actual displacement of the vehicle is obtained by using the pixel distance of the vehicle movement through the pixel ratio r calculated in step S32, and then the frame difference between the initial and end positions of the vehicle speed measurement is divided by the video frame rate to estimate the average speed of the vehicle passing through the speed measurement area. The specific calculation steps are as follows:
[0047] The actual displacement S of the vehicle is obtained by the pixel ratio r calculated in the above step S32 and the pixel distance d moved by the vehicle. The calculation formula is as follows:
[0048]
[0049] Where d is the pixel distance the vehicle moves, r is the ratio of the pixel distance to the actual distance, represents the center of mass coordinates of the i-th car in the k-th frame, represents the center of mass coordinates of the i-th car in the k+n-th frame;
[0050] The time t when the vehicle passes through the speed measurement area is calculated by the frame difference and the video frame rate. The formula is as follows:
[0051]
[0052] Among them, frame1 is the frame number of the initial speed measurement point, frame2 is the frame number of the end speed measurement point, and fps is the video frame rate;
[0053] Finally, the vehicle speed is obtained by calculating the relationship between the actual displacement of the vehicle and time, and the formula is:
[0054] v t =S / t.
[0055] Among them, S is the actual displacement within a specific number of frames, t is the time it takes to pass through the speed measurement area, and v t is the calculated vehicle speed.
[0056] Compared with the prior art, the present invention has the following beneficial effects:
[0057] 1. Aiming at the problem of data ambiguity in materials collected from special scenes in coal mines, the present invention adopts the improved target detection algorithm YOLOv8-GS. By using the slim-neck design paradigm to transform the neck of the baseline model and adding the SE attention mechanism module, the computational complexity and inference time of the detector are reduced, and feature extraction is better completed. The problems of ambient light interference and diverse and complex targets are well solved, and the detection accuracy is improved.
[0058] 2. The present invention adopts a joint loss function to classify targets, which improves the ability to classify similar targets. Compared with the traditional DeepSort tracking algorithm, it effectively improves the ability of neural network to extract features, reduces the phenomenon of ID jumping to a certain extent, and improves the accuracy and stability of target tracking.
[0059] 3. The present invention calculates the vehicle speed in real time based on video, does not require the use of professional speed measuring equipment, and does not damage the road surface. It only needs to process the collected video images in real time to complete the detection of the vehicle's driving speed. Compared with the method of manually calibrating the speed measurement area to determine the vehicle's driving speed, the speed measurement method of the present invention can calculate the vehicle's driving speed efficiently and quickly. The method is feasible and effective, and has certain practical application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 It is a flowchart of the overall process of the present invention.
[0061] Figure 2 Schematic diagram of the structure of SE attention mechanism.
[0062] Figure 3 This is an example diagram of YOLOv8 detection results.
[0063] Figure 4 This is an example diagram of the detection results of the improved algorithm YOLOv8-GS.
[0064] Figure 5 This is a flow chart of speeding behavior detection.
[0065] Figure 6 It is an example diagram of vehicle speed detection results in a specific implementation manner. DETAILED DESCRIPTION
[0066] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0067] like Figure 1 As shown, a method for identifying overspeeding of a mining trackless rubber-tyred vehicle based on machine vision, the specific steps are as follows:
[0068] Step S1, image source data acquisition: using a camera installed in the material collection area, obtain the real-time video stream of the monitoring point of the trackless rubber-tyred vehicle transportation scene, intercept the video of the trackless rubber-tyred vehicle driving in multiple time periods, and then filter the intercepted video, and intercept the video frame by frame to obtain a data set for subsequent processing.
[0069] Furthermore, step S1 includes the following steps:
[0070] S11, installing the camera: the camera is installed at the oblique rear of the trackless rubber-tyred vehicle transport lane to facilitate the subsequent calculation of the vehicle speed, and is also installed directly above the trackless rubber-tyred vehicle transport lane to facilitate the calculation of the perspective transformation and pixel ratio between the two camera images mentioned in the following steps (see step S3 for details).
[0071] S12, intercepting the video stream for playback. Set the task of intercepting the video stream, and set the video stream storage parameters in advance: video saving format, video stream capture time period, video stream frame number, etc. According to the specific time of the trackless rubber-tyred vehicle transportation, capture the video stream of the trackless rubber-tyred vehicle transportation through the lane.
[0072] Step S2, data enhancement preprocessing: The data collected in the above step S1 is transformed and expanded to generate more and richer samples, which can improve the diversity of training samples, reduce the model's dependence on specific data distribution, and enhance the generalization ability and robustness of the model.
[0073] Data enhancement refers to the method of transforming and expanding limited data to generate more and richer samples. It is an effective means to improve the diversity of training samples, reduce the model's dependence on specific data distribution, and enhance the generalization ability and robustness of the model. The present invention performs data enhancement by flipping the image horizontally 90°, rotating it clockwise 15°, adding 5% noise, 25% grayscale and 40% brightness, etc., which can increase the diversity of the training data set and improve the robustness of the model.
[0074] Step S3, image perspective transformation and pixel ratio calculation: perform perspective transformation on the acquired data and calculate the ratio r of the pixel distance to the actual distance.
[0075] Furthermore, step S3 comprises the following steps:
[0076] Step S31, perspective transformation is a transformation algorithm that projects an image onto a new viewing plane. Its basic principle is to use the condition that the perspective center, image point, and target point are collinear, and according to the perspective rotation law, the image receiving surface (perspective surface) is rotated around the trace (perspective axis) by a certain angle, destroying the original projection light beam, while still maintaining the projection geometry on the image receiving surface unchanged. The essence of perspective transformation is to project the image onto a new viewing plane, and its general transformation formula is:
[0077]
[0078] Among them, (x, y, z) is the homogeneous coordinates of the original image pixel, and (X, Y, Z) is the homogeneous coordinates of the transformed image pixel. After normalizing Z, the normalized homogeneous coordinates (X′, Y′, 1) are obtained, that is, the two-dimensional plane coordinates of the pixel corresponding to the original image after transformation are:
[0079]
[0080] Therefore, given the four pairs of pixel coordinates corresponding to the perspective transformation, the perspective transformation matrix can be obtained; conversely, given the perspective transformation matrix, the perspective transformation of the image or pixel coordinates can be completed.
[0081] Step S32, calculation of the ratio of pixel distance to actual distance (hereinafter collectively referred to as pixel ratio r): place an object of known size under the monitoring point for obtaining source image data, and convert the image into a top view using the perspective transformation of step S31, then calculate the pixel distance using pixel points, and calculate the pixel ratio r based on the actual distance of the reference target and the calculated pixel distance.
[0082] The present invention uses an image with the same resolution of 1920×1080 to calculate the pixel ratio, wherein the pixel distance is calculated by the pixel coordinates, and the formula is: (x1, y1) and (x2, y2) are the pixel coordinates of the two points, respectively, and the actual distance is obtained by referring to the known size of the object. The pixel distance between the two points measured in the image is 50 pixels, and the actual distance is 2 meters, so the pixel ratio r of the present invention is 25.
[0083] Step S4, preparation of training data set: annotate and divide the data set processed by the above step S3, use LabelImg software to annotate the rubber-tired trackless vehicle on the image after data preprocessing, and annotate 2000 images in total, and name the category as "rubber-tired_vehicle". After annotation, a txt file with position coordinates and category name will be generated; the annotated data set is divided into training set, test set and validation set in proportion, wherein the training data accounts for 80% (1400 images) of the whole data set, the test set accounts for 20% (400 images) of the whole data set, and the validation set accounts for 10% (400 images) of the whole data set.
[0084] Step S5, model training: In order to solve the problems of complex calculation, large number of parameters, long reasoning time and difficult feature extraction of the existing YOLOv8 target detection model, the present invention adopts the improved YOLOv8-GS algorithm to realize the recognition of trackless rubber-wheeled vehicles. Although the YOLOv8 algorithm has made significant progress in detection accuracy and speed, when performing target detection on images of complex environments in coal mines, there are problems such as severe occlusion, much background interference, and poor image quality in the images. Therefore, it is necessary to further optimize the YOLOv8 algorithm to improve the overall performance of target detection.
[0085] The improvements to the YOLOv8 algorithm mainly involve the following two aspects: First, in terms of reducing the model's computational complexity, the complexity of the network structure, and enhancing network flexibility, the slim-neck design paradigm is used to transform the neck of the baseline model, and the GSBottleneck module is used to enhance the network's ability to process features. The GSConv module stacking is used to improve the model's learning ability, improve feature utilization efficiency, and network performance; then, the channel attention (SE) module is introduced to improve the efficiency and performance of the detector, and improve the model's ability to extract features between different channels in the entire image. Through the above optimization strategies, accurate identification of underground trackless rubber-wheeled vehicles is finally achieved, and the improved algorithm is named YOLOv8-GS. The comparison of the detection results of YOLOv8 and YOLOv8-GS is shown below. Figure 3 , Figure 4 shown.
[0086] Furthermore, step S5 comprises the following steps:
[0087] Step S51: Use the slim-neck design paradigm to transform the neck of the baseline model to reduce the computational complexity and inference time of the detector while maintaining accuracy.
[0088] Specifically, the lightweight convolution method GSConv is used to replace SC to speed up the prediction calculation of images in CNN. GSConv aims to keep the hidden connections between channels as much as possible while maintaining low time complexity. Then, based on GSConv, GSBottleneck and one-time aggregation methods are introduced to design the cross-level partial network (GSCSP) module VoV-GSCSP, which not only reduces the complexity of calculation and network structure, but also maintains sufficient accuracy. This architecture uses GSConv and VoV-GSCSP modules to build an efficient Neck. In this architecture, feature maps of different scales are first processed by GSConv, and then combined with feature maps of other scales through upsampling and splicing operations. The processed feature maps pass through the GSConv module again, and finally the VoV-GSCSP module is used to further extract and fuse features, and finally enter the Head for detection. Through this modular and layered approach, the slim-neck architecture can reduce computational complexity and inference time while maintaining high accuracy. The comparison results of the detection results after the transformation of the model neck are shown in Table 1:
[0089] Table 1 Comparison of test results after modification of the model neck
[0090]
[0091] Compared with the original YOLOv8 network, the improved network model (Yolov8-G) has an accuracy and recall rate of 0.8% and 0.7% respectively, and an average precision rate of 0.9%. At the same time, the detection speed is increased to 89.24FPS, and the calculation amount is reduced by 10.6G compared with the original YOLOv8 model. Finally, the model size is reduced by 5.9MB. The relevant data in Table 1 prove that the model after the neck of the baseline model is modified using the slim-neck design paradigm is efficient, lightweight and accurate.
[0092] Step S52, the present invention introduces an attention mechanism (SE) into the neck network to improve the model's ability to extract features between different channels in the entire image. The schematic diagram of the SE attention mechanism structure is as follows: Figure 2 shown.
[0093] This module performs pooling on each channel of the feature map to obtain a one-dimensional vector about the channel. The vector is then sent to two fully connected layers with different activation functions. In the second fully connected layer, the length of the output vector is the same as the length of the input one-dimensional vector. Finally, it is multiplied with the input feature map to obtain a feature map after the attention mechanism. Under the premise of ensuring that the general structure remains unchanged, the network's ability to share features between different channels is improved. The process is as follows Figure 3 The detection effect of adding the attention mechanism model is shown in Table 2:
[0094] Table 2 Comparison of detection results of adding SE attention mechanism model
[0095]
[0096] The results in Table 2 show that although the detection results of adding the SE attention mechanism are not as good as Yolov8-G in terms of computational complexity, detection speed and model size, the accuracy, recall rate and average precision have all been improved.
[0097] Step S53: In order to test the detection performance of the improved model of the present invention, the present invention uses accuracy, recall rate, mAP@0.5, model parameter scale, model calculation amount (FLOPs), model training time and detection speed (unit: frame / s) as evaluation indicators.
[0098] Precision is the ratio of the number of positive samples predicted by the model to the number of all detected samples, and is calculated as follows:
[0099]
[0100] In the formula, TP represents the number of positive samples predicted by the model, and FP represents the number of negative samples predicted by the model.
[0101] Recall is the ratio of the number of positive samples correctly predicted by the model to the number of positive samples that actually occurred. The recall calculation is as follows:
[0102]
[0103] In the formula, TP represents the number of positive samples predicted by the model, and FN represents the number of positive samples predicted incorrectly by the model.
[0104] The average precision (AP) is the area under the precision and recall curves. The mean average precision (mAP@0.5) is the result obtained by the weighted average of the AP values of all sample categories. It is used to evaluate the detection performance of the model in all categories. The threshold of the intersection ratio of the predicted box and the true box is set to 0.5. The formula is as follows:
[0105]
[0106] Where N is the sample category, AP represents the area under the precision and recall curves, and @0.5 means that the IoU threshold used when calculating the area under the precision and recall curves is 0.5.
[0107] Step S6: Implement multi-target tracking based on DeepSort tracking algorithm. When tracking targets, mutual occlusion between targets and changes in light in the environment may lead to tracking failure, which in turn affects the determination of vehicle speed. Therefore, it is particularly necessary to effectively utilize multi-view information to improve the accuracy and stability of target tracking. The present invention adopts a joint loss function to improve the classification performance of similar targets.
[0108] Step S61, when the softmax loss function is used for classification, different types of vehicles can basically be classified, but the classification performance for targets with high similarity is not very good. The softmax loss function is as follows:
[0109]
[0110] Among them, m is the number of samples, n is the number of categories, Is the discriminant X i The weight vector of the true label, Indicates the discrimination X i The weight vector of the class, X i Represents the feature vector of the i-th image, y i Represents X i The true label of , b is the bias term;
[0111] The center loss function is used to train the vehicle data. The calculation formula of the center loss function is as follows:
[0112]
[0113] Among them, m is the number of samples, xi represents the feature value of the i-th image, and yi is the output of the network feature. Represents the central value of the feature, the central value of the feature Varies with the characteristics of depth.
[0114] Therefore, the softmax loss function combined with the center loss function defines the final loss function L veh_L , the calculation formula is as follows:
[0115]
[0116] Among them, λ represents the fusion relationship between the two, L S represents the softmax loss function, L C represents the center loss function;
[0117] The present invention uses the fusion coefficient λ to balance the center loss and the softmax loss. C With L S Combined, the ability of neural network to extract features is effectively improved and the ID Switch phenomenon is reduced to a certain extent.
[0118] Step S62, CLEAR MOT (a standardized evaluation framework for evaluating the performance of multi-target tracking algorithms) indicators proposes two comprehensive indicators: multi-target tracking accuracy MOTA and multi-target tracking precision MOTP. These two indicators can measure the overall performance of the multi-target tracker.
[0119] Regarding the multi-target tracking accuracy (MOTA), MOTA is the ratio of correctly predicted samples to all samples except for false positives (FP), lost targets (FN), and abnormal ID switches (ID Switch). It measures the performance of the tracker in detecting targets and maintaining tracks. The calculation formula is as follows:
[0120]
[0121] Among them, Fn t is the number of missed targets in the tth frame, Fp t is the number of error detections in frame t, ID st is the number of ID Switches in frame t, GT t Represents the number of correctly labeled training data samples in the tth frame.
[0122] Regarding the multi-object tracking precision (MOTP), the ratio of the correctly predicted targets to the total number of predicted targets under the detection box overlap threshold measures the positioning accuracy of the detector. The calculation formula is as follows:
[0123]
[0124] Among them, d t,i represents the average metric distance between target i and the annotation box at time t, i.e., the matching error, c t Indicates the number of successful matches at time t; the larger the MOTP, the closer the tracking box is to the manually labeled box.
[0125] Step S63, comparative analysis of tracking results, the DeepSort algorithm and the improved DeepSort tracking algorithm are analyzed and compared, the detectors all use the detection algorithm optimized in step S5, wherein 5 videos are selected as data sets for tracking result comparison, and the comparison results are shown in Table 3 below:
[0126] Table 3 Comparison of tracking results
[0127]
[0128] It can be seen from Table 3 above that the MOTA and MOTP of the improved DeepSort algorithm of the present invention are higher than those of the original DeepSort algorithm, which proves that the tracking performance of the improved algorithm is better than that of the original DeepSort algorithm, and its tracking accuracy is improved, and stable tracking can be achieved in the transport lane scenario of the trackless rubber-tyred vehicle of the present invention.
[0129] Step S7, calculate the speed of the tracked target (rubber-tyred trackless vehicle): Based on the improved target detection algorithm YOLOv8-GS and DeepSort algorithm, complete the acquisition of the vehicle speed and speeding behavior detection. The speeding behavior detection flow chart is as follows: Figure 5 shown.
[0130] Step S71, real-time position acquisition of the vehicle: based on the tracking result, the center point position of the object in each frame is obtained, and the calculation formula is as follows:
[0131]
[0132] Where u represents the horizontal coordinate of the center of the vehicle detection frame; v represents the vertical coordinate of the center of the vehicle detection frame; w represents the aspect ratio of the vehicle detection frame; h represents the height of the vehicle detection frame; (x k ,y k ) represents the final calculated vehicle center of mass.
[0133] It can be obtained that the vehicle center of mass trajectory set is as follows:
[0134]
[0135] in, It represents the center of mass coordinates of the i-th car in the k-th frame, and frames is the total number of frames.
[0136] S72, determination of vehicle speed: using the pixel distance of the vehicle movement through the pixel ratio r calculated in step S32 to obtain the actual displacement of the vehicle, and then dividing the frame number difference between the initial and end positions of the vehicle speed measurement by the video frame rate to estimate the average speed of the vehicle passing through the speed measurement area. Figure 6 The example diagram of the vehicle speed detection result is shown. The label in the upper left corner indicates the vehicle speed. The specific calculation steps of the vehicle speed are as follows:
[0137] The actual displacement S of the vehicle is obtained by the pixel ratio r calculated in the above step S32 and the pixel distance d moved by the vehicle. The calculation formula is as follows:
[0138]
[0139] Where d is the pixel distance the vehicle moves, r is the ratio of the pixel distance to the actual distance, represents the center of mass coordinates of the i-th car in the k-th frame, Represents the center of mass coordinates of the i-th car in the k+n-th frame.
[0140] The time t when the vehicle passes through the speed measurement area is calculated by the frame difference and the video frame rate. The formula is as follows:
[0141]
[0142] Among them, frame1 is the frame number of the initial speed measurement point, frame2 is the frame number of the end speed measurement point, and fps is the video frame rate.
[0143] Finally, the vehicle speed is obtained by calculating the relationship between the actual displacement of the vehicle and time, and the formula is:
[0144] v t =S / t
[0145] Among them, S is the actual displacement within a specific number of frames, t is the time it takes to pass through the speed measurement area, and v t is the calculated vehicle speed.
[0146] The present invention calculates the vehicle speed in real time based on video, does not require the use of professional speed measuring equipment and does not damage the road surface. It only needs to process the collected video images in real time to complete the detection of the vehicle's driving speed. Compared with the method of manually calibrating the speed measurement area to determine the vehicle's driving speed, the speed measurement method of the present invention can calculate the vehicle's driving speed efficiently and quickly. The method is feasible and effective, and has certain practical application value.
[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims
1. A method for identifying speeding of mining trackless rubber-tyred vehicles based on machine vision. It is characterized in that The specific steps are as follows: Step 1: Image source data acquisition: Use existing cameras in actual coal mine application scenarios to obtain real-time video streams from monitoring points in the trackless rubber-tyred vehicle transportation scene, intercept videos of the trackless rubber-tyred vehicle driving in multiple time periods, and then filter the intercepted videos, intercept the videos frame by frame, and obtain data sets for subsequent processing; Step 2: Data enhancement preprocessing: transform and expand the data collected in step 1; Step 3: Image perspective transformation and pixel ratio calculation: Perform perspective transformation on the acquired data and calculate the ratio r of pixel distance to actual distance; Step 4: Preparation of training data set: Label and divide the data set processed in step 3, use LabelImg software to label the rubber-tyred trackless vehicles on the images after data preprocessing, and generate a txt file with location coordinates and category names after labeling. The labeled data set is divided into training set, test set and validation set in a ratio of 7:2:1; Step 5: Model training: Use the improved YOLOv8-GS algorithm to realize the recognition of trackless rubber-wheeled vehicles; Step 6: Realize multi-target tracking based on DeepSort tracking algorithm; Step 7: Calculate the speed of the target to be tracked. Based on the YOLOv8-GS improved algorithm and the DeepSort tracking algorithm, complete the acquisition of vehicle speed and speeding behavior detection.
2. According to the method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision according to claim 1, It is characterized in that In step three, the specific steps are as follows: Step S31: The specific perspective transformation formula is: Among them, (x, y, z) is the homogeneous coordinates of the original image pixel, (X, Y, Z) is the homogeneous coordinates of the transformed image pixel, and after normalizing Z, the normalized homogeneous coordinates (X′, Y′, 1) are obtained, that is, the two-dimensional plane coordinates of the pixel corresponding to the original image after the transformation are: Step S32, take an object of known size and place it under the monitoring point for obtaining source image data, and convert the image into a top view using the perspective transformation in step S31, calculate the pixel distance using pixel points, and calculate the pixel ratio r based on the actual distance of the reference target and the calculated pixel distance.
3. According to the method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision as described in claim 2, It is characterized in that In step five, the specific training steps are: Step S51, modifying the neck of the baseline model; Step S52, introducing an attention mechanism into the neck network; Step S53, the present invention uses accuracy, recall rate, mAP@0.5, model parameter scale, model calculation amount, model training time and detection speed as evaluation indicators.
4. According to the method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision as described in claim 3, It is characterized in that In step six, the specific steps are as follows: Step S61: When the softmax loss function is used for classification, the softmax loss function is: Among them, m is the number of samples, n is the number of categories, Is the discriminant X i The weight vector of the true label, Indicates the discrimination X i The weight vector of the class, X i Represents the feature vector of the i-th image, y i Represents X i The true label of , b is the bias term; The center loss function is used to train the vehicle data. The calculation formula of the center loss function is as follows: Where m is the number of samples, x i Represents the eigenvalue of the i-th image, y i is the output of the network features, Represents the central value of the feature, the central value of the feature Variations with depth characteristics; Therefore, the softmax loss function combined with the center loss function defines the final loss function L veh_L , the calculation formula is as follows: Among them, λ represents the fusion relationship between the two, L S represents the softmax loss function, L C represents the center loss function; Step S62, the calculation formula of multi-target tracking accuracy MOTA is as follows: Among them, Fn t is the number of missed targets in the tth frame, Fp t is the number of error detections in frame t, ID st is the number of IDSwitches in frame t, GT t Indicates the number of correctly labeled training data samples in the tth frame; The multi-target tracking accuracy MOTP is the ratio of the correctly predicted targets to the total number of predicted targets under the detection frame overlap threshold. The specific calculation formula is as follows: Among them, d t,i represents the average metric distance between target i and the annotation box at time t, i.e., the matching error, c t Indicates the number of successful matches at time t.
5. According to the method for identifying overspeed of a mining trackless rubber-tyred vehicle based on machine vision as described in claim 4, It is characterized in that In step seven, the specific steps are: Step S71, real-time position acquisition of the vehicle: based on the tracking result, the center point position of the object in each frame is obtained, and the calculation formula is as follows: Where u represents the horizontal coordinate of the center of the vehicle detection frame; v represents the vertical coordinate of the center of the vehicle detection frame; w represents the aspect ratio of the vehicle detection frame; h represents the height of the vehicle detection frame; (x k ,y k ) represents the final calculated vehicle center of mass; It can be obtained that the vehicle center of mass trajectory set is as follows: in, represents the coordinates of the center of mass of the i-th car in the k-th frame, and frames is the total number of frames; S72, determination of vehicle speed: using the pixel distance of the vehicle movement through the pixel ratio r calculated in step S32 to obtain the actual displacement of the vehicle, and then using the frame difference between the initial and end positions of the vehicle speed measurement divided by the video frame rate to estimate the average speed of the vehicle passing through the speed measurement area. The specific calculation steps are as follows: The actual displacement S of the vehicle is obtained by the pixel ratio r calculated in the above step S32 and the pixel distance d moved by the vehicle. The calculation formula is as follows: Where d is the pixel distance the vehicle moves, r is the ratio of the pixel distance to the actual distance, represents the center of mass coordinates of the i-th car in the k-th frame, represents the center of mass coordinates of the i-th car in the k+n-th frame; The time t when the vehicle passes through the speed measurement area is calculated by the frame difference and the video frame rate. The formula is as follows: Among them, frame 1 is the frame number of the initial speed measurement point, frame 2 is the number of frames at the end of the speed measurement point, and fps is the video frame rate; Finally, the vehicle speed is obtained by calculating the relationship between the actual displacement of the vehicle and time, and the formula is: v t =S / t Among them, S is the actual displacement within a specific number of frames, t is the time it takes to pass through the speed measurement area, and v t is the calculated vehicle speed.
Citation Information
Cited By
Trackless rubber-tyred vehicle safety guarantee system and method based on Internet of Things
CN121981521A
