Automatic tracking and auditing system for whole-process operation of muck truck
By integrating LPR license plate recognition, GPS positioning and waste loading audit modules on the waste truck, combined with cloud computing and deep learning models, the problems of poor environmental adaptability and low efficiency in waste truck operation management are solved, real-time Monitor and efficient management.
Patent Information
- Application Number
- CN202411011470.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing automatic tracking and audit system for the full-process operation of dump trucks has problems such as poor environmental adaptability, delayed and unstable data transmission, high safety and environmental protection inspection costs, and low work efficiency.
Using LPR license plate recognition module, GPS global satellite positioning module, slag loading audit module and cloud computing analysis module, data processing and analysis are carried out through radar sensors, cameras and 5G network communication, combined with deep learning models, and loading and unloading operation analysis report is generated And detect abnormal situations.
Real-time monitoring and accurate recording of dump truck operations is realized, operation transparency and safety is improved, resource management is optimized, human errors and labor costs are reduced, and social and economic management benefits are significantly improved.
Smart Images

Figure CN118966933A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of muck truck operation management, and in particular to an automatic tracking and auditing system for the entire operation of a muck truck. Background Art
[0002] With the acceleration of urbanization, the management of dump trucks has become an important part of urban environmental governance. In order to effectively manage dump trucks, urban management departments usually require all dump trucks to be registered before operation, and install GPS equipment on each dump truck to report the vehicle location in real time, so that the regulatory authorities can monitor the entire process. However, the traditional management method of dump trucks has many problems, such as dump trucks not being fully loaded, dumping the dump trucks at places other than designated locations, etc., and because dump trucks usually operate at night, supervisors often need to accompany the trucks for inspection at night, resulting in high manual management costs and low regulatory efficiency.
[0003] In order to improve the efficiency and accuracy of dump truck management, advanced artificial intelligence and automation technologies are currently used to effectively reduce labor costs, improve supervision effects, and realize full-process automatic tracking and management of dump trucks.
[0004] The existing technology has the following shortcomings: At present, the automatic tracking and auditing system for the entire operation of muck trucks mainly relies on high-end technology and equipment, including the use of LPR license plate recognition technology, GPS global positioning system combined with electronic fence technology, and in-depth analysis and real-time tracking of the operation data of muck trucks through wireless communication and cloud computing technology. Although the tracking of muck trucks throughout the operation has improved management efficiency, there is a lack of auditing of the muck truck's operation volume, and there are problems such as poor environmental adaptability, delayed and unstable data transmission, high management costs for safety and environmental protection testing, and low work efficiency. Summary of the invention
[0005] The purpose of the present invention is to provide an automatic tracking and auditing system for the entire operation of a slag truck. The present invention solves the problems in the above-mentioned background technology by setting an LPR license plate recognition module, a slag loading auditing module and a cloud computing analysis module.
[0006] In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: an automatic tracking and auditing system for the whole operation of a muck truck, comprising a plurality of muck trucks, a radar sensor, a CPU central processing unit, a camera, an indicator light and a monitoring terminal device, as well as 5G network communication using virtualization technology, a cloud server and an electronic fence of an unloading point;
[0007] It also includes an LPR license plate recognition module: using a camera device to collect license plate image data, using a license plate recognition algorithm to process the license plate image data, extracting license plate character information to perform license plate detection and recognition, generating vehicle identity recognition information, and transmitting the vehicle identity recognition information to the cloud computing analysis module through a mobile network transparent transmission module;
[0008] GPS global satellite positioning module: A GPS device is installed on each muck truck to locate and record the vehicle's driving trajectory and position changes in real time, obtain the muck truck's GPS data, and use the GPS positioning algorithm combined with the Kalman filter algorithm to process the GPS data to generate vehicle operation signals, which are then transmitted to the cloud computing analysis module through the mobile network transmission module;
[0009] Muck loading audit module: Use cameras and radar sensors to collect, monitor and record the loading images of the muck truck and the location and shape data of the loaded muck, establish a volume model of the muck in the car, estimate the loading weight of the muck truck, generate operation data, and make the operation data of the muck truck within the preset standard range to prevent overloading, underloading or non-compliant operation;
[0010] Mobile network transparent transmission module: using the MQTT transmission protocol of the mobile communication network, it realizes the real-time data transmission between the LPR license plate recognition module, the GPS global satellite positioning module, the muck loading audit module and the cloud computing analysis module, ensuring the network communication between the GPS, radar sensor, camera, CPU device on the muck truck and the cloud server, and between the cloud server and the monitoring terminal;
[0011] Cloud computing analysis module: The cloud server device receives the vehicle identification information in the LPR license plate recognition module, the vehicle operation signal in the GPS global satellite positioning module, and the operation data in the muck loading audit module through the mobile network transmission module, and uses the deep learning model to store, process and analyze the data, generate an analysis report on the loading and unloading operation of the muck truck, and generate alarm information based on abnormal conditions detected in the loading and unloading operation analysis report of the muck truck, and then transmit the loading and unloading operation analysis report and alarm information of the muck truck to the AI recognition feedback module of the supervision terminal;
[0012] Supervision terminal AI recognition and feedback module: Utilizing the user interaction interface provided by the supervision terminal device, real-time AI recognition displays the storage data, loading and unloading operation analysis reports and alarm information of the cloud computing analysis module, supporting supervisors to conduct real-time monitoring queries and decision-making feedback.
[0013] Optionally, the steps of processing image data by the license plate recognition algorithm are as follows:
[0014] The license plate image data is collected by a camera and calibrated as Lpid. The license plate image data Lpid is preprocessed by grayscale, noise reduction, and histogram equalization to obtain the preprocessed image information, which is calibrated as Pid. The grayscale preprocessing calculation formula is I(Lpid)=0.299×I R +0.587×I G +0.114×I B , where I(Lpid) represents the preprocessed image after grayscale conversion, I R ,I G ,I B They represent the pixel values of the red, green and blue channels corresponding to the original color image, and the pixel value range is 0-255. 0.299, 0.587 and 0.114 represent the weighting coefficients determined according to the red, green and blue sensitivities corresponding to the human eye, respectively.
[0015] The preprocessing calculation formula for noise reduction is N(Lpid)=I(Lpid)×G(x,y), and Where N(Lpid) represents the preprocessed image after denoising, G(x,y) represents the Gaussian kernel function for smoothing the image, σ represents the standard deviation of the Gaussian kernel, x and y represent pixel coordinates, and e represents an exponential function;
[0016] The preprocessing calculation formula for histogram equalization is: And k∈[1,K],K=256, where G(Lpid) k It is represented by the pixel value of gray level k in the preprocessed image after histogram equalization. K and k represent the total number of gray levels and the kth gray level respectively. M N(Lpid) It is expressed as the total number of pixels of the pre-processed image N(Lpid) after denoising, N(Lpid) k It is expressed as the number of pixels with gray level k in the pre-processed image N (Lpid) after denoising, that is, the pre-processed image information of grayscale, denoising and histogram equalization is calibrated as Pid;
[0017] Use the license plate recognition algorithm to perform license plate area positioning, character segmentation, character recognition and information extraction on the pre-processed image information Pid to obtain the complete license plate number and mark it as Ipn;
[0018] The license plate recognition algorithm includes the Sobel operator edge detection algorithm, morphological operation algorithm, vertical projection method, and convolutional neural network (CNN) model to achieve the reprocessing of license plate area positioning, character segmentation, character recognition, and information extraction. The calculation formula of the Sobel operator edge detection algorithm is: and In the formula, Gx (Pid)It is represented as the gradient of the preprocessed image information Pid in the x direction, Gy (Pid) It is represented as the gradient of the preprocessed image information Pid in the y direction. It is represented as a 3×3 matrix of the convolution kernel smoothed on the positive half axis of the image information Pid on the x-axis. It is represented as a 3×3 matrix of the convolution kernel with the image information Pid smoothed on the positive half axis of the y-axis, G′ (Pid) It is expressed as the gradient amplitude of the preprocessed image information Pid, that is, the edge strength of the image information Pid, and the image after edge detection is calibrated as eg G′(Pid) ;
[0019] The calculation formula of the morphological operation algorithm is: ed (Pid) =dl (Pid) ΘB, where dl (Pid) Represented as after edge detection eg G′(Pid) The image after the expansion operation, eg G′(Pid) Represented as the image after edge detection, represents the XOR operator, B represents the structural element, ed (Pid) Represented as after edge detection eg G′(Pid) The image after the corrosion closing operation, Θ represents the inverse operation operator of XOR, then the image after edge detection eg G′(Pid) After morphological processing, the license plate area positioning image is obtained and calibrated as BL (Pid) ;
[0020] The calculation formula of the vertical projection method is Pvd = BL (Pid) ×Pid (x,y) , where Pvd represents the license plate area positioning image BL (Pid) The vertical projection value, BL (Pid) Represents the license plate area positioning image, Pid (x,y) The pixel coordinates of the preprocessed image information Pid are expressed as follows. Character segmentation is performed based on the peak and valley values of the projection value Pvd, and the character image is calibrated as CS (Pid) ;
[0021] The calculation formula of the convolutional neural network CNN model is Ycs = W0 × CS (Pid) +b0, where Ycs represents the probability distribution of the recognized characters output by the trained CNN model, W0 represents the weight of the fully connected layer in the CNN model, and b0 represents the bias of the fully connected layer in the CNN model. The characters segmented from the character image are re-recognized using the pre-trained CNN model, and the character features are extracted through the convolutional layer, pooling layer and fully connected layer. The probability distribution of the characters Ycs is output, and the recognized characters are determined and combined to obtain the license plate number Ipn.
[0022] Optionally, the process of obtaining the vehicle identification information is as follows:
[0023] The license plate number Ipn obtained after processing the image data according to the license plate recognition algorithm is affixed with a timestamp and a position, which are calibrated as Ts and Ps respectively, to generate vehicle identification information, which is calibrated as VIN, then VIN = {Ipn, Ts, Ps};
[0024] The generated vehicle identification information VIN is transmitted to the cloud server in real time using the MQTT protocol.
[0025] Optionally, the steps of generating the vehicle running signal are as follows:
[0026] Use the vehicle-mounted GPS device to collect and record the GPS data of the muck truck in real time and calibrate it as GPS d ;
[0027] According to the GPS data of the muck truck d , and determine that n number of satellite signals have been received, and use the triangulation method in the GPS positioning algorithm to calculate the three-dimensional coordinates of the current position of the muck truck. The calculation formula of the triangulation method is: And c 0 =3×10 8 m / s, where (x 0 ,y 0 、z 0 ) represents the three-dimensional coordinates of the current position of the muck truck, Expressed as the position coordinates corresponding to n satellites, They are respectively represented by the time when the satellite signals received by n satellites arrive at the receiver from the corresponding satellites, c 0 Expressed as the speed of light in meters per second, that is, the GPS data of the muck truck GPS d =(x 0 ,y 0 、z 0 );
[0028] Using Kalman filter algorithm to analyze GPS data d Smoothing and optimization, including measuring the initialization state, prediction and updating of GPS data, to generate vehicle operation signals, calibrated as VOS, used to reduce the noise and error of GPS data and improve GPS positioning accuracy. The calculation formula of the Kalman filter algorithm is H(GPS d ) t =F(GPS d )×H(GPS d ) t-1 +B(GPS d)×U(GPS d ) t 、P(GPS d ) t =F(GPS d )×P(GPS d ) t-1 ×F(GPS d ) T +Q(GPS d ), and K(GPS d ) t =P(GPS d ) t ×C(GPS d ) T ×(C(GPS d )×P(GPS d ) t ×C(GPS d ) T +R(GPS d )) -1 , and H(GPS d ) t+1 =H(GPS d ) t +K(GPS d ) t ×(Z(GPS d ) t -C(GPS d )×H(GPS d ) t )、P(GPS d ) t+1 =(IK(GPS d ) t ×C(GPS d ))×P(GPS d ) t , where H(GPS d )、P(GPS d ) represent the measured GPS data GPS d The estimated vector and covariance matrix of the initial state are set, H(GPS d ) t Represented as measured GPS data GPS d The state vector predicted at the current time step t, F(GPS d ) indicates the measurement of GPS data GPS d The state transfer matrix, H(GPS d ) t-1 Represented as measured GPS data GPS dThe state vector predicted at the previous time step t-1, B(GPS d ) indicates the measurement of GPS data GPS d The control input matrix, U(GPS d ) t Represented as measured GPS data GPS d At the current time step t, the control input, P(GPS d ) t Represented as measured GPS data GPS d The covariance matrix predicted at the current time step t, P(GPS d ) t-1 Represented as measured GPS data GPS d The covariance matrix predicted at the previous time step t-1, T represents the transpose, Q(GPS d ) indicates the measurement of GPS data GPS d The process noise covariance matrix, K(GPS d ) t Represented as measured GPS data GPS d The Kalman gain at the current time step t, C(GPS d ) indicates the measurement of GPS data GPS d The measurement matrix, R(GPS d ) indicates the measurement of GPS data GPS d The measurement noise covariance matrix, H(GPS d ) t+1 Represented as measured GPS data GPS d The predicted state vector after updating at the next time step t+1, P(GPS d ) t+1 Represented as measured GPS data GPS d The predicted covariance matrix is updated at the next time step t+1, and I is represented as the unit matrix. Then, according to the prediction update of the Kalman filter algorithm, the output H (GPS d ) t+1 As the vehicle operation signal VOS, that is, VOS = H (GPS d ) t+1 ;
[0029] The vehicle operation signal VOS is transmitted to the cloud server via the mobile network for storage and analysis.
[0030] Optionally, the steps of establishing the volume model of the slag in the carriage are as follows:
[0031] The image data of the muck loaded in the muck carriage and the position and shape data of the muck are collected by using a camera and a radar sensor, which are calibrated as Sid and Ssd respectively;
[0032] The Canny edge detection algorithm is used to pre-process the image data Sid for the edge detection of the slag, and the contour information of the slag is extracted by combining the position shape data Ssd of the slag. The calculation formula of the Canny edge detection algorithm is E Sid (x,y)=Canny(Sid(x,y)), where E Sid (x, y) represents the binary image after edge detection of the image data Sid, Canny represents the Canny edge detection function, including Gaussian filtering and gradient calculation, Sid(x, y) represents the grayscale image after processing the image data Sid at pixel (x, y);
[0033] The calculation formula for extracting the contour information of the muck is: In the formula, OI represents the contour information of the extracted slag, Ssd represents the location shape data of the slag, and is composed of a series of boundary points;
[0034] The 3D model of the muck in the carriage is reconstructed using multi-view images, and the 3D point cloud data of the muck surface is obtained using structured light scanning technology. The reconstruction calculation formula of the 3D model is: And 3D pcd =OI∩Dv, where Dv is the depth value of the 3D point, bl is the baseline length of the camera light scan, cf is the focal length of the camera, and pv is the parallax value. pcd It is represented as three-dimensional point cloud data, ∩ represents the intersection, and OI∩Dv represents the point cloud data set composed of the common elements of the contour information of the muck and the depth value of the three-dimensional point;
[0035] According to the known length, width and height of the muck car, they are calibrated as Lrc, Wrc and Hrc respectively, and the volume of the reconstructed 3D point cloud data is calculated. The volume calculation formula is Vms = ∑3D pcd ×Lrc×Wrc×(Hrc-Dv), where Vms is the volume of the muck in the muck car, ∑3D pcd It is represented by the sum of the volume values corresponding to each point data in the point cloud dataset, Lrc×Wrc is represented by the bottom area of the slag, and (Hrc-Dv) is represented by the height value of the slag surface in the point cloud dataset;
[0036] The weight of the slag is estimated by using its volume and density. The weight calculation formula is Wms = ρms × Vms, where Wms is the weight of the slag, ρms is the density of the slag, and Vms is the volume of the slag.
[0037] The workload data is generated based on the weight of the slag Wms, and the preset weight standard range is [W min, W max ], to determine whether the operation volume of the muck truck is compliant, the calculation formula for generating the corresponding operation volume data is Hvd = Wms ± 2% × Wms, where Hvd represents the operation volume data, ± 2% × Wms represents the absolute error value affected by the density of the muck or the transportation factor. When (Hvd = 102% Wms) < W min When (Hvd=98%Wms)>W min When W is greater than the preset weight standard, it is judged that the operation volume of the muck truck exceeds the preset weight standard range, and the muck truck has a large vehicle wear and tear and a high transportation risk; when W min ≤Hvd≤W max , it is judged that the operating volume of the muck truck meets the preset weight standard range.
[0038] Optionally, the data processing steps of the deep learning model are as follows:
[0039] Receive the vehicle identification information VIN from the LPR license plate recognition module, the vehicle operation signal VOS from the GPS global satellite positioning module, and the operation volume data Hvd from the muck loading volume audit module as characteristic data of the muck truck's loading and unloading operations;
[0040] The feature weighted fusion algorithm is used to integrate the feature data to obtain feature fusion data, which provides comprehensive information for the deep learning model. The calculation formula of the feature fusion data is F = α × VIN + β × VOS + χ × Hvd, where F represents the feature fusion data obtained by fusing the vehicle identity information VIN, the vehicle operation signal VOS and the workload data Hvd, and α, β, and χ represent the weight coefficients of the corresponding vehicle identity information VIN, the vehicle operation signal VOS and the workload data Hvd features, respectively.
[0041] The convolutional neural network (CNN) algorithm in the deep learning model is used to train and predict the feature fusion data F to generate a loading and unloading operation analysis report. The calculation formula of the convolutional layer operation of CNN is: And i∈[1,k], j∈[1,k], where Y F represents the output value of the convolution operation, k represents the size of the convolution kernel, F{x+i-1,y+j-1} represents the input data on the convolution layer, k{i,j} represents the weight of the convolution kernel, and b F It is represented as the bias term of the convolution layer, i and j represent the position and channel in the convolution kernel respectively, and x and y represent the position of the feature fusion data F respectively;
[0042] The maximum pooling calculation formula of the pooling layer is Where Y F′ represents the output value of the pooling layer, and Max represents the maximum value function;
[0043] The calculation formula of the fully connected layer is Y F =W F ×Y F ′+b Y , where Y F ″ represents the output value of the fully connected layer, W F Expressed as the corresponding output value Y F ′’s weight matrix, b Y Represented as the bias term of the fully connected layer;
[0044] The loss function is calculated as In the formula, loss is the mean square error loss, n and N are the nth sample and the total number of samples, respectively, and F and Y are F ″ represents the feature fusion data as the actual value and the predicted value output by the fully connected layer of the CNN model;
[0045] The calculation formula for gradient update is Where W F ′ represents the updated weight value, λ represents the learning rate, Expressed as the gradient of the loss function with respect to the weight;
[0046] The prediction formula of CNN after training is Y F ″′=W F ′×F+b′ Y ,and Where Y F ″′ represents the predicted output value after CNN model training, b′ Y Represents the updated bias value after CNN model training, It is expressed as the gradient of the loss function with respect to the bias.
[0047] Optionally, the steps for generating the loading and unloading operation analysis report and warning information are as follows:
[0048] The trained CNN model is used to predict the newly input feature fusion data and generate a loading and unloading operation analysis report;
[0049] Perform anomaly detection on the generated loading and unloading operation analysis report to identify abnormal situations such as overloading, underloading, and irregular operations;
[0050] Set anomaly detection thresholds based on historical data and business requirements;
[0051] Compare real-time data with set thresholds to generate anomaly detection results and determine whether there is an anomaly;
[0052] Based on the abnormal detection results, alarm information including alarm type, severity, and recommended measures is generated, and then the alarm information is formatted into a standard report and transmitted to the supervision terminal.
[0053] Optionally, the method steps for automatically tracking and auditing the entire operation of the muck truck are as follows:
[0054] S1. Radar sensors, cameras and parking / passing signal indicators are installed at the exit of the muck truck loading station. GPS locators are installed on the muck trucks to collect data information such as license plate images, location and loading images in real time;
[0055] S2, using MQTT transmission protocol and 5G network communication technology to transmit the collected data information to the central server and cloud server for processing, analysis and storage;
[0056] S3. Setting a virtualized electronic fence of the unloading point where the muck truck is working on the cloud server, and setting an indicator light for controlling the muck truck on the electronic fence of the unloading point to record the working status of the muck truck;
[0057] S4. Estimated audit analysis of the loading volume of the muck truck is performed, and the integration, analysis and prediction of comprehensive data are processed through the cloud server to identify the loading behavior pattern of the muck truck and detect abnormal situations;
[0058] S5. Use the deep learning model to process the fusion data, generate the job analysis report and alarm signal, and push it to the supervisor for system feedback maintenance and optimization.
[0059] In the above technical solution, the technical effects and advantages provided by the present invention are:
[0060] The present invention arranges an LPR license plate recognition module, a GPS global satellite positioning module, a slag loading audit module and a cloud computing analysis module, that is, a radar sensing device, a camera and a parking / passing signal indicator light are arranged at the exit of the slag truck loading place, and a GPS locator is installed on the slag truck, so as to realize the automatic unmanned tracking management function during the operation of the slag truck, and further ensure that the whole operation of the slag truck can be monitored in real time, accurately recorded, and effectively audited and analyzed. It not only improves the transparency and safety of the slag truck operation, but also optimizes resource management and scheduling, reduces human errors and labor costs, and brings significant social and economic management benefits to the slag transportation industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0062] Figure 1 It is a module schematic diagram of the automatic tracking and auditing system for the whole operation of a muck truck of the present invention.
[0063] Figure 2 This is a diagram for implementing the equipment of the automatic tracking and auditing system for the entire operation of a muck truck according to the present invention.
[0064] Figure 3 The present invention is a method flow chart of the automatic tracking and auditing system for the entire operation of a muck truck. DETAILED DESCRIPTION
[0065] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.
[0066] The present invention provides Figure 1 The system for automatically tracking and auditing the whole operation of a muck truck shown in the figure includes a number of muck trucks, radar sensors, CPU central processing units, cameras, indicator lights and monitoring terminal devices, as well as 5G network communications using virtualization technology, cloud servers and electronic fences at unloading points;
[0067] It also includes an LPR license plate recognition module: using a camera device to collect license plate image data, using a license plate recognition algorithm to process the license plate image data, extracting license plate character information to perform license plate detection and recognition, generating vehicle identity recognition information, and transmitting the vehicle identity recognition information to the cloud computing analysis module through a mobile network transparent transmission module;
[0068] Specifically, the steps for the license plate recognition algorithm to process image data are as follows:
[0069] The license plate image data is collected by a camera and calibrated as Lpid. The license plate image data Lpid is preprocessed by grayscale, noise reduction, and histogram equalization to obtain the preprocessed image information, which is calibrated as Pid. The grayscale preprocessing calculation formula is I(Lpid)=0.299×I R +0.587×I G +0.114×I B , where I(Lpid) represents the preprocessed image after grayscale conversion, IR ,I G ,I B They are respectively represented as the pixel values of the red, green and blue channels corresponding to the original color image, and the pixel value range is 0-255, and 0.299, 0.587 and 0.114 are respectively represented as weighting coefficients determined according to the red, green and blue sensitivities corresponding to the human eye;
[0070] The preprocessing calculation formula for noise reduction is N(Lpid)=I(Lpid)×G(x,y), and Where N(Lpid) represents the preprocessed image after denoising, G(x,y) represents the Gaussian kernel function for smoothing the image, σ represents the standard deviation of the Gaussian kernel, x and y represent pixel coordinates, and e represents an exponential function;
[0071] The preprocessing calculation formula for histogram equalization is: And k∈[1,K],K=256, where G(Lpid) k It is represented by the pixel value of gray level k in the preprocessed image after histogram equalization. K and k represent the total number of gray levels and the kth gray level respectively. M N(Lpid) It is expressed as the total number of pixels of the pre-processed image N(Lpid) after denoising, N(Lpid) k It is expressed as the number of pixels with gray level k in the pre-processed image N (Lpid) after denoising, that is, the pre-processed image information of grayscale, denoising and histogram equalization is calibrated as Pid;
[0072] Use the license plate recognition algorithm to perform license plate area positioning, character segmentation, character recognition and information extraction on the pre-processed image information Pid to obtain the complete license plate number and mark it as Ipn;
[0073] The license plate recognition algorithm includes the Sobel operator edge detection algorithm, morphological operation algorithm, vertical projection method, and convolutional neural network (CNN) model to achieve the reprocessing of license plate area positioning, character segmentation, character recognition, and information extraction. The calculation formula of the Sobel operator edge detection algorithm is: and In the formula, Gx (Pid) It is represented as the gradient of the preprocessed image information Pid in the x direction, Gy (Pid) It is represented as the gradient of the preprocessed image information Pid in the y direction. It is represented as a 3×3 matrix of the convolution kernel smoothed on the positive half axis of the image information Pid on the x-axis. It is represented as a 3×3 matrix of the convolution kernel with the image information Pid smoothed on the positive half axis of the y-axis, G′ (Pid)It is expressed as the gradient amplitude of the preprocessed image information Pid, that is, the edge strength of the image information Pid, and the image after edge detection is calibrated as eg G′(Pid) ;
[0074] The calculation formula of the morphological operation algorithm is: ed (Pid) =dl (Pid) ΘB, where dl (Pid) Represented as after edge detection eg G′(Pid) The image after the expansion operation, eg G′(Pid) Represented as the image after edge detection, represents the XOR operator, B represents the structural element, ed (Pid) Represented as after edge detection eg G′(Pid) The image after the corrosion closing operation, Θ represents the inverse operation operator of XOR, then the image after edge detection eg G′(Pid) After morphological processing, the license plate area positioning image is obtained and calibrated as BL (Pid) ;
[0075] The calculation formula of the vertical projection method is Pvd = BL (Pid) ×Pid (x,y) , where Pvd represents the license plate area positioning image BL (Pid) The vertical projection value, BL (Pid) Represents the license plate area positioning image, Pid (x,y) The pixel coordinates of the preprocessed image information Pid are expressed as follows. Character segmentation is performed based on the peak and valley values of the projection value Pvd, and the character image is calibrated as CS (Pid) ;
[0076] The calculation formula of the convolutional neural network CNN model is Ycs = W0 × CS (Pid) +b0, where Ycs represents the probability distribution of the recognized characters output by the trained CNN model, W0 represents the weight of the fully connected layer in the CNN model, and b0 represents the bias of the fully connected layer in the CNN model. The characters segmented from the character image are re-recognized using the pre-trained CNN model, and the character features are extracted through the convolutional layer, pooling layer and fully connected layer. The probability distribution of the characters Ycs is output, and the recognized characters are determined and combined to obtain the license plate number Ipn.
[0077] Specifically, the process of obtaining vehicle identification information is as follows:
[0078] The license plate number Ipn obtained after processing the image data according to the license plate recognition algorithm is affixed with a timestamp and a position, which are calibrated as Ts and Ps respectively, to generate vehicle identification information, which is calibrated as VIN, then VIN = {Ipn, Ts, Ps};
[0079] The generated vehicle identification information VIN is transmitted to the cloud server in real time using the MQTT protocol.
[0080] GPS global satellite positioning module: A GPS device is installed on each muck truck to locate and record the vehicle's driving trajectory and position changes in real time, obtain the muck truck's GPS data, and use the GPS positioning algorithm combined with the Kalman filter algorithm to process the GPS data to generate vehicle operation signals, which are then transmitted to the cloud computing analysis module through the mobile network transmission module;
[0081] Specifically, the steps for generating the vehicle operation signal are as follows:
[0082] Use the vehicle-mounted GPS device to collect and record the GPS data of the muck truck in real time and calibrate it as GPS d ;
[0083] According to the GPS data of the muck truck d , and determine that n number of satellite signals have been received, and use the triangulation method in the GPS positioning algorithm to calculate the three-dimensional coordinates of the current position of the muck truck. The calculation formula of the triangulation method is: And c 0 =3×10 8 m / s, where (x 0 ,y 0 、z 0 ) represents the three-dimensional coordinates of the current position of the muck truck, Expressed as the position coordinates corresponding to n satellites, They are respectively represented by the time when the satellite signals received by n satellites arrive at the receiver from the corresponding satellites, c 0 Expressed as the speed of light in meters per second, that is, the GPS data of the muck truck GPS d =(x 0 ,y 0 、z 0 );
[0084] Using Kalman filter algorithm to analyze GPS data d Smoothing and optimization, including measuring the initialization state, prediction and updating of GPS data, to generate vehicle operation signals, calibrated as VOS, used to reduce the noise and error of GPS data and improve GPS positioning accuracy. The calculation formula of the Kalman filter algorithm is H(GPS d ) t =F(GPS d )×H(GPS d ) t-1 +B(GPS d )×U(GPSd ) t 、P(GPS d ) t =F(GPS d )×P(GPS d ) t-1 ×F(GPS d ) T +Q(GPS d ), and K(GPS d ) t =P(GPS d ) t ×C(GPS d ) T ×(C(GPS d )×P(GPS d ) t ×C(GPS d ) T +R(GPS d )) -1 , and H(GPS d ) t+1 =H(GPS d ) t +K(GPS d ) t ×(Z(GPS d ) t -C(GPS d )×H(GPS d ) t )、P(GPS d ) t+1 =(IK(GPS d ) t ×C(GPS d ))×P(GPS d ) t , where H(GPS d )、P(GPS d ) represent the measured GPS data GPS d The estimated vector and covariance matrix of the initial state are set, H(GPS d ) t Represented as measured GPS data GPS d The state vector predicted at the current time step t, F(GPS d ) indicates the measurement of GPS data GPS d The state transfer matrix, H(GPS d ) t-1 Represented as measured GPS data GPS d The state vector predicted at the previous time step t-1, B(GPSd ) indicates the measurement of GPS data GPS d The control input matrix, U(GPS d ) t Represented as measured GPS data GPS d At the current time step t, the control input, P(GPS d ) t Represented as measured GPS data GPS d The covariance matrix predicted at the current time step t, P(GPS d ) t-1 Represented as measured GPS data GPS d The covariance matrix predicted at the previous time step t-1, T represents the transpose, Q(GPS d ) indicates the measurement of GPS data GPS d The process noise covariance matrix, K(GPS d ) t Represented as measured GPS data GPS d The Kalman gain at the current time step t, C(GPS d ) indicates the measurement of GPS data GPS d The measurement matrix, R(GPS d ) indicates the measurement of GPS data GPS d The measurement noise covariance matrix, H(GPS d ) t+1 Represented as measured GPS data GPS d The predicted state vector after updating at the next time step t+1, P(GPS d ) t+1 Represented as measured GPS data GPS d The predicted covariance matrix is updated at the next time step t+1, and I is represented as the unit matrix. Then, according to the prediction update of the Kalman filter algorithm, the output H (GPS d ) t+1 As the vehicle operation signal VOS, that is, VOS = H (GPS d ) t+1 ;
[0085] The vehicle operation signal VOS is transmitted to the cloud server via the mobile network for storage and analysis.
[0086] Muck loading audit module: Use cameras and radar sensors to collect, monitor and record the loading images of the muck truck and the location and shape data of the loaded muck, establish a volume model of the muck in the car, estimate the loading weight of the muck truck, generate operation data, and make the operation data of the muck truck within the preset standard range to prevent overloading, underloading or non-compliant operation;
[0087] Specifically, the steps for establishing the volume model of the muck in the carriage are as follows:
[0088] The image data of the muck loaded in the muck carriage and the position and shape data of the muck are collected by using a camera and a radar sensor, which are calibrated as Sid and Ssd respectively;
[0089] The Canny edge detection algorithm is used to pre-process the image data Sid for the edge detection of the slag, and the contour information of the slag is extracted by combining the position shape data Ssd of the slag. The calculation formula of the Canny edge detection algorithm is E Sid (x,y)=Canny(Sid(x,y)), where E Sid (x, y) represents the binary image after edge detection of the image data Sid, Canny represents the Canny edge detection function, including Gaussian filtering and gradient calculation, Sid(x, y) represents the grayscale image after processing the image data Sid at pixel (x, y);
[0090] The calculation formula for extracting the contour information of the muck is: In the formula, OI represents the contour information of the extracted soil, Ssd represents the location and shape data of the soil, and is composed of a series of boundary points;
[0091] The 3D model of the muck in the carriage is reconstructed using multi-view images, and the 3D point cloud data of the muck surface is obtained using structured light scanning technology. The reconstruction calculation formula of the 3D model is: And 3D pcd =OI∩Dv, where Dv is the depth value of the 3D point, bl is the baseline length of the camera light scan, cf is the focal length of the camera, and pv is the parallax value. pcd It is represented as three-dimensional point cloud data, ∩ represents the intersection, and OI∩Dv represents the point cloud data set composed of the common elements of the contour information of the muck and the depth value of the three-dimensional point;
[0092] According to the known length, width and height of the muck car, they are calibrated as Lrc, Wrc and Hrc respectively, and the volume of the reconstructed 3D point cloud data is calculated. The volume calculation formula is Vms = ∑3D pcd ×Lrc×Wrc×(Hrc-Dv), where Vms is the volume of the muck in the muck car, ∑3D pcd It is represented by the sum of the volume values corresponding to each point data in the point cloud dataset, Lrc×Wrc is represented by the bottom area of the slag, and (Hrc-Dv) is represented by the height value of the slag surface in the point cloud dataset;
[0093] The weight of the slag is estimated by using its volume and density. The weight calculation formula is Wms = ρms × Vms, where Wms is the weight of the slag, ρms is the density of the slag, and Vms is the volume of the slag.
[0094] The workload data is generated based on the weight of the slag Wms, and the preset weight standard range is [W min , W max ], to determine whether the operation volume of the muck truck is compliant, the calculation formula for generating the corresponding operation volume data is Hvd = Wms ± 2% × Wms, where Hvd represents the operation volume data, ± 2% × Wms represents the absolute error value affected by the density of the muck or the transportation factor. When (Hvd = 102% Wms) < W min When (Hvd=98%Wms)>W min When W is greater than the preset weight standard, it is judged that the operation volume of the muck truck exceeds the preset weight standard range, and the muck truck has a large vehicle wear and tear and a high transportation risk; when W min ≤Hvd≤W max , it is judged that the operating volume of the muck truck meets the preset weight standard range.
[0095] Mobile network transparent transmission module: Using the MQTT transmission protocol of the mobile communication network, the real-time data transmission between the LPR license plate recognition module, the GPS global satellite positioning module, the muck loading audit module and the cloud computing analysis module is realized, ensuring 5G network communication between the GPS, radar sensor, camera, CPU device on the muck truck and the cloud server, and between the cloud server and the monitoring terminal;
[0096] Specifically, the MQTT transport protocol is a communication protocol for data exchange, which synchronizes data between modules;
[0097] 5G network communication is a communication network technology that provides high-speed data transmission, low latency and a large number of connections, and supports a large number of devices online at the same time and real-time data transmission.
[0098] Cloud computing analysis module: The cloud server device receives the vehicle identification information in the LPR license plate recognition module, the vehicle operation signal in the GPS global satellite positioning module, and the operation data in the muck loading audit module through the mobile network transmission module, and uses the deep learning model to store, process and analyze the data, generate an analysis report on the loading and unloading operation of the muck truck, and generate alarm information based on abnormal conditions detected in the loading and unloading operation analysis report of the muck truck, and then transmit the loading and unloading operation analysis report and alarm information of the muck truck to the AI recognition feedback module of the supervision terminal;
[0099] Specifically, the data processing steps of the deep learning model are as follows:
[0100] Receive the vehicle identification information VIN from the LPR license plate recognition module, the vehicle operation signal VOS from the GPS global satellite positioning module, and the operation volume data Hvd from the muck loading volume audit module as characteristic data of the muck truck's loading and unloading operations;
[0101] The feature weighted fusion algorithm is used to integrate the feature data to obtain feature fusion data, which provides comprehensive information for the deep learning model. The calculation formula of the feature fusion data is F = α × VIN + β × VOS + χ × Hvd, where F represents the feature fusion data obtained by fusing the vehicle identity information VIN, the vehicle operation signal VOS and the workload data Hvd, and α, β, and χ represent the weight coefficients of the corresponding vehicle identity information VIN, the vehicle operation signal VOS and the workload data Hvd features, respectively.
[0102] The convolutional neural network (CNN) algorithm in the deep learning model is used to train and predict the feature fusion data F to generate a loading and unloading operation analysis report. The calculation formula of the convolutional layer operation of CNN is: And i∈[1,k], j∈[1,k], where Y F represents the output value of the convolution operation, k represents the size of the convolution kernel, F{x+i-1,y+j-1} represents the input data on the convolution layer, k{i,j} represents the weight of the convolution kernel, and b F It is represented as the bias term of the convolution layer, i and j represent the position and channel in the convolution kernel respectively, and x and y represent the position of the feature fusion data F respectively;
[0103] The maximum pooling calculation formula of the pooling layer is Where Y F ′ represents the output value of the pooling layer, and Max represents the maximum value function;
[0104] The calculation formula of the fully connected layer is Y F ″=W F ×Y F ′+b Y , where Y F ″ represents the output value of the fully connected layer, W F Expressed as the corresponding output value Y F ′’s weight matrix, b Y Represented as the bias term of the fully connected layer;
[0105] The loss function is calculated as In the formula, loss is the mean square error loss, n and N are the nth sample and the total number of samples, respectively, and F and Y are F″ represents the feature fusion data as the actual value and the predicted value output by the fully connected layer of the CNN model;
[0106] The calculation formula for gradient update is Where W F ′ represents the updated weight value, λ represents the learning rate, Expressed as the gradient of the loss function with respect to the weight;
[0107] The prediction formula of CNN after training is Y F ″′=W F ′×F+b′ Y ,and Where Y F ″′ represents the predicted output value after CNN model training, b′ Y Represents the updated bias value after CNN model training, It is expressed as the gradient of the loss function with respect to the bias.
[0108] Specifically, the steps for generating the loading and unloading operation analysis report and warning information are as follows:
[0109] The trained CNN model is used to predict the newly input feature fusion data and generate a loading and unloading operation analysis report;
[0110] Perform anomaly detection on the generated loading and unloading operation analysis report to identify abnormal situations such as overloading, underloading, and irregular operations;
[0111] Set anomaly detection thresholds based on historical data and business requirements;
[0112] Compare real-time data with set thresholds to generate anomaly detection results and determine whether there is an anomaly;
[0113] Based on the abnormal detection results, alarm information including alarm type, severity, and recommended measures is generated, and then formatted into a standard report and transmitted to the supervision terminal for real-time viewing and feedback control.
[0114] Supervision terminal AI recognition and feedback module: using the user interaction interface provided by the supervision terminal device, real-time AI recognition displays the storage data, loading and unloading operation analysis reports and alarm information of the cloud computing analysis module, supporting supervisors to conduct real-time monitoring queries and decision-making feedback;
[0115] Specifically, the cloud computing analysis module stores the analysis results and alarm information in a SQL database;
[0116] Parse the loading and unloading operation analysis report data into an easy-to-understand format and use data visualization tools to generate charts and graphs to display the analysis results from the supervisory terminal;
[0117] At the same time, based on the preset alarm rules, the alarm information is identified and classified, and the alarm information is formatted and displayed on the user interface;
[0118] Based on the analysis results of real-time data and historical data, predictive analysis and decision-making methods are provided to help regulators make decisions, enabling regulators to conduct effective real-time monitoring and decision feedback.
[0119] The present invention provides Figure 1 The automatic tracking and auditing system for the whole operation of a muck truck shown in the figure comprises the following steps:
[0120] S1. Radar sensors, cameras and parking / passing signal indicators are installed at the exit of the muck truck loading station, and GPS locators are installed on the muck trucks to collect data information such as license plate images, location and loading images in real time;
[0121] S2, using MQTT transmission protocol and 5G network communication technology to transmit the collected data information to the central server and cloud server for processing, analysis and storage;
[0122] S3. Setting a virtualized electronic fence of the unloading point where the muck truck is working on the cloud server, and setting an indicator light for controlling the muck truck on the electronic fence of the unloading point to record the working status of the muck truck;
[0123] S4. Estimated audit analysis of the loading volume of the muck truck is performed, and the integration, analysis and prediction of comprehensive data are processed through the cloud server to identify the loading behavior pattern of the muck truck and detect abnormal situations;
[0124] S5. Use the deep learning model to process the fusion data, generate the job analysis report and alarm signal, and push it to the supervisor for system feedback maintenance and optimization.
[0125] The specific method and process of the automatic tracking and auditing system for the entire operation of a muck truck provided by an embodiment of the present invention are detailed in the above-mentioned embodiment and will not be repeated here.
[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An automatic tracking and auditing system for the whole operation of a muck truck, characterized in that: It includes an LPR license plate recognition module: using a camera device to collect license plate image data, processing the license plate image data by using a license plate recognition algorithm, extracting license plate character information to perform license plate detection and recognition, generating vehicle identity recognition information, and transmitting the vehicle identity recognition information to a cloud computing analysis module through a mobile network transparent transmission module; GPS global satellite positioning module: A GPS device is installed on each muck truck to locate and record the vehicle's driving trajectory and position changes in real time, obtain the muck truck's GPS data, and use the GPS positioning algorithm combined with the Kalman filter algorithm to process the GPS data to generate vehicle operation signals, which are then transmitted to the cloud computing analysis module through the mobile network transmission module; Muck loading audit module: Use cameras and radar sensors to collect, monitor and record the loading images of the muck truck and the location and shape data of the loaded muck, establish a volume model of the muck in the car, estimate the loading weight of the muck truck, generate operation data, and make the operation data of the muck truck within the preset standard range to prevent overloading, underloading or non-compliant operation; Mobile network transparent transmission module: Using the MQTT transmission protocol of the mobile communication network, the real-time data transmission between the LPR license plate recognition module, the GPS global satellite positioning module, the muck loading audit module and the cloud computing analysis module is realized, ensuring 5G network communication between the GPS, radar sensor, camera, CPU device on the muck truck and the cloud server, and between the cloud server and the monitoring terminal; Cloud computing analysis module: The cloud server device receives the vehicle identification information in the LPR license plate recognition module, the vehicle operation signal in the GPS global satellite positioning module, and the operation data in the muck loading audit module through the mobile network transmission module, and uses the deep learning model to store, process and analyze the data, generate an analysis report on the loading and unloading operation of the muck truck, and generate alarm information based on abnormal conditions detected in the loading and unloading operation analysis report of the muck truck, and then transmit the loading and unloading operation analysis report and alarm information of the muck truck to the AI recognition feedback module of the supervision terminal; Supervision terminal AI recognition and feedback module: Utilizing the user interaction interface provided by the supervision terminal device, real-time AI recognition displays the storage data, loading and unloading operation analysis reports and alarm information of the cloud computing analysis module, supporting supervisors to conduct real-time monitoring queries and decision-making feedback.
2. According to claim 1, the automatic tracking and auditing system for the whole operation of a muck truck is characterized in that: The steps of the license plate recognition algorithm to process image data are as follows: The license plate image data is collected by a camera and calibrated as Lpid, and the license plate image data Lpid is preprocessed by grayscale, noise reduction, and histogram equalization to obtain the preprocessed image information, which is calibrated as Pid. The grayscale preprocessing calculation formula is I(Lpid)=0.299×I R +0.587×I G +0.114×I B , where I(Lpid) represents the preprocessed image after grayscale conversion, I R ,I G ,I B They are respectively represented as the pixel values of the red, green and blue channels corresponding to the original color image, and the pixel value range is 0-255, and 0.299, 0.587 and 0.114 are respectively represented as weighting coefficients determined according to the red, green and blue sensitivities corresponding to the human eye; The preprocessing calculation formula for noise reduction is N(Lpid)=I(Lpid)×G(x,y), and Where N(Lpid) represents the preprocessed image after denoising, G(x,y) represents the Gaussian kernel function for smoothing the image, σ represents the standard deviation of the Gaussian kernel, x and y represent pixel coordinates, and e represents an exponential function; The preprocessing calculation formula for histogram equalization is: And k∈[1,K],K=256, where G(Lpid) k It is represented by the pixel value of gray level k in the preprocessed image after histogram equalization. K and k represent the total number of gray levels and the kth gray level respectively. M N(Lpid) It is expressed as the total number of pixels of the pre-processed image N(Lpid) after denoising, N(Lpid) k It is expressed as the number of pixels with gray level k in the pre-processed image N (Lpid) after denoising, that is, the pre-processed image information of grayscale, denoising and histogram equalization is calibrated as Pid; Use the license plate recognition algorithm to perform license plate area positioning, character segmentation, character recognition and information extraction on the pre-processed image information Pid to obtain the complete license plate number and mark it as Ipn; The license plate recognition algorithm includes the Sobel operator edge detection algorithm, morphological operation algorithm, vertical projection method, and convolutional neural network (CNN) model to achieve the reprocessing of license plate area positioning, character segmentation, character recognition, and information extraction. The calculation formula of the Sobel operator edge detection algorithm is: and In the formula, Gx (Pid) It is represented as the gradient of the preprocessed image information Pid in the x direction, Gy (Pid) It is represented as the gradient of the preprocessed image information Pid in the y direction. It is represented as a 3×3 matrix of the convolution kernel smoothed on the positive half axis of the x-axis for the image information Pid. It is represented as a 3×3 matrix of the convolution kernel with the image information Pid smoothed on the positive half axis of the y-axis, G ( ' Pid) It is expressed as the gradient amplitude of the preprocessed image information Pid, that is, the edge strength of the image information Pid, and the image after edge detection is calibrated as eg G′(Pid) ; The calculation formula of the morphological operation algorithm is: Where, dl (Pid) Represented as after edge detection eg G′(Pid) The image after the expansion operation, eg G′(Pid) Represented as the image after edge detection, represents the XOR operator, B represents the structural element, ed (Pid) Represented as after edge detection eg G′(Pid) The image after the corrosion closing operation, Θ represents the inverse operation operator of XOR, then the image after edge detection eg G′(Pid) After morphological processing, the license plate area positioning image is obtained and calibrated as BL (Pid) ; The calculation formula of the vertical projection method is Pvd = BL (Pid) ×Pid (x,y) , where Pvd represents the license plate area positioning image BL (Pid) The vertical projection value, BL (Pid) Represents the license plate area positioning image, Pid (x,y) The pixel coordinates of the preprocessed image information Pid are expressed as follows. Character segmentation is performed based on the peak and valley values of the projection value Pvd, and the character image is calibrated as CS (Pid) ; The calculation formula of the convolutional neural network CNN model is Ycs = W0 × CS (Pid) +b0, where Ycs represents the probability distribution of the recognized characters output by the trained CNN model, W0 represents the weight of the fully connected layer in the CNN model, and b0 represents the bias of the fully connected layer in the CNN model. The characters segmented from the character image are re-recognized using the pre-trained CNN model, and the character features are extracted through the convolutional layer, pooling layer and fully connected layer. The probability distribution of the characters Ycs is output, and the recognized characters are determined and combined to obtain the license plate number Ipn.
3. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 2 is characterized in that: The process of obtaining the vehicle identification information is as follows: The license plate number Ipn obtained after processing the image data according to the license plate recognition algorithm is affixed with a timestamp and a position, which are calibrated as Ts and Ps respectively, to generate vehicle identification information, which is calibrated as VIN, then VIN = {Ipn, Ts, Ps}; The generated vehicle identification information VIN is transmitted to the cloud server in real time using the MQTT protocol.
4. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 3 is characterized in that: The steps of generating the vehicle running signal are as follows: Use the vehicle-mounted GPS device to collect and record the GPS data of the muck truck in real time and calibrate it as GPS d ; According to the GPS data of the muck truck d , and determine that n number of satellite signals have been received, and use the triangulation method in the GPS positioning algorithm to calculate the three-dimensional coordinates of the current position of the muck truck. The calculation formula of the triangulation method is: And c 0 =3×10 8 m / s, where (x 0 ,y 0 、z 0 ) represents the three-dimensional coordinates of the current position of the muck truck, Expressed as the position coordinates corresponding to n satellites, They are respectively represented by the time when the satellite signals received by n satellites arrive at the receiver from the corresponding satellites, c 0 Expressed as the speed of light in meters per second, that is, the GPS data of the muck truck GPS d =(x 0 ,y 0 、z 0 ); Using Kalman filter algorithm to analyze GPS data d Smoothing and optimization, including measuring the initialization state, prediction and updating of GPS data, to generate vehicle operation signals, calibrated as VOS, the calculation formula of Kalman filter algorithm is H(GPS d ) t =F(GPS d )×H(GPS d ) t-1 +B(GPS d )×U(GPS d ) t 、P(GPS d ) t =F(GPS d )×P(GPS d ) t-1 ×F(GPS d ) T +Q(GPS d ), and K(GPS d ) t =P(GPS d ) t ×C(GPS d ) T ×(C(GPS d )×P(GPS d ) t ×C(GPS d ) T +R(GPS d )) -1 , and H(GPS d ) t+1 =H(GPS d ) t +K(GPS d ) t ×(Z(GPS d ) t -C(GPS d )×H(GPS d ) t )、P(GPS d ) t+1 =(IK(GPS d ) t ×C(GPS d ))×P(GPS d ) t , where H(GPS d )、P(GPS d ) represent the measured GPS data GPS d The estimated vector and covariance matrix of the initial state are set, H(GPS d ) t Represented as measured GPS data GPS d The state vector predicted at the current time step t, F(GPS d ) indicates the measurement of GPS data GPS d The state transfer matrix, H(GPS d ) t-1 Represented as measured GPS data GPS d The state vector predicted at the previous time step t-1, B(GPS d ) indicates the measurement of GPS data GPS d The control input matrix, U(GPS d ) t Represented as measured GPS data GPS d At the current time step t, the control input, P(GPS d ) t Represented as measured GPS data GPS d The covariance matrix predicted at the current time step t, P(GPS d ) t-1 Represented as measured GPS data GPS d The covariance matrix predicted at the previous time step t-1, T represents the transpose, Q(GPS d ) indicates the measurement of GPS data GPS d The process noise covariance matrix, K(GPS d ) t Represented as measured GPS data GPS d The Kalman gain at the current time step t, C(GPS d ) indicates the measurement of GPS data GPS d The measurement matrix, R(GPS d ) indicates the measurement of GPS data GPS d The measurement noise covariance matrix, H(GPS d ) t+1 Represented as measured GPS data GPS d The predicted state vector after updating at the next time step t+1, P(GPS d ) t+1 Represented as measured GPS data GPS d The predicted covariance matrix is updated at the next time step t+1, and I is represented as the unit matrix. Then, according to the prediction update of the Kalman filter algorithm, the output H (GPS d ) t+1 As the vehicle operation signal VOS, that is, VOS = H (GPS d ) t+1 ; The vehicle operation signal VOS is transmitted to the cloud server via the mobile network for storage and analysis.
5. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 4 is characterized in that: The steps for establishing the volume model of the muck in the carriage are as follows: The image data of the muck loaded in the muck carriage and the position and shape data of the muck are collected by using a camera and a radar sensor, which are calibrated as Sid and Ssd respectively; The Canny edge detection algorithm is used to pre-process the image data Sid for the edge detection of the slag, and the contour information of the slag is extracted by combining the position shape data Ssd of the slag. The calculation formula of the Canny edge detection algorithm is E Sid (x,y)=Canny(Sid(x,y)), where E Sid (x, y) represents the binary image after edge detection of the image data Sid, Canny represents the Canny edge detection function, including Gaussian filtering and gradient calculation, Sid(x, y) represents the grayscale image after processing the image data Sid at pixel (x, y); The calculation formula for extracting the contour information of the muck is: In the formula, OI represents the contour information of the extracted slag, Ssd represents the location shape data of the slag, and is composed of a series of boundary points; The 3D model of the muck in the carriage is reconstructed using multi-view images, and the 3D point cloud data of the muck surface is obtained using structured light scanning technology. The reconstruction calculation formula of the 3D model is: And 3D pcd =OI∩Dv, where Dv is the depth value of the 3D point, bl is the baseline length of the camera light scan, cf is the focal length of the camera, and pv is the parallax value. pcd It is represented as three-dimensional point cloud data, ∩ represents the intersection, and OI∩Dv represents the point cloud data set composed of the common elements of the contour information of the muck and the depth value of the three-dimensional point; According to the known length, width and height of the muck car, they are calibrated as Lrc, Wrc and Hrc respectively, and the volume of the reconstructed 3D point cloud data is calculated. The volume calculation formula is Vms = ∑3D pcd ×Lrc×Wrc×(Hrc-Dv), where Vms is the volume of the muck in the muck car, ∑3D pcd It is represented by the sum of the volume values corresponding to each point data in the point cloud dataset, Lrc×Wrc is represented by the bottom area of the slag, and (Hrc-Dv) is represented by the height value of the slag surface in the point cloud dataset; The weight of the slag is estimated by using its volume and density. The weight calculation formula is Wms = ρms × Vms, where Wms is the weight of the slag, ρms is the density of the slag, and Vms is the volume of the slag. The workload data is generated based on the weight of the slag Wms, and the preset weight standard range is [W min , W max ], to determine whether the operation volume of the muck truck is compliant, the calculation formula for generating the corresponding operation volume data is Hvd = Wms ± 2% × Wms, where Hvd represents the operation volume data, ± 2% × Wms represents the absolute error value affected by the density of the muck or the transportation factor. When (Hvd = 102% Wms) < W min When (Hvd=98%Wms)>W min When the operation volume of the muck truck exceeds the preset weight standard range, the muck truck will be subject to heavy wear and tear and high transportation risk; When W min ≤Hvd≤W max , it is judged that the operating volume of the muck truck meets the preset weight standard range.
6. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 5 is characterized in that: The data processing steps of the deep learning model are as follows: Receive the vehicle identification information VIN from the LPR license plate recognition module, the vehicle operation signal VOS from the GPS global satellite positioning module, and the operation volume data Hvd from the muck loading volume audit module as characteristic data of the muck truck's loading and unloading operations; The feature data are integrated together using a feature weighted fusion algorithm to obtain feature fusion data. The calculation formula for the feature fusion data is F = α × VIN + β × VOS + χ × Hvd, where F represents the feature fusion data obtained by fusing the vehicle identity information VIN, the vehicle operation signal VOS, and the workload data Hvd. α, β, and χ represent the weight coefficients of the corresponding vehicle identity information VIN, the vehicle operation signal VOS, and the workload data Hvd features, respectively. The convolutional neural network (CNN) algorithm in the deep learning model is used to train and predict the feature fusion data F to generate a loading and unloading operation analysis report. The calculation formula of the convolutional layer operation of CNN is: And i∈[1,k], j∈[1,k], where Y F represents the output value of the convolution operation, k represents the size of the convolution kernel, F{x+i-1,y+j-1} represents the input data on the convolution layer, k{i,j} represents the weight of the convolution kernel, and b F It is represented as the bias term of the convolution layer, i and j represent the position and channel in the convolution kernel respectively, and x and y represent the position of the feature fusion data F respectively; The maximum pooling calculation formula of the pooling layer is Where Y F ′ represents the output value of the pooling layer, and Max represents the maximum value function; The calculation formula of the fully connected layer is Y F =W F ×Y F ′+b Y , where Y F ″ represents the output value of the fully connected layer, W F Expressed as the corresponding output value Y F ′’s weight matrix, b Y Represented as the bias term of the fully connected layer; The loss function is calculated as In the formula, loss is the mean square error loss, n and N are the nth sample and the total number of samples, respectively, and F and Y are F ″ represents the feature fusion data as the actual value and the predicted value output by the fully connected layer of the CNN model; The calculation formula for gradient update is Where W F ′ represents the updated weight value, λ represents the learning rate, Expressed as the gradient of the loss function with respect to the weight; The prediction formula of CNN after training is Y F ″′=W F ′×F+b′ Y ,and Where Y F ″′ represents the predicted output value after CNN model training, b′ Y Represents the updated bias value after CNN model training, It is expressed as the gradient of the loss function with respect to the bias.
7. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 6 is characterized in that: The steps for generating the loading and unloading operation analysis report and warning information are as follows: The trained CNN model is used to predict the newly input feature fusion data and generate a loading and unloading operation analysis report; Perform anomaly detection on the generated loading and unloading operation analysis report to identify abnormal situations such as overloading, underloading, and irregular operations; Set anomaly detection thresholds based on historical data and business requirements; Compare real-time data with set thresholds to generate anomaly detection results and determine whether there is an anomaly; Based on the abnormal detection results, alarm information including alarm type, severity, and recommended measures is generated, and then the alarm information is formatted into a standard report and transmitted to the supervision terminal.
8. The automatic tracking and auditing system for the whole operation of a muck truck according to claim 7 is characterized in that: The method steps for automatic tracking and auditing the whole operation of the muck truck are as follows: S1. Radar sensors, cameras and parking / passing signal indicators are installed at the exit of the muck truck loading station, and GPS locators are installed on the muck trucks to collect data information such as license plate images, location and loading images in real time; S2, using MQTT transmission protocol and 5G network communication technology to transmit the collected data information to the central server and cloud server for processing, analysis and storage; S3. Setting a virtualized electronic fence of the unloading point where the muck truck is working on the cloud server, and setting an indicator light for controlling the muck truck on the electronic fence of the unloading point to record the working status of the muck truck; S4. Estimated audit analysis of the loading volume of the muck truck is performed, and the integration, analysis and prediction of comprehensive data are processed through the cloud server to identify the loading behavior pattern of the muck truck and detect abnormal situations; S5. Use the deep learning model to process the fusion data, generate the job analysis report and alarm signal, and push it to the supervisor for system feedback maintenance and optimization.
Citation Information
Cited By
Vehicle type size measuring method and system for unmanned intelligent loading
CN119354055A
