Highway Truck Crane Monitoring and Warning System Based on AI Deep Learning and Drone Applications

Abstract

This invention discloses a highway truck crane monitoring and warning system based on AI deep learning and drone applications, comprising a drone, an image acquisition module, and an information processing module. The drone's flight control module controls the operation of the drive mechanism, enabling the drone to perform crane identification and inspection flight tasks near the highway. The image acquisition module captures images of the vehicles' driving status on the highway, generating driving status images. The information processing module includes an image recognition module and a feature calculation module. The feature calculation module can perform image feature processing on driving status images to generate driving status feature information. The image recognition module can perform first and second image recognition processing on the driving status feature information to generate first recognition result information and second recognition result information. When the second recognition result information indicates that the truck crane boom is not properly retracted, a warning signal is output. This allows for the integration of drone and target image detection model technology, enabling real-time detection and identification of illegal trucks and issuing warnings to drivers on national highways. In addition to effectively improving national highway driving safety and protecting public facilities, it can also effectively solve the safety hazards and infrastructure damage caused by truck crane booms not being properly retracted on highways.

Claims

1. A highway truck crane boom monitoring and warning system based on AI deep learning and drone applications, comprising: A drone is equipped with a power drive mechanism and a flight control module. The flight control module controls the operation of the power drive mechanism according to the remote control command or flight control command issued by the remote control device, so that the drone can perform crane identification and inspection flight missions near highways. An image capture module, mounted on the drone, is used to capture images of the driving status of vehicles on the highway to generate driving status images; and an information processing module, coupled to the image capture module, includes an AI deep learning-based image recognition module and a feature calculation module. The feature calculation module is used to perform image feature processing on the real-time captured driving status images to generate corresponding driving status feature information. The image recognition module is used to perform first image recognition processing on the driving status feature information to generate first recognition result information. When the vehicles in the first recognition result information include trucks, the driving status feature information is then subjected to second image recognition processing to generate second result information. When the second recognition result information indicates that the truck crane boom is not properly retracted, a warning signal is output.

2. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 1, wherein, The information processing module includes an image detection database containing at least one vehicle image detection model and at least one truck crane image detection model. The at least one vehicle image detection model defines first identification result information, including vehicle type and license plate number. The at least one truck crane image detection model defines second identification result information, including whether the truck crane is not properly retracted and whether it is properly retracted. The image recognition module uses the at least one vehicle image detection model... The driving status feature information is subjected to the first image recognition processing to output the first recognition result information, such as the vehicle type and license plate number, which are corresponding to the real-time captured highway vehicle driving status. When the vehicle type in the first recognition result information includes a truck, the driving status feature information is subjected to the second image recognition processing using the at least one truck boom image detection model. When the second recognition result information indicates that the truck boom is not properly retracted, the warning signal is output, and the license plate number of the truck with the not properly retracted boom and the truck boom not properly retracted recognition result information are output.

3. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 2, wherein, The driving status feature information includes vehicle outline features and license plate outline features. The first image recognition processing performed by the image recognition module includes a vehicle image recognition and labeling step. When the vehicle image recognition and labeling step is executed, the image recognition module selects and marks each vehicle outline in the vehicle outline features one by one. The image recognition module uses the at least one vehicle image detection model to classify each selected and marked vehicle outline into a vehicle type to generate the first recognition result information that can identify the vehicle type. The module also performs image recognition processing on the license plate outline features to generate the first recognition result information that can identify the license plate number.

4. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 2, wherein, The driving status feature information includes vehicle outline features and boom outline features. The image recognition module uses the image detection model of at least one truck boom to perform the second image recognition processing on the real-time boom outline features. The second image recognition processing further includes a key point detection step. When the key point detection step is executed, key points are marked on each joint of the boom outline features, and each key point is substituted into an image coordinate positioning method to obtain the corresponding key point coordinates. Then, based on the truck's safety range, a rectangular safety range frame is set using the image coordinate positioning method. If the coordinates of one of the key points exceed the safety range frame, it is determined that the truck boom is not properly retracted.

5. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 4, wherein, When the coordinates of each key point do not exceed the safety range frame, a horizontal line detection step is performed. When the horizontal line detection step is performed, the center line feature of the boom is extracted from the boom outline feature. When the angle between the center line feature of the boom and a preset horizontal line is greater than the preset angle, it is determined that the truck boom is not properly retracted and a warning signal is issued. The preset angle is between 10 and 20 degrees.

6. The highway truck boom monitoring and warning system based on AI deep learning and drone applications as described in claim 1 further includes a warning device coupled to the information processing module for displaying the warning signal, the first identification result information and the second identification result information in the form of audio playback or text data.

7. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 1, wherein, The image feature processing performed by the feature calculation module includes a visual image feature extraction step. When the visual image feature extraction step is executed, edge detection technology and color analysis technology are used to initially extract vehicle outline features, license plate outline features and crane outline features as driving status feature information from the real-time driving status image.

8. The highway truck crane monitoring and warning system based on AI deep learning and drone applications as described in claim 1 further includes a GPS positioning module installed on the drone and coupled to the information processing module. The GPS positioning module generates positioning signals. The information processing module has built-in automatic patrol path data that enables the drone to perform crane identification and inspection flight tasks on a section of the highway. Based on the real-time positioning signal, the automatic patrol path data is continuously output as corresponding flight control commands, which are then transmitted to the flight control module, allowing the flight control module to control the operation of the power drive mechanism according to the flight control commands. The image recognition module further includes a vehicle tracking and analysis step. When the first... If the identification result indicates that the truck's crane boom is not properly retracted, the vehicle tracking and analysis step is executed. This involves selecting and marking the truck with the unretracted boom in the real-time driving status image, and using a perception analyzer and a perceptual analyzer to analyze and predict the driving trajectory of the selected truck, generating a corresponding driving trajectory prediction signal. The information processing module corrects the automatic patrol path data based on the real-time location signal and the driving trajectory prediction signal, and continuously outputs corresponding corrected flight control commands. The flight control module then controls the operation of the power drive mechanism based on the corrected flight control commands. When the drone approaches the vicinity of the truck with the unretracted boom, the image acquisition module is activated to take photos of the truck with the unretracted boom as evidence of the violation.

9. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 1, wherein, When the drone performs the crane identification and inspection flight mission on one section of the highway, the drone's direction of travel is the same as the direction of traffic on the highway. The image capture module is located at the rear of the drone, so that the image capture module can capture images of the traffic status of the highway from behind the drone.

10. The highway truck crane boom monitoring and warning system based on AI deep learning and drone applications as described in claim 9, wherein, The ratio of the drone's flight speed during the crane identification and inspection mission to the average speed of traffic on the highway is between 1:1.5 and 1:2.

Images