A 5G UAV dynamic AI inspection system

Abstract

The present invention discloses a 5G unmanned aerial vehicle dynamic AI inspection system, which includes: an unmanned aerial vehicle, which is used to implement the shooting of cruise videos; an unmanned aerial vehicle control system, which controls the cruise route, shoots and transmits back real-time cruise videos, and the unmanned aerial vehicle transmits back the position and power information; a video access and recognition output system, which receives real-time video data, identifies and analyzes safety issues in the video in real time through an AI algorithm, and provides an emergency plan function; and an unmanned aerial vehicle monitoring and early warning system, which displays the coordinates, problem pictures and emergency plans of safety issues of unmanned aerial vehicle cruises in real time; the unmanned aerial vehicle control system, the video access and recognition output system, and the unmanned aerial vehicle monitoring and early warning system are respectively displayed by a single screen. The present invention can effectively reduce labor costs, reduce safety risks, and provide assistance for the digital inspection of automated terminals.

Description

Technical Field

[0001] The present invention relates to the technical field of drone inspections, and in particular to a 5G drone dynamic AI inspection system. Background Art

[0002] Ports are transport hubs located along the coasts of seas, rivers, lakes and reservoirs, with water-land transport facilities and conditions for ships to enter, exit and berth safely. Ports are the gathering points and hubs of water and land transportation, the distribution centers of industrial and agricultural products and foreign trade import and export materials, and the places where ships berth, load and unload goods, get on and off passengers, and replenish supplies.

[0003] In China, coastal port construction focuses on transportation systems such as coal, containers, imported iron ore, grain, land-to-island roll-on / roll-off, and deep-water sea-going channels, and has particularly strengthened the construction of container transportation systems. However, the terminal equipment is generally large-scale, and there are major safety hazards in high-altitude operations. At the same time, the inspection work in the automated area has a greater impact on production and consumes a lot of manpower. Under such conditions, the inspection work of drones can replace a large amount of manual input, which is also of positive significance for reducing labor costs and personal safety protection.

[0004] Based on the above situation, we designed a 5G drone dynamic AI inspection system. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention aims to provide a 5G UAV dynamic AI inspection system.

[0006] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0007] A 5G UAV dynamic AI inspection system, comprising:

[0008] A drone, wherein the drone is used to shoot a cruise video;

[0009] UAV control system, which controls the cruise route, shoots and transmits back real-time cruise video, and the UAV transmits back information such as location and power;

[0010] A video access and recognition output system, which receives real-time video data, identifies and analyzes security issues in the video in real time through an AI algorithm, and provides an emergency plan function;

[0011] And a drone monitoring and early warning system, which displays the coordinates, problem pictures and emergency plans of safety issues of drone cruising in real time.

[0012] The drone control system, video access and identification output system, and drone monitoring and early warning system are respectively displayed on a single screen.

[0013] A further technical solution of the present invention is that the video access and recognition output system is mainly used to play the cruise video transmitted back by the drone control system, identify safety issues in the video through AI or manual, and automatically or manually select emergency plans, and push the recognition results and emergency plans to the drone monitoring and early warning system for three-dimensional all-round early warning and display.

[0014] A further technical solution of the present invention is that the drone monitoring and early warning system synchronously connects to the video and captured pictures of the corresponding position of the drone and automatically performs AI recognition, returns and saves the recognition result, and the recognition result is the identification of suspected safety hazards or the identification of serious safety hazard warnings.

[0015] A further technical solution of the present invention is set as follows: the visual module of the drone includes an inspection recognition module, and the inspection recognition module assists the drone in automatic inspection, and the inspection content includes box area and stacking inspection, operation area target inspection, and mechanical equipment and area location inspection.

[0016] A further technical solution of the present invention is that the box area and stack inspection includes:

[0017] 3D container detection, determining the orientation of each side of the container and drawing the 3D coordinates of the container in the 2D image;

[0018] Inspection of container area: after three-dimensional positioning of each container in the container area in the previous step, the coordinates of each vertex of the container and the container frame can be obtained, thereby judging whether the stacking and placement of the containers are correct;

[0019] Check the density and information of the container area. The stacking density of the container area can be obtained through the previous step. Furthermore, if the image quality is high, the container information such as the container number and danger mark can be identified;

[0020] Container lashing recognition: When the drone is inspecting the container area, it can identify whether the container lashing is completed and issue a warning based on the situation.

[0021] A further technical solution of the present invention is that the algorithm for the inspection of the box area and stacking is:

[0022] Container 3D positioning: By building a neural network and using algorithms such as Hough transformation, containers can be positioned in 3D. The model uses MobileNetV3 as the skeleton of the convolution operation, and then uses Unet to sample features at different sampling rates. The output results enter the Dual attention network to obtain the channel attention map and position attention map respectively, and the results are fused and merged with the intermediate layer output of MobileNetv3. Finally, after Upsamble to the same resolution as the original image, the semantic segmentation result is output and the corresponding pixel value of the container surface is derived.

[0023] 3D positioning of multiple containers in the container area: By performing 3D modeling on the vertices and border information of each container surface after positioning, the container placement position, angle and other information can be obtained, thereby judging whether the container position meets the specifications; for the 3D positioning of the above containers, by further performing instance segmentation on the image segmentation results, the 3D positioning results of different containers can be obtained;

[0024] Container information recognition: By extracting the containers and the segmented areas of each container surface in the previous step, if the image is clear enough, the number of containers stacked, the dangerous goods labels on the container surface, the container number and other information can be identified;

[0025] Binding identification: By identifying the main binding straps in the image, outlining the binding straps, and confirming whether each box has been reinforced, an early warning can be issued.

[0026] A further technical solution of the present invention is that the target inspection in the working area is performed by drone vision to detect targets in the working area, and the objects include road detection, AGV, staff and other targets that may exist in the working area;

[0027] The target inspection of the operation area includes:

[0028] Segmentation of work area types: using machine vision to divide the work area into different functional areas;

[0029] Target positioning: the system can realize the positioning of AGV, internal and external container trucks, and staff.

[0030] A further technical solution of the present invention is that during the drone inspection process, the mechanical equipment and regional location inspection requires manual inspection of the contents including the facades of high-rise buildings, steel structures, fenders on the sea side, box area tracks, diversion dikes and embankments.

[0031] A further technical solution of the present invention is to include a client and a server, wherein the client is provided with all display and control functions by a browser; and the server provides data display support for the client by connecting with a data system.

[0032] A further technical solution of the present invention is configured to also include cloud technology, wherein the cloud technology includes a basic resource layer and a containerization layer. The basic resource layer is constructed by multiple hosts with X86 architectures and GPU capabilities, and serves as the hardware foundation layer of cloud services, providing the most basic computing, storage, and network functions; the containerization layer uses Kubernetes as an overall solution, and all applications and algorithm services will provide services to the outside world in a containerized form.

[0033] In summary, the present invention includes the following beneficial technical effects:

[0034] The present invention can replace a large amount of manual input through the inspection work of drones, which is also of positive significance for reducing labor costs and personal safety protection; based on the rapid development of drone technology and 5G technology, drone inspections can greatly reduce the impact of conventional production and inspections on the production process of automated terminals, while reducing safety risks, and providing support for digital inspections of automated terminals. Collect video or image data of various business scenes captured by drone inspections, use artificial intelligence algorithms to analyze the content that can be machine-visually recognized, and combine drone dynamic cruise videos to perform AI machine vision recognition for container stacking, lashing, personnel safety, mechanical equipment location and equipment status, etc., effectively detect abnormal situations and issue warnings, identify potential inspection risks in a timely manner, and form special diagnostic reports for abnormal situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of the present invention;

[0036] Figure 2 It is the overall architecture diagram of the present invention;

[0037] Figure 3 is a system block diagram of the present invention;

[0038] Figure 4 This is the architecture diagram of the digital twin of the present invention

[0039] Figure 5 This is a schematic diagram of the algorithm for three-dimensional positioning of containers in the present invention;

[0040] Figure 6 Schematic diagram of the algorithm for container information recognition in the present invention. DETAILED DESCRIPTION

[0041] The following is combined with Figure 1-6 The present invention is described in further detail.

[0042] The present invention discloses a 5G UAV dynamic AI inspection system, including a UAV, a UAV control system, a video access and recognition output system, a UAV monitoring and early warning system, a client, a server and cloud technology.

[0043] Specifically, the drone is a DJI drone, which controls the cruise route, shoots and transmits real-time cruise video, and transmits information such as location and power. The video access and recognition output system receives real-time video data, and uses AI algorithms to identify and analyze safety issues in the video in real time (it also supports manual recognition and identification of safety issues); the video access and recognition output system provides an emergency plan function (automatically selects an emergency plan or manually processes an emergency plan according to the set emergency plan rules). The drone monitoring and early warning system (digital twin) displays the coordinates of safety issues of drone cruises, problem pictures and emergency plans in real time.

[0044] The main function of the video access and recognition output system is to play the cruise video sent back by the drone control system, identify safety issues in the video through AI or manual, and automatically or manually select emergency plans. The recognition results and emergency plans are pushed to the drone monitoring and early warning system (digital twin) for three-dimensional all-round early warning and display.

[0045] In the present invention, the drone control system, the video access and identification output system, and the drone monitoring and early warning system are respectively displayed by a single screen.

[0046] The digital twin interface focuses on business and drone flight paths, and can be synchronized with DJI drones in real time, combining virtual and real. It is a real-time drone cruise interface that displays flight status, drone equipment status, etc.

[0047] The drone control system interface consists of a top information bar, a left navigation bar, and a right work area. The right work area can be displayed in full screen, allowing users to intuitively focus on the current work area.

[0048] The video access and identification output interface provides inspection identification information, early warning information, etc.

[0049] The cruise monitoring interface is divided into four areas: real-time cruise video, real-time safety warning, emergency plan processing and emergency pre-processing status.

[0050] The real-time safety warning panel dynamically displays AI's real-time identification of safety issues. You can also manually click the "Capture Safety Hazards" button to capture the scene in the currently playing video, mark the problem location in the picture, and describe the safety issue.

[0051] The emergency plan processing panel displays a list of cruise safety issues that are pending by default (the system cannot automatically select an emergency plan based on the preset emergency plan rules). Users can click on the relevant safety issue photo to view it, and click the "Process Safety Issue" button to customize the emergency plan.

[0052] The emergency plan processing status dynamically displays a pie chart of the proportion of unresolved, under-processed and resolved safety issues.

[0053] Drone monitoring and early warning (in combination with digital twins), adding a new digital twin early warning icon, namely drone inspection early warning, synchronously docking the video and captured pictures of the corresponding position of the drone and automatically AI recognition, returning and saving the recognition results. The recognition result is the identification of suspected safety hazards or the identification of serious safety hazards early warning.

[0054] In addition, the drone includes a visually assisted inspection and recognition module. Through feasibility analysis of past projects, the inspection and recognition module has sorted out the following functions: container area and stacking inspection, operating area (personnel) target inspection, machinery and equipment and regional location inspection.

[0055] The inspection of the container area and stacking is carried out by using drone vision, which can include the following contents:

[0056] Container 3D detection: determine the orientation of each side of the container and draw the 3D coordinates of the container in the 2D image.

[0057] Inspection of container area: After three-dimensional positioning of each container in the container area in the previous step, the coordinates of each vertex of the container and the container frame can be obtained, thereby judging whether the stacking and placement of the containers are correct.

[0058] Checking the density and information of the container area: The stacking density of the container area can be obtained through the previous step. Furthermore, if the image quality is high, the container information such as the container number and danger mark can be identified.

[0059] Container lashing identification: When the drone is inspecting the container area, it can identify whether the container lashing is completed and issue a warning based on the situation.

[0060] The algorithm for inspection of box area and stacking is described as follows:

[0061] Container 3D positioning: By building a neural network and using algorithms such as Hough transformation, containers can be positioned in 3D. The model uses MobileNetV3 as the skeleton of the convolution operation, and then uses Unet to sample features at different sampling rates. The output results enter the Dual attention network to obtain the channel attention map and position attention map respectively, and the results are fused and merged with the intermediate layer output of MobileNetv3. Finally, after Upsamble to the same resolution as the original image, the semantic segmentation results are output and the corresponding pixel values ​​of the container surface are derived.

[0062] 3D positioning of multiple containers in the container area: By performing 3D modeling on the vertices and border information of each container surface after positioning, the container placement position, angle and other information can be obtained, thereby judging whether the container position meets the specifications. For the above-mentioned 3D positioning of containers, by further performing instance segmentation on the image segmentation results, the 3D positioning results of different containers can be obtained.

[0063] Container information recognition: By extracting the container and each container surface segmentation area in the previous step, if the image is clear enough, the number of container stacks, container surface danger labels, container numbers and other information can be identified. The following is a brief introduction to the recognition algorithm; the recognition algorithm also uses MobileNetV3 as the skeleton of the convolution operation to extract image features, obtains H / 8 vectors of length 512, and then performs self-attention operations on these H / 8 vectors to output vectors of the same size (H / 8,512), and then sequentially inputs these H / 8 vectors into the same LSTM Cell. A character prediction result can be obtained for each step of the output.

[0064] Binding identification: By identifying the main binding straps in the image, drawing the outline of the binding straps, confirming whether each box has been reinforced, and issuing an early warning. To meet the business needs of binding identification, you can first extract its main outline. You can use the second method of polar coordinates to represent the straight line of the outline in the image. The expression can be:

[0065]

[0066] The target inspection of the working area (personnel) uses the vision of the drone to detect the targets in the working area, including road detection (lanes, sidewalks, etc.), AGV, staff and other targets that may exist in the working area. In this regard, the following application scenarios are sorted out:

[0067] Segmentation of work area (road) types; dividing the work area into different functional areas through machine vision;

[0068] Target positioning: The system can realize the positioning of AGV, internal and external container trucks, and staff. Take staff as an example; by locating the coordinates of the personnel in the image and comparing the application area after the previous segmentation, it is determined whether the position of the personnel meets the requirements of safety regulations, and then an early warning is issued; by segmenting the working area in the image and detecting the staff, it is determined whether the position of the personnel is in compliance with the regulations.

[0069] The target inspection algorithm for the operation area (personnel) is described as follows:

[0070] Work area (road) type segmentation: For pixel-level classification of work areas in images, we plan to use a neural network for semantic segmentation. The neural network outputs a multi-dimensional matrix with the same resolution as the collected original image, and determines the area classification of the pixel at the corresponding position in the original image.

[0071] Target detection: For the detection of targets such as people and AGVs, a neural network can be used for bounding box regression. The model can be used to predict the coordinates of the target center point and the width and height values ​​to determine the coordinates of the target in the image.

[0072] During the drone inspection process, mechanical equipment and regional location (building) inspections require manual inspection of various types of content, including the facades of high-rise buildings, steel structures, sea-side fenders, box area tracks, dikes and embankments.

[0073] However, there is no effective deep learning algorithm that can realize the universal inspection function of multiple scenes and multiple targets. In this regard, as a first step, we can use traditional machine vision algorithms to realize that in a known scene, we can compare normal pictures taken in advance with pictures taken during inspections to find out the content with large differences between the two types of pictures, and then submit them for manual judgment to see if there are any abnormalities.

[0074] In the second step, after collecting enough abnormal type pictures, a neural network model can be built to learn such abnormal content. This will enable the system to autonomously discover abnormal situations in the future, thus reducing the workload of manually browsing normal pictures.

[0075] The inspection algorithm for mechanical equipment and regional locations (buildings) is described as follows: In the comparison between normal images and inspection images, the inspection images can be preprocessed by denoising, filtering, image correction, etc., and then the feature vectors of the normal images and inspection images are calculated and compared to obtain the areas with large differences in the images. Commonly used image correction algorithms through perspective transformation include rotation, translation, rigid body transformation, affine transformation, etc. The formula is:

[0076]

[0077] After obtaining enough abnormal images, we manually mark the abnormal parts, build a deep learning model, and use the model to learn various types of abnormalities.

[0078] The overall technical design of the 5G drone AI inspection system adopts a B / S (Browser / Server) structure: the browser on the client side assumes all display and control functions; the server provides data display support for the client side by connecting to the data system.

[0079] The client technology of the drone inspection system uses WebGL 2.0 technology to support the browser's 3D rendering capabilities. The client design requires the following three main goals:

[0080] (1) Display: It truly restores the control situation of the new area and can display and refresh in real time, smoothly and without any lag.

[0081] (2) Operation: It supports both embedded main system interface operation display and full-screen operation display, and provides a good operating experience; the operation is humanized and intuitive.

[0082] (3) Experience: The switching process of the main system is smooth, and there will be no long-term lag due to the loading of a large amount of resources.

[0083] In order to achieve the above goals, PR will be considered and the following technical functions will be implemented:

[0084] Key technical functions for display: data buffering, using a time-based data buffering dynamic interpolation method to buffer discrete data (especially position data) for a period of tens of seconds, and dynamically interpolate between data to ensure the smoothness of all 3D object display animations (for example, card collection and movement); rendering optimization, for 3D objects that affect rendering efficiency (for example, a large number of dynamic sub-objects): on the one hand, consider dynamic merging before publishing, and on the other hand, perform data-based dynamic mesh and material map generation and dynamic drawing for 3D objects that are sensitive to data form.

[0085] Key technical functions in operation: UI system, Web Assembly is used to drive the UI system in WebGL, all UI events only interact within the system, and will not interact with the outside through JavaScript; input and fonts, the internal UI supports Chinese fonts, Western fonts, and Chinese input methods; UI adaptation, the entire Canvas and UI system support detecting changes in browser size and achieve adaptation at the UI layer; key conflict, the main operation design avoids conflicts with browser shortcuts.

[0086] Key technical features in terms of experience: Cache mechanism, using the browser cache mechanism to cache the resources loaded for the first time locally, and directly load them from the local for the second time; in addition, the cache mechanism must support version changes and updates of resources. Dynamic loading, dynamic loading display in scene loading, thereby shortening the scene loading time. In addition, considering that the system will refresh the 3D scene in real time for a long time and with large amounts of data, and the browser has many restrictions on 3D display, the following core technical issues must also be considered comprehensively: Performance indicators, which can support smooth operation on mainstream mid-to-high-end graphics cards. Memory limit, the overall memory usage of the application cannot exceed the browser limit. Resource size, in order to improve download and loading speed and improve experience, the overall resource control size is below 256MB, and a more balanced resource compression algorithm is adopted. At the same time, quality requirements cannot be reduced due to resource saving. Network connection, due to browser restrictions, the WebSocket protocol is used to communicate with the server to ensure long connection and message push; at the same time, the design of disconnection and reconnection is considered.

[0087] The role of the drone inspection system server is mainly to serve as an intermediate buffer layer between the data processing system and the 3D WebGL client: extracting data and converting it into a data format that the client can understand, and then pushing it to the client.

[0088] Therefore, the server needs to be able to provide the following technical support:

[0089] (1) Status caching: By receiving the overall port data pushed by the data system, the status is cached and the data is updated based on the status difference, thereby reducing the overall data traffic.

[0090] (2) Session instance: supports different clients connecting at different time points and pushes the overall status to the client in the form of incremental data.

[0091] (3) Protocol conversion: Supports obtaining data from the data system through different protocols (most of which are based on the underlying Socket protocol), and ultimately provides data to the client using the unified WebSocket protocol.

[0092] (4) Event-driven: When a high-priority warning event is sent, it can actively push messages to the client.

[0093] 5G drone AI inspection private cloud technology solution, including basic resource layer and containerization layer.

[0094] The basic resource layer is built with multiple hosts with X86 architecture and GPU capabilities. As the hardware foundation of cloud services, it provides the most basic computing, storage, network and other functions.

[0095] The containerization layer is based on Kubernetes as an overall solution. All applications and algorithm services will be provided to the outside world in a containerized form.

[0096] Kubernetes is a complete distributed system support platform. Kubernetes has complete cluster management capabilities, including multi-level security protection and admission mechanisms, multi-tenant application support capabilities, transparent service registration and service discovery mechanisms, built-in intelligent load balancers, powerful fault detection and self-repair capabilities, service rolling upgrades and online expansion capabilities, scalable resource automatic scheduling mechanisms, and multi-granular resource quota management capabilities. At the same time, Kubernetes provides complete management tools that cover all aspects including development, deployment testing, operation and maintenance monitoring. Therefore, Kubernetes is a one-stop, complete distributed system development and support platform.

[0097] Kubernetes features:

[0098] (1) Portability: Supports public cloud, private cloud, hybrid cloud, and multi-cloud.

[0099] (2) Scalable: modular, plug-in, mountable, and combinable.

[0100] (3) Automation: automatic deployment, automatic restart, automatic replication, and automatic scaling / extension.

[0101] The points marked as work nodes in native Kubernetes are used as ordinary work nodes to meet container services such as service release, continuous program integration, continuous program delivery, automated testing, image warehouse, code warehouse, etc.

[0102] The GPU work node provides storage support for containers that need to use intelligent algorithms. The Storage node provides storage support for data persistence.

[0103] GPU support (Device Plugin), starting from version 1.8, introduces the Device Plugin (device plugin model), which provides a plugin-based way for device providers to enable external devices without modifying the Kubelet core code. Device providers only need to activate a device plugin in a DaemonSet on the computing node for Kubelet to call, and then use external devices.

[0104] Currently supported device types include GPU, high-performance NIC card, FPGA, InfiniBand, etc. This solution uses Device Plugin to implement containerized use of GPU devices. It allows applications that need GPU computing power to be dynamically allocated to the desired resources, and maximizes resource utilization and minimizes costs.

[0105] The service layer provides intelligent recognition algorithm API services:

[0106] The service layer provides RESTFUL, HTTP and other services of the recognition algorithm in the form of containers, and releases the final delivered software version after a series of tests and verifications. It can also provide API interfaces for other services and provide visual interfaces or SaaS services for derivative products of intelligent algorithms.

[0107] Kubernetes makes microservice architecture more convenient. It can easily provide CI / CD, automatic release, and automatic testing services in its architecture. Therefore, if resource consumption is acceptable, this architecture can also provide cloud platform support for other business systems.

[0108] The algorithm development layer provides customized development container images to quickly deploy the development environment and required controls, including the Tensorflow crossing recognition training set program and container number recognition training set program used in the project. It also supports lighter artificial intelligence and deep learning algorithm architectures such as Pytorch. It also provides development integrated WEB development interface (IDE), such as Zeppelin.

[0109] TensorFlow TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes represent mathematical operations in the graph, and edges represent multidimensional data arrays, or tensors, that are interconnected between nodes. Its flexible architecture allows you to perform computations on a variety of platforms, such as one or more CPUs (or GPUs) in desktop computers, servers, mobile devices, etc.

[0110] TensorFlow was originally developed by researchers and engineers at the Google Brain Group (part of Google's machine intelligence research organization) for research in machine learning and deep neural networks, but the versatility of the system makes it widely used in other computing fields.

[0111] Pytorch PyTorch's predecessor is Torch. Its underlying framework is the same as Torch, but a lot of content has been rewritten in Python. It is not only more flexible and supports dynamic graphs, but also provides Python interfaces. It was developed by the Torch7 team and is a Python-first deep learning framework that not only enables powerful GPU acceleration, but also supports dynamic neural networks, which are not supported by many mainstream deep learning frameworks such as Tensorflow. PyTorch can be seen as numpy with GPU support, but also as a powerful deep neural network with automatic differentiation. In addition to Facebook, it has been adopted by Twitter, CMU, and Salesforce.

[0112] Data requirements: In order to implement the 5G drone AI inspection system project more efficiently and reasonably, software vendors as data providers need to cooperate with the following:

[0113] Provide drone GPS coordinate data; video transmission protocols (RTMP, RTSP, etc.) are open to this system

[0114] The implementation principle of the present invention is:

[0115] The present invention can replace a large amount of manual input through the inspection work of drones, which is also of positive significance for reducing labor costs and personal safety protection; based on the rapid development of drone technology and 5G technology, drone inspections can greatly reduce the impact of conventional production and inspections on the production process of automated terminals, while reducing safety risks, and providing support for digital inspections of automated terminals. Collect video or image data of various business scenes captured by drone inspections, use artificial intelligence algorithms to analyze the content that can be machine-visually recognized, and combine drone dynamic cruise videos to perform AI machine vision recognition for container stacking, lashing, personnel safety, mechanical equipment location and equipment status, etc., effectively detect abnormal situations and issue warnings, identify potential inspection risks in a timely manner, and form special diagnostic reports for abnormal situations.

[0116] The embodiments of this specific implementation method are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A 5G UAV dynamic AI inspection system, characterized in that: include: A drone, wherein the drone is used to shoot a cruise video; UAV control system, which controls the cruise route, shoots and sends back real-time cruise video, and the UAV sends back location and power information; A video access and recognition output system, which receives real-time video data, identifies and analyzes security issues in the video in real time through an AI algorithm, and provides an emergency plan function; and a drone monitoring and early warning system, which displays coordinates, problem pictures and emergency plans of safety issues of drone cruise in real time; The drone control system, video access and identification output system, and drone monitoring and early warning system are respectively displayed on a single screen; The video access and recognition output system is mainly used to play the cruise video sent back by the drone control system, identify safety issues in the video through AI or manual, and automatically or manually select emergency plans, and push the recognition results and emergency plans to the drone monitoring and early warning system for three-dimensional all-round early warning and display; The drone monitoring and early warning system synchronously connects to the video and captured pictures of the corresponding position of the drone and automatically performs AI recognition, returns and saves the recognition result, and the recognition result is the recognition of suspected safety hazards or the recognition of serious safety hazard warnings; The visual module of the drone includes an inspection recognition module, which assists the drone in automatic inspection, and the inspection contents include inspection of box area and stacking, inspection of target in operation area, and inspection of mechanical equipment and regional position; The box area and stacking inspection includes: 3D container detection, determining the orientation of each side of the container and drawing the 3D coordinates of the container in the 2D image; Inspection of container area: after three-dimensional positioning of each container in the container area in the previous step, the coordinates of each vertex of the container and the container frame can be obtained, thereby judging whether the stacking and placement of the containers are correct; Check the density and information of the container area. The stacking density of the container area can be obtained through the previous step. Furthermore, if the image quality is high, the container number and dangerous goods container information can be identified. Container lashing identification: when the drone is inspecting the container area, it can identify whether the container lashing is completed and issue a warning according to the situation; The algorithm for the inspection of the box area and stacking is: Container 3D positioning: By building a neural network and using the Hough transform algorithm, the container can be positioned in 3D. The model uses MobileNetV3 as the skeleton of the convolution operation, and then uses Unet to sample the features at different sampling rates. The output results enter the Dual attention network to obtain the channel attention map and positionattention map respectively. The results are fused and merged with the intermediate layer output of MobileNetv3. Finally, the semantic segmentation results are output after Upsamble to the same resolution as the original image, and the corresponding pixel values ​​of the container surface are derived. 3D positioning of multiple containers in the container area: By performing 3D modeling on the vertices and border information of each container surface after positioning, the container placement and angle information can be obtained, thereby judging whether the container position meets the specifications; for the 3D positioning of the above containers, by further performing instance segmentation on the image segmentation results, the 3D positioning results of different containers can be obtained; Container information recognition: By extracting the containers and the segmented areas of each container surface in the previous step, if the image is clear enough, the number of containers stacked, the dangerous goods labels on the container surface, and the container number information can be identified; Binding identification: By identifying the main binding straps in the image, drawing the outline of the binding straps, confirming whether each box has been reinforced, and issuing an early warning; The target inspection in the working area is performed by drone vision to detect targets in the working area, including road detection, AGV, staff, and targets that may exist in other working areas; The target inspection of the operation area includes: Segmentation of work area types: using machine vision to divide the work area into different functional areas; Target positioning: the system can realize the positioning of AGV, internal and external container trucks, and staff; During the inspection of the mechanical equipment and regional locations by drones, manual inspection is required for the facades of high-rise buildings, steel structures, fenders on the sea side, rails in the container area, diversion dikes and levees; It also includes a client and a server, wherein the client is provided with all display and control functions by a browser; the server provides data display support for the client by connecting with a data system; It also includes cloud technology, which includes a basic resource layer and a containerization layer. The basic resource layer is built with multiple hosts with X86 architecture and GPU capabilities. As the hardware foundation layer of cloud services, it provides the most basic computing, storage, and network functions; the containerization layer uses Kubernetes as an overall solution, and all applications and algorithm services will provide services to the outside world in a containerized form.

Images