Intelligent inspection method and system for iron tower base station

Abstract

The invention relates to the technical field of base station inspection, and discloses an intelligent inspection method and system for an iron tower base station, and the method comprises the steps: firstly, obtaining real-time video information and multi-sensing data through the inspection of an unmanned plane, and transmitting the real-time video information and multi-sensing data to an edge gateway of the iron tower base station; secondly, identifying the real-time video information and the multi-sensing data by the iron tower base station edge gateway, and judging whether risk abnormity exists or not; then, if the risk abnormity does not exist, the iron tower base station edge gateway sends a first unmanned aerial vehicle information request to the unmanned aerial vehicle; then, the unmanned aerial vehicle sends first unmanned aerial vehicle information to an iron tower base station edge gateway based on the received first unmanned aerial vehicle information request; and finally, the iron tower base station edge gateway provides new route information for the unmanned aerial vehicle based on the received first unmanned aerial vehicle information. According to the method, the synchronization of inspection and analysis is realized, in addition, the inspection path and the inspection target of the unmanned aerial vehicle each time are not preset, the route is planned in real time, and the reliability and the inspection efficiency of unmanned aerial vehicle inspection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent inspection of communication towers, and in particular relates to an intelligent inspection method and system for tower base stations. Background Art

[0002] Existing intelligent inspections of communication towers typically use drones. These inspections typically follow the following procedures: If multiple inspection locations are required, the inspection route is typically pre-stored within the drone, which then inspects the entire tower along the route. However, existing drone inspections fail to account for the impact of wind speed and direction on the drone's power consumption, potentially preventing the inspection from completing. Furthermore, data from drone inspections cannot be transmitted until the inspection is complete, limiting the amount of data that can be stored. Furthermore, video data is typically large and cannot be transmitted over limited bandwidth (especially in wireless environments), making it impossible for drones to simultaneously capture and analyze video. Summary of the Invention

[0003] In response to the above problems, the present invention provides a tower base station intelligent inspection method and system, which realizes the synchronization of inspection and analysis, and plans route information in real time to improve inspection efficiency.

[0004] The purpose of the present invention is to provide an intelligent inspection method for tower base stations, comprising: The drone inspection obtains real-time video information and multi-sensor data and sends it to the tower base station edge gateway; The tower base station edge gateway identifies real-time video information and multi-sensor data to determine whether there are any risk anomalies; If there is no risk anomaly, the tower base station edge gateway sends a first drone information request to the drone; The drone sends the first drone information to the tower base station edge gateway based on the received first drone information request; The tower base station edge gateway provides new route information to the drone based on the received first drone information.

[0005] Furthermore, the drone inspection obtains real-time video information and sends it to the tower base station edge gateway including, The drone compresses the real-time video information obtained during the inspection based on the first video processing model and sends it to the tower base station edge gateway; The tower base station edge gateway decompresses the compressed real-time video information based on the second video processing model to obtain the decompressed real-time video information.

[0006] Furthermore, the first video processing model and the second video processing model are both video processing models, and the video processing model includes a spatiotemporal feature extractor, a classifier, and a decoder, wherein: Spatiotemporal feature extractor, used for spatial feature extraction and spatiotemporal feature fusion of video data frames; A classifier is used to receive video spatial features and video spatiotemporal features as input, and further fuse the video spatial features and video spatiotemporal features to obtain a stage classification result; The decoder is used to receive the video spatiotemporal features as input and decode the video spatiotemporal features back into video frame data.

[0007] Furthermore, it also includes setting a classification label in the classifier, wherein the classification label is risk anomaly, including illegal intruders, damaged devices, open flames, and open smoke. The tower base station edge gateway identifies the real-time video information and determines whether there is a risk anomaly, including: Check whether the classification label exists in the stage classification results output by the classifier. If so, there is a risk anomaly.

[0008] Furthermore, the method further includes obtaining a video data training set, and training a video processing model using the obtained video data training set to obtain a trained video processing model, which specifically includes the following steps: Initialize the video processing model; Inputting the acquired video data training set into the spatiotemporal feature extractor to obtain video spatial features and video spatiotemporal features; Inputting the video spatial features and the video spatiotemporal features into a classifier to obtain a predicted video frame category result; Obtaining a first classification loss function according to the predicted video frame category result; Input the video spatiotemporal features into the decoder to obtain the reconstructed video frames; Obtaining a second classification loss function based on the reconstructed video frame; Obtain a multi-task joint loss function based on the first classification loss function and the second classification loss function; Repeat the above steps until the iteration is completed or the loss value tends to be stable, then the training is completed, and the model parameters at the time of training completion are obtained.

[0009] Furthermore, the tower base station edge gateway identifies multi-sensor data including: Based on the acquired multi-sensor data, the tower base station edge gateway constructs a benchmark 3D scene model of the tower base station through self-evolving neural radiation fields; Acquire updated multi-sensor data, update the tower base station baseline three-dimensional scene model, and obtain an updated tower base station three-dimensional scene model; The initial scene of the tower base station benchmark three-dimensional scene model is compared with the updated scene of the updated tower base station three-dimensional scene model to identify whether the tower base station has structural abnormalities.

[0010] Furthermore, updated multi-sensor data is obtained to update the tower base station benchmark three-dimensional scene model, and the updated tower base station three-dimensional scene model includes: Add the updated multi-sensor data to the scene memory library M and form a multi-sensor data block; Extract new data blocks corresponding to specific scene areas from multi-sensor data blocks; Based on the extracted new data blocks corresponding to the specific scene area, the specific scene area is updated based on the local scene loss function; Based on the back propagation algorithm, the corresponding parameters of the specific scene area are updated, and the parameters of other areas except the specific scene area in the tower base station benchmark three-dimensional scene model are kept unchanged to obtain the updated tower base station three-dimensional scene model.

[0011] Furthermore, the initial scene of the tower base station benchmark three-dimensional scene model is compared with the updated scene of the updated tower base station three-dimensional scene model to identify whether the tower base station has structural abnormalities, including identifying one or more differences among volume density difference, color value difference, and geometric shape difference, including: A threshold value of the corresponding difference is set, wherein if the corresponding difference calculation result exceeds the threshold, it is marked as a potential abnormal area and located.

[0012] Furthermore, after the tower base station edge gateway completes the recognition of the real-time video information and the multi-sensor data, it deletes the real-time video information and the multi-sensor data.

[0013] Furthermore, the first drone information includes current drone power information. The tower base station edge gateway provides the drone with new route information based on the received first drone information, including: Obtain the current wind speed information, wind direction information, and the wind speed information and wind direction information within the future estimated time period of the weather forecast respectively; Inputting the current wind speed information, the wind speed information for the future estimated time period from the weather forecast, and the current UAV power information into a power consumption prediction model to obtain an estimated power consumption; Acquire a wind direction coefficient database, wherein the wind direction coefficient database includes multiple wind direction coefficients and angle information between the actual flight direction and the wind direction corresponding to each wind direction coefficient; Acquire a navigation path database, wherein the navigation path database includes at least one navigation path and a flight direction and a flight distance of the corresponding navigation path; The wind direction coefficient is obtained according to the flight direction of the navigation path and the wind direction coefficient database; Multiply the estimated power consumption by the wind direction coefficient to obtain the actual estimated power consumption; Get the flight distance based on the current drone power information and the actual estimated power consumption; A navigation path whose flight distance is less than the flightable distance is selected as the next navigation data of the UAV and sent to the UAV.

[0014] Furthermore, the power consumption model is an Elman-based neural network, and the input of the power consumption model during training is wind speed information and second UAV information, wherein the second UAV information includes flight speed information and flight altitude information.

[0015] Another object of the present invention is to provide an intelligent inspection system for a tower base station, comprising a drone and an edge gateway for a tower base station, wherein: A drone, configured to obtain real-time video information and multi-sensor data and send the data to the tower base station edge gateway, and send first drone information to the tower base station edge gateway based on the received first drone information request; The tower base station edge gateway is used to identify real-time video information and multi-sensor data to determine whether there is a risk anomaly. If there is no risk anomaly, a first drone information request is sent to the drone, and new route information is provided to the drone based on the received first drone information.

[0016] In the intelligent inspection method of the present invention, during the drone inspection process, the drone sends the inspection video data to the tower base station edge gateway. The tower base station edge gateway can directly process the video data during the drone inspection process, thereby achieving synchronization between inspection and analysis. In addition, the drone's inspection path and inspection target are not pre-set each time. Instead, new route information is provided to the drone based on the first drone information, thereby preventing the situation where the inspection path is pre-set but the inspection task cannot be completed, thereby improving the reliability and efficiency of the drone inspection.

[0017] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flow chart of an intelligent inspection method for a tower base station according to an embodiment of the present invention is shown; Figure 2 A flow chart of a model training method for video processing according to an embodiment of the present invention is shown; Figure 3 A schematic structural diagram of an intelligent inspection system for a tower base station in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0021] like Figure 1 As shown, an embodiment of the present invention introduces a tower base station intelligent inspection method. The inspection method includes: first, a drone inspection obtains real-time video information and multi-sensor data and sends it to the tower base station edge gateway; second, the tower base station edge gateway identifies the real-time video information and multi-sensor data to determine whether there is a risk anomaly; then, if there is no risk anomaly, the tower base station edge gateway sends a first drone information request to the drone; then, based on the received first drone information request, the drone sends the first drone information to the tower base station edge gateway; finally, the tower base station edge gateway provides the drone with new route information based on the received first drone information. During the drone inspection process, the drone sends the inspection video data to the tower base station edge gateway. The tower base station edge gateway can directly process the video data during the drone inspection process, thereby achieving synchronization of inspection and analysis. In addition, the drone's inspection path and inspection target are not pre-set each time. Instead, new route information is provided to the drone based on the first drone information, thereby preventing the situation where a pre-set inspection path fails to complete the inspection task.

[0022] Specifically, the drone inspection process acquires real-time video information and sends it to the tower base station edge gateway. First, a first video processing model and a second video processing model are established. The first and second video processing models can be the same video processing model. The video processing model includes a spatiotemporal feature extractor, a classifier, and a decoder. The spatiotemporal feature extractor extracts and fuses spatial features from video data frames. The classifier receives video spatial features and spatiotemporal features as input and further fuses the spatial and spatiotemporal features to generate a stage-by-stage classification result. The stage-by-stage classification result is based on a classification label. The classifier sets the classification label, which includes risk anomalies, including illegal intruders, damaged equipment, open flames, and open smoke. The decoder receives the spatiotemporal features as input and decodes them back into video frame data. The drone compresses the real-time video information acquired during the inspection based on the video processing model and sends it to the tower base station edge gateway. The tower base station edge gateway decompresses the compressed real-time video information based on the video processing model to obtain the decompressed real-time video information. That is, the aforementioned video processing model is used for compression, decompression, and real-time video information recognition. Specifically, compression utilizes a spatiotemporal feature extractor, enabling the video processing model to significantly compress transmitted video data, thereby saving bandwidth. Furthermore, decompression utilizes a decoder, while recognition utilizes the spatiotemporal feature extractor and classifier. The final output indicates risk anomalies, including the presence of intruders, device damage, open flames, and visible smoke.

[0023] In the embodiment of the present invention, an alarm is issued if there is a risk or the structure appearance is abnormal.

[0024] In an embodiment of the present invention, the method further comprises obtaining a video data training set, and then training a video processing model using the obtained video data training set, thereby obtaining a trained video processing model, such as Figure 2 As shown, the specific steps include: Step S1, initializing the video processing model, that is, initializing the parameters and training parameters in the video processing model; Step S2: inputting the acquired video data training set into the spatiotemporal feature extractor to obtain video spatial features and video spatiotemporal features; Step S3: inputting the video spatial features and the video spatiotemporal features into a classifier to obtain a predicted video frame category result; Step S4: obtaining a first classification loss function based on the predicted video frame category result; wherein, first, the video spatial features and the video spatiotemporal features are input into the classifier; secondly, the Trans layer (network layer of the Transformer structure, Transformer is a neural network structure based on the "self-attention mechanism") in the classifier further fuses the spatial and temporal features using a multi-head self-attention mechanism, and classifies the fused features using a multi-layer perceptron (MLP, a basic artificial neural network model) to obtain a predicted video frame category result; then, based on the obtained predicted video frame category result and the true label of the video frame in the video data training set, a loss calculation is performed to minimize the difference between the classification result of the video frame and the true label value. The obtained classification loss function is as follows:

[0025] Among them, L class ( ) represents the classification loss value, i refers to the i-th category sample, is the true label value of the video frame, is the predicted video frame classification result, and n is the number of categories. The loss value is calculated through the loss function, and the network parameters are updated in the direction of decreasing the loss value. The network parameters that need to be updated are the classifier and spatiotemporal feature extractor.

[0026] Step S5: input the video spatiotemporal features into the decoder to obtain a reconstructed video frame; Step S6: Obtain a second classification loss function based on the reconstructed video frame; wherein, the loss is calculated between the reconstructed video frame and the real video frame in the acquired video data training set to minimize the difference between the reconstructed video frame and the real video frame. The obtained classification loss function is as follows:

[0027] Among them, L recon represents the reconstruction loss value, is a real video frame in the video data training set, is the reconstructed video frame generated by the model, and C is the number of video frame categories. The loss value is calculated through the loss function, and the network parameters are updated in the direction of decreasing the loss value. The network parameters that need to be updated include the spatiotemporal feature extractor and decoder. i refers to the i-th category sample.

[0028] Step S7: According to the first classification loss function And the second classification loss function Obtain the multi-task joint loss function; that is, the classification loss and reconstruction loss of the integrated stage are formed to form a multi-task joint loss function:

[0029] in, and is the weight parameter used to balance the two losses. The entire model parameters, including the spatiotemporal feature extractor, classifier, and decoder, are optimized through back propagation.

[0030] Step S8: Repeat the above steps until the iteration is completed or the loss value tends to be stable, then the training is completed, and the model parameters at the time of training completion are obtained, and then the model parameters at the time of training completion are used to obtain the trained video processing model.

[0031] In an embodiment of the present invention, the tower base station edge gateway identifies multi-sensor data, including: first, based on the acquired multi-sensor data, the tower base station edge gateway constructs a three-dimensional scene model through a self-evolving neural radiation field; then, based on the constructed three-dimensional scene model, an initial scene and an updated scene are obtained; finally, the initial scene and the updated scene are compared to identify whether there is any structural abnormality in the tower base station.

[0032] Specifically, a Self-Evolving Neural Radiance Field (SENRF) model is deployed on each tower base station edge gateway. First, the SENRF model constructs an initial 3D scene of the tower base station based on multi-sensor data transmitted by drones. This 3D scene includes the tower base station's surroundings. This multi-sensor data includes multimodal data such as LiDAR point clouds, image information, and infrared images. This multimodal fusion enhances the model's ability to capture scene details and improves the quality of 3D reconstruction. Furthermore, as drones continue to patrol, new data continuously arrives at the edge gateway. The SENRF model continuously learns from this new multi-sensor data through incremental scene updates and online adaptive optimization mechanisms, dynamically updating the 3D scene representation to adapt to environmental changes.

[0033] Furthermore, the core of the SENRF model is a multi-layer perceptron (MLP) network that maps the input spatial coordinates and viewing direction to the output volume density and color values, thereby implicitly representing the three-dimensional scene. The mathematical expression of the MLP network is: ; in, Represents the position coordinates in three-dimensional space; Indicates the viewing direction, expressed in spherical coordinates, θ is the azimuth, is the elevation angle; Indicates the volume density, that is, the degree of light blocking at this location. ≥0. Represents the color value (ie RGB value), the color seen when observing position x from the viewing direction d, r, g, b ∈ [0, 1], F represents a mapping function, Represents the trainable parameters of the MLP network.

[0034] In order to enhance the network's ability to model high-frequency details, the input coordinate x and viewing direction d are encoded with high-frequency positions:

[0035] Where p represents the input coordinate x, y, z or the viewing direction component θ, , L is the number of encoding functions, which controls the dimension after encoding, and the dimension after encoding is 2L.

[0036] After encoding, the input of the MLP network becomes:

[0037] Among them, ⊕ represents the vector concatenation operation, and the weights of the MLP network are initialized using the He initialization method (also known as Kaiming initialization). For the fully connected layer, the weight W is sampled from the normal distribution. 、 Represent the encoded coordinate position and viewing direction respectively.

[0038] Given a camera ray , o is the camera center, and t is a scalar parameter, equivalent to the "distance" of light propagation. The SENRF model renders the pixel color C(r) corresponding to the light by sampling and integrating along the light:

[0039] Where o is the camera center, and are the depths of the near and far clipping planes respectively; is the transmittance, which indicates the light The probability of not being blocked at t, r( s ) indicates the depth of the ray s The spatial location of the .

[0040] In actual calculations, the integral is approximated by a discrete sum:

[0041] Where N is the sampling point, i represents the i-th sampling point, is the sampling step size of the i-th sampling point; is the cumulative transmittance, j represents the jth sampling point, 、 represents the volume density of the i-th and j-th sampling points, Represents the color value of the i-th sampling point, The sampling step size of the jth sample point.

[0042] The mean square error (MSE) is used as the reconstruction loss function to measure the difference between the rendered color and the true color: , Where: r is a single pixel element in the set R, and R is the rendered pixel set; is the actual pixel color of the image. , optimize the network parameters Θ, make the rendered image close to the real image, complete the initial scene reconstruction, and establish the benchmark 3D scene model of the tower base station.

[0043] In an embodiment of the present invention, when a scene is updated, it is primarily updated through the scene memory. Specifically, the newly acquired multi-sensor data stream Dstream (discretized data stream), i.e., the multi-sensor data, is added to the scene memory M. A first-in-first-out (FIFO) or scene importance-based strategy is used to manage the memory capacity, retaining the most representative historical scene information and providing a data foundation for subsequent global consistency maintenance. The pre-processed multi-sensor data stream Dstream, acquired in real time, includes image information, lidar point clouds, infrared images (optional), and other data acquired by the drone during this inspection, and has been pre-processed to ensure data quality. The image information is high-definition.

[0044] Add the newly acquired multi-sensor data stream Dstream to the scene memory M. The scene memory M is a data buffer with limited capacity, which is used to store historical scene data and provide long-term memory and context information for the model. The memory capacity is managed using a first-in-first-out (FIFO) or scene importance-based strategy. When the memory reaches its capacity limit, the earliest data that enters the memory or the data that is judged to be less important will be removed to make room for new data, ensuring that the most representative historical scene information is always retained in the memory. Updated scene memory M updated This contains representative scene data from the start of the inspection to the current moment, providing a data foundation for subsequent global consistency maintenance. When new data arrives, the incremental scene update mechanism is triggered, performing local scene updates in the areas covered by the new data, improving the model's adaptability to new data while avoiding the high computational cost of global retraining.

[0045] Furthermore, the newly collected pre-processed multi-sensor data block Dnew is extracted from Dstream, which is related to the specific scene area. The current SENRF neural radiation field network parameter Θ represents the modeling ability of the SENRF model for the current scene and is the basis for incremental updates. The updated scene memory library M updated : Provides historical scene context to assist in local updates and global consistency maintenance, including the following processing steps: Construction of local scene loss function: Construct a local scene loss function L for the scene area covered by the new data block Dnew local , focusing on the reconstruction error of the new data area, L local Only the difference between the rendering result and the real data in the new data area is calculated to guide the network parameters to perform local updates. The local scene loss function is defined as: ; in, Represents the new data block D new The number of pixels contained in ; Represents a piece of data in a data block, r is a pixel element, is the actual collected pixel color; is the pixel color rendered using the SENRF model, These are some parameters in the network that are responsible for the representation of new data areas.

[0046] Back propagation and local parameter update: Calculate the local scene loss function through the back propagation algorithm Gradient with respect to the network parameters Θ Only the parameters in the network responsible for the representation of the new data area are updated , while keeping other regional parameters unchanged. The parameter update formula is: ; in, is the updated local network parameter, is the learning rate of local update, which controls the step size of local parameter update.

[0047] Output: Updated SENRF neural radiation field network parameters : After the local scene is updated, the network parameters responsible for the representation of the new data area are updated, making the model's scene representation of the new data area more accurate.

[0048] In order to make full use of the multi-sensor information carried by UAVs, the SENRF model is adopted to improve the accuracy and robustness of three-dimensional scene reconstruction, providing a more reliable basis for structural shape anomaly detection.

[0049] This also includes feature extraction from multi-sensor data. For example, a pre-trained or online-learned convolutional neural network (CNN) is used as an image feature extractor. It takes a high-definition image as input and outputs an image feature map (FRGB), capturing texture, edge, and semantic information. A graph neural network (GNN) or a point cloud processing network such as PointNet is used as a point cloud feature extractor. It takes a lidar point cloud as input and outputs a point cloud feature map (FLiDAR), capturing geometric shape and spatial structure. If infrared images are used, a similar CNN structure is used as an infrared feature extractor. It takes an infrared image as input and outputs an infrared feature map (FIR), capturing temperature distribution information.

[0050] Using sensor calibration parameters and spatiotemporal synchronization information, the features of different modalities are spatially and temporally aligned and calibrated to ensure that the features correspond to the same scene area and time. Design a decoder network and input the fused multimodal features F fused , decoding the volume density of the scene and color values , updating the scene representation. The decoded volume density and color values ​​are incorporated into SENRF's neural radiance field to refine the scene representation, improving reconstruction accuracy and detail richness. Feature extraction, cross-modal attention fusion, and fused feature decoding are integrated into SENRF's end-to-end training framework for joint optimization, minimizing the final reconstruction loss and improving overall performance.

[0051] In this embodiment, the difference between the updated scene and the initial scene is calculated. This difference calculation can be performed at multiple levels, including volume density difference: comparing the volume density values ​​of the two scenes at the same spatial location. Significant changes in volume density may indicate changes in the structural shape, such as missing or deformed components. Volume density differences reflect changes in the material distribution of the scene in three-dimensional space and can be used to detect missing structural components, deformation, or the addition of new objects.

[0052] Volume density query: For each sampling point , respectively, using the updated scene neural radiance field and the initial scene neural radiation field Query the volume density value of the point, where the scene volume density is updated: , Where d is an arbitrary viewing direction (in volume density comparison, viewing direction does not affect the volume density value).

[0053] In addition, the initial scene volume density: .

[0054] Volume density difference calculation: Calculate each sampling point x i The difference in body density The degree of difference can be measured using absolute or relative differences.

[0055] Absolute difference: .

[0056] Color value difference: compare the color values ​​of two scenes at the same viewing direction. Color differences may reflect changes in surface material or foreign matter. , along the camera ray starting from the camera origin o , respectively, using the updated scene neural radiance field and the initial scene neural radiation field Render the pixel color value at this viewpoint. Update the scene color value: ; Initial scene color value: ; In actual calculation, the integral is approximated by discrete summation. Color value difference calculation: Calculate the value for each viewing angle The color value difference Color differences can be measured using Euclidean distance or distance after color space conversion (such as Lab color space difference).

[0057] Geometric shape difference: Extract 3D geometric shapes from the volume density field through methods such as isosurface extraction, and compare the differences in geometric shapes before and after the update, such as changes in surface normals and component size.

[0058] Based on the scene difference calculation results, an appropriate threshold is set to identify structural anomalies. Areas with differences exceeding the threshold are marked as potential anomalies and located. Based on the anomaly detection results, a risk assessment is conducted, an anomaly report is generated, and relevant operations and maintenance personnel are notified. The risk assessment can consider the severity, scope, and potential risk level of the anomaly. The anomaly report should include the 3D coordinates and extent of the anomaly area, a description of the anomaly type, an assessment of the anomaly's severity and risk level, an image or video clip of the anomaly area (optional), and recommended countermeasures. This model can continuously learn from new data collected during continuous drone inspections, dynamically optimizing and updating the neural radiation field, enabling the model to adapt to environmental changes, such as new facilities around tower base stations and seasonal vegetation changes. This self-evolutionary capability enables SENRF to maintain the accuracy and timeliness of scene representation over the long term without requiring full retraining.

[0059] In an embodiment of the present invention, the first drone information includes current drone power information, so that the tower base station edge gateway provides new route information to the drone based on the first drone information, including the following steps: Step S11: After receiving the current drone power information, the drone obtains the current wind speed information, wind direction information, and the wind speed information and wind direction information within the future estimated time period of the weather forecast; wherein the estimated time period can be 1 hour or half an hour, but is not limited thereto, and other time periods such as 2 hours are also applicable to the present invention.

[0060] Step S12: Input the current wind speed information, the wind speed information for the estimated future time period from the weather forecast, and the received current drone power information into a power consumption prediction model to obtain an estimated power consumption. The power consumption unit can be per minute or per hour, typically determined based on training data. Furthermore, the method includes obtaining a trained power consumption prediction model from the tower base station edge gateway. The power consumption prediction model can be based on an Elman neural network, with the wind speed information and the second drone information being the training inputs. The second drone information can include flight speed information, flight altitude information, and other information.

[0061] Step S13: Obtain a wind direction coefficient database, which includes multiple wind direction coefficients and the angle information between the actual flight direction and the wind direction corresponding to each wind direction coefficient. The wind direction coefficient can be obtained based on experiments. For example, if the estimated flight direction of the drone is south and the wind direction is north (i.e., the angle information is 180 degrees), then the distance of the actual flight preset time and the distance of the flight preset time (1 hour) in a windless state are obtained at a predetermined flight speed. The ratio of the two distances is the wind direction coefficient for the estimated flight direction being south and the wind direction being north. The preset time can be, but is not limited to, 1 hour. A time of 30 minutes, etc., is also applicable to the present invention.

[0062] Step S14: Acquire a navigation path database, wherein the navigation path database includes at least one navigation path and a flight direction and a flight distance of the corresponding navigation path; Step S15: Obtaining the wind direction coefficient according to the flight direction of the navigation path and the wind direction coefficient database; Step S16: multiply the estimated power consumption by the wind direction coefficient to obtain the actual estimated power consumption; Step S17: Obtain the flight distance based on the current UAV power information and the actual estimated power consumption; Step S18: Select a navigation path whose flight distance is less than the permitted flight distance as the next navigation data for the drone and transmit it to the drone. Furthermore, if multiple navigation paths have flight distances less than the permitted flight distance, the smallest navigation path is selected. Drone power consumption is generally not linear and varies depending on the weather conditions (particularly wind speed and direction). Such variations are difficult to predict. Therefore, the above method eliminates the need for such a prediction. The route to the next inspection location is transmitted after reaching one inspection base station. This method, therefore, makes tower base station inspections more intelligent and efficient.

[0063] In an embodiment of the present invention, after the tower base station edge gateway analyzes the video data, the drone can delete the acquired real-time video information and multi-sensor data, thereby saving space and making room for inspection at the next location.

[0064] like Figure 3 As shown, an embodiment of the present invention also introduces an intelligent inspection system for tower base stations that can execute the above-mentioned method, wherein the system includes a drone and a tower base station edge gateway, wherein the drone is used to obtain real-time video information and multi-sensor data, and send it to the tower base station edge gateway, and based on the received first drone information request, send the first drone information to the tower base station edge gateway; the tower base station edge gateway is used to identify the real-time video information and multi-sensor data, and determine whether there is a risk anomaly, wherein if there is no risk anomaly, send the first drone information request to the drone, and provide new route information to the drone based on the received first drone information. When the tower base station edge gateway completes the recognition of real-time video information and believes that there is no risk, in theory the drone should fly to the next inspection target or return to a drone station, or directly stop at the tower base station edge gateway of this application. However, if these cruise paths are pre-designed, there is no way to consider the actual situation of the drone. For example, most of the existing technologies directly set the entire inspection path for the drone. If the drone’s power is not enough to support the next inspection point, it will automatically return, etc., which results in the drone having only the choice of going to the next inspection point or returning. The above-mentioned system requests the first drone information (such as power information, etc.) from the drone after the drone has inspected a tower base station edge gateway, and sets other inspection paths for the drone based on the current drone’s power information. In this way, the drone’s inspection path can be changed according to the actual inspection situation, making the inspection of the tower base station more intelligent and efficient.

[0065] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tower base station intelligent inspection method, characterized in that: include, The drone inspection obtains real-time video information and multi-sensor data and sends it to the tower base station edge gateway; The tower base station edge gateway identifies real-time video information and multi-sensor data to determine whether there are any risk anomalies; If there is no risk anomaly, the tower base station edge gateway sends a first drone information request to the drone; The drone sends the first drone information to the tower base station edge gateway based on the received first drone information request; The tower base station edge gateway provides new route information to the drone based on the received first drone information.

2. The intelligent inspection method for tower base stations according to claim 1, characterized in that: The drone inspection obtains real-time video information and sends it to the tower base station edge gateway including: The drone compresses the real-time video information obtained during the inspection based on the first video processing model and sends it to the tower base station edge gateway; The tower base station edge gateway decompresses the compressed real-time video information based on the second video processing model to obtain the decompressed real-time video information.

3. The intelligent inspection method for tower base stations according to claim 2, characterized in that: The first video processing model and the second video processing model are both video processing models, and the video processing model includes a spatiotemporal feature extractor, a classifier, and a decoder, wherein: Spatiotemporal feature extractor, used for spatial feature extraction and spatiotemporal feature fusion of video data frames; A classifier is used to receive video spatial features and video spatiotemporal features as input, and further fuse the video spatial features and video spatiotemporal features to obtain a stage classification result; The decoder is used to receive the video spatiotemporal features as input and decode the video spatiotemporal features back into video frame data.

4. The intelligent inspection method for tower base stations according to claim 3, characterized in that: It also includes setting classification labels in the classifier, wherein the classification labels are risk anomalies, including illegal intruders, damaged devices, open flames, and open smoke. The tower base station edge gateway identifies the real-time video information to determine whether there is a risk anomaly. include, Check whether the classification label exists in the stage classification results output by the classifier. If so, there is a risk anomaly.

5. The intelligent inspection method for tower base stations according to claim 4, characterized in that: The method further includes obtaining a video data training set, and training a video processing model using the obtained video data training set to obtain a trained video processing model, specifically including the following steps: Initialize the video processing model; Inputting the acquired video data training set into the spatiotemporal feature extractor to obtain video spatial features and video spatiotemporal features; Inputting the video spatial features and the video spatiotemporal features into a classifier to obtain a predicted video frame category result; Obtaining a first classification loss function according to the predicted video frame category result; Input the video spatiotemporal features into the decoder to obtain the reconstructed video frames; Obtaining a second classification loss function based on the reconstructed video frame; Obtain a multi-task joint loss function based on the first classification loss function and the second classification loss function; Repeat the above steps until the iteration is completed or the loss value tends to be stable, then the training is completed, and the model parameters at the time of training completion are obtained.

6. The intelligent inspection method for tower base stations according to claim 1, characterized in that: The tower base station edge gateway identifies multi-sensor data including: Based on the acquired multi-sensor data, the tower base station edge gateway constructs a benchmark 3D scene model of the tower base station through self-evolving neural radiation fields; Acquire updated multi-sensor data, update the tower base station baseline three-dimensional scene model, and obtain an updated tower base station three-dimensional scene model; The initial scene of the tower base station benchmark three-dimensional scene model is compared with the updated scene of the updated tower base station three-dimensional scene model to identify whether the tower base station has structural abnormalities.

7. The intelligent inspection method for tower base stations according to claim 6, characterized in that: Obtain updated multi-sensor data and update the tower base station benchmark three-dimensional scene model to obtain an updated tower base station three-dimensional scene model including: Add the updated multi-sensor data to the scene memory library M and form a multi-sensor data block; Extract new data blocks corresponding to specific scene areas from multi-sensor data blocks; Based on the extracted new data blocks corresponding to the specific scene area, the specific scene area is updated based on the local scene loss function; Based on the back propagation algorithm, the corresponding parameters of the specific scene area are updated, and the parameters of other areas except the specific scene area in the tower base station benchmark three-dimensional scene model are kept unchanged to obtain the updated tower base station three-dimensional scene model.

8. The intelligent inspection method for tower base stations according to claim 7, characterized in that: Comparing the initial scene of the tower base station benchmark 3D scene model with the updated scene of the updated tower base station 3D scene model, identifying whether the tower base station has structural abnormalities includes identifying one or more differences among volume density difference, color value difference, and geometric shape difference, including: A threshold value of the corresponding difference is set, wherein if the corresponding difference calculation result exceeds the threshold, it is marked as a potential abnormal area and located.

9. The intelligent inspection method for tower base stations according to any one of claims 1 to 8, characterized in that: After the tower base station edge gateway completes the recognition of real-time video information and multi-sensor data, it deletes the real-time video information and multi-sensor data.

10. The intelligent inspection method for tower base stations according to any one of claims 1 to 8, characterized in that: The first drone information includes the current drone power information. The tower base station edge gateway provides the drone with new route information based on the received first drone information, including: Obtain the current wind speed information, wind direction information, and the wind speed information and wind direction information within the future estimated time period of the weather forecast respectively; Inputting the current wind speed information, the wind speed information for the future estimated time period from the weather forecast, and the current UAV power information into a power consumption prediction model to obtain an estimated power consumption; Acquire a wind direction coefficient database, wherein the wind direction coefficient database includes multiple wind direction coefficients and angle information between the actual flight direction and the wind direction corresponding to each wind direction coefficient; Acquire a navigation path database, wherein the navigation path database includes at least one navigation path and a flight direction and a flight distance of the corresponding navigation path; The wind direction coefficient is obtained according to the flight direction of the navigation path and the wind direction coefficient database; Multiply the estimated power consumption by the wind direction coefficient to obtain the actual estimated power consumption; Get the flight distance based on the current drone power information and the actual estimated power consumption; A navigation path whose flight distance is less than the flightable distance is selected as the next navigation data of the UAV and sent to the UAV.

11. The intelligent inspection method for tower base stations according to claim 10, characterized in that: The power consumption model is an Elman-based neural network. The input of the power consumption model during training is wind speed information and second UAV information, wherein the second UAV information includes flight speed information and flight altitude information.

12. An intelligent inspection system for tower base stations, characterized in that: Including drones and tower base station edge gateways, among which, A drone, configured to obtain real-time video information and multi-sensor data and send the data to the tower base station edge gateway, and send first drone information to the tower base station edge gateway based on the received first drone information request; The tower base station edge gateway is used to identify real-time video information and multi-sensor data to determine whether there is a risk anomaly. If there is no risk anomaly, a first drone information request is sent to the drone, and new route information is provided to the drone based on the received first drone information.

Images