A data processing terminal based on Beidou satellite data and AI graphics fusion

Abstract

The application discloses a kind of data processing terminal based on Beidou satellite data and AI graphics fusion, comprising: main control unit, imaging module, Beidou positioning module, Beidou timing module, communication module, short message module, storage module and power module, the terminal is set in iron tower body, the imaging module is continuously obtained with set time interval adjacent iron tower image, and image timestamp generated by Beidou timing module is marked, image with timestamp is sent to main control unit, and the power module other modules in terminal power supply;Main control unit pre-processes received image, uses sparse light flow algorithm to calculate the offset vector of corner point in adjacent two images, and uses offset vector to judge whether iron tower is inclined or not.The application can accurately and quickly identify the inclination of iron tower by combining Beidou timing and Beidou positioning, and accurately report fault point.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a data processing terminal based on Beidou satellite data and AI graphics fusion. BACKGROUND

[0002] The iron tower refers to a high tower made of steel materials, and the traditional iron tower can be used in technical fields such as railways, electric power and communication. The iron tower is generally built in deep mountains, remote areas, and due to reasons such as earthquakes and disasters, the location of the iron tower may change in geology, and the iron tower is easy to tilt with the foundation, when the tilt angle reaches a certain degree, the root opening and the leg height difference of the tower body change, the tower body structure produces a large additional stress, and when the external force such as wind blowing and icing is serious, the tower body may be locally damaged or even collapsed, which directly threatens the safety of the iron tower and the stable operation of the line. The traditional detection of the tilt condition of the iron tower mainly relies on artificial periodic line inspection, and there is a problem of high detection difficulty.

[0003] Optical flow is the instantaneous velocity of the pixel motion of a spatial moving object on an observation imaging plane. The optical flow method is a method for finding the correspondence between the previous frame and the current frame by using the change of the pixels in the time domain and the correlation between the adjacent frames, so as to calculate the motion information of the object between the adjacent frames. The Chinese invention patent with the publication number CN105023278B proposes a motion target tracking method and system based on the optical flow method, which includes providing a video image and pre-processing the image to generate a pre-processed image; performing edge detection on the pre-processed image and extracting target information from the pre-processed image by using the optical flow method, and fusing the information obtained by the edge detection and the extracted target information to generate a complete motion target; using the optical flow method to estimate and analyze the motion target, and using a forward-backward error algorithm based on feature point trajectories to remove the error matching points caused by the light. A template image is created and template image matching is performed to track the motion target. The optical flow method detects the displacement amplitude and displacement direction of the object in the image, and according to the displacement direction, the interference of normal swinging can be removed, but the conventional optical flow method needs to detect all the pixels in the gray image, and there is a problem of low detection efficiency. SUMMARY

[0004] The application provides a data processing terminal based on Beidou satellite data and AI graphics fusion, which fuses Beidou satellite data and image processing data when detecting the tilt of the iron tower, and improves the optical flow method, aiming to solve the problems of low detection efficiency and insufficient accuracy when using the conventional optical flow method to detect the tilt of the iron tower.

[0005] The data processing terminal provided by the application comprises a master control unit, an imaging module, a Beidou positioning module, a Beidou timing module, a communication module, a short message module, a storage module and a power module, the terminal is arranged on a tower body, the imaging module continuously acquires images of adjacent towers at a set time interval, and the images are marked with time stamps generated by the Beidou timing module, and the images with the time stamps are sent to the master control unit, and the power module supplies power to other modules in the terminal.

[0006] The master control unit is configured to perform the following tower tilt detection method:

[0007] S1: The master control unit receives the images sent by the imaging module, pre-processes the images, generates edge images, and then saves the edge images in time sequence to the storage module, the pre-processing comprises sequentially arranged graying, twice downsampling and edge detection, the edge detection detects the images output by the graying, the first downsampling and the second downsampling respectively, and outputs three-scale edge images.

[0008] S2: The master control unit compares every two continuous three-scale edge images in the storage module, and the comparison method adopts a sparse optical flow algorithm, first calculates the corner points in the first edge image, and calculates the displacement amplitude and direction of the corner points between the two continuous edge images.

[0009] S3: The master control unit compares the positioning offset value of the terminal on the photographed tower and the displacement amplitude of the corresponding corner point, first converts the offset amplitude obtained by the Beidou positioning module into an offset amplitude in the direction of the edge image according to the included angle between the offset direction obtained by the Beidou positioning module and the offset direction of the edge image, then converts the converted offset amplitude into an offset value in pixel unit, obtains the positioning offset value, and takes the edge image with a displacement amplitude deviation of no more than 3 pixels at the corresponding corner point as an offset image.

[0010] S4: The master control unit determines whether the tower tilts according to the offset image, calculates the corresponding offset distance according to the offset vectors of all corner points of the offset image, and then calculates the standard deviation of the offset distance, and determines the edge image with a standard deviation of no less than 2 as a tilt, and determines the edge image with a standard deviation less than 2 as a whole offset. If it is determined that the tower tilts, then it is determined whether it is normal swinging according to the displacement direction, if it is determined to be abnormal swinging, then alarm information is sent to the management platform through the communication module and the short message module respectively, and the alarm information further comprises position information generated by the Beidou positioning module.

[0011] Preferably, the edge detection adopts a Canny algorithm with an adaptive threshold, and a suitable threshold is automatically selected according to the statistical characteristics of the image, and the detection step comprises:

[0012] Gaussian filter is used to smooth the input gray image and filter out noise to obtain a filtered image.

[0013] The Sobel operator is used to calculate the gradient magnitude and direction of each pixel in the filtered image.

[0014] Non-maximum suppression is applied to all gradient magnitudes to eliminate the spurious responses caused by edge detection.

[0015] Adaptive high-low threshold detection is applied to determine strong edges, non-edges and weak edges.

[0016] Edge linking is used to connect weak edge points with adjacent strong edge points.

[0017] Preferably, the method for obtaining the high-low threshold of the adaptive threshold is as follows: first, the gray histogram of the input image is calculated, then the median of the gray value is calculated according to the gray histogram, and finally the high threshold and the low threshold are calculated according to the median and a preset sigma value, and the calculation formula is as follows:

[0018] min = (1-σ)×median

[0019] max = (1+σ)×median

[0020] In the formula, min is the low threshold for the Canny algorithm, max is the high threshold for the Canny algorithm, σ is the preset sigma value, and median is the median of the gray value calculated according to the gray histogram. If the calculated high threshold max>255, then max is set to 255.

[0021] Preferably, the method for detecting the corner point is as follows:

[0022] S2-1: Select a white pixel point P with a gray value of 255 in the edge image.

[0023] S2-2: Draw a discretization circle with a radius of 3 pixels with the white pixel as the center, and there are 16 boundary pixels on the boundary of the discretization circle.

[0024] S2-3: If there are 10 continuous pixels with a gray value of 0 among the 16 boundary pixels of the discretization circle, then the point P is determined as a corner point.

[0025] S2-4: Repeat steps S2-1 to S2-3 to traverse all pixel points in the edge image to obtain all corner points in the edge image.

[0026] Preferably, the sparse optical flow algorithm is as follows:

[0027] For a corner point in the first edge image, a first neighborhood window of a square is defined in the first edge image with the corner point as the center.

[0028] An initial offset vector (u=0, v=0) is constructed, and the initial offset vector is used to obtain a second neighborhood window of the same size as the first neighborhood window in the second edge image.

[0029] The offset vector is iteratively adjusted until the second neighborhood window obtained in the second edge image is the same as the first neighborhood window, and the offset vector used in this iteration is used as the optimal offset vector of the corner point.

[0030] The optimal displacement vector of the corner point is visualized as an arrow.

[0031] Preferably, the size of the neighborhood window is set to 7x7 pixels.

[0032] Preferably, the communication module communicates with the management platform through a mobile network, a wireless bridge, or a power integrated data network, and sends alarm information or images to the management platform.

[0033] Preferably, the communication module also receives control instructions from the management platform and forwards the control instructions to the main control unit, which controls the corresponding modules according to the received control instructions.

[0034] Preferably, the power module is a solar power supply system, which includes a solar panel and a storage battery connected to the solar panel.

[0035] Compared with the prior art, the present application has the following technical effects:

[0036] 1. The data processing terminal of the present application uses a main control unit to calculate the displacement of the tower in the input image in real time, and issues an alarm message according to the calculation result, and combines the accurate time obtained by the Beidou timing module to comprehensively determine whether the tower is tilted by fusing the calculation tilt amplitude obtained by image processing and the offset value obtained by Beidou positioning, effectively reducing the error of single means determination of tower tilt and improving the accuracy of tower tilt detection; the improved optical flow method is used to calculate the tower displacement, which can effectively improve the efficiency of image processing.

[0037] 2. The iron tower inclination detection method provided by the application carries out preprocessing on the input image, including grayscale, twice downsampling and edge detection arranged in sequence, the edge detection algorithm can effectively detect the edges in the image, therefore, using the edge image as the input of the optical flow method can provide more accurate edge information, which helps the optical flow method to more accurately calculate the motion vector of the pixel, so as to obtain more reliable motion calculation results; the conventional optical flow method carries out pixel-level calculation on the whole image, while using the edge image can limit the optical flow calculation to the edge area of the image, so that the calculation amount can be reduced, the running speed can be improved, and the motion of the edge area can be more focused; therefore, using the edge image as the input of the optical flow method can provide more accurate, faster and more fine motion calculation results.

[0038] 3. The iron tower inclination detection method provided by the application adopts adaptive high and low threshold values when carrying out edge detection, so that manual selection of threshold values and multiple parameter adjustments to determine the division of edge pixels and non-edge pixels are no longer needed, different threshold values can be automatically calculated for different images and application scenarios, and appropriate threshold values can be automatically selected according to the statistical characteristics or local pixel information of the image, so that good edge detection effects can be obtained for images acquired in different environments, the method can adapt to various different weather conditions in the working environment of the iron tower, and the robustness of edge detection on the iron tower image is improved.

[0039] 4. The iron tower inclination detection method provided by the application uses a sparse optical flow algorithm to calculate the displacement amplitude and displacement direction of a plurality of points on the iron tower in two continuous images, the displacement amplitude is used to determine whether the iron tower has displacement, and the displacement direction can be used to exclude normal swinging of the iron tower due to environmental factors, reduce false positives, and improve the accuracy of iron tower inclination detection.

[0040] 5. The iron tower inclination detection method provided by the application uses a sparse optical flow algorithm, and only needs to calculate the displacement amplitude and direction of a plurality of corner points in the image, compared with the dense optical flow algorithm which detects the displacement amplitude and direction of all pixels in the image, the calculation cost can be greatly saved, and the calculation efficiency is improved.

[0041] 6. The data processing terminal provided by the application adopts a mobile network, a wireless bridge or a power integrated data network to communicate, a suitable communication mode can be selected according to the actual environment of the iron tower, and the flexibility of deployment is effectively improved.

[0042] 7. The data processing terminal provided by the application includes a Beidou short message module, which can send alarm information using the Beidou short message module, provides a redundant communication link in the case that the communication module cannot work normally, ensures the timely sending of the alarm information, and improves the robustness of the iron tower inclination detection system. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1is a data processing terminal structure schematic diagram of an embodiment of the present application;

[0044] Figure 2 is an imaging terminal deployment and imaging module imaging schematic diagram of an embodiment of the present application;

[0045] Figure 3 is a main control unit processing flow schematic diagram in a data processing terminal of an embodiment of the present application;

[0046] Figure 4 is a preprocessing flow schematic diagram of an embodiment of the present application;

[0047] Figure 5 is a gray scale histogram of a gray scale image of an embodiment of the present application;

[0048] Figure 6 is a corner point detection schematic diagram of an embodiment of the present application;

[0049] Figure 7 is a discretization circle schematic diagram in a corner point detection process of an embodiment of the present application;

[0050] Figure 8 is a corner point schematic diagram of an edge image of an embodiment of the present application;

[0051] Figure 9 is a neighborhood window schematic diagram of an embodiment of the present application;

[0052] Figure 10 is a partial corner point offset vector schematic diagram of an embodiment of the present application;

[0053] Figure 11 is a partial corner point offset visualization schematic diagram of an embodiment of the present application.

[0054] The figure mark: 1, main control unit;2, imaging module;3, Beidou positioning module;4, Beidou timing module;5, communication module;6, short message module;7, storage module;8, power module;91, first iron tower;92, second iron tower;93, third iron tower. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described clearly and completely below in combination with specific embodiments of the present application and by referring to the drawings.

[0056] As shown in Figure 1 A data processing terminal based on Beidou satellite data and AI graphics fusion, comprising: main control unit 1, imaging module 2, Beidou positioning module 3, Beidou timing module 4, communication module 5, short message module 6, storage module 7 and power module 8.

[0057] The deployment of iron tower and terminal is as shown inFigure 2 As shown, the second tower 92 has two adjacent towers: the first tower 91 and the third tower 93. The terminal is arranged on the tower body, the imaging module 2 continuously acquires images of adjacent towers at a set time interval, and the time interval at which the imaging module 2 acquires images can be set to any value between 1-120 minutes, which can be adjusted by the main control unit 1 according to the instructions of the management platform if necessary. The imaging module 2 also labels the acquired images with a timestamp, which is generated by the Beidou timing module 4. After labeling the timestamp, the imaging module 2 sends the image with the timestamp to the main control unit 1. The terminal deployed on the second tower 92 can acquire and process images of the first tower 91 and the third tower 93. Correspondingly, the terminal deployed on the first tower 91 can acquire and process images of the second tower 92 and another tower adjacent to the first tower 91, and the terminal deployed on the third tower 93 can acquire and process images of the second tower 92 and another tower adjacent to the third tower 93.

[0058] The power module 8 supplies power to other modules in the terminal, and the power module 8 is a solar power system including a solar panel and a storage battery connected to the solar panel. The communication module 5 communicates with the management platform through a mobile network, a wireless network bridge, or a power integrated data network. The communication mode can be selected according to the actual environment of the tower to effectively improve the flexibility of deployment and send alarm information or images to the management platform. The communication module 5 also receives control instructions from the management platform and forwards the control instructions to the main control unit 1, which controls the corresponding modules according to the received control instructions.

[0059] As shown in Figure 3 The main control unit 1 is configured to perform the following tower tilt detection method:

[0060] S1: The main control unit 1 receives the images sent by the imaging module 2, pre-processes the images, generates edge images, and then saves the edge images in chronological order to the storage module 7. The preprocessing includes sequentially arranged grayscale, twice downsampling, and edge detection. The edge detection detects the images output by the grayscale, the first downsampling, and the second downsampling, respectively, to output edge images of three scales.

[0061] S2: The main control unit 1 compares every two consecutive three-scale edge images in the storage module 7, and for each image, the three-scale edge images generated are compared using a sparse optical flow algorithm, and only when at least two comparison results are the same, the majority comparison result is adopted. The comparison method uses a sparse optical flow algorithm, which first calculates the corner points in the first edge image, and calculates the displacement amplitude and direction of the corner points between two consecutive edge images. In computer vision, corner points are also called interest points, key points or feature points, and are widely used to solve problems in object recognition, image matching, visual tracking, 3D reconstruction and other fields. The corner point is usually defined as the intersection of two edges, and more strictly, the local neighborhood of the corner point should have two different regions with different direction boundaries. In practical applications, most corner detection methods detect image points with specific features, not just "corner points". These feature points have specific coordinates in the image and have certain mathematical features such as local maximum or minimum gray value, certain gradient features, etc.

[0062] S3: The main control unit 1 compares the positioning offset value of the terminal on the photographed tower with the displacement amplitude of the terminal corresponding corner point. First, according to the included angle between the positioning offset direction obtained by the Beidou positioning module 3 and the offset direction of the edge image, the offset amplitude obtained by the Beidou positioning module 3 is converted into the offset amplitude in the direction of the edge image, and then the converted offset amplitude is expressed as an offset value in pixel units, to obtain the positioning offset value. The positioning offset value is compared with the edge image whose displacement amplitude deviation at the corresponding corner point does not exceed 3 pixels as the offset image.

[0063] S4: The main control unit 1 determines whether the tower has tilted according to the offset image. The offset distance corresponding to the offset vector of all corner points of the offset image is calculated, and then the standard deviation of the offset distance is calculated. The edge image with a standard deviation not less than 2 is determined to be tilted, and the edge image with a standard deviation less than 2 is determined to be a whole offset. If it is determined that the tower has tilted, it is further determined whether it is normal swing according to the displacement direction. If it is determined to be abnormal swing, an alarm information is sent to the management platform through the communication module 5 and the short message module 6, respectively. The alarm information also includes the position information generated by the Beidou positioning module 3.

[0064] In step S1, as Figure 4As shown, the preprocessing includes gray scale, twice down-sampling and edge detection arranged in sequence. After the image of the iron tower collected by the imaging module 2 is transmitted to the main control unit 1, the main control unit 1 first performs gray scale on the input image, converts the color input image of the RGB channel into a gray scale image; the gray scale image is subjected to a down-sampling operation once to generate a first down-sampled image; the first down-sampled image is subjected to a down-sampling operation again to generate a second down-sampled image. The gray scale image, the first down-sampled image and the second down-sampled image are subjected to edge detection respectively to obtain three edge images of different scales. By down-sampling the original gray scale image to obtain the first down-sampled image and the second down-sampled image, image information of different scales can be obtained. The images of different scales contain details and structural information of different scales. When performing edge detection, the use of multi-scale information can provide more comprehensive edge detection results and capture edge features of different scales. Moreover, image down-sampling can smooth the image and reduce high-frequency noise in the image, which can reduce the influence of noise on edge detection and improve the accuracy and robustness of edge detection.

[0065] The conventional edge detection needs to manually set two threshold values, i.e., a high threshold value and a low threshold value, and then uses the two set threshold values to detect the edges in the image by using the Canny algorithm. The two threshold values are fine-tuned according to the edge detection effect until the best detection effect is obtained. In this embodiment, the Canny algorithm with adaptive threshold value is used to automatically select appropriate high and low threshold values according to the statistical characteristics of the image. The specific detection method is the following S1-1 to S1-5 steps:

[0066] S1-1: Gaussian filtering, used to smooth the input gray scale image and filter out noise to obtain a filtered image;

[0067] A Gaussian filter with a standard deviation of 0.2 is constructed using a 3x3 size Gaussian kernel. The constructed Gaussian filter is convolved with the input image. The convolution operation performs a weighted average of the neighboring pixels around each pixel with the Gaussian kernel to obtain the blurred pixel value. Gaussian blur can effectively smooth the noise in the image, especially high-frequency noise. By reducing the details and irregularities of the noise, the image becomes clearer. Gaussian blur can make the details of the image blurred, producing a smoothing effect, which can effectively reduce the jagged edges of the image or reduce the intensity of the texture for image processing tasks and visual effects.

[0068] S1-2: using the Sobel operator to calculate the gradient amplitude and direction of each pixel point in the filtered image;

[0069] The Sobel operator is a 3x3 size convolution kernel. The convolution kernel is used to perform convolution operation on each pixel in the image to obtain the gradient G x in the horizontal direction and the gradient G yCalculate the gradient magnitude of each pixel based on the gradients in the horizontal and vertical directions. and the gradient direction of each pixel atan2(G) y G x The gradient direction is specifically represented as:

[0070]

[0071] S1-3: Apply non-maximum suppression to all gradient magnitudes to suppress the response in non-maximum regions, thereby eliminating stray responses caused by edge detection;

[0072] S1-4: Apply adaptive high and low threshold detection to determine strong edges, non-edges, and weak edges. If the gradient magnitude is greater than the high threshold, the pixel is marked as a strong edge point. If the gradient magnitude is less than the low threshold, the pixel is excluded and considered not to be an edge point. If the gradient magnitude is between the low and high thresholds, the pixel is marked as a weak edge point.

[0073] S1-5: Edge connection, connecting weak edge points with neighboring strong edge points.

[0074] The method for obtaining the high and low thresholds of the adaptive threshold in steps S1-4 is as follows: First, calculate the grayscale histogram of the input image. Figure 5 This is a grayscale histogram calculated from the grayscale image used in this embodiment. The vertical axis represents the number of pixels, and the horizontal axis represents the grayscale value. The median of the grayscale values ​​is calculated based on the grayscale histogram. Then, the high threshold and low threshold are calculated based on the median grayscale values ​​and a preset sigma value. The calculation formula is as follows:

[0075] min=(1-σ)×median

[0076] max=(1+σ)×median

[0077] In the formula, min is the low threshold used in the Canny algorithm, max is the high threshold used in the Canny algorithm, σ is the preset sigma value, and median is the median of gray values ​​calculated from the gray-level histogram. If the calculated high threshold max > 255, then max is set to 255. In this embodiment, the value of σ is set to 0.3, through... Figure 5 The grayscale histogram shown calculates a median grayscale value of 152. Therefore, the value of the low threshold min is (1-0.3)×152=106, and the value of the high threshold max is (1+0.3)×152=197. Traversing all pixels of the grayscale image, pixels with grayscale values ​​no greater than 106 are marked as weak edge points, pixels with grayscale values ​​no less than 197 are marked as strong edge points, and pixels with grayscale values ​​between 106 and 197 are marked as non-edge points.

[0078] After processing in step S1, the tower area and the background are clearly distinguishable in the edge image. Since the optical flow algorithm used in this embodiment is a sparse optical flow algorithm, it does not calculate the offset vector of all pixels in the edge image, but only the offset vector of the corner points. Therefore, before applying the sparse optical flow algorithm, it is necessary to calculate the corner points of the edge image. In step S2, the method for detecting the corner points is as follows:

[0079] S2-1: As Figure 6 As shown, a white pixel P with a grayscale value of 255 is selected in the edge image.

[0080] S2-2: As Figure 7 As shown, a discretized circle with a radius of 3 pixels is drawn with the white pixel as the center. There are 16 boundary pixels on the boundary of the discretized circle, namely 3 pixels directly above, below, to the left and to the right of point P, for a total of 12 pixels, and 4 pixels each at 45°, 135°, 225° and 315° of point P. The above 16 pixels are marked as 1 to 16 in the figure.

[0081] S2-3: If there are 10 consecutive pixels with a gray value of 0 (i.e., black) among the 16 boundary pixels of the discretized circle, then point P is determined to be a corner point. Figure 7 In the graph, pixels 16, 1, 2 to 11 are 12 consecutive pixels with a gray value of 0, which meets the criteria for a corner point: there are 10 consecutive pixels with a gray value of 0. Therefore, point P is determined to be a corner point.

[0082] S2-4: Repeat steps S2-1 to S2-3 to traverse all pixels in the edge image and obtain all corner points in the edge image, such as... Figure 8 As shown.

[0083] For the edge images at the other two scales, corner points are also selected respectively.

[0084] The sparse optical flow algorithm compares two adjacent edge images in storage module 7. In this embodiment, the first edge image and the second edge image are obtained by preprocessing two consecutively captured images. The first edge image was captured earlier than the second edge image. The sparse optical flow algorithm is as follows:

[0085] For a corner point in the first edge image, a square first neighborhood window is defined with the corner point as the center in the first edge image, such as... Figure 9 As shown, point P is a corner point in the first edge image. A neighborhood window of size 7×7 pixels is constructed in the first edge image with point P as the center.

[0086] An initial offset vector (u = 0, v = 0) is constructed. Using this initial offset vector, a second neighboring window of the same size as the first neighboring window is obtained in the second edge image. If the second neighboring window is not the same as the first neighboring window, the offset vector is changed to re-obtain the second neighboring window. This process is iteratively adjusted until the second neighboring window obtained in the second edge image is the same as the first neighboring window. The offset vector used in this iteration is taken as the optimal offset vector for the corner point. The iteration method is as follows:

[0087] Starting with (u,v) = (0,0), the values ​​of u and v are first iterated within the integer range of -1 ≤ u ≤ 1 and -1 ≤ v ≤ 1. Then, u and v are expanded to -2 ≤ u ≤ 2 and -2 ≤ v ≤ 2, and the values ​​of u and v are iterated again. This process is repeated to gradually expand the range of u and v until the offset vector exceeds the pixel range of the second edge image. In each iteration, a second neighborhood window of size 7×7 is obtained in the second edge image using the offset vector (u,v). The second neighborhood window is compared pixel by pixel with the first neighborhood window. If the second neighborhood window is the same as the first neighborhood window, the iteration stops, and the offset vector (u,v) used in this iteration is used as the offset vector of that corner point.

[0088] Iterate through all corner points of the first edge image and obtain the optimal offset vector for each corner point, such as... Figure 10 The image shows an example of offset vectors calculated from some corner points. The offset vectors for some corner points are listed in the image: A(9,-2), B(9,-2), C(7,-1), D(7,-1), E(4,0), F(3,0), G(1,0), and H(0,0). Here, A(9,-2) indicates that, relative to the first edge image, corner point A in the second edge image is offset 9 pixels to the right and 2 pixels down.

[0089] like Figure 11 As shown, the optimal displacement vector of the corner point is visualized as an arrow, and this visualization image can be saved.

[0090] In step S3, the positioning offset value is the offset value after conversion based on the offset amplitude and offset direction obtained by the Beidou positioning module. Based on the angle between the offset direction obtained by the Beidou positioning module and the offset direction of the edge image, the offset amplitude obtained by the Beidou positioning module is converted into the offset amplitude in the edge image direction, which is the positioning offset value.

[0091] In step S4, when the host unit 1 detects that the tower has displacement according to the offset image, first, it is judged whether it is the overall offset of the image according to the offset vectors of all the corner points. The method for detecting the overall offset is as follows: the offset distance of the corresponding offset vector of all the corner points of the offset image is calculated, and then the standard deviation of the offset distance is calculated. If the standard deviation is not less than 2, the offset image is judged to be inclined, and if the standard deviation is less than 2, the offset image is judged to be overall offset but not inclined. Taking the example of the example corner points A(9, -2), B(9, -2), C(7, -1), D(7, -1), E(4, 0), F(3, 0), G(1, 0), and H(0, 0), first, the offset distance is calculated:

[0092]

[0093]

[0094] …

[0095]

[0096] Then, the average value is calculated According to all the offset distances and the average value, the standard deviation of the offset distance of the above eight example corner points is 3.3568126040810657, so according to the judgment rule, the offset image is judged to be inclined. If the calculated standard deviation is less than 2, it is judged that the tower is overall offset, and no processing is performed.

[0097] When the host unit 1 detects that the tower has inclination, it can issue a continuous photographing instruction to the imaging module 2, continuously image in 2-4 swing periods of the tower, compare the continuous images in the 2-4 swing periods, and if the comparison result shows a left-right regular inclination, it is judged that the tower has normal swing, and if it does not show a left-right regular inclination, it issues an alarm information, offset data, and visualized picture to the management platform through the communication module 5 and the short message module 6. The judgment of normal swing is as follows: taking the vertical line as the symmetry axis, all the continuous images in the 2-4 swing periods are divided into two groups according to the symmetry principle of the offset direction, the offset distance of the corner points in all the images is calculated, and for any image in one group of images, the first offset distance matrix of all the corner points is obtained. If an image in the other group of images can be obtained, the offset distance matrix of the corner points of the image is the same as the first offset distance matrix, then it is considered that the tower has normal swing.

[0098] In other embodiments of the present application, a magnitude threshold of the offset vector can be set, and if the magnitude of the offset vector detected exceeds the set threshold, an alarm information, offset data, and visualized picture are issued to the management platform through the communication module 5 and the short message module 6.

[0099] The above merely describes the preferred embodiments of the present application, and it should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept, and these all belong to the protection scope of the present application.

Claims

1. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics, characterized in that, include: The terminal is equipped with a main control unit (1), an imaging module (2), a Beidou positioning module (3), a Beidou timing module (4), a communication module (5), a short message module (6), a storage module (7), and a power supply module (8). The terminal is installed on the tower body. The imaging module (2) continuously acquires images of adjacent towers at set time intervals and marks the images with timestamps generated by the Beidou timing module (4). The images with timestamps are sent to the main control unit (1). The power supply module (8) supplies power to the other modules in the terminal. The main control unit (1) is configured to perform the following tower tilt detection method: S1: The main control unit (1) receives the image sent by the imaging module (2), preprocesses the image, generates an edge image, and then saves the edge image to the storage module (7) in chronological order. The preprocessing includes grayscale conversion, two downsampling operations and edge detection set sequentially. The edge detection detects the images output by grayscale conversion, the first downsampling and the second downsampling respectively, and outputs edge images at three scales. S2: The main control unit (1) compares every two consecutive edge images of three scales in the storage module (7). The comparison method adopts the sparse optical flow algorithm. First, the corner points in the first edge image are calculated, and the displacement amplitude and direction of the corner points between the two consecutive edge images are calculated. S3: The main control unit (1) compares the positioning offset value of the terminal on the photographed iron tower with the displacement amplitude at the corresponding corner point of the terminal. First, based on the angle between the positioning offset direction obtained by the Beidou positioning module (3) and the offset direction of the edge image, the offset amplitude obtained by the Beidou positioning module (3) is converted into the offset amplitude in the edge image direction. Then, the converted offset amplitude is expressed as the offset value in pixel units to obtain the positioning offset value. The edge image with a positioning offset value and a displacement amplitude deviation of no more than 3 pixels at the corresponding corner point is used as the offset image. S4: The main control unit (1) determines whether the tower is tilted based on the offset image, calculates the corresponding offset distance based on the offset vector of all corner points of the offset image, and then calculates the standard deviation of the offset distance. Edge images with a standard deviation of not less than 2 are judged as tilted, and edge images with a standard deviation of less than 2 are judged as overall offset. If the tower is determined to be tilted, the direction of displacement is used to determine whether it is a normal sway. If it is determined to be an abnormal sway, alarm information is sent to the management platform through the communication module (5) and the short message module (6). The alarm information also includes the location information generated by the Beidou positioning module (3).

2. The data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1, characterized in that, The edge detection employs the Canny algorithm with adaptive thresholding, automatically selecting an appropriate threshold based on the statistical features of the image. The detection steps include: Gaussian filtering is used to smooth the input grayscale image, filter out noise, and obtain a filtered image. The Sobel operator is used to calculate the gradient magnitude and direction of each pixel in the filtered image; Non-maximum suppression is applied to all gradient magnitudes to eliminate stray responses caused by edge detection; Adaptive high and low dual threshold detection is used to determine strong edges, non-edges, and weak edges; Edge connectivity connects weak edge points to neighboring strong edge points.

3. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 2, characterized in that, The method for obtaining the high and low thresholds of the adaptive threshold is as follows: First, calculate the gray-level histogram of the input image; calculate the median of gray-level values ​​based on the gray-level histogram; then calculate the high threshold and low threshold based on the median of gray-level values ​​and a preset sigma value. The calculation formula is as follows: min=(1-σ)×mediam max=(1+σ)×mediam In the formula, min is the low threshold used for the Canny algorithm, max is the high threshold used for the Canny algorithm, σ is the preset sigma value, and median is the median of gray values ​​calculated from the gray histogram; if the calculated high threshold max>255, then max is set to 255.

4. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1, characterized in that, The method for detecting corner points is as follows: S2-1: Select a white pixel P with a gray value of 255 in the edge image; S2-2: Draw a discretized circle with a radius of 3 pixels centered on the white pixel, and the discretized circle has 16 boundary pixels on its boundary. S2-3: If there are 10 consecutive pixels with a gray value of 0 among the 16 boundary pixels of the discretized circle, then point P is determined to be a corner point. S2-4: Repeat steps S2-1 to S2-3 to traverse all pixels in the edge image and obtain all corner points in the edge image.

5. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1, characterized in that, The sparse optical flow algorithm is specifically as follows: For a corner point in the first edge image, a square first neighborhood window is defined in the first edge image with the corner point as the center; Construct an initial offset vector (u=0, v=0), and use the initial offset vector to obtain a second neighboring window of the same size as the first neighboring window in the second edge image. If the second neighboring window is not the same as the first neighboring window, change the offset vector and re-obtain the second neighboring window. Iterate and adjust the offset vector until the second neighboring window obtained in the second edge image is the same as the first neighboring window. The offset vector used in this iteration is the optimal offset vector for the corner point. Iterate through all corner points of the first edge image and obtain the optimal offset vector for each corner point; The optimal displacement vector of the corner point is visualized as an arrow.

6. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 5, characterized in that, The size of the neighborhood window is set to 7×7 pixels.

7. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1, characterized in that, The communication module (5) communicates with the management platform through a mobile network, wireless bridge or power integrated data network, and sends alarm information or images to the management platform.

8. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1 or 7, characterized in that, The communication module (5) also receives control instructions from the management platform and forwards the control instructions to the main control unit (1), which then controls the corresponding module to work according to the received control instructions.

9. A data processing terminal based on the fusion of BeiDou satellite data and AI graphics as described in claim 1, characterized in that, The power module (8) is a solar power supply system, including a solar panel and a battery connected to the solar panel.

Images