Intelligent lamp pole-based outdoor billboard deformation monitoring method, system and device

Abstract

This application belongs to the field of urban public facility safety monitoring and machine vision, specifically involving a method, system, and device for monitoring the deformation of outdoor billboards based on smart light poles. The method acquires current image data and corresponding reference image data collected at the deployment points of the smart light poles, and generates standardized image data through preprocessing. The standardized image data is then used to extract the pixel regions of the billboard, determining the current deformation calculation region and the reference deformation calculation region under the same image coordinate system. Candidate deformation data are generated based on feature point displacement and regional cross-correlation displacement, and deviation rate data is generated based on the candidate deformation data corresponding to the same deformation type for consistency verification, resulting in deformation monitoring results. When the actual physical deformation data that passes the consistency verification exceeds a preset safety threshold, alarm data is generated and output. This invention can be used for monitoring and providing safety warnings for the deformation state of billboards in complex outdoor scenarios.

Description

Technical Field

[0001] This application belongs to the field of urban public facility safety monitoring and machine vision, specifically involving a method, system and equipment for monitoring the deformation of outdoor billboards based on smart light poles. Background Technology

[0002] With the continuous increase in outdoor advertising facilities along urban roads, commercial streets, transportation hubs, and around public buildings, outdoor billboards, as public ancillary facilities exposed to the open environment for extended periods, are susceptible to structural damage from factors such as wind loads, rain and snow, temperature changes, loosening of support connections, foundation deformation, and variations in human maintenance. When billboards experience overall shifting, localized warping, panel tilting, or deformation of connecting parts without timely detection, it can affect the normal display effect of the advertising facility and further create public safety hazards. Therefore, continuous, timely, and reliable deformation monitoring of outdoor billboards is of great significance for the safety management of urban public facilities.

[0003] Existing image deformation monitoring methods typically acquire images of the target using a camera and obtain displacement or deformation information of the target through image matching, correlation calculation, or stereo vision. For example, existing methods can acquire a first image and a second image of the target using a binocular camera, determine the Hamming distance between the first and second images, perform cost aggregation based on the Hamming distance to determine the depth information of the monitoring image, determine the initial displacement value using a normalized cross-correlation criterion, and then combine iterative algorithms to determine the planar displacement value of the monitoring image. Such methods can achieve image-based deformation measurement of the monitoring target to a certain extent and improve the accuracy of displacement calculation in the depth and planar directions.

[0004] However, the actual monitoring environment of outdoor billboards differs from that of relatively regular structural experimental scenarios or fixed target measurement scenarios. Outdoor billboards are typically located along roadsides, on building facades, on the exterior walls of commercial complexes, or around public squares. Their images often simultaneously include road vehicles, pedestrians, greenery, building facades, light poles, cables, sky backgrounds, reflective areas, and other advertising facilities. Furthermore, different billboards vary significantly in material, size, boundary shape, display method, and installation posture. Ordinary flat billboards, illuminated billboards, grid billboards, and electronic display billboards are not entirely consistent in image texture and edge features. In these complex open scenarios, relying solely on conventional image matching or single displacement calculation methods can easily lead to the misidentification of background areas, occluded areas, or areas of changing lighting as effective monitoring areas. This results in unstable extraction of the billboard target area, which in turn affects the judgment of the billboard's displacement, tilt, warping, or local deformation. Summary of the Invention

[0005] Firstly, in view of the shortcomings of the existing technology, the purpose of this application is to provide a method for detecting deformation of outdoor billboards based on smart light poles. This method solves the problem that the existing technology relies solely on conventional image matching or single displacement calculation, which easily misidentifies background areas, occluded areas, or areas with changes in lighting as effective monitoring areas. This leads to unstable extraction of the target area of ​​the billboard, which in turn affects the judgment of the billboard's displacement, tilt, warping, or local deformation state.

[0006] The objective of this application can be achieved through the following technical solutions: A method for monitoring the deformation of outdoor billboards based on smart light poles includes: Acquire image data of the outdoor billboard to be monitored. The image data includes current image data collected at the smart light pole deployment points, as well as baseline image data corresponding to the outdoor billboard to be monitored. Preprocess the current image data and the reference image data to generate the current standardized image data and the reference standardized image data; The pixel regions of the billboard are extracted from the current standardized image data and the benchmark standardized image data respectively, generating the target region data of the current billboard and the target region data of the benchmark billboard. Based on the current billboard target area data and the reference billboard target area data, determine the current deformation calculation area and the reference deformation calculation area that are in the same image coordinate system; Based on the current deformation calculation region and the benchmark deformation calculation region, at least two sets of candidate deformation data are generated respectively. The at least two sets of candidate deformation data include the first candidate deformation data generated based on the displacement of feature points and the second candidate deformation data generated based on the cross-correlation displacement of the region. Deviation rate data is generated based on at least two sets of candidate deformation data corresponding to the same deformation type, and consistency verification is performed on at least two sets of candidate deformation data based on the deviation rate data to generate deformation monitoring results. When the deformation monitoring results include real physical deformation data that has passed consistency verification, and the real physical deformation data is greater than the corresponding preset safety threshold, alarm data for the outdoor billboard to be monitored is generated and output.

[0007] Furthermore, image data of the outdoor billboard to be monitored is acquired, including: Obtain the video stream address and billboard identifier corresponding to the outdoor billboard to be monitored; Obtain the video stream of the outdoor billboard based on the video stream address; Extract the current frame image from the video stream of the outdoor billboard according to a preset sampling frequency; The current frame image is associated with the billboard logo, acquisition time, and acquisition location information to generate the current image data; Based on the billboard markings, read the baseline image data corresponding to the outdoor billboard to be monitored; Output the current image data and the reference image data to the preprocessing step; The baseline image data consists of pre-collected billboard image data that has been marked as qualified.

[0008] Furthermore, the current image data and the reference image data are preprocessed to generate current standardized image data and reference standardized image data, including: Receive current image data and reference image data; Based on preset image cropping parameters, the current image data and the reference image data are cropped respectively to generate the current cropped image data and the reference cropped image data; The current cropped image data and the reference cropped image data are converted to grayscale respectively to generate the current grayscale image data and the reference grayscale image data; Denoising is performed on the current grayscale image data and the reference grayscale image data respectively to generate the current denoised image data and the reference denoised image data; Based on the grayscale distribution of the current denoised image data and the baseline denoised image data, the current denoised image data is subjected to illumination normalization processing to generate the current illumination normalized image data. Scale normalization is performed on the current illumination-normalized image data and the baseline denoised image data to generate current normalized image data and baseline normalized image data with the same image size and the same pixel coordinate system.

[0009] Furthermore, the pixel regions of the billboard are extracted from the current standardized image data and the benchmark standardized image data respectively, generating the current billboard target region data and the benchmark billboard target region data, and determining the current deformation calculation region and the benchmark deformation calculation region, including: Input the current standardized image data into the trained U-Net semantic segmentation model to generate the current billboard target mask; Based on the current billboard target mask, extract the current billboard pixel region from the current standardized image data, and generate the current billboard target region data including the current billboard target mask and the current target region boundary; The benchmark normalized image data is input into the U-Net semantic segmentation model to generate a benchmark billboard target mask; Based on the benchmark billboard target mask, extract the benchmark billboard pixel region from the benchmark normalized image data, and generate benchmark billboard target region data including the benchmark billboard target mask and the benchmark target region boundary; Based on the current target region boundary and the reference target region boundary, determine the ROI region for deformation calculation in the same pixel coordinate system; Based on the ROI region, the current deformation calculation region is extracted from the current normalized image data, and the baseline deformation calculation region is extracted from the baseline normalized image data.

[0010] Furthermore, after receiving the current standardized image data or the benchmark standardized image data, the U-Net semantic segmentation model generates an encoded feature map, calculates the channel weights of the encoded feature map, and performs weighted processing on the encoded feature map according to the calculated channel weights to generate a weighted feature map. The billboard target mask is then generated based on the weighted feature map.

[0011] Furthermore, based on the displacement of feature points, first candidate deformation data is generated, including: Extract the baseline ORB feature points within the baseline deformation calculation area, and extract the current ORB feature points within the current deformation calculation area; Calculate the ORB response values ​​for each baseline ORB feature point and each current ORB feature point; ORB feature points are filtered based on the maximum ORB response value in the neighborhood of each ORB feature point to obtain the filtered baseline ORB feature points and the filtered current ORB feature points. Perform feature descriptor matching between the filtered baseline ORB feature points and the filtered current ORB feature points to generate matching feature point pairs. Based on the coordinates of the reference feature points in the matching feature point pairs and the coordinates of the current feature point Generate horizontal pixel displacement and vertical pixel displacement ; The average gray value of the region is calculated based on the reference deformation. and the average gray value of the current deformation calculation area , for horizontal pixel displacement and vertical pixel displacement Perform illumination robustness correction and generate the corrected lateral pixel displacement. and corrected vertical pixel displacement ; Based on the actual width of the outdoor billboard to be monitored and the pixel width of the outdoor billboard to be monitored in the image Generate proportionality coefficient ; According to the proportionality coefficient Corrected horizontal pixel displacement and corrected vertical pixel displacement Converted into actual physical displacement; Based on the matching distance of each matching feature point pair Generate corresponding weights ; Based on weight The weighted calculation of the corresponding real physical displacements of multiple matching feature points is performed to generate the first candidate deformation data; Among them, horizontal pixel displacement and vertical pixel displacement Generate according to the following method: proportionality coefficient Generate according to the following method: Lighting robustness correction is performed as follows: Weight Generate according to the following method: in, ; Weighted true physical displacement Generate according to the following method: in, Let N be the k-th matching feature point, and N be the number of matching feature point pairs.

[0012] Furthermore, second candidate deformation data are generated based on regional cross-correlation displacements, including: Gaussian smoothing is applied to the current deformation calculation region and the reference deformation calculation region respectively to generate the current smoothed region data and the reference smoothed region data. Select from the baseline smoothed region data The texture-rich areas of a pixel are used as template sub-regions; Determine the search region corresponding to the template sub-region in the current smoothed region data; Use the template sub-region as the template image. The candidate sub-regions in the current smoothed region data are used as real-time images. ; template image and real-time images Perform grayscale mean normalization and grayscale standard deviation normalization; Calculate template image With each real-time image cross-relationships between ; Based on cross-relation numbers The location of the largest candidate sub-region generates the best matching position for an integer pixel. Subpixel accuracy is improved based on the cross-correlation coefficient within the neighborhood of the best matching position of the whole pixel, and subpixel displacement data of the template sub-region in the current deformation calculation region is generated. According to the proportionality coefficient The subpixel displacement data is converted into real physical displacement, and the second candidate deformation data is generated based on the converted real physical displacement. Among them, the number of cross-relationships Generate according to the following method: in, For template image, For real-time images, The grayscale mean of the corresponding region in the template image. This represents the average grayscale value of the corresponding region in the real-time image. Gaussian smoothing uses the following Gaussian smoothing kernel function: in, For smoothing coefficients; Subpixel precision enhancement is performed as follows: in, , For the best matching position of integer pixels, , They are respectively The coefficients of the first and second terms of the directional fitting quadratic curve. , They are respectively The coefficients of the first and second terms of the directional fitted quadratic curve.

[0013] Furthermore, after generating the current billboard target area data and the baseline billboard target area data, the process also includes: Read the boundary of the benchmark target area from the benchmark billboard target area data, and read the boundary of the current target area from the current billboard target area data; Map the boundary of the reference target region and the boundary of the current target region to the same pixel coordinate system; Calculate the boundary position difference between the baseline target area boundary and the current target area boundary, and generate target area boundary displacement data; According to the proportionality coefficient The boundary displacement data of the target area is converted into real physical displacement, and the third candidate deformation data is generated based on the converted real physical displacement. The third candidate deformation data is used for consistency verification with at least one of the first candidate deformation data and the second candidate deformation data.

[0014] Furthermore, deviation rate data is generated based on at least two sets of candidate deformation data corresponding to the same deformation type. Consistency verification is then performed on the at least two sets of candidate deformation data based on the deviation rate data to generate deformation monitoring results. Additionally, alarm data for the monitored outdoor billboard is generated and output, including: Obtain data values ​​belonging to the same deformation type from at least two sets of candidate deformation data. The deformation type includes at least one of global translation, tilt, and local deformation. Using the data values ​​of the same deformation type in the first candidate deformation data as the benchmark data values, calculate the deviation rate of the data values ​​of the same deformation type in other candidate deformation data relative to the benchmark data values, and generate deviation rate data. When the deviation rate data is less than or equal to the first deviation threshold, a deformation monitoring result including real physical deformation data that has passed the consistency verification is generated based on the first candidate deformation data. When the deviation rate data is greater than the first deviation threshold and less than or equal to the second deviation threshold, a re-detection instruction is generated. The re-detection instruction is used to trigger the reacquisition of the current image data and the regeneration of candidate deformation data. When the deviation rate data is greater than the second deviation threshold, a deformation monitoring result is generated, including abnormal data markings and review prompts. When the deformation monitoring results include real physical deformation data that has passed the consistency verification, at least one of the overall translation, tilt, and local deformation variables in the real physical deformation data is obtained, and preset safety thresholds corresponding to the overall translation, tilt, and local deformation variables are obtained respectively. In response to at least one of the overall translation, tilt, and local deformation exceeding the corresponding preset safety threshold, at least one of the following is written into the preset alarm data structure: billboard identification, acquisition time, acquisition location information, deformation type, actual physical deformation data value, current image data, current billboard target area data, environmental parameters, abnormal data markers, and review prompt data, and alarm data is generated and output. The deformation monitoring results and alarm data are encapsulated into upload messages and sent according to the preset remote management address to generate historical monitoring record data and visualization display data.

[0015] Secondly, in view of the shortcomings of the existing technology, the purpose of this application is to provide a deformation monitoring method for outdoor billboards based on smart light poles. This solves the problem that the existing technology relies solely on conventional image matching or single displacement calculation methods, which easily misidentifies background areas, occluded areas, or areas with changes in lighting as effective monitoring areas, resulting in unstable extraction of the target area of ​​the billboard and thus affecting the judgment of the billboard's displacement, tilt, warping, or local deformation state.

[0016] The objective of this application can be achieved through the following technical solutions: An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the deformation monitoring method for outdoor billboards based on smart light poles, as described in the first aspect.

[0017] The beneficial effects of this application are: This invention does not rely solely on a single image matching result to determine whether a billboard is deformed. Instead, it generates first candidate deformation data based on feature point displacement and second candidate deformation data based on regional cross-correlation displacement, based on the current deformation calculation region and a baseline deformation calculation region. The first candidate deformation data is obtained through ORB feature point filtering, feature descriptor matching, illumination robustness correction, scaling factor k to convert to true physical displacement, and weighted calculation based on matching distance. The second candidate deformation data is obtained through texture region cross-correlation, integer pixel optimal matching position, and sub-pixel accuracy improvement. Therefore, this invention can obtain billboard deformation information from two different perspectives: feature point changes and local texture correlation. It converts pixel-level changes into true physical deformation data that can be compared with a safety threshold, avoiding the problem that single pixel matching or single cross-correlation results cannot reliably reflect the actual structural state of outdoor billboards.

[0018] This invention further generates deviation rate data based on at least two sets of candidate deformation data corresponding to the same deformation type, and performs consistency verification on the candidate deformation data based on the deviation rate data. Alarm data is only generated and output when the deformation monitoring result includes real physical deformation data that has passed the consistency verification, and the real physical deformation data is greater than the corresponding preset safety threshold. Compared with the existing technology that directly uses the result of a single displacement calculation as the monitoring output, this invention performs cross-validation on deformation results from different sources before alarming, which can reduce the risk of false alarms caused by local occlusion, changes in lighting, unstable texture, mismatch of feature points, or abnormal target boundaries, making the monitoring results more reliable in supporting safety early warning for outdoor billboards. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall process of an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Example 1: This invention provides a method for monitoring the deformation of outdoor billboards based on smart light poles. The method is applicable to continuous image monitoring of outdoor billboards in open scenarios such as roadsides, commercial streets, public squares, transportation hubs, and building facades. The outdoor billboards can be ordinary flat billboards, LED billboards, grid billboards, lightbox billboards, or other outdoor advertising facilities that require structural status monitoring.

[0023] Outdoor billboards are exposed to open environments for extended periods, making them susceptible to factors such as wind, temperature variations, rain, snow, loose support connections, foundation deformation, vehicle vibration, and maintenance discrepancies. If structural risks such as overall translation, tilting, localized warping, boundary shifts, or deformation of the support area are not detected promptly, they can gradually escalate. Existing image-based deformation monitoring methods typically focus on displacement measurement of general structures. When directly applied to urban outdoor billboard scenarios, they are easily affected by complex backgrounds, lighting variations, occlusion, differences in billboard type, and instability in the target area of ​​the image, making it difficult to reliably obtain the target area and its deformation state. This invention collects current image data through smart light pole deployment points and performs target area extraction, deformation calculation, consistency verification of multiple candidate results, and alarm output in the same coordinate system with reference image data, enabling reliable monitoring results to support safety early warning for outdoor billboards.

[0024] like Figure 1 As shown in this embodiment, the method for monitoring the deformation of outdoor billboards based on smart light poles includes the following steps: S1, acquire image data of the outdoor billboard to be monitored; S2, preprocess the current image data and the reference image data to generate the current standardized image data and the reference standardized image data; S3, extract the billboard pixel region from the current standardized image data and the benchmark standardized image data respectively, and generate the current billboard target region data and the benchmark billboard target region data; S4. Based on the current billboard target area data and the reference billboard target area data, determine the current deformation calculation area and the reference deformation calculation area in the same image coordinate system; S5, Generate the first candidate deformation data based on the displacement of feature points; S6, Generate second candidate deformation data based on regional cross-correlation displacement; S7, optionally, generates third candidate deformation data based on the boundary displacement of the target region; S8. Generate deviation rate data based on at least two sets of candidate deformation data corresponding to the same deformation type, and generate deformation monitoring results based on the deviation rate data. S9. When the deformation monitoring results include real physical deformation data that has passed the consistency verification and the real physical deformation data is greater than the corresponding preset safety threshold, generate and output alarm data.

[0025] Specifically: S1. Acquire image data of the outdoor billboard to be monitored. The image data includes current image data collected at the smart light pole deployment points and baseline image data corresponding to the outdoor billboard to be monitored.

[0026] In S1, the smart light pole deployment point refers to the road light pole, integrated pole, multi-functional light pole, or image acquisition point that is matched with the smart light pole and can cover the area where the outdoor billboard to be monitored is located; the current image data is the billboard image data collected within the current monitoring period; the reference image data is the billboard image data that has been pre-collected and marked as qualified. The qualified state refers to the state corresponding to when the billboard has not experienced overall translation, tilting, local warping, boundary offset, or other structural risk states exceeding the preset safety threshold.

[0027] In one embodiment, acquiring image data of the outdoor billboard to be monitored includes: acquiring the video stream address and billboard identifier corresponding to the outdoor billboard to be monitored; acquiring the outdoor billboard video stream according to the video stream address; extracting the current frame image from the outdoor billboard video stream according to a preset sampling frequency; associating the current frame image with the billboard identifier, acquisition time, and acquisition location information to generate current image data; reading the reference image data corresponding to the outdoor billboard to be monitored according to the billboard identifier; and outputting the current image data and the reference image data to the preprocessing step.

[0028] In one specific implementation, the video stream from the outdoor billboard is acquired via the RTSP protocol, and the sampling frequency can be determined based on monitoring accuracy requirements and computing resources. For ordinary billboards, LED billboards, or grid billboards along roadsides or in commercial districts, a resolution of [resolution missing] can be used. The high-definition image acquisition parameters include a frame rate of 25fps. When the outdoor billboard to be monitored is far from the smart light pole deployment point or the deformation to be identified is small, the image acquisition resolution can be increased, or the subsequent image cropping area can be reduced to ensure that the billboard edges, fixed support parts, and surface texture areas have sufficient pixel width in the image. The parameters described above are used to illustrate an feasible acquisition configuration and do not constitute the sole limitation on the scope of protection of this invention.

[0029] In one implementation, the smart light pole deployment locations are determined based on the camera's coverage area, the distribution characteristics of outdoor billboards, surrounding environmental conditions, and monitoring blind spots. Specifically, the image acquisition direction can be determined first based on the relative positional relationship between the smart light pole and the outdoor billboard to be monitored. Then, the image acquisition direction, image acquisition height, and image acquisition angle can be adjusted based on road greenery, building facades, vehicle obstructions, roadside poles, adjacent advertising facilities, and monitoring blind spots, so that the current frame image can cover the key deformation areas of the outdoor billboard to be monitored. The key deformation areas may include the billboard edges, corner areas, support connection areas, fixed border areas, and surface texture areas.

[0030] In one implementation, the acquisition time of the current frame image can be generated by GPS time synchronization or network time synchronization; each current image data establishes a data index including billboard identification, acquisition time, acquisition location information and image source information; the image source information may include at least one of video stream address, smart light pole deployment point identification, image acquisition parameters, image acquisition height and image acquisition angle; through the data index, the subsequently generated current standardized image data, current billboard target area data, candidate deformation data, deformation monitoring results and alarm data can all be associated with the same outdoor billboard to be monitored.

[0031] S2 preprocesses the current image data and the reference image data to generate the current standardized image data and the reference standardized image data.

[0032] In S2, preprocessing is used to reduce the impact of illumination variations, image noise, background interference, image scale differences, and cropping region differences in the outdoor acquisition environment on subsequent deformation calculations. Preprocessing may include at least one of image cropping, grayscale conversion, denoising, illumination normalization, and scale normalization. Preferably, it is performed in the order of image cropping, grayscale conversion, denoising, illumination normalization, and scale normalization.

[0033] In one implementation, S2 includes: Receive current image data and reference image data; Based on preset image cropping parameters, the current image data and the reference image data are cropped respectively to generate the current cropped image data and the reference cropped image data; The current cropped image data and the reference cropped image data are converted to grayscale respectively to generate the current grayscale image data and the reference grayscale image data; Denoising is performed on the current grayscale image data and the reference grayscale image data respectively to generate the current denoised image data and the reference denoised image data; Based on the grayscale distribution of the current denoised image data and the baseline denoised image data, the current denoised image data is subjected to illumination normalization processing to generate the current illumination normalized image data. Scale normalization is performed on the current illumination-normalized image data and the baseline denoised image data to generate current normalized image data and baseline normalized image data with the same image size and the same pixel coordinate system.

[0034] The preset image cropping parameters can be determined based on the approximate position of the outdoor billboard to be monitored in the image, the preset monitoring area corresponding to the billboard sign, the relative positional relationship between the smart light pole deployment point and the billboard, or the target area of ​​the historical billboard. Grayscale processing is used to convert the current cropped image data and the reference cropped image data into grayscale images, which facilitates subsequent ORB feature point extraction, grayscale normalization, cross-correlation calculation, and illumination robustness correction. Denoising processing is used to remove random noise, compression noise, weak texture area noise, and local noise caused by environmental interference in the image. Illumination normalization processing is used to reduce the grayscale difference between the reference image data and the current image data caused by day and night changes, shadows, reflections, and weather changes. Scale normalization processing is used to ensure that the current standardized image data and the reference standardized image data have the same image size and the same pixel coordinate system.

[0035] In one implementation, preprocessing further includes image format normalization. Specifically, image data that has undergone cropping, grayscale conversion, denoising, and illumination normalization can be converted into a preset image format, retaining the data index corresponding to the billboard logo, acquisition time, and acquisition location information within the preset image format. The preset image format can be JPEG format, or other image formats that can be read by the subsequent U-Net semantic segmentation model, feature point displacement calculation process, and region cross-correlation displacement calculation process. Through the above processing, the current normalized image data and the baseline normalized image data not only have the same image size and the same pixel coordinate system, but also have a unified index relationship corresponding to the outdoor billboard to be monitored.

[0036] S3 extracts the billboard pixel region from the current standardized image data and the benchmark standardized image data respectively, and generates the current billboard target region data and the benchmark billboard target region data.

[0037] In S3, billboard pixel region extraction is used to determine the target region corresponding to the outdoor billboard in a complex open scene. The outdoor billboard image may contain vehicles, pedestrians, roads, building facades, greenery, sky background, cables, light poles, adjacent billboards, or other non-target regions. If deformation calculation is performed directly on the entire image, non-target regions can easily affect the displacement recognition results. Therefore, this implementation method first generates the target region data of the billboard, and then performs deformation calculation within the ROI region corresponding to the target region of the billboard.

[0038] In one implementation, S3 includes: Input the current standardized image data into the trained U-Net semantic segmentation model to generate the current billboard target mask; Based on the current billboard target mask, extract the current billboard pixel region from the current standardized image data, and generate the current billboard target region data including the current billboard target mask and the current target region boundary; The benchmark normalized image data is input into the U-Net semantic segmentation model to generate a benchmark billboard target mask; Based on the benchmark billboard target mask, the benchmark billboard pixel region is extracted from the benchmark normalized image data, and benchmark billboard target region data including the benchmark billboard target mask and the benchmark target region boundary is generated.

[0039] The U-Net semantic segmentation model is used to extract the target region of a billboard from current normalized image data and benchmark normalized image data. The input of the U-Net semantic segmentation model is either current normalized image data or benchmark normalized image data, and the output is a billboard target mask. The target pixels in the billboard target mask are used to represent the billboard pixel region, and the non-target pixels are used to represent roads, buildings, sky, greenery, pedestrians, vehicles and other background regions. The boundary of the target region can be obtained based on the billboard target mask, and the ROI region can be further determined.

[0040] In one implementation, the U-Net semantic segmentation model receives current standardized image data or benchmark standardized image data, generates an encoded feature map, calculates channel weights on the encoded feature map, and performs weighted processing on the encoded feature map according to the calculated channel weights to generate a weighted feature map. Based on the weighted feature map, a billboard target mask is generated. Through this processing, the U-Net semantic segmentation model is used to form the billboard target mask and the target region boundary, rather than being directly used as a deformation calculation algorithm parallel to feature point displacement calculation and region cross-correlation displacement calculation.

[0041] In one implementation, the U-Net semantic segmentation model is trained using labeled outdoor billboard image samples. These image samples include ordinary billboards, LED billboards, and grid billboards, covering different lighting conditions, background conditions, and shooting angles. The total number of samples is no less than 10,000, and they are divided into training, validation, and test sets in an 8:1:1 ratio. During training, the labels for the image samples are pixel-level binary annotations of the billboard target areas, where the billboard areas are labeled as foreground pixels and the remaining areas as background pixels. During training, data-augmented training samples are used, including random flipping, random cropping, brightness perturbation, and Gaussian blur. The loss function is a weighted composite of the Dice loss function and the cross-entropy loss function, expressed as: in, For Dice's loss, For cross-entropy loss, the weight coefficient α ranges from 0.3 to 0.7, preferably 0.5. A lightweight U-Net architecture is used, with depthwise separable convolutions replacing standard convolutions in the encoder. After training, the model is pruned. Based on the L1 norm importance ranking of convolutional layer channels, channels with importance below a set threshold are pruned, with the pruning rate controlled between 30% and 50%. Fine-tuning is then performed after pruning. Subsequently, INT8 quantization is used, and quantization error is calibrated using a calibration set to reduce model size and inference latency. During training, the initial learning rate is set to... The model employs the Adam optimizer with a batch size of 16 and 200 training epochs. Training is terminated early and the optimal weight file is saved when the Dice coefficients on the validation set show no improvement for 10 consecutive epochs. This lightweight model can be deployed locally or remotely. When deployed locally, the single-frame inference latency is controlled to within 100ms, and the GPU memory usage does not exceed 256MB, making it suitable for target region segmentation tasks involving billboard images.

[0042] In one implementation, the U-Net semantic segmentation model is pre-loaded with a training weight file before deployment. This training weight file is obtained from a training dataset containing images of outdoor billboards and their labeled target regions. The training dataset can include images of ordinary billboards, LED billboards, grid billboards, and billboards under different lighting conditions, shooting angles, and background interference. During training, a composite loss function consisting of the Dice loss function and the cross-entropy loss function can be used to ensure that the billboard target mask output by the model corresponds to the labeled target region. During model deployment, the trained U-Net semantic segmentation model can be pruned and quantized to reduce the number of model parameters and inference computation, making it suitable for image processing tasks on smart light poles or at edges.

[0043] In one implementation, the U-Net semantic segmentation model includes an encoder and a decoder. The encoder performs layer-by-layer feature extraction on current normalized image data or benchmark normalized image data to generate an encoded feature map. The decoder performs feature fusion and upsampling on the encoded feature map to generate a billboard target mask corresponding to the input image size. A channel attention module can also be set between the encoder and decoder to generate channel weights based on the channel responses of the encoded feature map, and use the channel weights to weight the encoded feature map to form a weighted feature map. After processing, the decoder generates the billboard target mask based on the weighted feature map.

[0044] In one implementation, feature extraction during the encoding phase is performed as follows: in, For the coding stage The encoded feature map output by the layer. This is the feature map of the previous layer. This indicates double convolution processing. This indicates max pooling.

[0045] In one implementation, feature fusion during the decoding stage is performed as follows: in, For the decoding stage The decoded feature map output by the layer. This is the feature map decoded from the previous layer. This indicates transpose convolution processing. This indicates that the decoded feature map after transpose convolution is concatenated and fused with the encoded feature map of the corresponding encoding layer.

[0046] In one implementation, the final segmentation output is executed as follows: in, A pixel classification probability map for the billboard target mask. express Convolution processing, This indicates pixel category probability normalization processing. Based on the pixel classification probability map, the target pixel region of the billboard can be determined, and a current billboard target mask or a reference billboard target mask can be generated.

[0047] In one implementation, the channel attention module is executed as follows: in, The feature map after channel attention weighting. For global average pooling, For global max pooling, It is a multi-layer fully connected network. The Sigmoid activation function is used; through the channel attention module, channel weights can be generated based on the response intensity of different channels, and the encoded feature maps can be weighted using the channel weights.

[0048] S4. Based on the current billboard target area data and the reference billboard target area data, determine the current deformation calculation area and the reference deformation calculation area that are in the same image coordinate system.

[0049] In S4, the ROI region for deformation calculation is determined in the same pixel coordinate system based on the current target region boundary and the reference target region boundary; based on the ROI region, the current deformation calculation region is extracted from the current normalized image data, and the reference deformation calculation region is extracted from the reference normalized image data.

[0050] The Region of Interest (ROI) can cover the overlapping area between the current billboard target area and the reference billboard target area, or it can cover the union of the boundaries of their target areas. To enable coordinate comparison between the current deformation calculation area and the reference deformation calculation area, the ROI should be located in the same pixel coordinate system, and the truncated current deformation calculation area and the reference deformation calculation area should have corresponding pixel coordinate ranges. The current deformation calculation area and the reference deformation calculation area serve as the input for calculating the subsequent first and second candidate deformation data.

[0051] S5. Based on the current deformation calculation area and the benchmark deformation calculation area, generate the first candidate deformation data based on the displacement of the feature points.

[0052] In S5, the first candidate deformation data is used to characterize the deformation information of outdoor billboards obtained by feature point matching. Specifically, benchmark ORB feature points are extracted within the benchmark deformation calculation region, and current ORB feature points are extracted within the current deformation calculation region; the ORB response values ​​of each benchmark ORB feature point and each current ORB feature point are calculated; ORB feature points are filtered based on the maximum ORB response value in the neighborhood of each ORB feature point to obtain filtered benchmark ORB feature points and filtered current ORB feature points; feature descriptor matching is performed on the filtered benchmark ORB feature points and filtered current ORB feature points to generate matching feature point pairs.

[0053] In one implementation, to avoid duplicate feature points interfering with calculation accuracy, a new feature point non-maximum suppression formula is added to screen out the feature points with the strongest local responses. The screening of RB response values ​​is performed as follows: in, ORB feature points 8 neighborhoods ORB feature points The response value; by filtering, feature points that meet the filtering rules are retained in each local neighborhood, reducing the participation of duplicate feature points and low-response feature points in displacement calculation.

[0054] For any pair of matching feature points, the coordinates of the reference feature points in the pair are used as a basis. and the coordinates of the current feature point Generate horizontal pixel displacement and vertical pixel displacement Specifically, lateral pixel displacement and vertical pixel displacement Generate according to the following method: After obtaining the pixel displacement of each matching feature point pair, the average pixel displacement can be calculated based on multiple matching feature point pairs; for For each pair of matching feature points, the average lateral pixel displacement can be calculated separately. and vertical average pixel displacement Horizontal average pixel displacement The average vertical pixel displacement can be obtained by averaging the horizontal pixel displacements of each matching feature point pair. It can be obtained by averaging the vertical pixel displacements of each matching feature point pair, as follows: Average displacement calculation: Select matching feature points and calculate the average pixel displacement. In outdoor scenes, variations in lighting can cause differences in grayscale distribution between the current deformation calculation area and the reference deformation calculation area, potentially leading to shifts in feature point detection positions or matching results. Therefore, this implementation method uses the average grayscale value of the reference deformation calculation area... and the average gray value of the current deformation calculation area , for horizontal pixel displacement and vertical pixel displacement Perform illumination robustness correction and generate the corrected lateral pixel displacement. and corrected vertical pixel displacement ; Lighting robustness correction is performed as follows: in, The average gray value of the ROI region in the baseline image. The average gray value of the ROI region corresponding to the current image is used to correct the displacement deviation caused by changes in illumination by adjusting the gray value ratio.

[0055] After obtaining the corrected pixel displacement, based on the actual width of the outdoor billboard to be monitored... and the pixel width of the outdoor billboard to be monitored in the image Generate proportionality coefficient Proportionality coefficient Pixel displacement is converted to true millimeter displacement using the following method: According to the proportionality coefficient The corrected horizontal pixel displacement and corrected vertical pixel displacement Converted into actual physical displacement. Specifically, and Multiply to obtain the actual lateral physical displacement; and Multiplying these values ​​yields the true vertical physical displacement. Thus, the first candidate deformation data is no longer merely a difference in pixel coordinates, but is transformed into true physical deformation data that can be used for comparison with safety thresholds.

[0056] To reduce the impact of feature point pairs with large matching distances on the overall displacement results, this implementation method adjusts the matching distance of each matching feature point pair. Generate corresponding weights Weight Generate according to the following method: in, The matching distance for the k-th feature point. Avoid denominator of 0, weight It is inversely proportional to the matching distance, increasing the contribution of high-confidence feature points to the results.

[0057] Furthermore, based on weights Weighted calculations are performed on the corresponding real physical displacements of multiple matching feature points to generate the first candidate deformation data. (The process of obtaining the first...) Each matching feature point corresponds to a real physical displacement. and weight Based on the corresponding real physical displacements of multiple matching feature points. and weight Generate weighted real physical displacement Based on weighted real physical displacement Generate the first candidate deformation data. Weighted true physical displacement. Generate according to the following method: in, This represents the number of feature point pairs to be matched.

[0058] For the longitudinal true physical displacement, the same weighting calculation method as described above can be used to generate the longitudinal weighted true physical displacement; through weighting processing, the matching distance is... Smaller matching feature point pairs have higher weights in the generation of the first candidate deformable data, and the matching distance... Larger matching feature points have a reduced impact on the final result.

[0059] The first candidate deformation data may include at least one of the following: lateral weighted true physical displacement, longitudinal weighted true physical displacement, overall translation, tilt, and local deformation generated from multiple point displacement data. Specifically, the overall translation can be determined by the average or weighted true physical displacement of multiple matched feature point pairs; the tilt can be determined by the displacement difference of feature points located at different boundaries of the deformation calculation region; and the local deformation can be determined by the displacement difference of feature points within different sub-regions of the deformation calculation region.

[0060] S6. Based on the current deformation calculation area and the benchmark deformation calculation area, generate second candidate deformation data based on the cross-correlation displacement of the regions.

[0061] In S6, the second candidate deformation data is used to characterize the deformation information of the outdoor billboard obtained by cross-correlation matching of local texture regions. Specifically, Gaussian smoothing is applied to the current deformation calculation region and the reference deformation calculation region respectively to generate current smoothed region data and reference smoothed region data; the reference smoothed region data is then selected... The texture-rich region of a pixel is used as a template sub-region; the search region corresponding to the template sub-region is determined in the current smooth region data.

[0062] In one implementation, Gaussian smoothing uses the following Gaussian smoothing kernel function: in, This is the smoothing coefficient. Gaussian smoothing can preserve edge information while smoothing noise, allowing the gray-level gradient of the edge region to be used for subsequent cross-correlation matching.

[0063] Use the template sub-region as the template image. The candidate sub-regions in the current smoothed region data are used as real-time images. ; for template image and real-time images Perform grayscale mean normalization and grayscale standard deviation normalization; calculate the template image. With each real-time image cross-relationships between .

[0064] In one implementation, the cross-correlation coefficient Generate according to the following method: in, For template image, For real-time images, The grayscale mean of the corresponding region in the template image. This represents the average grayscale value of the corresponding region in the real-time image.

[0065] Based on cross-relation numbers The location of the largest candidate sub-region is used to generate the best matching position for integer pixels. To obtain more refined displacement data based on the best matching position for integer pixels, sub-pixel accuracy is improved based on the cross-correlation coefficient within the neighborhood of the best matching position for integer pixels, generating sub-pixel displacement data of the template sub-region in the current deformation calculation area.

[0066] In one implementation, subpixel precision enhancement is performed as follows: in, , The optimal matching position for an integer pixel; , They are respectively The coefficients of the first and second terms of the directional fitted quadratic curve; , They are respectively The coefficients of the first and second terms of the quadratic curve fitted in the direction are used as a correction term to compensate for the systematic error of the parabola fitting, so that the subpixel positioning error is less than 0.01 pixels.

[0067] In one implementation, the sub-pixel displacement data obtained from the regional cross-correlation displacement calculation can be first determined to determine the cross-correlation coefficient before being converted into actual physical displacement. Does it meet the preset correlation conditions? When the cross-relationship number... When the correlation coefficient is below a preset correlation threshold, the displacement data corresponding to the template sub-region can be excluded as valid input for the second candidate deformation data, and a corresponding low correlation marker can be generated; when the cross-correlation coefficient is below a preset threshold, the displacement data can be excluded as valid input for the second candidate deformation data, and a corresponding low correlation marker can be generated; When the preset correlation conditions are met, according to the proportional coefficient Subpixel displacement data is converted into real physical displacement, and second candidate deformation data is generated. This process avoids the involvement of unstable textures, occlusions, or locally reflective areas in the generation of regional cross-correlation displacement results.

[0068] After obtaining the subpixel displacement data, based on the scaling factor generated in S5 Subpixel displacement data is converted into true physical displacement, and second candidate deformation data is generated based on the converted true physical displacement. The second candidate deformation data may include at least one of the following: local true physical displacement generated based on template sub-region displacement, global translation, tilt, and local deformation.

[0069] S7. After generating the target area data of the current billboard and the target area data of the benchmark billboard, generate the third candidate deformation data based on the boundary displacement of the target area.

[0070] In one implementation, third candidate deformation data participates in consistency verification as an optional data source. Its generation process includes: reading the baseline target area boundary from the baseline billboard target area data and reading the current target area boundary from the current billboard target area data; mapping the baseline target area boundary and the current target area boundary to the same pixel coordinate system; calculating the boundary position difference between the baseline target area boundary and the current target area boundary to generate target area boundary displacement data; and then, based on a scaling factor... The boundary displacement data of the target area is converted into real physical displacement, and a third candidate deformation data is generated based on the converted real physical displacement.

[0071] In one implementation, based on the current target region boundary and the reference target region boundary output by the U-Net semantic segmentation model, third candidate deformation data can be further generated. Specifically, the current target region boundary is first read from the current billboard target region data, and the reference target region boundary is read from the reference billboard target region data; then, the current target region boundary and the reference target region boundary are mapped to the same pixel coordinate system; subsequently, the boundary position difference between the two is calculated to generate target region boundary displacement data; finally, based on the scaling factor... The boundary displacement data of the target area is converted into real physical displacement to generate third candidate deformation data.

[0072] It should be noted that the U-Net semantic segmentation model in this implementation is mainly used to generate the billboard target mask, target region boundary, and ROI region. The third candidate deformation data is not directly generated by replacing the segmentation result of the U-Net semantic segmentation model, but is obtained by further calculating the boundary position difference based on the target region boundary output by the U-Net semantic segmentation model. This avoids the erroneous understanding of the region extraction model as a deformation calculation algorithm at the same level as feature point displacement calculation and region cross-correlation displacement calculation.

[0073] S8. Generate deviation rate data based on at least two sets of candidate deformation data corresponding to the same deformation type, and perform consistency verification on at least two sets of candidate deformation data based on the deviation rate data to generate deformation monitoring results.

[0074] In S8, candidate deformation data may include at least two sets of first candidate deformation data, second candidate deformation data, and third candidate deformation data. Deformation type may include at least one of global translation, tilt, and local deformation. To enable comparison between different candidate deformation data, the data participating in the consistency verification should belong to the same deformation type and have all been converted into real physical deformation data.

[0075] In one implementation, S8 includes: acquiring data values ​​belonging to the same deformation type from at least two sets of candidate deformation data; using the data values ​​belonging to the same deformation type in the first candidate deformation data as the reference data value, calculating the deviation rate of the data values ​​belonging to the same deformation type in other candidate deformation data relative to the reference data value, and generating deviation rate data; when the deviation rate data is less than or equal to a first deviation threshold, generating a deformation monitoring result including real physical deformation data that has passed consistency verification based on the first candidate deformation data; when the deviation rate data is greater than the first deviation threshold and less than or equal to a second deviation threshold, generating a re-detection instruction, the re-detection instruction being used to trigger the reacquisition of the current image data and the regeneration of candidate deformation data; when the deviation rate data is greater than the second deviation threshold, generating a deformation monitoring result including abnormal data markings and review prompt data.

[0076] In one implementation, the first candidate deformation data, the second candidate deformation data, and the third candidate deformation data can be mutually verified. During mutual verification, data values ​​belonging to the same deformation type are first selected, such as overall translation, tilting, or local deformation; then, the data values ​​belonging to the deformation type in the first candidate deformation data are used as the reference data values, and the deviation rate of the second candidate deformation data and / or the third candidate deformation data relative to the reference data values ​​is calculated to generate deviation rate data.

[0077] When the deviation rate is less than or equal to 5%, at least two sets of candidate deformation data are determined to meet the consistency requirements, and a deformation monitoring result including the actual physical deformation data that has passed the consistency check is generated based on the first candidate deformation data. When the deviation rate is greater than 5% but less than or equal to 10%, a re-detection instruction is generated to reacquire the current image data, and preprocessing, billboard pixel region extraction, candidate deformation data generation, and consistency check are performed again based on the reacquired current image data. When the deviation rate is greater than 10%, a deformation monitoring result including abnormal data marking and review prompt data is generated, and the current image data, billboard identification, acquisition time, acquisition location information, image acquisition parameters, and environmental parameters are recorded for manual review.

[0078] During the aforementioned mutual verification process, the first, second, and third candidate deformation data should all be converted into real physical deformation data under the same physical dimensions before the deviation rate is calculated. If any candidate deformation data fails the validity judgment, for example, if the cross-correlation coefficient is lower than the preset correlation threshold, the target area boundary is missing, or the number of matching feature points is insufficient, then the candidate deformation data will not be used as the consistency verification input for the current monitoring period, and a corresponding invalid data marker will be generated.

[0079] In one alternative implementation, the confidence level of the deformation monitoring results can also be quantified; the confidence level is used to characterize the reliability of the deformation identification results within the current monitoring period. The confidence level can be calculated based on the number of true positive events (TP) where the actual deformation is correctly identified, the number of false positive events (FP) where no deformation occurs but is misidentified as deformation, and the number of false negative events (FN) where actual deformation occurs but is not identified. The manual annotation method uses on-site inspection and verification results as the true value benchmark. Measured data from professional equipment (such as total stations and displacement gauges) is used to categorize each monitoring result into three types: "true deformation," "false deformation," and "missed detection." The sample update cycle is set to quarterly, and the sample library is iteratively optimized by combining newly added inspection data, historical false alarm records, and structural safety assessment reports to ensure the timeliness and accuracy of the confidence model. The correspondence with single monitoring results is as follows: each deformation monitoring output result is classified into TP, FP, or FN categories based on its match with the labeled true value, serving as a valid sample for confidence statistics. Simultaneously, the confidence level participates in alarm triggering decisions; only when the confidence level meets a preset threshold will the monitoring result be included in the alarm logic judgment. Results below the threshold are only marked as suspicious data and do not directly trigger an alarm. If confidence level is used as an auxiliary basis for alarm output, the above rules should be determined based on manual verification results, historical annotated samples, or inspection verification data.

[0080] In one implementation, when the confidence level is greater than or equal to a preset alarm confidence threshold, the actual physical deformation data that has passed consistency verification can be compared with a preset safety threshold, and alarm data can be output. When the confidence level is within a preset warning confidence interval, warning data can be generated. When the confidence level is lower than a preset review confidence threshold, the current monitoring data can be marked as suspicious data, and re-detection can be triggered. The preset alarm confidence threshold, preset warning confidence interval, and preset review confidence threshold can be determined based on sample verification results, inspection review results, or management requirements.

[0081] In one alternative implementation, confidence levels are calculated using TP, FP, and FN. TP is a true positive, indicating that the actual deformation was correctly identified; FP is a false positive, indicating that no deformation was misjudged; and FN is a false negative, indicating that the deformation was not identified. The confidence level can be calculated as follows: When the confidence level is greater than or equal to 90%, the actual physical deformation data that has passed the consistency verification can be compared with the preset safety threshold and an alarm can be triggered. When the confidence level is between 80% and 90%, early warning data can be generated; When the confidence level is below 80%, suspicious data can be marked and a re-inspection can be triggered.

[0082] S9, when the deformation monitoring results include real physical deformation data that has passed the consistency verification, and the real physical deformation data is greater than the corresponding preset safety threshold, generate and output alarm data for the outdoor billboard to be monitored.

[0083] In S9, at least one of the overall translation, tilt, and local deformation variables in the real physical deformation data is first acquired, and a preset safety threshold corresponding to the overall translation, tilt, and local deformation variables is acquired respectively. Then, the real physical deformation data is compared with the corresponding preset safety threshold. In response to at least one of the overall translation, tilt, and local deformation variables being greater than the corresponding preset safety threshold, at least one of the following is written into the preset alarm data structure: billboard identification, acquisition time, acquisition location information, deformation type, real physical deformation data value, current image data, current billboard target area data, environmental parameters, abnormal data markers, and review prompt data. Alarm data is then generated and output.

[0084] Environmental parameters can include wind speed, temperature, rainfall, light intensity, or other data reflecting the environmental conditions of the outdoor billboard. The data source for these environmental parameters is a meteorological monitoring terminal integrated with the smart light pole; the acquisition devices are integrated wind speed and direction sensors, temperature and humidity sensors, rainfall sensors, and light intensity sensors; the acquisition frequency is configured according to scenario requirements, defaulting to synchronization with the image acquisition cycle, and can automatically increase to once per minute under extreme weather conditions; the time synchronization method uses the NTP network time protocol to ensure that the timestamp error between the environmental parameter data and the current image data is controlled within 1 second, achieving time sequence alignment. Environmental parameters can serve as auxiliary fields in alarm data for subsequent review and handling. For example, when a deformation alarm occurs, the wind speed and temperature data for the corresponding time period can be combined to determine the cause of the deformation, assisting in manual on-site verification; if a certain implementation scenario does not collect environmental parameters, they can be omitted from the alarm data structure. In one implementation, environmental parameters can also be used as part of the review prompt data to mark conditions prone to false alarms, such as high wind loads and heavy rainfall, triggering the manual review process.

[0085] In one implementation, after generating alarm data, the deformation monitoring results and alarm data are encapsulated into an upload message, which is then sent according to a preset remote management address to generate historical monitoring record data and visualization display data. The upload message may include at least one of the following: billboard identifier, acquisition time, acquisition location information, deformation monitoring results, alarm data, current image data, current billboard target area data, abnormal data markers, review prompt data, and environmental parameters. The upload message can be sent to the remote management platform via the MQTT protocol, or other communication methods capable of remote data transmission can be used.

[0086] In one implementation, the remote management platform can perform data storage, trend analysis, multi-source data fusion, early warning strategy management, and visualization. The remote management platform can adopt a B / S architecture, allowing the management end to access historical monitoring records, alarm data, current image data, and current billboard target area data via a browser. Historical monitoring records can be used to display the overall translation, tilt, and local deformation trends of the same monitored outdoor billboard over multiple monitoring periods. Multi-source data fusion can correlate real physical deformation data with wind force, temperature, rainfall, lighting conditions, or inspection notes to help determine the environmental background and handling priority corresponding to alarm data.

[0087] In a specific application scenario, the outdoor billboard to be monitored is located on one side of an urban road, and the smart light poles are deployed on the opposite or diagonally opposite side of the billboard. The system pre-collects image data of the billboard in a qualified state as baseline image data and records the billboard identification, acquisition time, and acquisition location information. In subsequent monitoring cycles, the system extracts the current frame image from the billboard video stream and associates the current frame image with the billboard identification, acquisition time, and acquisition location information to generate the current image data. Subsequently, standardization processing, billboard target area extraction, deformation calculation, candidate deformation data consistency verification, and alarm output are performed through S2 to S9. When the billboard experiences overall translation, tilting, or local warping due to wind load, loose connectors, or bracket deformation, this implementation can convert the changes into real physical deformation data and generate alarm data based on a preset safety threshold.

[0088] In another implementation, if the outdoor billboard to be monitored is an LED billboard, the current image data may contain changes in displayed content, localized bright areas, and dynamic scenes. In this case, grayscale conversion, noise reduction, and illumination normalization in preprocessing can reduce the impact of changes in displayed content on subsequent feature point displacement and region cross-correlation displacement, and ROI regions can be preferentially determined in the billboard edge area, fixed structure area, or texture-stable area. In this scenario, the first candidate deformation data can mainly reflect the feature point displacement at the billboard boundary or fixed structure position, while the second candidate deformation data can mainly reflect the local displacement in texture-stable areas.

[0089] In another implementation, if the outdoor billboard to be monitored is a grid billboard, the current image data may contain repetitive textures, partial occlusion, and background perspective interference. In this case, ORB response value filtering, matching distance weighting, and candidate deformation data consistency verification can be used to reduce erroneous matches caused by repetitive textures, and a cross-correlation algorithm can be used to select the appropriate models. Local matching is performed on regions with rich pixel textures to form second candidate deformation data that is cross-validated with the displacement calculation of feature points.

[0090] Example 2: Based on Embodiment 1, this invention can be implemented through an edge-cloud collaborative approach. The edge side is used to acquire video streams or current frame images from outdoor billboards; the edge side is used to perform current image data generation, preprocessing, billboard pixel region extraction, ROI region determination, candidate deformation data generation, consistency verification, and alarm data generation; the cloud side is used to receive deformation monitoring results and alarm data, and generate historical monitoring record data and visualization display data, such as: S100, Smart Light Pole Equipment Deployment and Image Data Acquisition The S101 uses a high-definition camera with a resolution of 3840×2160 and a frame rate of 25fps, paired with a GPS synchronization module to ensure accurate synchronization of the acquisition time. The camera lens parameters are adjusted according to the monitoring distance to ensure coverage of key areas of the LED billboard. It is equipped with an edge computing smart box (equipped with an NVIDIA Jetson Xavier NX processor and 8GB of memory), which has the functions of real-time data processing and parallel deployment of multiple algorithms. S102. Based on the camera's coverage area, the distribution characteristics of LED billboards, and the surrounding environmental conditions, the equipment is deployed on the smart light pole directly in front of the billboard. The deployment height and angle are based on the principle of unobstructed and full coverage, ensuring that there are no blind spots in the key areas of the LED billboard surface. S103: The device collects LED billboard image data at a preset frequency of 25 frames per second. The edge device receives the data through the local area network, completes temporary caching and JPEG format standardization processing, and establishes data association tags containing device identification, collection time, and location. The cloud receives image data that needs to be stored for a long time or analyzed globally through a high-speed communication module and uploads it to the urban public facility security cloud platform server to ensure that the data is traceable and accessible. S200, data preprocessing and parallel processing of three types of algorithms and result verification S201 employs a combination of techniques such as Gaussian smoothing noise reduction, histogram equalization, image cropping, and region of interest (ROI) extraction for preprocessing to remove environmental noise and lighting interference, convert the image to standard JPEG format, and establish a data index containing device ID, acquisition time, and location. S202, parallel deployment of template matching feature point detection algorithm, cross-correlation algorithm and U-Net semantic segmentation algorithm, all load pre-trained weight files and complete lightweight processing to reduce the resource consumption of edge computing smart box; S203. Three algorithms were run simultaneously to calculate deformation: the template matching feature point detection algorithm calculated millimeter-level deformation through ORB feature extraction and matching; the cross-correlation algorithm calculated deformation through gray-level cross-correlation analysis and sub-pixel optimization; and the U-Net semantic segmentation algorithm first segmented the target region and then calculated the deformation. Comparing the results of the three types, the deformation deviation rate was 3.2% ≤ 5%, and the result was deemed valid, with the result of the template matching feature point detection algorithm as the benchmark. S204. When the baseline deformation exceeds the preset safety threshold and the confidence level is 93%≥90%, a graded alarm is triggered and the deformation area is selected in the image; the alarm information is synchronously transmitted to the cloud platform for urban public facility safety and backed up locally on the edge computing smart box to avoid information loss. S205. Three types of algorithms are deployed using containerization technology to improve the efficiency of algorithm updates and troubleshooting; the CPU utilization, memory usage and communication link status of the edge computing smart box are monitored in real time, and maintenance reminders are triggered when the device is malfunctioning to ensure the stable operation of edge devices; algorithm parameters are updated regularly to adapt to feature changes in different environments; S300 and cloud platform construction and function output S301: Receive alarm data, verification results and image data uploaded from the edge terminal, record the transmission status through the data receiving log, verify the data integrity, and store the data using a "device-time-space" three-dimensional index; S302. Mark the location and extent of deformation on the visual dashboard, automatically generate maintenance suggestion plans, push early warning information to the urban management department, and support filtering and exporting monitoring reports by "deformation occurrence time period"; S400 and monitoring system integration and implementation S401. Constructing an edge-cloud collaborative architecture: The terminal camera transmits video streams to the edge computing smart box via the RTSP protocol; the edge computing smart box completes data preprocessing, runs three types of algorithms and verifies results, and uploads structured data via the MQTT protocol; the cloud server builds a data management platform to realize data storage, trend analysis and visualization. S402. Complete system integration: Cameras are fixedly deployed at preset locations to ensure no blind spots in monitoring; the edge computing smart box supports automatic startup of multi-algorithm verification mode; the cloud platform realizes real-time updates and queries of monitoring data, and the system automatically cleans abnormal data such as jump points and missing values. When the deformation exceeds the threshold, a dual warning is triggered via SMS and platform pop-up.

[0091] Example 3: This embodiment also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the deformation monitoring method for outdoor billboards based on smart light poles described in S1 to S9 above. The processor executes data processing steps such as data acquisition, image preprocessing, billboard pixel region extraction, ROI region determination, candidate deformation data generation, consistency verification, alarm data generation, and message encapsulation. On-site construction activities such as smart light pole deployment, billboard installation, and physical installation of camera equipment should not be interpreted as being directly executed by the processor.

[0092] In one implementation, the electronic device can be installed in a smart light pole enclosure, a roadside local control box, a monitoring and management node, or a remote server. If the electronic device is installed locally, image preprocessing, billboard pixel region extraction, and candidate deformation data generation can be performed locally, reducing the amount of data uploaded in the original video stream. If the electronic device is installed remotely, it can receive the uploaded current image data and reference image data, and execute subsequent deformation monitoring processes. The above deployment methods are only used to illustrate different implementation methods and do not change the data stream processing logic in the method of this invention.

[0093] This implementation places the extraction of the billboard's target area before the deformation calculation process, ensuring that subsequent feature point displacement calculations and regional cross-correlation displacement calculations are performed within the ROI region, thus reducing the impact of complex road backgrounds on billboard deformation calculations. Feature point selection is based on ORB response values, and... and Perform robust lighting correction, based on and Generate proportionality coefficient And convert the actual physical displacement, based on generate By performing weighted calculations, the first candidate deformation data can be made more suitable for monitoring scenarios involving changes in outdoor lighting and differences in local matching quality. Through... By performing cross-correlation calculations on texture-rich regions of pixels and improving sub-pixel accuracy, the second candidate deformation data can complement the first candidate deformation data. Consistency checks are performed on at least two sets of candidate deformation data using deviation rate data. This allows for the identification of inconsistencies between candidate data before alarm output, and the generation of valid monitoring results, re-detection instructions, or review prompts when the deviation rate data falls within different threshold ranges.

[0094] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0095] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of the claims of this application.

Claims

1. A method for monitoring deformation of an outdoor billboard based on a smart light pole, characterized in that, include: Acquire image data of the outdoor billboard to be monitored, including current image data collected at the smart light pole deployment points and reference image data corresponding to the outdoor billboard to be monitored; The current image data and the reference image data are preprocessed to generate current standardized image data and reference standardized image data; The current standardized image data and the benchmark standardized image data are respectively processed to extract the billboard pixel region, generating the current billboard target region data and the benchmark billboard target region data; Based on the current billboard target area data and the reference billboard target area data, determine the current deformation calculation area and the reference deformation calculation area that are in the same image coordinate system; Based on the current deformation calculation region and the reference deformation calculation region, at least two sets of candidate deformation data are generated respectively. The at least two sets of candidate deformation data include first candidate deformation data generated based on feature point displacement and second candidate deformation data generated based on regional cross-correlation displacement. Deviation rate data is generated based on at least two sets of candidate deformation data corresponding to the same deformation type, and consistency verification is performed on the at least two sets of candidate deformation data based on the deviation rate data to generate deformation monitoring results. When the deformation monitoring results include real physical deformation data that has passed consistency verification, and the real physical deformation data is greater than the corresponding preset safety threshold, alarm data for the outdoor billboard to be monitored is generated and output. 2.The smart pole-based outdoor billboard morphing monitoring method of claim 1, wherein, The acquisition of image data of the outdoor billboard to be monitored includes: Obtain the video stream address and billboard identifier corresponding to the outdoor billboard to be monitored; Obtain the outdoor billboard video stream based on the video stream address; The current frame image is extracted from the video stream of the outdoor billboard according to a preset sampling frequency; The current frame image is associated with the billboard identifier, acquisition time, and acquisition location information to generate the current image data; Based on the billboard identification, read the reference image data corresponding to the outdoor billboard to be monitored; The current image data and the reference image data are output to the preprocessing step; The reference image data is pre-collected billboard image data that has been marked as qualified. 3.The smart pole-based outdoor billboard morphing monitoring method of claim 1, wherein, The preprocessing of the current image data and the reference image data to generate current standardized image data and reference standardized image data includes: Receive the current image data and the reference image data; Based on preset image cropping parameters, the current image data and the reference image data are cropped respectively to generate current cropped image data and reference cropped image data; The current cropped image data and the reference cropped image data are respectively processed into grayscale to generate current grayscale image data and reference grayscale image data; Denoising processing is performed on the current grayscale image data and the reference grayscale image data respectively to generate current denoised image data and reference denoised image data; Based on the grayscale distribution of the current denoised image data and the reference denoised image data, the current denoised image data is subjected to illumination normalization processing to generate current illumination normalized image data. The current illumination-normalized image data and the reference denoised image data are subjected to scale normalization processing to generate the current normalized image data and the reference normalized image data with the same image size and the same pixel coordinate system. 4.The smart pole-based outdoor billboard metamorphic monitoring method of claim 1, wherein, The step of extracting the billboard pixel region from the current standardized image data and the reference standardized image data respectively, generating current billboard target region data and reference billboard target region data, and determining the current deformation calculation region and the reference deformation calculation region includes: The current standardized image data is input into the trained U-Net semantic segmentation model to generate the current billboard target mask; Based on the current billboard target mask, extract the current billboard pixel region from the current normalized image data, and generate the current billboard target region data including the current billboard target mask and the current target region boundary; The benchmark normalized image data is input into the U-Net semantic segmentation model to generate a benchmark billboard target mask; Based on the benchmark billboard target mask, the benchmark billboard pixel region is extracted from the benchmark normalized image data, and the benchmark billboard target region data including the benchmark billboard target mask and the benchmark target region boundary is generated; Based on the current target region boundary and the reference target region boundary, determine the ROI region for deformation calculation in the same pixel coordinate system; Based on the ROI region, the current deformation calculation region is extracted from the current normalized image data, and the reference deformation calculation region is extracted from the reference normalized image data.

5. The smart pole based outdoor billboard metamorphosis monitoring method of claim 4, wherein, After receiving the current standardized image data or the baseline standardized image data, the U-Net semantic segmentation model generates an encoded feature map, calculates channel weights on the encoded feature map, and performs weighted processing on the encoded feature map according to the calculated channel weights to generate a weighted feature map. Based on the weighted feature map, an advertising billboard target mask is generated.

6. The smart pole based outdoor billboard metamorphic monitoring method of claim 1, wherein, The generation of the first candidate deformation data based on feature point displacement includes: Extract the baseline ORB feature points within the baseline deformation calculation region, and extract the current ORB feature points within the current deformation calculation region; Calculate the ORB response values ​​for each baseline ORB feature point and each current ORB feature point; ORB feature points are filtered based on the maximum ORB response value in the neighborhood of each ORB feature point to obtain the filtered baseline ORB feature points and the filtered current ORB feature points. Feature descriptor matching is performed on the filtered baseline ORB feature points and the filtered current ORB feature points to generate matching feature point pairs; generating a lateral pixel displacement and a longitudinal pixel displacement based on the reference feature point coordinates and the current feature point coordinates and the current feature point coordinates ;​​ The average gray value of the region is calculated based on the reference deformation. and the average gray value of the current deformation calculation area For the lateral pixel displacement and the vertical pixel displacement Perform illumination robustness correction and generate the corrected lateral pixel displacement. and corrected vertical pixel displacement ; Based on the actual width of the outdoor billboard to be monitored and the pixel width of the outdoor billboard to be monitored in the image Generate proportionality coefficient ; According to the proportionality coefficient The corrected lateral pixel displacement and the corrected vertical pixel displacement Converted into actual physical displacement; Based on the matching distance of each matching feature point pair Generate corresponding weights ; Based on the weight The first candidate deformation data is generated by weighting the corresponding real physical displacements of multiple matching feature points. Wherein, the lateral pixel displacement and the vertical pixel displacement Generate according to the following method: The proportionality coefficient Generate according to the following method: The illumination robustness correction is performed as follows: The weight Generate according to the following method: in, ; The weighted real physical displacement Generate according to the following method: in, Let N be the k-th matching feature point, and N be the number of matching feature point pairs.

7. The method for monitoring deformation of outdoor billboards based on smart light poles according to claim 1, characterized in that, The generation of second candidate deformation data based on regional cross-correlation displacement includes: Gaussian smoothing is applied to the current deformation calculation region and the reference deformation calculation region respectively to generate current smoothed region data and reference smoothed region data; Select from the reference smoothing region data The texture-rich areas of a pixel are used as template sub-regions; Determine the search region corresponding to the template sub-region in the current smoothed region data; The template sub-region is used as the template image. The candidate sub-regions in the current smoothed region data are used as real-time images. ; For the template image and the real-time image Perform grayscale mean normalization and grayscale standard deviation normalization; Calculate the template image With each real-time image cross-relationships between ; Based on cross-relation numbers The location of the largest candidate sub-region generates the best matching position for an integer pixel. Based on the cross-correlation coefficient within the neighborhood of the best matching position of the whole pixel, sub-pixel accuracy is improved to generate sub-pixel displacement data of the template sub-region in the current deformation calculation region. According to the proportionality coefficient The subpixel displacement data is converted into real physical displacement, and the second candidate deformation data is generated based on the converted real physical displacement. Wherein, the cross-relation number Generate according to the following method: in, For template image, For real-time images, The grayscale mean of the corresponding region in the template image. This represents the average grayscale value of the corresponding region in the real-time image. The Gaussian smoothing process uses the following Gaussian smoothing kernel function: in, For smoothing coefficients; The sub-pixel precision enhancement is performed as follows: in, , For the best matching position of integer pixels, , They are respectively The coefficients of the first and second terms of the directional fitting quadratic curve. , They are respectively The coefficients of the first and second terms of the directional fitted quadratic curve.

8. The method for monitoring deformation of outdoor billboards based on smart light poles according to claim 6, characterized in that, After generating the current billboard target area data and the baseline billboard target area data, the method further includes: The boundary of the benchmark target area is read from the benchmark billboard target area data, and the boundary of the current target area is read from the current billboard target area data; Map the boundary of the reference target region and the boundary of the current target region to the same pixel coordinate system; Calculate the boundary position difference between the reference target region boundary and the current target region boundary, and generate target region boundary displacement data; According to the proportionality coefficient The boundary displacement data of the target area is converted into real physical displacement, and a third candidate deformation data is generated based on the converted real physical displacement. The third candidate deformation data is used to perform consistency verification with at least one of the first candidate deformation data and the second candidate deformation data.

9. The method for monitoring deformation of outdoor billboards based on smart light poles according to claim 1, characterized in that, The process of generating deviation rate data based on at least two sets of candidate deformation data corresponding to the same deformation type, performing consistency verification on the at least two sets of candidate deformation data based on the deviation rate data, generating deformation monitoring results, and generating and outputting alarm data for the outdoor billboard to be monitored includes: Obtain data values ​​belonging to the same deformation type from the at least two sets of candidate deformation data, wherein the deformation type includes at least one of global translation, tilt, and local deformation. Using the data value belonging to the same deformation type in the first candidate deformation data as the benchmark data value, calculate the deviation rate of the data value belonging to the same deformation type in other candidate deformation data relative to the benchmark data value, and generate the deviation rate data. When the deviation rate data is less than or equal to the first deviation threshold, the deformation monitoring result, which includes real physical deformation data that has passed the consistency verification, is generated based on the first candidate deformation data. When the deviation rate data is greater than the first deviation threshold and less than or equal to the second deviation threshold, a re-detection instruction is generated. The re-detection instruction is used to trigger the reacquisition of the current image data and the regeneration of candidate deformation data. When the deviation rate data is greater than the second deviation threshold, the deformation monitoring result, including abnormal data markers and review prompts, is generated. When the deformation monitoring result includes real physical deformation data that has passed the consistency verification, at least one of the overall translation, tilt, and local deformation in the real physical deformation data is obtained, and a preset safety threshold corresponding to the overall translation, the tilt, and the local deformation is obtained respectively. In response to at least one of the overall translation, the tilt, and the local deformation exceeding a corresponding preset safety threshold, at least one of the following is written into a preset alarm data structure: billboard identifier, acquisition time, acquisition location information, deformation type, actual physical deformation data value, current image data, current billboard target area data, environmental parameters, abnormal data marker, and review prompt data; alarm data is generated and output. The deformation monitoring results and the alarm data are encapsulated into an upload message, and the upload message is sent according to a preset remote management address to generate historical monitoring record data and visualization display data.

10. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the deformation monitoring method for outdoor billboards based on smart light poles as described in any one of claims 1 to 9.

Images