Municipal street light dim warning dispatching system based on unmanned aerial vehicle vision analysis

Abstract

The present application relates to the technical fields of operation and maintenance, and discloses a municipal street lamp dim light early warning dispatching system based on unmanned aerial vehicle vision analysis, comprising: controlling the unmanned aerial vehicle to fly along the road according to the preset route to collect continuous video image streams; constructing a street lamp arrangement sequence based on municipal lamp post asset points and road center lines and generating a lamp arm extension offset; extracting a lamp head light spot sequence from the image stream to locate the dim light lamp head image position, mapping it to the lamp head installation height plane, and using the lamp arm extension offset to perform position offset compensation processing to obtain the geographic coordinates of the fault lamp post; associating the geographic coordinates with the asset data in the street lamp arrangement sequence to generate a dim light event, generating a maintenance dispatching business field, and sending a dim light maintenance dispatching work order containing the geographic coordinates of the fault lamp post to a mobile terminal receiving object. The present application overcomes the problem of incorrect dispatching of work orders caused by the spatial separation of lamp heads and lamp posts, and realizes automatic fault anchoring and dispatching operation closed loop.

Description

Technical Field

[0001] This invention relates to the field of operation and maintenance management technology, and more specifically, to a municipal street light dimming warning and dispatch system based on UAV visual analysis. Background Technology

[0002] In the operation and maintenance system of municipal road lighting, the method of troubleshooting street light malfunctions is gradually evolving from manual inspections to drone-borne visual equipment patrols. At present, conventional drone night patrol solutions mainly use airborne cameras to collect images of road lighting, identify unlit areas in the images, and then project the image location points directly onto the ground plane, or match them to nearby equipment locations according to the plane distance.

[0003] However, urban streetlights generally use a combination of poles and cantilevered extensions. In this configuration, the lamp head, acting as the light-emitting actuator, is suspended above the lane, while the pole, serving as the asset for management and the anchor point for maintenance personnel, stands within the sidewalk or central median, resulting in a fixed spatial discrepancy between the two. When the system identifies unlit spaces solely based on image recognition, it only obtains the location of the lamp head. If conventional data processing methods are used to project this directly onto the road surface, the fault point is often pinpointed within the lane. Furthermore, if the system is based on proximity, in scenarios with symmetrical arrangements on both sides of the road, double-arm arrangements in the central median, and dense pole positions at intersections, maintenance instructions are easily misassigned to adjacent streetlights, lamplights in the opposite lane, or another lamp position on the same pole.

[0004] This coordinate mapping error, caused by the spatial separation of the light-emitting components and the pole base, prevents the visual alarm results generated at the front end from being directly converted into asset coordinates that can be imported into the business system and guide the maintenance team to the target pole position. Currently, the business process still requires back-end personnel to compare video recordings with ledger drawings when intervening in the work order assignment stage, limiting the closed-loop automation level of converting inspection data into front-line maintenance work orders. Summary of the Invention

[0005] This invention provides a municipal street light dimming warning and dispatch system based on UAV visual analysis, which solves the technical problems mentioned in the background art.

[0006] This invention provides a municipal street light dimming warning and dispatch system based on UAV visual analysis, applied to an operation and maintenance architecture that includes image acquisition equipment and a platform system, comprising:

[0007] Control the drone equipped with the image acquisition device to fly along the road along a preset route and acquire a continuous video stream; The street light arrangement sequence is constructed based on the municipal light pole asset points and the road centerline, and the arm extension offset of each light position is generated. Extract the bright spot sequence of the lamp head from the continuous video image stream, and locate and extract the image position of the missing lamp head in the street light arrangement sequence; The image position of the missing lamp head is mapped to the lamp head installation height plane, and the position offset is compensated by the lamp arm extension offset. The geographical coordinates of the faulty lamp pole are anchored to avoid coordinate mapping errors. The geographical coordinates of the faulty light pole are associated with the asset data in the street light arrangement sequence to generate a missing light event and identify the corresponding unique repair object; Based on the aforementioned brightness loss event, a repair dispatch order business field is generated in the platform system, and the mobile terminal receiving object is specified; The maintenance dispatch service field is used to send a maintenance dispatch order for the faulty street light containing the geographical coordinates of the faulty light pole to the receiving object of the mobile terminal.

[0008] Beneficial effects include: mapping the location of the missing lamp head image extracted from continuous video to the lamp head installation height plane, and using the lamp arm extension offset to perform position offset compensation processing, thus reversely anchoring the geographical coordinates of the faulty lamp pole. This invention effectively overcomes the mapping deviation caused by the spatial separation of the light source and the asset pole position in cantilever street light scenarios, avoiding the situation where work orders are mistakenly assigned to adjacent lamp poles, opposite lamp poles, or other lamp heads on the same pole in sections such as double-armed central medians or densely packed pole positions at intersections; at the same time, this invention directly binds the corrected geographical coordinates to the municipal asset sequence, automatically generating and sending maintenance dispatch business data containing coordinate positioning and evidence images, realizing an automated closed loop from drone patrol image acquisition to the maintenance team's mobile terminal receiving the work order, improving the operational efficiency of municipal lighting maintenance management. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the fault anchoring and dispatching mechanism for streetlights with missing lights based on lamp arm extension offset compensation according to the present invention. Detailed Implementation

[0010] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0011] A municipal street light dimming warning and dispatch system based on UAV visual analysis is applied to an operation and maintenance architecture that includes image acquisition equipment and a platform system, including: Control the drone equipped with the image acquisition device to fly along the road along a preset route and acquire a continuous video stream; The street light arrangement sequence is constructed based on the municipal light pole asset points and the road centerline, and the arm extension offset of each light position is generated. Extract the bright spot sequence of the lamp head from the continuous video image stream, and locate and extract the image position of the missing lamp head in the street light arrangement sequence; The image position of the missing lamp head is mapped to the lamp head installation height plane, and the position offset is compensated by the lamp arm extension offset. The geographical coordinates of the faulty lamp pole are anchored to avoid coordinate mapping errors. The geographical coordinates of the faulty light pole are associated with the asset data in the street light arrangement sequence to generate a missing light event and identify the corresponding unique repair object; Based on the aforementioned brightness loss event, a repair dispatch order business field is generated in the platform system, and the mobile terminal receiving object is specified; The maintenance dispatch service field is used to send a maintenance dispatch order for the faulty street light containing the geographical coordinates of the faulty light pole to the receiving object of the mobile terminal.

[0012] S201, During the designated operating period when the streetlights are set to be lit, control the drone equipped with the image acquisition device to fly along the designated inspection flight path to obtain scene images.

[0013] Specifically, the designated operating time for streetlights is the time window during which municipal road lighting facilities are scheduled to be powered on and operational, typically during nighttime hours, spanning from 6 PM to 6 AM the following morning. The image acquisition device is a visible light sensor lens fixed to the bottom of the drone. In some optional implementations, the designated inspection flight corridor is a low-altitude flight corridor pre-planned using a 3D map, distributed along the centerline of the road or the space above the outer edge of the carriageway. Furthermore, the flight altitude of the designated inspection flight corridor is set between 10 and 30 meters above the ground, maintaining a horizontal safety distance of at least 5 meters from municipal lampposts and roadside tree canopies to avoid physical collisions. The flight speed must match the video capture frame rate to ensure that the image overlap of the same lamphead between adjacent frames meets tracking requirements. Finally, the ground control platform issues the flight path command for the designated inspection flight corridor to the drone, which then enters the space above the corresponding road segment and begins a uniform-speed patrol, continuously capturing images of the road surface and surrounding facilities to form scene images.

[0014] S202, the image acquisition device is used to acquire video of the road lighting scene at a fixed frame rate to form video frames, and the camera exposure parameters are fixed before entering the road.

[0015] Specifically, a fixed frame rate of 25 or 30 frames per second is set to enable the image acquisition device to continuously and stably capture the nighttime environment along the road, generating multiple consecutive single-frame images, i.e., video frames. In some optional implementations, the camera exposure parameters include at least one of the camera's ISO, shutter speed, and aperture size. Before the drone enters the current preset road segment and begins recording, the values ​​of the camera exposure parameters are locked and remain unchanged. Furthermore, in operation scenarios with a large number of cantilevered streetlights arranged continuously, if the device's built-in automatic exposure mode is used, when the image acquisition device approaches or passes over a bright, lit streetlight, the camera's light-sensing calculation module will lower the exposure value, causing a sharp decrease in the overall brightness of the image. This results in the loss of dark area structural details in the video image when the drone subsequently passes over unlit streetlight spaces. Finally, by fixing the values ​​of the camera exposure parameters, sudden changes in the overall brightness of the image caused by the high intensity of streetlight illumination are avoided, preserving the visibility of the background structure in dark areas.

[0016] S203, synchronously record the corresponding UAV position information, camera attitude information, gimbal attitude information and timestamp information for each video frame.

[0017] Specifically, at the same moment each video frame is generated, the UAV's built-in satellite positioning module is invoked to obtain the absolute three-dimensional coordinate coefficient values ​​of the current instant as the UAV's position information. The UAV's position information includes the longitude, latitude, and altitude of the UAV's location. In some optional implementations, gimbal attitude information records the pitch, yaw, and roll angles of the mechanical gimbal used by the camera on the UAV at the corresponding moment. Further, camera attitude information is calculated by superimposing the UAV's own flight three-dimensional attitude angles with the angle values ​​of the gimbal attitude information. All angle parameters are converted into a unified calculation standard within the system to reflect the absolute three-dimensional spatial line of sight extension pointed to by the image acquisition device when capturing each video frame. Finally, the timestamp information is a precise time synchronization tag uniformly allocated and issued by the system to the UAV terminal. Using the system's unified high-precision timing standard, the UAV position information, camera attitude information, gimbal attitude information, and timestamp information are encapsulated and packaged with the corresponding video frame into a data stream file with spatial attribute identifiers and time attribute identifiers for persistent storage.

[0018] S204, establish the data association between the video frame, the flight mileage along the road, and the street light arrangement sequence.

[0019] Specifically, the flight mileage along the road is the cumulative distance traveled from the starting point, calculated by projecting the UAV's position information onto a pre-marked road centerline using a vertical geometric projection. In some optional embodiments, the street light arrangement sequence is a set of numbers extracted from the municipal asset ledger, consisting of multiple physical street light positions belonging to the same road segment and arranged sequentially according to spatial distribution mileage. Further, the video frames carrying the timestamp information are retrieved, and the calculated flight mileage along the road is extracted and compared with the preset mileage coordinates corresponding to each street light position in the street light arrangement sequence. Finally, the resulting data association is a hierarchical matching index table. Using this index table, each continuous image captured can accurately fall into the mileage segment interval of the actual road space and be bound to the candidate street light entity asset code distributed within that interval, forming a data link that spans from dynamic visual images to the static municipal entity grid hierarchical affiliation.

[0020] S301, map each of the municipal light pole assets to the center line of the road to obtain the corresponding cumulative mileage and the side location of the road.

[0021] Specifically, the municipal light pole asset point refers to the absolute geographical coordinates of the physical location of the light pole base registered in the lighting business system logbook, reflecting the fixed base landing point of the physical pole in the real physical environment. The road centerline is a virtual vector line segment representing the geometric extension route of the current patrol and inspection section. In some optional implementations, the shortest geometric distance from each of the municipal light pole asset points to the road centerline is calculated, and a virtual projection line that intersects and is perpendicular to each other is established, with the perpendicular contact point as the target position of the mapping action. Further, the length of the curved space traversed by smoothly traveling from the planned starting point of the road section along the road centerline to the perpendicular contact point is calculated, and this length is taken as the cumulative mileage. Finally, using the direction of the continuously increasing cumulative mileage as the reference reference direction, it is determined whether the basic orientation of the municipal light pole asset point is located in the left half area, right half area, or central median green belt area coinciding with the road centerline. Based on this, a judgment label summarizing the lateral distribution attribute of the light pole relative to the road extension centerline is generated, which is taken as the side position of the road.

[0022] S302, based on the accumulated mileage, the streetlights located at the same side position of the road are sorted to obtain the streetlight serial number, and the distance between adjacent streetlights is obtained.

[0023] Specifically, a layout calculation queue is constructed by extracting all groups of streetlights that share a specific lateral location label belonging to the same road, prohibiting the mixing of streetlights belonging to the left and right halves of the area into the same queue. In some optional implementations, within the established single-lateral layout calculation queue, streetlights are arranged in ascending order from the smallest to the largest value according to the numerical scale of their independently attached cumulative mileage. Further, based on the order of their arrangement, a sequence of positive integer values, continuously increasing by a single increment, is assigned to the streetlights in the queue from the first to the last, generating streetlight serial numbers. This generates the topological continuity logic for the sequential connection of the various municipal lighting streetlights along the road's lateral direction. Finally, the cumulative mileage of two adjacent streetlights in the same layout calculation queue is extracted, and the absolute difference between the two sets of cumulative mileage values ​​is calculated. This absolute difference is used as the spacing between adjacent streetlights, which numerically reflects the standard physical span when the current lateral municipal facilities of this road segment are periodically and repeatedly arranged.

[0024] S303, determine the direction of the road cross-section based on the road extension direction, and generate the extension direction of the lamp arm for each lamp position in combination with the side position of the road.

[0025] Specifically, the road extension direction is the tangential spatial vector derived from the perpendicular contact point on the road centerline along the direction of travel reference. In some optional implementations, the tangential spatial vector contained in the road extension direction is geometrically transformed by rotating it 90 degrees clockwise or counterclockwise in a two-dimensional horizontal coordinate system plane to obtain a transverse normal vector perpendicular to the current tangential travel trajectory. This transverse normal vector is defined as the direction of the road cross-section. Further, the orientation of the lamp arm structure attached to the street light has an objective physical law of covering the roadway surface inward. The pre-determined side position of the road is retrieved. If the determination label is the left half area, a subset of vectors pointing from the road cross-section direction to the road center is allocated as the extension determination orientation. For the double cantilever assembly scenario in the central median green belt area, the left and right extension determination orientations are assigned according to the left or right cantilever bracket attributes recorded in the ledger. Finally, the matched extension determination orientation is converted into a horizontal unit spatial vector with a length metric value of 1 to generate the lamp arm extension direction for each lamp position.

[0026] S304, using the lamp arm extension distance and lamp arm extension direction included in the municipal lamp pole asset point to perform vector combination processing to generate the lamp arm extension offset corresponding to each lamp position.

[0027] Specifically, the lamp arm extension distance of the municipal light pole asset point is either a scalar value representing the horizontal length of the corresponding cantilever metal component pre-recorded in the underlying facility data, or a scalar value representing the lateral span of the geometric center point of the light source's luminous component deviating from the vertical axis of the upright pole, calculated by actual measurement of similar standard installation model light positions. In some optional embodiments, the aforementioned lateral span scalar value is extracted, and it is mathematically multiplied with the lamp arm extension direction representing the entity's pointing attribute by a scalar value and a spatial unit vector, thus completing the vector combination processing action, thereby calculating a composite attribute three-dimensional vector that has both a predetermined length modulus and a specific horizontal three-dimensional spatial direction. The lamp arm extension offset includes components in both horizontal and vertical directions, fully reflecting the spatial positional relationship between the lamp head and the light pole. Further, the lamp arm extension offset corresponding to each light position is generated from this composite attribute three-dimensional vector. Finally, the lamp arm extension offset quantitatively reconstructs and characterizes the structured displacement deviation vector generated by the luminous component of the lighting system relative to the center of the underlying fixed pole support component across the horizontal plane on the coordinate plane.

[0028] S305, establish an asset index containing the street light serial number, the municipal light pole asset point and the theoretical location information of the light head.

[0029] Specifically, the underlying three-dimensional coordinates of the municipal light pole asset point representing the base are first converted into Cartesian coordinates. This is then combined with the vector offset difference included in the generated lamp arm extension offset, and superimposed with the vertical installation elevation value of the lamp head designed by the facility data record. Finally, the calculation result is converted back to the original three-dimensional geographic coordinates. In some optional implementations, the absolute expected coordinate point where the high-altitude light-emitting component should be suspended in the three-dimensional world is obtained through vector and coordinate summation in three-dimensional space, and this absolute expected coordinate point is used as the theoretical position information of the lamp head. Further, the street light serial number, the municipal light pole asset point representing the core base of the asset anchorage, and the theoretical position information of the lamp head representing the light-emitting execution end, calculated for the same physical entity lamp position, are data-bound and structured. Finally, a relational mapping database table structure accommodating the above three types of digital identifiers and spatial characteristic values ​​is established and persistently stored, generating an asset index. This asset index serves as a digital skeleton reflecting the mechanical extension structure of the light array, establishing a benchmark for handling visual image offset misalignment.

[0030] S401, retrieve the theoretical position information of the lamp head containing the absolute coordinates in three-dimensional space, and perform perspective projection spatial transformation in combination with the internal and external parameter matrix of the image acquisition device attached to the video frame at the corresponding time.

[0031] Specifically, perspective projection calculations are used to reduce the dimensionality of three-dimensional world space coordinates to two-dimensional plane pixel coordinates, obtaining the position of these two-dimensional plane pixel coordinates within the currently processed video frame. In some optional implementations, this position is used as the geometric center, and a circular viewing boundary is defined with a radius of 50 to 100 pixels. Further, this radius range is determined based on the drone's flight altitude and the lens's field of view, used to encompass the reasonable imaging size of normal municipal lighting fixtures in the image. All pixels falling within this boundary are independently cropped, separating the corresponding lamp head image area. Finally, by defining the area boundary, background pixel interference from road surface reflections, window light from surrounding buildings, and headlights of moving vehicles can be eliminated.

[0032] S402, the extracted lamp head image area is subjected to image grayscale transformation processing, so that each pixel in the area is mapped to a brightness scalar value between 0 and 255.

[0033] Specifically, the pixel brightness value adopts the industry-standard 8-bit quantization. The luminance threshold condition is a baseline value for determining if the pixel brightness scalar value is greater than or equal to 200. This value is determined based on the high-frequency luminous characteristics of municipal streetlights after they are turned on at night, and is used to separate the pixels of the light source itself from the pixels reflected by ambient light in the dark background. In some optional implementations, all pixels within the image area of ​​the lamp head are traversed, retaining the bright pixels that meet the baseline value and discarding the low grayscale pixels. Further, the retained bright pixels are extracted and connected component search aggregation operations are performed in a two-dimensional coordinate system. The connected components that meet the preset aggregation area conditions are extracted as a whole to form the pixel region. Finally, the center of the weighted average luminance coordinates of all pixels in the extracted region is calculated within the two-dimensional image pixel coordinate grid system. This represents the location of the highest optical energy concentration of the luminous physical entity, and the center of the weighted average coordinates is set as the center point of the bright spot.

[0034] S403, following the continuously increasing integer direction of the street light serial numbers recorded in the constructed sequence, all the extracted bright spot center points are retrieved sequentially from one side of the video image to the other in spatial extension order, and this set of center point coordinates arranged in spatial extension order is constructed and combined into a light head bright spot sequence.

[0035] Specifically, in the extraction and combination operations, the missing slots in the image plane spatial position discontinuity jump in the comparison sequence are identified, and it is determined that a certain intermediate sequence light position failed to generate the corresponding coordinate center using pixel calculation. In some optional implementations, the distance between adjacent streetlights recorded with the physical span period of normally lit light positions on the same side is retrieved. Combined with the current flight altitude of the UAV and the imaging characteristics of the camera, the two-dimensional pixel step value corresponding to this physical spatial distance under the current perspective depth field of view of the camera is calculated. Further, along the spatial extension direction of the line connecting the extracted center points in the two-dimensional image, starting from the image plane coordinates of the effective point before the discontinuity jump missing slot, one pixel step value is translated and superimposed. Finally, using the image plane coordinate interpolation extrapolation mathematical calculation method, the corresponding light position should be calculated purely based on the structural periodicity within the dark image space without any optical luminescence features.

[0036] S404: For municipal facilities registered in the digital ledger that did not emit light at the time of shooting due to physical damage to their own equipment or power outages, the system cannot obtain any luminous area that meets the optical brightness conditions within the corresponding pixel processing area.

[0037] Specifically, for the theoretical position information of the lamp head that does not match the center point of the bright spot, the two-dimensional plane pure numerical coordinate point extrapolated by image plane coordinate interpolation at the processing node is directly read to extract the position that should emit light. In some optional embodiments, the coordinate values ​​of the inferred point with specific two-dimensional image plane pixel row and column index numerical attributes are directly assigned and defined as the image position of the lamp head with missing light. Further, by extracting the continuation rule to force the filling of the dark gap coordinates caused by physical damage on the two-dimensional image grid, the damaged fault components that disappear in visual presentation features are obtained as digital positioning coordinate anchor points with visual spatial arrangement feature attributes. Finally, the necessary image viewpoint plane reference coordinate source is sent to the inverse spatial position coordinate projection compensation calculation stage.

[0038] S405, when the UAV system is performing a patrol mission, it is in a state of continuous translational motion. The same fixed pole component that is in a fault-off state appears to be in a state of relative motion with displacement in multiple single still images of the video stream.

[0039] Specifically, for the same light position, the image position of the missing light head, calculated sequentially under the current processing frame number sequence number and subsequent multiple adjacent frame number sequences, is retrieved sequentially. In some optional embodiments, based on the spatial proximity distance judgment rule under different frame plane pixel coordinate system grids, combined with the pixel motion optical flow offset scale converted from the UAV flight velocity vector, it is determined that the inferred points extracted from adjacent video frames all belong to the same physical target object from different shooting time perspectives. Further, the values ​​of multiple independent two-dimensional plane pixel coordinate points belonging to the same faulty target object and appearing across multiple frames at different times are sequentially connected in the order of their occurrence during recording. Finally, inter-frame correlation tracking is performed to generate a missing light image trajectory composed of a set of discrete inferred points connected in the time dimension. This trajectory set can accommodate multiple sets of visual judgment spatial coordinate results under different tilted observation angles.

[0040] S501, retrieve the UAV location information and camera attitude information corresponding to the timestamp information.

[0041] Specifically, the UAV location information is represented as three-dimensional spatial point coordinates containing longitude, latitude, and altitude coordinates. Physically, it serves as the spatial origin of the image perspective projection calculation model. First, the UAV location information is converted into Cartesian coordinates. Then, pre-calibrated internal reference matrix parameters are extracted from the image acquisition device. Using the inverse mapping rules of these internal reference matrix parameters, combined with the three-dimensional rotation matrix data carried by the camera attitude information, the position of the missing lamp head image, which has two-dimensional attributes, is transformed into a direction vector extending outward from the aforementioned spatial origin along a straight line in three-dimensional space. This direction vector is defined as the camera's line-of-sight direction. In some optional embodiments, the lamp head installation height plane is a virtual horizontal cross-sectional slice with a specific altitude obtained from the lamp post hardware design drawings recorded in the municipal facilities ledger. All altitude data is uniformly converted to the elevation benchmark used in the municipal asset ledger to ensure consistent height calculations. A logic for solving the intersection coordinates of the three-dimensional spatial ray and the horizontal cross-sectional slice is established. Further, specifically, this includes: calculating the arithmetic difference between the altitude value carried by the plane where the lamp head is installed and the vertical height component value of the UAV's position information to obtain an altitude difference value; extracting the vector component value of the camera's line of sight along the vertical coordinate axis; dividing the altitude difference value by the vector component value along the vertical coordinate axis to calculate the scaling scalar of the ray's spatial extension; multiplying the entire three-dimensional vector contained in the camera's line of sight by the scaling scalar to obtain a spatial displacement compensation vector; then performing vector addition on the spatial displacement compensation vector and the three-dimensional spatial coordinates of the UAV's position information to calculate the three-dimensional intersection coordinates of the line of sight ray penetrating the height section; and converting these three-dimensional intersection coordinates back to the original three-dimensional geographic coordinates as the spatial position where the lamp head should emit light. Finally, this calculation logic prohibits the direct projection of the line of sight ray onto a horizontal ground section with a height of 0, thereby avoiding interference from false road intersections caused by reflections from lane puddles or moving vehicle light sources.

[0042] S502, the center of the light-emitting component of the cantilevered municipal street light is naturally separated from its upright load-bearing pole base in space. After obtaining the light-emitting spatial position of the lamp head, which reflects the suspended and void position, it is necessary to use structural parameters to map it back to the location where the physical asset is located.

[0043] Specifically, the three-dimensional absolute coordinates of the spatial position where the lamp head should emit light are retrieved, and the outward offset of the lamp arm belonging to the same lamp column number is simultaneously adjusted. The outward offset of the lamp arm carries the lateral three-dimensional length and direction attributes extending from the center of the pole to the side of the lane. In some optional embodiments, the spatial position where the lamp head should emit light is first converted into Cartesian coordinates, and then the three-dimensional vector value contained in the outward offset of the lamp arm is subtracted to complete the coordinate inverse compensation process. Finally, the calculation result is converted back to the original three-dimensional geographic coordinates. Further, this vector subtraction action is the coordinate inverse compensation process, which simulates the spatial crossing process of climbing back along the solid cantilever metal pole to the top of the main upright pole in a virtual three-dimensional coordinate system. Finally, the three-dimensional digital coordinate points with latitude, longitude and elevation attributes derived from this reverse backtracking are generated and extracted as single-frame analytical lamp pole coordinates, which are used to anchor and obtain the orientation of the solid load-bearing base from the image perspective at that moment, avoiding the image positioning deviation caused by the separation of the emitting point and the base.

[0044] When calculating target coordinates for a single image, the positioning accuracy of the S503 is affected by the shooting angle of the airborne camera and the image quality.

[0045] Specifically, the camera's line-of-sight vector and the lamp arm's outward extension vector in a three-dimensional coordinate system are retrieved, and the degree between these two sets of spatial straight line vectors is calculated. The corresponding cosine trigonometric function value is then calculated from this degree. In some optional implementations, since the closer the camera is to a vertical overhead view, the smaller the error in calculating the lateral cantilever structure, the larger value is selected as the numerator for calculating the weight evaluation value by comparing the cosine trigonometric function value with the number 0. Further, the grayscale gradient change feature data of adjacent pixels in the current video frame caused by the aircraft's unstable attitude is extracted and converted into a normalized blur score between 0 and 1. The proportion of pixels exceeding the upper limit of the photosensitive threshold due to local overexposure of strong light in the frame is counted as the pixel saturation overflow value. The normalized blur score and the pixel saturation overflow value are multiplied by preset weight coefficients and then added together to obtain a comprehensive error variable. The sum of the number 1 and this comprehensive error variable is used as the denominator for calculating the weight evaluation value. Finally, by dividing the obtained numerator value by the denominator value, the quotient value is generated and defined as the coordinate evaluation weight. This value can assign a high degree of confidence to video images with low blur and reasonable line-of-sight angles.

[0046] S504, for the same lamp position, since the image acquisition device is in a uniform motion recording state, the system will calculate the coordinates of multiple single-frame parsed lamp posts belonging to the same entity object from the continuous video data stream.

[0047] Specifically, the three-dimensional coordinate values ​​of all single-frame analyzed lamppost coordinates within the associated time period of the entity object are extracted, along with the coordinate evaluation weights independently corresponding to each frame. All coordinate evaluation weights need to be normalized to ensure a reasonable contribution ratio for each frame during multi-frame fusion. In some optional implementations, each single-frame analyzed lamppost coordinate is first converted to Cartesian coordinates, and then independently multiplied by its corresponding coordinate evaluation weight to obtain multiple sets of weighted elevation and latitude / longitude coordinate components. Further, the weighted coordinate components of all frames are cumulatively added along their dimensions to obtain a spatial weighted sum vector value. The values ​​of all coordinate evaluation weights participating in this fusion operation are arithmetically cumulatively added to obtain the denominator of the weighted sum representing the overall credibility. Finally, the values ​​of the spatial weighted sum vector in each of the three spatial coordinate dimensions are divided by the denominator of the weighted sum, and the resulting three-dimensional absolute geographic coordinates with averaging convergence characteristics are the geographic coordinates of the faulty light pole. This multi-frame spatial location fusion processing step weakens and suppresses the accidental spatial jump jitter caused by sudden airflow disturbances or local light source obstruction in a single frame image, thereby generating geographic location correction coordinate data that can be smoothly connected to the external business management dispatch system.

[0048] S601, retrieve the street light serial number, and initiate a data retrieval within the constructed asset index.

[0049] Specifically, the search scope is limited to segmented digital intervals within the same road entity, the same lateral extension, and the same structural type. Physical structural isolation is used to prevent matching actions from crossing intersections or penetrating the central median strip. In some optional implementations, within this search interval, a search logic is executed based on numerical comparison to search for index entries equal to the integer value of the streetlight serial number. Further, after a successful match, the specific hardware device encoding text sequence attached to the entry is extracted to generate the asset number. The asset number includes the hardware code of the upright column, the hardware code of the cantilever component, and the sequential number of the last light head to distinguish the left and right light head branches on the same column. Finally, the original latitude and longitude digital positioning point of the base corresponding to the hardware device, registered in the static data database, is synchronously read and defined as the corresponding ledger-related coordinates.

[0050] S602, retrieve the geographical coordinates of the faulty light pole that carry the spatial dimension reverse compensation correction results and the extracted ledger-related coordinates.

[0051] Specifically, the calculation logic is derived using spherical geometric arc length. Physical ground distances are converted for two sets of three-dimensional spatial coordinate points with longitude and latitude attributes, resulting in a scalar value of the distance measured in meters, which is recorded as the spatial distance. The spherical distance calculation uses the industry-standard Earth's average radius parameter to ensure accuracy. In some optional implementations, the distance between adjacent streetlights is retrieved, and one-third of this physical distance scalar value is calculated. Furthermore, one-third is chosen as the calculation coefficient because this proportion effectively isolates the risk of unauthorized assignment by the system positioning to jump to the preceding or following streetlight entity at a mathematical level. This generates a safety boundary limit that can accommodate floating-point errors in high-altitude calculations but rejects misaligned binding logic of adjacent facilities. This safety boundary limit is set as the distance tolerance range. The distance tolerance range is dynamically adjusted according to the actual interval between adjacent streetlights to ensure that calculation errors are accommodated without affecting adjacent light poles. Finally, the system's underlying comparator compares the spatial distance value with the distance tolerance range value to verify whether the spatial distance scalar is less than or equal to the upper limit threshold set for the tolerance range.

[0052] S603 receives the numerical logic judgment conclusion fed back from the comparison and verification process.

[0053] Specifically, if the determination branch shows that the relationship between the two conforms to the defined logic, that is, the spatial distance is indeed within the distance tolerance range, it confirms that the coordinates calculated by back-derived from the high-altitude image are completely consistent with the geographical affiliation of the lighting facility in the corresponding sequence in the physical world, and there is no misalignment of the facilities. In some optional implementations, at this time, the system generates a judgment text feature character to describe the loss of nighttime luminous efficiency of the equipment, and uses it as the result of the brightness deficiency identification. Further, the brightness deficiency identification result is updated and bound to the dynamic attribute monitoring list under the corresponding asset number via a memory write operation. Finally, by performing the above precise matching and attribute writing operations, the unique physical equipment that has experienced physical power failure is identified from the spatial logic level in the massive hardware library of the complex municipal road network, and the equipment is given a directional identity for manual investigation and intervention, and is designated as the only maintenance object.

[0054] S604, a status overwrite instruction is sent to the data node location of the asset number whose identity has been locked, changing the original attribute field code representing normal power supply operation to an abnormal warning code numerical value representing light source failure, and triggering the street light off status indicator through the code update action.

[0055] Specifically, the original still viewpoint image containing locally dark areas, captured during the early pixel brightness threshold logic interception, is retrieved and used as the evidence image. In some optional implementations, multiple sets of calculated two-dimensional pixel coordinates for missing dark areas, arranged sequentially and connected in chronological order, are extracted as the trajectory of the missing brightness image. Further, the absolute time node number string of the moment when the aircraft passes over the target hardware and performs photosensitive recording is captured to generate the video acquisition time. Finally, the underlying data serialization assembly logic is invoked to merge and encapsulate the street light extinguishing status identifier, the evidence image with visual evidentiary effect, the missing brightness image trajectory, and the video acquisition time carrying a time stamp, assembling them into a data transaction entity package with a single query primary key, generating the missing brightness event, and providing digital evidence containing independent object attribution and a complete chain of factual evidence for entering the administrative assignment and scheduling stage.

[0056] S701 triggers the business data parsing interface within the platform system to perform layer-by-layer reading operations on the data packets corresponding to the loaded missing light event.

[0057] Specifically, an asset number representing the unique identity of the faulty entity is identified and separated from the attribute set of the data packet. In some optional implementations, the geographical coordinates of the faulty light pole, carrying reverse anchoring location information, are extracted simultaneously. These coordinates will serve as the core spatial positioning benchmark guiding maintenance personnel to the maintenance location. Furthermore, the street light extinguishing status indicator, representing the conclusion that the equipment has lost its luminous function, is captured in parallel. Finally, the extracted three types of digital parameters are temporarily stored in the dynamic memory execution area of ​​the platform system as the basic data material for assembling the dispatch business entity.

[0058] S702, the platform system's default data format is a standardized latitude and longitude spatial expression system or projected coordinate reference specification that can be accepted and supported by the underlying architecture of municipal lighting operation and maintenance management business flow, which is used to eliminate spatial data reading and writing barriers between heterogeneous subsystems.

[0059] Specifically, the system retrieves the temporarily stored geographic coordinates of faulty light poles and uses a spatial coordinate system transformation function library to convert their coordinate reference and expression into parameter forms required by a preset data format. In some optional implementations, the geographic coordinate values ​​after format standardization are loaded into the corresponding data key-value pairs in the newly generated business form, specifically assigned to a storage space named "Maintenance On-Site Coordinates". Furthermore, the light pole coordinates derived from the actual image are set as the on-site reference, avoiding the location guidance failure problem that might be caused by calling the original coordinates from the ledger. Finally, this allows field personnel to accurately reach the location of the pole containing the asset number based on this field.

[0060] S703 extracts the equipment ownership identifier carried by the asset number and performs a structured query matching in the static maintenance configuration database associated with the municipal asset ledger.

[0061] Specifically, the responsibility area code corresponding to the device under the administrative region slicing and grid-based management plan is retrieved and used as the maintenance grid attribute. In some optional implementations, the management mapping table structure corresponding to the maintenance grid attribute is invoked to query and extract the organizational structure identifier of the grassroots field operation team or personnel responsible for the daily inspection and emergency repair of the grid, thus obtaining the corresponding maintenance team. Further, the network addressing address or account identifier of the work allocation smart communication device of the obtained maintenance team is set as the data push terminal node for executing the maintenance task instruction, and designated as the receiving object of the mobile terminal. Finally, in this conversion link, the management unit division of labor structure built into the asset ledger is directly used to determine the recipient, replacing the action of temporarily or randomly generating the maintenance personnel list by the visual discrimination model, ensuring the business compliance of the assignment level.

[0062] S704 wakes up the platform system storage module and retrieves the visual recording file of the image dimension associated with the brightness loss event.

[0063] Specifically, evidence images representing localized dark areas are selected, along with the trajectory of the missing light image generated by connecting spatial coordinates from different shooting angles. In some optional implementations, the platform system's built-in data compression and format packaging algorithms are used to integrate the two types of graphical media files into a file package with a unified file extension and a fixed index header format, generating work order attachment data. Further, this work order attachment data is attached to the side of the final dispatched data stream file. Finally, it is opened and viewed by on-site personnel arriving at the actual road section for on-site comparison and verification of the specific lamppost with the malfunction and the physical location of the corresponding cantilever lamp head on that post experiencing the missing light.

[0064] S705 calls the form assembly service layer interface of the business system to perform a structured splicing operation on all maintenance dispatch business fields, including attributes such as asset number and maintenance arrival coordinates, to construct the task body information block.

[0065] Specifically, a summary text with attributes indicating the alarm level and event type is concatenated and written into the header to form the task title. In some optional implementations, previously packaged work order attachments are embedded at the end of the task body information block as data pointers or object links. Further, the task title, task body, and work order attachments are subjected to overall data aggregation operations to encapsulate and generate a mobile terminal message to be dispatched that is suitable for cellular wireless network transmission protocols and can be recognized and displayed as a pop-up window by the mobile communication terminal operating system. Finally, this achieves the hierarchical transformation of visual anomaly alarm signals into interactive administrative and maintenance management data processing objects.

[0066] S801, wake up and trigger the pre-set dispatch interaction interface inside the platform system.

[0067] Specifically, the dispatching interaction interface is an application programming interface data gateway opened within the service layer of the municipal lighting business system, used to listen for and receive maintenance dispatch request data messages transmitted from the identification module. In some optional implementations, for the missing light event whose asset identity has been analyzed, identified, and locked, the system uses a built-in serial number generation algorithm or hash calculation rule to generate a unique identifier sequence composed of numbers, which serves as the work order number. The work order number has a globally unique characteristic and is used to independently mark and track the entire lifecycle processing progress of this maintenance dispatch task in the complex business database. Further, the underlying communication protocol stack is invoked to bind the work order number with the asset number, which represents the unique identity of the damaged physical equipment, into a digital structure. Simultaneously, the system retrieves the maintenance arrival coordinate field, which has been standardized and used to guide field workers to accurately arrive at the maintenance physical location, as well as the work order attachment data constructed by merging and packaging the on-site viewpoint image and the calculated dark area trajectory from multiple frames. Finally, the above four types of parameters are loaded and written into the data payload information segment of the network request message, and a submission instruction is executed to the dispatch interaction interface to achieve cross-level data transformation from machine vision early warning data to administrative maintenance business dispatch instructions.

[0068] S802, Load and render the system monitoring page located on the display screen in the dispatch center.

[0069] Specifically, the system monitoring page is a graphical user interface that integrates electronic geographic information system (GIS) layer components. It is used to macroscopically present the operational health status of all underlying physical facilities of the municipal lighting network within the jurisdiction, as well as the spatial distribution of current dispatch tasks. In some optional implementations, the electronic map layer rendering engine on the system monitoring page reads the absolute longitude and latitude values ​​associated with the incoming geographic coordinates of the faulty light pole. This set of longitude and latitude positioning values ​​is set as the reference coordinates for map graphic coloring, using this virtual three-dimensional spatial point as the core focus of visual presentation. Further, the graphics library rendering function is called to overlay and draw highlighted color blocks, flashing icons, or customized spatial pins, or other graphical visual components, on the map layer pixel grid area centered on the reference coordinates, thereby generating the map positioning marker. Finally, through this rendering process, the underlying abstract elevation and longitude / latitude numerical sets are transformed into intuitive and eye-catching map positioning markers, explicitly presenting the location of the faulty facility to backend administrators within the global road network map view, and intuitively indicating the dispatch location.

[0070] S803, establish a wireless data transmission link between the platform system server and the mobile terminal receiving object.

[0071] Specifically, the network is a wide-area data interaction transmission channel built based on fourth-generation mobile communication technology, fifth-generation mobile communication technology, or wireless local area network communication protocols. The mobile terminal receiving object is a portable intelligent handheld communication device carried by the grassroots field maintenance team, as determined by the preliminary parsing and matching. In some optional implementations, the mobile terminal message information to be dispatched, which integrates the task execution content and on-site graphic record data, is re-encoded into a data stream file packet suitable for long-distance encrypted wireless transmission, and pushed to the target device's operation terminal task management application receiving end through the established network node. Further, in the attribute configuration header of the pushed data stream file packet, the message title used to trigger the display of the handheld communication device's screen notification banner pop-up is uniformly configured and filled with a standardized text character sequence: a work order for repairing a street light with a missing lamp. Finally, after the mobile terminal receives and parses the file packet, its device screen will first pop up an assignment prompt with the standardized text character sequence, prompting front-line maintenance personnel to check the embedded geographical coordinates of the faulty light pole and the complete set of evidence. By directly sending an assignment message carrying the reverse anchor coordinates to the task execution end, an automated workflow system is achieved from the discovery of missing light sources by aerial drone video sequence acquisition to the maintenance team receiving precise navigation and work order assignment.

[0072] like Figure 1As shown in this embodiment, the drone flies along the road and collects continuous video streams of streetlights on both sides of the road. The platform system constructs a streetlight arrangement sequence based on the municipal light pole asset points and the road centerline, and marks the pole position, serial number, and extension direction of the lamp arm of each light pole accordingly. For light heads in a normal lighting state, they form a continuous bright spot sequence in the image; when a light head corresponding to a certain serial number does not form a bright spot, the system determines the image position of the missing light head based on the arrangement relationship of adjacent bright spots. Subsequently, the system does not directly use the image position of the missing light head as the repair arrival location, but first maps it to the light head installation height plane to obtain the spatial position where the light head should emit light, and then performs reverse compensation based on the extension offset of the lamp arm corresponding to the light position, so that the positioning point moves back from the position of the suspended light head to the actual light pole asset point position, thereby obtaining the geographical coordinates of the faulty light pole. The platform system further associates the geographical coordinates of the faulty light pole with the asset data in the street light arrangement sequence to identify a unique repair target, and generates a repair dispatch order for the missing street light containing the asset number, geographical coordinates of the faulty light pole, and evidence images, which is then sent to the corresponding mobile terminal recipient.

[0073] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A municipal street light dimming warning and dispatch system based on UAV visual analysis, applied to an operation and maintenance architecture that includes image acquisition equipment and a platform system, characterized in that... include: Control the drone equipped with the image acquisition device to fly along the road along a preset route and acquire a continuous video stream; The street light arrangement sequence is constructed based on the municipal light pole asset points and the road centerline, and the arm extension offset of each light position is generated. Extract the bright spot sequence of the lamp head from the continuous video image stream, and locate and extract the image position of the missing lamp head in the street light arrangement sequence; The image position of the missing lamp head is mapped to the lamp head installation height plane, and the position offset is compensated by the lamp arm extension offset. The geographical coordinates of the faulty lamp pole are anchored to avoid coordinate mapping errors. The geographical coordinates of the faulty light pole are associated with the asset data in the street light arrangement sequence to generate a missing light event and identify the corresponding unique repair object; Based on the aforementioned brightness loss event, a repair dispatch order business field is generated in the platform system, and the mobile terminal receiving object is specified; The maintenance dispatch service field is used to send a maintenance dispatch order for the faulty street light containing the geographical coordinates of the faulty light pole to the receiving object of the mobile terminal.

2. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 1, characterized in that, Controlling a drone equipped with the image acquisition device to fly along a road along a preset route and acquire a continuous video stream includes: During the designated operating period when the streetlights are set to be lit, the drone equipped with the image acquisition device is controlled to fly along the designated inspection flight path to obtain scene images; The image acquisition device is used to acquire video of the road lighting scene at a fixed frame rate to form video frames, and the camera exposure parameters are fixed before entering the road; For each video frame, the corresponding UAV position information, camera attitude information, gimbal attitude information, and timestamp information are recorded synchronously. Establish a data association between the video frame, the flight mileage along the road, and the street light arrangement sequence.

3. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 2, characterized in that, Based on the municipal light pole asset points and the road centerline, a street light arrangement sequence is constructed, and the arm extension offset of each light position is generated, including: Map each of the municipal light pole assets to the center line of the road to obtain the corresponding cumulative mileage and the side location of the road; Based on the accumulated mileage, the streetlights located on the same side of the road are sorted to obtain the streetlight serial number, and the spacing between adjacent streetlights is obtained. The direction of the road cross-section is determined based on the road's extension direction, and the extension direction of the lamp arm for each lamp position is generated by combining the side position of the road. By performing vector combination processing on the lamp arm extension distance and the lamp arm extension direction included in the municipal lamp pole asset point, the lamp arm extension offset corresponding to each lamp position is generated; Establish an asset index that includes the street light serial number, the municipal light pole asset point, and the theoretical location information of the light head.

4. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 3, characterized in that, Extracting the bright spot sequence of lamp heads from the continuous video image stream, and locating and extracting the image positions of missing lamp heads in the street light arrangement sequence, includes: The theoretical position information of the lamp head is mapped to the currently processed video frame to divide the corresponding lamp head image region; Extract the pixel region that meets the light emission threshold condition within the image area of ​​the lamp head to set the center point of the bright spot; Extract the lamp head bright spot sequence formed by the center point of the bright spot along the increasing direction of the street lamp number, and combine it with the distance between the adjacent street lamps corresponding to the bright spots in the lit state to calculate the light emission position of the corresponding lamp position. For the theoretical position information of the lamp head that does not match the center point of the bright spot, the position that should emit light is extracted as the position of the image of the lamp head with missing light. For the same lamp position, inter-frame correlation tracking is performed on the image positions of the lamp head that appears across multiple frames to generate the image trajectory of the lamp head that is missing a lamp head.

5. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 4, characterized in that, Mapping the image location of the faulty lamp head to the lamp head installation height plane, and using the lamp arm extension offset for position offset compensation processing, anchoring the obtained geographical coordinates of the faulty lamp pole to avoid coordinate mapping errors, including: Based on the UAV location information and camera attitude information corresponding to the timestamp information, the camera shooting line of sight is constructed, and the position of the image of the missing lamp head is projected along the camera shooting line of sight onto the lamp head installation height plane to obtain the spatial position where the lamp head should emit light. The coordinate inverse compensation process is performed on the spatial position of the lamp head to be illuminated by the lamp arm extension offset to obtain the single-frame analytical lamp post coordinates; Based on the spatial angle between the camera's shooting line of sight and the lamp arm's outward extension direction, and combined with image motion blur parameters and pixel saturation overflow values, coordinate evaluation weights are constructed. For multiple single-frame parsed lamp post coordinates associated with the same lamp position, multi-frame spatial location fusion processing is performed using the coordinate evaluation weights to generate the geographical coordinates of the faulty lamp post to achieve geographical location correction.

6. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 5, characterized in that, The geographical coordinates of the faulty light pole are associated with asset data in the street light arrangement sequence to generate a missing light event, and the corresponding unique repair object is identified, including: The asset index is traversed to find the asset number that matches the street light serial number and the corresponding ledger-related coordinates; Calculate the spatial distance between the geographical coordinates of the faulty light pole and the coordinates associated with the ledger, and verify whether the spatial distance is within the distance tolerance range set based on the distance between adjacent streetlights; If it is within the distance tolerance range, the missing light identification result will be entered into the corresponding asset number to designate it as the unique repair object; The street light is triggered to be off based on the asset number, and the evidence image extracted from the video frame, the trajectory of the missing light image, and the video acquisition time are packaged together into the missing light event.

7. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 6, characterized in that, Based on the aforementioned brightness loss event, a repair dispatch order field is generated in the platform system, specifying the mobile terminal receiving object, including: Extract the asset number, the geographical coordinates of the faulty light pole, and the street light off status identifier from the built-in information of the missing light event; The platform system performs a format conversion on the geographical coordinates of the faulty light pole according to the preset data format, and enters the on-site maintenance coordinates field. The maintenance grid attribute to which the asset number belongs is parsed and mapped to the corresponding maintenance team, which is then designated as the receiving object of the mobile terminal. The evidence image and the trajectory of the image with missing brightness are constructed as work order attachment data; The repair dispatch business fields and the work order attachments are aggregated to generate a mobile terminal message to be dispatched, which includes a task title and a task body.

8. The municipal street light dimming warning and dispatch system based on UAV visual analysis according to claim 7, characterized in that, Using the aforementioned maintenance dispatch service field, a maintenance dispatch order for a street light with missing lights, containing the geographical coordinates of the faulty light pole, is sent to the mobile terminal receiving object, including: By calling the dispatch interaction interface of the platform system, the work order number generated for the missing light event, the asset number, the maintenance arrival coordinates field, and the work order attachment information are submitted. On the system monitoring page, a map location marker is generated using the geographical coordinates of the faulty light pole as the reference coordinates to indicate the dispatch location. The message information to be dispatched from the mobile terminal is sent to the receiving object of the mobile terminal via the network, and the message title is configured as the work order for the repair of the street light with missing light.

Images