New energy charging station inspection management system based on multi-view feature fusion

Abstract

This invention relates to the field of inspection and maintenance management technology, and discloses a new energy charging station inspection and management system based on multi-view feature fusion, including: firstly, establishing a station coordinate system and registering charging stations as workstation units, and locking the mapping relationship between camera-generated images and spatial coordinates; real-time acquisition of multiple video streams to generate an image set arranged in time slices; decomposing fixed components and dynamic components within the workstation unit, and extracting the center line of the charging gun containing the fixed end and the vehicle-side charging end from the dynamic component; calculating the fusion weight of the camera according to the cross-workstation cycle topology attribute to aggregate panoramic pixels and target coordinates; based on the panoramic pixels, using the path constraints of the workstation unit boundary and the center line of the charging gun to splice seams to generate a dynamic panoramic image; identifying anomalies in the panoramic image, binding them to the corresponding workstation unit and target coordinates to generate jump instructions, and finally outputting the panoramic image and instructions to the terminal.

Description

Technical Field

[0001] This invention relates to the field of inspection and maintenance management technology, and more specifically, to a new energy charging station inspection and management system based on multi-view feature fusion. Background Technology

[0002] Currently, with the rapid growth in the number of new energy vehicles, the centralized operation and maintenance and inspection management of large-scale new energy charging stations has become a pain point in the industry. In order to reduce the cost of manual inspection and improve the operational efficiency of the stations, managers usually rely on existing conventional monitoring equipment within the stations for remote inspection and work order scheduling. However, existing inspection management systems face challenges of scheduling and positioning errors and low management efficiency in the commercial operation scenario of charging stations.

[0003] First, the refined operation of charging stations relies on the accurate management of charging workstations. However, the stations are filled with a large number of rigid and repetitive infrastructures that look similar (such as equidistant canopy columns and identical anti-collision posts). When the existing operation and maintenance system attempts to conduct a global status inspection, the spatial allocation in the management view is easily confused due to the interference of the repetitive structures. This will prevent the back-end management system from accurately binding the abnormal event to the actual workstation when it detects anomalies (such as vehicles illegally parked or occupying space) and attempts to issue inspection instructions.

[0004] Secondly, with the widespread use of high-power liquid-cooled supercharging equipment, heavy-duty charging gun cables often sag, bend, or lie flat on the ground, and their low-level field of view is frequently obstructed by vehicle bodies. When maintenance anomalies such as cables dragging on the ground or not being returned to their designated positions occur, conventional inspection systems are prone to incorrectly assigning the cable status to adjacent parking spaces. This causes the back-end dispatch system to output incorrect work order location instructions when receiving alarms and guiding maintenance personnel to troubleshoot, which not only increases the ineffective troubleshooting time for on-site personnel but also prolongs the downtime of faulty parking spaces, significantly reducing the asset turnover rate of the depot.

[0005] To address the business needs for precise scheduling and positioning, adding mobile guide rails, thermal imagers, or automated inspection robots to the management solution would not only significantly increase the overall operation and renovation costs of the charging station, but these hardware components could also easily cause physical interference and obstruction with routine commercial operations such as vehicle traffic scheduling and charging cable recovery devices. Therefore, there is an urgent need for a new energy charging station inspection and management system that can overcome interference from complex environments and accurately bind and automatically guide abnormal on-site conditions with underlying workstation management units, thereby effectively improving the accuracy of back-end operation and maintenance scheduling and the overall business management efficiency of the charging station. Summary of the Invention

[0006] This invention provides a new energy charging station inspection and management system based on multi-view feature fusion, which solves the technical problems mentioned in the background art.

[0007] This invention provides a new energy charging station inspection and management system based on multi-view feature fusion, applied to new energy charging stations. The new energy charging station is equipped with charging stations, inspection terminals, and cameras, including:

[0008] The inspection monitoring and management module is used to identify inspection anomalies in the dynamic panoramic image in order to carry out the inspection and maintenance management of the new energy charging station, including an anomaly binding unit and a scheduling output unit. The exception binding unit is used to bind the exception target to the corresponding workstation unit and target coordinates, and generate a jump instruction; The scheduling output unit is used to output the dynamic panoramic image and the jump command to the inspection terminal to lock the abnormal object.

[0009] Beneficial effects include: 1. This invention enables efficient and centralized management of station operations by simply reusing existing traditional monitoring equipment. It avoids the high capital investment and operational disruptions associated with introducing expensive automated inspection hardware such as moving guide rails and robots.

[0010] 2. By deeply coupling the charging cable trajectory with the physical spatial topology of the charging station, this invention effectively overcomes the management problems caused by repetitive infrastructure (such as similar anti-collision posts and columns) and flexible cable obstruction within the site. The system can accurately anchor detected maintenance anomalies (such as unauthorized occupation, facility damage, and cable sagging) to the actual physical business unit (workstation unit), thereby avoiding misjudgment and mismanagement of adjacent and close stations.

[0011] 3. This invention can directly output a digital panoramic inspection view with global business constraints to a remote operation and maintenance scheduling system, and automatically generate and issue jump instructions for abnormal points carrying physical coordinates and observation orientation. This not only shortens the event confirmation cycle for back-end management personnel, but also provides business instruction guidance for front-line operation and maintenance personnel to conduct on-site troubleshooting, effectively improving the remote supervision efficiency and fault response speed of large new energy charging stations in complex scenarios. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the inspection management system mechanism based on multi-view feature fusion of the present invention; Figure 2 This is a schematic diagram of the topology fusion and anomaly attribution mechanism of the present invention. Detailed Implementation

[0013] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0014] Figure 1 The process sequence is shown, including the site coordinate system and workstation unit, camera acquisition, local image and dynamic structure extraction, multi-view feature fusion, dynamic panoramic image stitching, anomaly binding and terminal redirection. Figure 2 The correspondence between cross-station features, fusion weights, splicing seams, anomalous targets, target coordinates, and observation orientation is shown.

[0015] Example 1: like Figure 1 As shown, a new energy charging station inspection and management system based on multi-view feature fusion is applied to a new energy charging station, which is equipped with charging stations, inspection terminals, and cameras. The system includes an inspection monitoring and management module, which further includes an anomaly binding unit and a scheduling output unit.

[0016] The inspection monitoring and management module is used to establish a site coordinate system, generate an image-to-physical coordinate mapping relationship, acquire multiple video streams in real time, generate an image set arranged by time slices, decompose fixed and dynamic structures within the workstation unit, extract the center line of the gun line, calculate the fusion weight of the camera based on cross-workstation features, aggregate panoramic pixels, determine target coordinates, generate a dynamic panoramic image, and identify inspection anomalies. The anomaly binding unit is used to bind abnormal targets to the corresponding workstation unit and target coordinates and generate jump instructions. The scheduling output unit is used to output a dynamic panoramic image and jump instructions to the inspection terminal to lock the abnormal target.

[0017] In one implementation, the inspection monitoring and management module, the anomaly binding unit, and the scheduling output unit can be configured as different functional units within the same processing flow, or as data processing units capable of calling each other. Regardless of the configuration method, the dynamic panoramic view output by the inspection monitoring and management module, the jump instructions output by the anomaly binding unit, and the content output by the scheduling output unit to the inspection terminal remain consistent.

[0018] In one implementation, the inspection monitoring and management module, the anomaly binding unit, and the scheduling output unit are all deployed on a local edge server, employing a centralized deployment architecture, with data processing completed within the local edge server. After completing image acquisition, fixed and dynamic structure decomposition, gun line centerline extraction, fusion weight calculation, dynamic panoramic image generation, and inspection anomaly identification, the inspection monitoring and management module outputs the dynamic panoramic image and anomaly target information to the anomaly binding unit. After completing workstation unit binding and jump instruction generation, the anomaly binding unit outputs the jump instruction to the scheduling output unit. The scheduling output unit outputs the dynamic panoramic image and jump instruction to the inspection terminal.

[0019] In one implementation, the index table is stored in the form of a structured data table and includes a camera index table, a workstation unit index table, a structural anchor point index table, and an anomaly target index table. The camera index table stores camera coordinates, equipment parameters, the identifier of the bound workstation unit, and the mapping table storage path; the workstation unit index table stores the identifier of the workstation unit, physical boundary coordinates, a list of fixed structures, and the background template storage path; the structural anchor point index table stores the identifier of the structural anchor point, the identifier of the corresponding workstation unit, pixel position, and physical point; the anomaly target index table stores the identifier of the anomaly target, the identifier of the corresponding workstation unit, the anomaly type, the target coordinates, and the observation orientation.

[0020] Example 2: Reference Figure 1 The station coordinate system and workstation unit section establish a station coordinate system based on the physical structure of the new energy charging station. This physical structure is one whose spatial position is fixed during inspection. The station coordinate system is used to uniformly represent charging workstations, workstation units, cameras, fixed structures, dynamic structures, gun centerlines, target coordinates, and camera coordinates.

[0021] In one implementation, the station coordinate system uses the ground surface of the charging station as the reference plane, selects the ground corner point at the entrance of the charging station as the origin, and uses the horizontal direction parallel to the long side of the main building of the charging station as the coordinate system. The positive direction of the axis, perpendicular to The horizontal direction of the axis is The positive direction of the axis is defined as the direction perpendicular to the ground and upwards. Positive axis direction. The panoramic coordinate system is the ground plane projection coordinate system of the station coordinate system, retaining... shaft and The axis components, panoramic coordinates, and ground coordinates in the station coordinate system correspond one-to-one.

[0022] In one implementation, the local coordinate system takes the lower left corner of the physical boundary of the corresponding workstation unit as its origin, and the direction parallel to the long side of the workstation as its coordinate system. The positive direction of the axis is the direction parallel to the shorter side of the parking space. In the positive direction of the axis, the transformation between the local coordinate system and the station coordinate system is achieved through a preset translation and rotation matrix.

[0023]

[0024] in, This represents the rotation matrix of the local coordinate system relative to the station coordinate system. Represents the translation vector. Represents coordinates in a local coordinate system. Represents the coordinates in the station coordinate system.

[0025] Each charging station is registered as a station unit, and the physical boundaries and fixed structures of each station unit are also registered. The physical boundary is the boundary of the station unit in the site coordinate system. The fixed structure is a structure with a fixed spatial position within or around the station unit. The fixed structure includes structures with fixed spatial positions such as charging terminals, anti-collision posts, pillars, parking lines, and pile-side gun mounts. After registration, the station unit, physical boundary, fixed structure, and station unit identifier are written into an index table.

[0026] The device parameters of the camera are acquired and bound to the corresponding workstation unit. These device parameters include camera coordinates, field of view angle, focal length, lens distortion parameters, frame rate, and reference brightness. For cameras capable of rotation, their field of view angle is locked to ensure stable acquisition during inspection.

[0027] When determining the binding relationship between the camera and the workstation unit, the camera's field of view is projected onto the site coordinate system, and the overlap ratio between the field of view and the physical boundary of the workstation unit is calculated.

[0028]

[0029] in, Indicates a camera. Indicates a workstation unit. This represents the projection range of the camera's field of view into the station's coordinate system. Represents the physical boundary of the workstation unit. Indicates the size of the region. This indicates a positive number used to avoid a denominator of zero. This indicates the coverage level of the workstation unit by the camera. When When the value is not less than the preset value, the camera will be bound to the corresponding workstation unit.

[0030] In one implementation, the preset value for the coverage ratio of the camera and the workstation unit is taken as... ,when When the camera is in use, determine that it covers the corresponding workstation unit and bind the camera to the covered workstation unit.

[0031] The pixel positions of structural anchor points in the fixed structure are extracted, and these anchor points are then mapped to physical points in the station coordinate system. Each structural anchor point is a point in the fixed structure that can be repeatedly identified in the image and has a fixed spatial position in the station coordinate system. Each anchor point corresponds to one pixel position and one physical point.

[0032] In one embodiment, the structural anchor points are selected from points in the fixed structure that have obvious corner features and unique spatial locations, including the top corner of the anti-collision post, the bottom corner of the charging terminal, the corner endpoint of the parking space line, and the edge inflection point of the pile side gun seat. All of the above points meet the requirements of repeatability and fixed position.

[0033] In one implementation, the automatic extraction of structural anchor points employs the Harris corner detection algorithm. First, corner points are detected in the corrected image to obtain candidate corner points. Then, these candidate corner points are matched with the physical points corresponding to preset fixed structures. Successfully matched candidate corner points are considered valid structural anchor points. For structural anchor points that fail automatic detection, manual annotation is used to supplement them. During annotation, the pixel center position of the corner point is selected as the pixel position of the structural anchor point. The corresponding physical point in the field station coordinate system is obtained through field measurements. The pixel position of each structural anchor point is stored in a one-to-one correspondence with a physical point.

[0034] A planar mapping table for images to ground coordinates in a panoramic coordinate system, a panoramic mapping table for images to panoramic coordinates, and an index table for associating workstations and structural anchor points are generated for the camera, serving as the mapping relationship from images to physical coordinates. The planar mapping table is used to map the pixel positions in the images captured by the camera to ground coordinates in the panoramic coordinate system; the panoramic mapping table is used to map the pixel positions in the images captured by the camera to panoramic coordinates; and the index table is used to associate the coordinates of the camera, workstation, structural anchor point, physical boundary, fixed structure, and camera.

[0035]

[0036] in, Indicates the pixel position in the image. This represents the planar mapping table corresponding to the camera. Indicates pixel position Ground coordinates in the panoramic coordinate system obtained through a plane mapping table.

[0037]

[0038] in, This represents the panoramic mapping table corresponding to the camera. Indicates pixel position The panoramic coordinates are obtained through the panoramic mapping table. Therefore, the mapping direction of both the planar mapping table and the panoramic mapping table is from the pixel position in the image to the corresponding coordinates. This is used when performing the reverse mapping on the pixel coordinates later. Finish.

[0039] In one implementation, the ground coordinates output by the planar mapping table are two-dimensional ground plane coordinates in the station coordinate system, while the panoramic coordinates output by the panoramic mapping table are two-dimensional coordinates in the panoramic coordinate system. Both have the same dimension and are applicable to target physical positioning and panoramic image stitching, respectively. When generating the planar and panoramic mapping tables, each workstation unit selects at least four non-collinear structural anchor points as calibration references. The mapping relationship from image pixels to physical coordinates is calculated based on the pinhole camera model and homography matrix solution method.

[0040]

[0041] in, Represents the homography matrix. Represents the homogeneous form of image pixel coordinates. Representing homogeneous physical coordinates. After obtaining the homography matrix by solving for at least four sets of structural anchor point correspondences, a corresponding mapping table is generated and stored by traversing all pixel positions of the image.

[0042] In one implementation, the reverse mapping is achieved using a lookup table interpolation method. For a given panoramic coordinate, the corresponding image pixel grid position is first looked up using the reverse index of the mapping table, and then the sub-pixel precision image pixel coordinates are calculated using bilinear interpolation.

[0043]

[0044] in, This represents a pre-generated reverse mapping lookup table that stores the correspondence between panoramic coordinates and image pixel coordinates. Representing panoramic coordinates In the camera Collect the corresponding pixel positions in the image.

[0045] Example 3: Reference Figure 1 In the camera acquisition section, frame sequences are retrieved from the cameras to generate time slices with errors within a preset time threshold. Each video frame in the frame sequence carries a timestamp. The timestamp difference between any two video frames corresponding to any two cameras within the same time slice does not exceed the preset time threshold.

[0046]

[0047] in, Indicates the first A time slice, Indicates camera In time The captured video frames, Indicates the first The set of cameras that participated in the data collection within a specific time frame. and These represent cameras. and cameras The corresponding video frame timestamps, This indicates a preset time threshold.

[0048] In one implementation, the preset time threshold of the time slice Pick ,correspond The time interval between single frames at the capture frame rate is used to ensure that all video frames within the same time slice are in the same capture cycle. The time slice is generated using a sliding window method with a fixed step size, which is consistent with the video capture frame rate. A new time slice is generated each time a new video frame is captured.

[0049] In one implementation, the time reference for each time slice is selected from the frame timestamp of the main camera, and the timestamps of the other cameras are selected with a difference of no more than [value missing] from the reference timestamp. The most recent video frame is included in the current time slice. When a camera drops a frame, the previous valid frame closest to the reference timestamp of that camera is selected as the replacement frame and included in the time slice, and this frame is marked as a frame drop compensation frame in the quality tag; if the number of consecutive frame drops exceeds three frames, the camera is marked as unavailable in the corresponding time slice and is not included in the subsequent fusion calculation.

[0050] Lens distortion correction is performed on the video frames within the time slice. Specifically, the camera's device parameters are read, and the pixel positions in the video frames are resampled according to the lens distortion parameters to ensure that the fixed structure maintains a spatial relationship consistent with the station coordinate system in the corrected video frames.

[0051] In one implementation, lens distortion correction employs a Brownian distortion model, calculating the remapping coordinates of each pixel based on the camera's intrinsic distortion parameters. The resampling process uses a bilinear interpolation algorithm to keep the edges of the corrected image smooth and to ensure that the geometry of the fixed structure remains consistent with the physical shape in the field coordinate system.

[0052] The coverage area of ​​the corresponding workstation unit is cropped using an index table to obtain a partial image. Specifically, based on the relationship between workstation units, structural anchor points, and cameras in the index table, the area covering the corresponding workstation unit is extracted from the corrected video frame to obtain the partial image corresponding to the workstation unit.

[0053] In one implementation, when cropping a local image, the physical boundary vertices of the workstation unit are first converted into pixel coordinates in the image using a planar mapping table to obtain the polygonal projection area of ​​the physical boundary in the image. Then, the corrected image is cropped along the polygonal boundary. The pixels within the polygonal area are the local images of the corresponding workstation unit. The cropped local images retain the pixel coordinate system of the original image and correspond one-to-one with the physical boundary of the workstation unit.

[0054] Brightness normalization is performed on a local area of ​​the image based on the camera's preset baseline brightness. Brightness normalization is performed on the brightness component corresponding to the local area of ​​the image.

[0055]

[0056] in, Indicates camera Pixel position in a partial image The brightness component, This represents the luminance component after luminance normalization. Indicates camera The reference brightness, Indicates camera The average brightness of a local area of ​​the image. This indicates a positive number used to avoid a denominator of zero.

[0057] In one implementation, the reference brightness Obtained through offline camera calibration. During calibration, the camera is pointed at a standard grayscale chart to capture images, and the average brightness of the grayscale chart area is taken as the reference brightness for the camera. When calculating the average brightness, images with grayscale values ​​less than a certain threshold are discarded. and greater than For overly dark or overexposed pixels, the average value is calculated only for pixels within the effective grayscale range.

[0058] The image blur, structural redundancy, and occlusion ratio of a local area are obtained and written into the quality label. To ensure stable calculation of the fusion weights, the image blur, structural redundancy, and occlusion ratio are all normalized.

[0059]

[0060] in, This represents the intermediate value used when calculating image blur. Represented by pixel position The pixel range centered on express Pixel position within, Represents the Laplace operator. This represents the luminance component after luminance normalization. Indicates in Calculate the variance within the internal calculation.

[0061] In one implementation, by pixel position Centered pixel window use take Square pixel neighborhood, normalized parameters Pick To make the image blurry The output range falls within Interval.

[0062]

[0063] in, Indicates camera At pixel position Image blur within the corresponding range, This represents the intermediate value used when calculating image blur. This represents a positive number used for normalization. The closer , indicating that the local area of ​​the image is more blurred.

[0064]

[0065] in, Indicates camera At pixel position Structural repeatability within the corresponding range This represents the set of displacements used to compare repeating structures. Represents the displacements in the displacement set. Indicates camera At pixel position Image features at the location, Indicates according to displacement Image features after movement Indicates in Internal correlation calculation. The closer This indicates that the fixed structure is more likely to overlap with the fixed structure in the adjacent workstation unit within the corresponding range.

[0066] In one implementation, displacement set Take the positive and negative values ​​for both the horizontal and vertical directions. The correlation calculation uses a normalized cross-correlation algorithm for integer shift combinations of pixels.

[0067]

[0068] in, This represents the mean of the features of the original image within the window. This represents the mean of the image features after displacement within the window. This indicates a positive number used to avoid a denominator of zero.

[0069]

[0070] in, Indicates camera At pixel position The occlusion ratio within the corresponding range, express The range of occluded pixels caused by internal dynamic structure coverage, lack of effective image pixels, or inability to identify structural anchor points. Indicates the size of the occluded pixel range. express Size, This indicates a positive number used to avoid a denominator of zero.

[0071] In one implementation, the occluded pixel range It includes three types of pixels: first, fixed structure pixels covered by dynamic structures; second, dark area pixels at the edge of the field of view with no effective image pixels; and third, blurry area pixels where structural anchor points cannot be identified. These three types of pixels together constitute the occlusion pixel range, which is used to calculate the occlusion ratio within the corresponding window.

[0072] A set of images is formed by combining partial frames, workstation units, and quality markers. The quality markers include at least image blur, structural repetition, and occlusion ratio. The image set is arranged according to time slices, and each partial frame is associated with a corresponding workstation unit through an index table.

[0073] Example 4: Reference Figure 1 In the partial image and dynamic structure extraction section, the image set is projected onto the local coordinate system of the corresponding workstation unit. The local coordinate system is established based on the physical boundary of the corresponding workstation unit and is used to uniformly represent the fixed structure, dynamic structure, gun centerline, fixed end, and vehicle-side charging end within the same workstation unit.

[0074] The dynamic structure is extracted by erasing overlapping textures in the local coordinate system according to a preset background template. The background template is formed by the physical boundary and fixed structure of the workstation unit, and the overlapping texture is the pixel content in the local image that matches the background template in terms of position and texture.

[0075]

[0076] in, Indicates workstation unit Inner pixel position The difference results Indicates camera Data collection workstation unit A partial image at pixel position pixel values, Indicates workstation unit Background template at pixel position The pixel value.

[0077]

[0078] in, This represents the pixel value after erasing the overlapping texture. This represents the preset value used to determine overlapping textures. When At that time, pixel position Corresponding overlapping textures are erased; when At that time, preserve pixel position The corresponding pixel value. The continuous range of pixels that have not been erased forms a dynamic structure.

[0079] In one implementation, a preset value for determining overlapping textures is provided. Pick ,correspond The pixel difference threshold for a grayscale image, where the absolute value of the difference between pixel positions is no greater than [value missing]. When the textures overlap, they are identified as overlapping textures and the erasure is performed.

[0080] In one implementation, the background template for a workstation unit is generated through multi-frame fusion. Ten consecutive frames of idle workstation images without dynamic targets are selected, and the median value of the corresponding pixels in each frame is taken as the pixel value of the background template. The generated background template is aligned with the local coordinate system, and each workstation unit has an independent background template. The background template adopts a periodic update strategy, selecting a frame of workstation image without dynamic targets every thirty minutes to perform a weighted update of the background template.

[0081]

[0082] in, Pick Background weight coefficient, This represents the pixel values ​​of the original background template. This represents the image pixel value where there are currently no moving targets. This represents the updated background template pixel values.

[0083] In one implementation, the connectivity determination of the dynamic structure adopts the 8-connectivity rule, that is, the top, bottom, left, right and four diagonal directions of a pixel are all considered connected, and the minimum area threshold of the dynamic structure is taken as... Pixels with an area smaller than the threshold are considered noise and removed. After dynamic structure extraction, morphological post-processing is performed, first using... take The rectangular structural element is subjected to erosion operation to remove noise points, and then the same structural element is used to perform expansion operation to fill the holes inside the connected domain, finally obtaining a complete dynamic structural region.

[0084] Continuous edges and strip-shaped regions are extracted from the dynamic structure to form candidate line segments. The continuous edges are edge pixels continuously distributed within the dynamic structure, and the strip-shaped regions are pixel ranges with a strip-like shape within the dynamic structure. Candidate line segments are formed by extracting the central skeleton from the continuous edges and strip-shaped regions.

[0085] In one implementation, continuous edge extraction employs the Canny edge detection algorithm. First, the grayscale image of the dynamic structural region is smoothed using Gaussian filtering. Then, the image gradient magnitude and direction are calculated, and thinned edges are obtained through non-maximum suppression. Finally, a double thresholding method is used to filter and obtain continuous edge pixels. Strip region recognition uses a strip detection algorithm, applying directional filtering to the dynamic structural region to extract pixels with a width within... to pixel range, length greater than The strip-shaped pixel region of a pixel is called the strip region.

[0086] In one implementation, the central skeleton extraction employs the Zhang-Suen parallel thinning algorithm, which performs thinning processing on continuous edges and strip regions respectively to obtain skeleton lines with a single pixel width. Then, the skeleton lines are broken and short branches are removed to finally obtain continuous candidate line segments. Each candidate line segment records the coordinates and direction information of its two endpoints.

[0087] The system determines whether the candidate line segment passes through the pile-side gun mount, extends to the vehicle side, and is parallel to the extension direction of the fixed structure. The vehicle side refers to the region on the side where the vehicle is located in the dynamic structure, and the vehicle-side charging terminal is the pixel position on the vehicle side adjacent to the candidate line segment and used to determine the charging connection relationship.

[0088] In one implementation, image localization of the pile-side gun mount employs a template matching method. A standard image of the pile-side gun mount is pre-acquired as a matching template. Normalized cross-correlation template matching is then performed within the pile body region of a local image, achieving the highest matching degree (greater than...). The area is the image location of the pile-side gun seat, and the center pixel of this area is taken as the positioning point of the pile-side gun seat. The vehicle side area is divided based on the physical boundary of the workstation unit. The boundary on the side away from the pile-side gun seat is the vehicle entrance side. The corresponding one-third area away from the pile-side gun seat in the local image is defined as the vehicle side, which is used to determine whether the candidate line segment extends to the vehicle side.

[0089]

[0090] in, Indicates candidate line segments, and Representing candidate line segments respectively The two endpoints, Indicates candidate line segments The direction.

[0091]

[0092] in, Indicates workstation unit The set of extension directions of internal fixation structures Indicates one direction of extension of a fixed structure. This represents a function that groups angular differences into the same range for comparison. Indicates candidate line segments The minimum directional difference between the candidate line segment and the extension direction of the fixed structure. When the candidate line segment passes the pile-side gun mount and extends to the vehicle side, and... When the value is not less than the preset value, the candidate line segment is not parallel to the extension direction of the fixed structure.

[0093] In one embodiment, the set of extension directions of the fixed structure The results were obtained by fitting straight lines to the edges of the fixed structure within the workstation unit, using the Hough linear transform to extract straight line segments from the fixed structure, calculating the direction angle of each straight line, and categorizing them into a set. Direction angle calculation based on image The positive direction of the axis is the reference, and counterclockwise rotation is the positive direction.

[0094]

[0095] in, This represents the angle normalization result. This represents the angle difference to be normalized. This represents the rounding function.

[0096] In one implementation, the minimum directional difference between the candidate line segment and the extension direction of the fixed structure is preset to a value of 1. ,when When the candidate line segment is not parallel to the extension direction of the fixed structure, it is determined that the candidate line segment is not parallel to the extension direction of the fixed structure.

[0097] Based on the judgment results, the centerline of the charging gun is extracted from the candidate line segments, and the fixed end, vehicle-side charging end, and centerline direction of the centerline are determined. To ensure that the endpoint fit has a discriminative effect, the fixed end is independently determined by the pixel position of the charging gun mount on the pile side, and the vehicle-side charging end is independently determined by the pixel position of the charging connection on the vehicle side; neither of these uses the endpoints of the candidate line segment itself as a priori. After the candidate line segment is determined as the centerline of the charging gun, the ends of the centerline closest to the fixed end and the vehicle-side charging end, respectively, are designated as the fixed end and the vehicle-side charging end of the centerline.

[0098] In one implementation, the pixel position of the fixed end is determined independently as follows: first, the entire area of ​​the pile side gun seat is located by template matching, and then edge detection is performed at the outlet position of the pile side gun seat area to extract the lower edge midpoint of the outlet as the pixel coordinate of the fixed end. This positioning process does not depend on the position information of the candidate line segment.

[0099] In one implementation, the pixel position of the vehicle-side charging terminal is determined independently as follows: First, feature detection of the charging gun head is performed in the vehicle-side region of the dynamic structure. The charging gun head region is identified using directional gradient histogram features combined with a support vector machine classifier. The center point of the contact position between the charging gun head region and the vehicle body is taken as the pixel coordinate of the vehicle-side charging terminal. This localization process does not rely on the position information of candidate line segments. After independently determining the fixed end and the vehicle-side charging terminal, they are matched with candidate line segments. The pixel distance from the fixed end and the vehicle-side charging terminal to the candidate line segment is calculated, and the directional deviation is used to determine whether the candidate line segment is the center line of the charging gun.

[0100] Example 5: Reference Figure 2 The mid-span feature section combines the pixel distances from the fixed end and the vehicle-side charging end to the center line of the charging gun to determine the endpoint fit. For any candidate line segment or the already determined center line of the charging gun, the endpoint fit is used to represent the degree of fit between the fixed end and the vehicle-side charging end and the center line of the charging gun.

[0101]

[0102] in, Indicates the center line of the gun. Corresponding endpoint fit This indicates the pixel position of a fixed, independently determined end. This indicates the pixel position of the independently determined vehicle-side charging terminal. This represents the pixel distance from the fixed end to the center line of the gun line. This indicates the pixel distance from the charging terminal on the vehicle side to the center line of the charging gun. Represents positive numbers used for normalization. This indicates a positive number used to avoid a denominator of zero.

[0103] In one implementation, the normalized parameter in the endpoint fit calculation Pick The normalized reference for the corresponding pixel distance.

[0104] The analytical directional deviation is based on the angle between the centerline direction and the extension direction of the fixed structure. The directional deviation indicates the degree of deviation of the gun's centerline from the extension direction of the fixed structure.

[0105]

[0106] in, Indicates the center line of the gun. The corresponding directional deviation, Indicates the direction of the centerline of the gun's center line. Indicates one direction of extension of a fixed structure. Indicates workstation unit The set of extension directions of internal fixation structures This represents a positive number used to normalize the angle difference. This indicates a positive number used to avoid a denominator of zero. The larger the value, the less likely the center line of the gun line is to be replaced by the repeated extension direction of the fixed structure.

[0107] In one implementation, the angle normalization parameter in the orientation deviation calculation Pick This is used to control the decay rate of directional deviation.

[0108] Topological feature values ​​are constructed based on endpoint fit and directional deviation. These topological feature values ​​represent the ability of the gun line centerline to break through repetitive structures across workstations.

[0109]

[0110] in, Indicates camera At pixel position The corresponding topological eigenvalues, Indicates camera At pixel position The center line of the gun line obtained within the corresponding range This indicates the degree of fit between the endpoints of the centerline of the gun. This indicates the degree of directional deviation of the centerline of the gun.

[0111] A fusion weight is constructed by combining topological feature values, structural repetition in quality markers, and image blur. To avoid inconsistencies in the numerical ranges of different indicators, the topological feature values, structural repetition, and image blur are all normalized.

[0112]

[0113] in, Indicates camera At pixel position The corresponding fusion intermediate value, , and All are preset positive numbers. Represents topological eigenvalues. Indicates structural repetition. Indicates the degree of image blur.

[0114] In one implementation, a preset positive number is incorporated into the intermediate value calculation. Pick , Pick , Pick These correspond to the positive gain coefficient of the topological eigenvalue and the negative penalty coefficients of structural redundancy and image blur, respectively.

[0115]

[0116] in, Indicates camera At pixel position The corresponding fusion weights, Indicates the pixel location that can be covered. A collection of cameras, express The camera in the middle, Indicates camera At pixel position The corresponding intermediate value of the fusion. When When empty, pixel position Not participating in panoramic pixel aggregation; when When not empty, the sum of the fusion weights of all cameras is .

[0117] In one implementation, the camera coverage set The criteria for determination are: panoramic coordinates The image falls within the effective field of view of the camera through reverse mapping, and the image blur at that location is less than [value missing]. The occlusion ratio is less than Cameras that meet the above conditions are included in the camera coverage set. .

[0118] Image features and pixels from various viewpoints are aggregated using fusion weights to render panoramic pixels. To avoid coordinate domain confusion, the pixel positions in the camera images corresponding to the panoramic coordinates are first determined using a panoramic mapping table.

[0119]

[0120] in, Represents panoramic coordinates. Represents the panoramic mapping table The reverse addressing relationship, Representing panoramic coordinates In the camera Collect the corresponding pixel positions in the image.

[0121]

[0122] in, Representing panoramic coordinates Corresponding panoramic pixels, Indicates camera At pixel position Image pixels at that location, Indicates camera In panoramic coordinates The fusion weight at the location.

[0123] In one implementation, panoramic pixel fusion is applied to color images. The three channels are weighted and fused separately, with each channel independently calculating the weighted pixel value, and finally synthesizing the color panoramic pixels.

[0124]

[0125] in, Representing panoramic coordinates Corresponding image features Indicates camera At pixel position Image features at the location.

[0126] In one implementation, image features To determine the gradient magnitude features of a grayscale image, the Sobel operator is used to calculate the image. direction and After calculating the gradient in the direction, the gradient magnitude is taken as the image feature value.

[0127]

[0128] in, express directional gradient value, express The directional gradient value, structural repeatability calculation, and panoramic feature aggregation all use the same gradient magnitude feature.

[0129] By utilizing fusion weights, inverse mapping and aggregation are performed on the pixel coordinates of the inspected object from various viewpoints to obtain the target coordinates. The inspected object refers to the object whose target coordinates need to be determined, including abnormal targets. Here, inverse mapping refers to mapping the pixel coordinates in the image captured by the camera to the ground coordinates in the panoramic coordinate system through a planar mapping table.

[0130]

[0131] in, Indicates the target coordinates. This refers to the set of cameras that can observe the objects being inspected. Indicates the object of inspection is under the camera. Acquire pixel coordinates from the image. Indicates camera In pixel coordinates The corresponding fusion weights, Represents pixel coordinates Ground coordinates in the panoramic coordinate system obtained through a planar mapping table. When When there is only one camera, the target coordinates are determined by the plane mapping table corresponding to that camera; when... When there are multiple cameras, the target coordinates are obtained by aggregating the results from each viewpoint according to the fusion weight.

[0132] In one implementation, single-camera positioning is applicable when the target is located on the ground plane, and is suitable for calculating the coordinates of targets with abnormal occupancy or facility anomalies. For non-planar targets such as the center line of the gun emplacement, multi-camera aggregation is preferred to calculate the target coordinates. The multi-camera weighted aggregation target coordinate calculation method is applicable to targets covered by at least two cameras. Targets covered by a single camera are located using a single-plane mapping table. The selection of cable perpendicular points using single-camera results is applicable when only a single camera can observe the complete center line of the gun emplacement.

[0133] Example 6: Reference Figure 1 The mid-range dynamic panoramic image stitching section, combined with... Figure 2 In the integration of weights and splicing seam constraints, the workstation units are expanded into the panoramic coordinate system according to the preset site orientation. During expansion, the display area of ​​each workstation unit in the panoramic coordinate system is determined based on its physical boundaries, while maintaining the spatial order between adjacent workstation units consistent with the site coordinate system.

[0134] In one implementation, the dynamic panoramic image is presented in a planar unfolding format. Using the ground plane of the station coordinate system as the unfolding reference, the images of each workstation unit are tiled and unfolded onto the panoramic coordinate system according to their physical spatial positions. The mapping between the physical boundaries of the workstation units and the panoramic pixel coordinates uses a fixed scale, with the scale set to correspond to a certain value per meter. Each workstation is represented by a single panoramic pixel. Its position in the panoramic coordinate system is calculated based on the site coordinates of the physical boundary of the workstation unit, thus determining the display area of ​​each workstation unit in the panoramic image. The arrangement logic of multiple workstations in the panoramic image is consistent with the actual spatial layout of the site. Adjacent workstation units are arranged sequentially according to their physical positions, and a spacing consistent with the actual distance is maintained between workstation units.

[0135] The process analyzes the crossing costs of gun lines, dynamic structures, and fixed structures for stitching paths. The stitching paths are located in the overlapping areas of adjacent images or adjacent workstations. To ensure that the crossing costs are compared under the same rules, crossing gun lines, dynamic structures, and fixed structures are all converted to normalized scales.

[0136]

[0137]

[0138]

[0139] in, Indicates the splicing path. This represents the normalized proportion of the splicing path crossing the gun line. This represents the normalized proportion of the splicing path traversing the dynamic structure. This represents the normalized proportion of the splicing path traversing a fixed structure. This represents the function for calculating path length. Indicates the center line of the gun. Indicates workstation unit The dynamic structure within, Indicates workstation unit Internal fixed structure, This indicates a positive number used to avoid a denominator of zero.

[0140]

[0141] in, Indicates the splicing path The cost of time travel , and All are preset positive numbers, used to adjust the impact of crossing gun lines, crossing dynamic structures, and crossing fixed structures on the crossing cost, respectively.

[0142] In one implementation, a preset positive weight is used in the calculation of splicing traversal cost. Pick , Pick , Pick These correspond to the cost weights for crossing a gun line, crossing a dynamic structure, and crossing a fixed structure, respectively, maximizing the penalty for crossing a gun line.

[0143] A path with a buffer zone distributed along the physical boundary and a crossing cost that meets the set parameters is selected as the splicing seam. The buffer zone along the physical boundary refers to the area on both sides of the physical boundary of adjacent workstations used to select the splicing path.

[0144]

[0145] in, This indicates that the seam has been determined. This represents the set of splicing paths within the buffer zone of the physical boundary. This indicates whether the crossing cost meets the preset value. If there exists a value that meets the preset value... If the splicing path is such that the path with the lowest crossing cost is selected as the splicing seam, then the splicing path with the lowest crossing cost will be selected as the splicing seam.

[0146] In one implementation, the preset value for determining the crossing cost is... Pick This is used to select splicing paths that meet the low crossing cost requirement. The width of the buffer zone at the physical boundary is taken as a fraction of the width of the overlapping area of ​​adjacent workstation units. The splicing path is selected only within the buffer zone of this physical boundary.

[0147]

[0148] When there is no condition within the buffer zone of the physical boundary that satisfies When determining the splicing path, the splicing path with the lowest crossing cost within the buffer zone of the physical boundary is selected as the determined splicing seam, and the crossing cost corresponding to the determined splicing seam is recorded in the index table.

[0149] To generate a dynamic panoramic image, panoramic pixels are extracted from the corresponding regions along the defined stitching seams. Specifically, the image source corresponding to each panoramic coordinate is determined using the defined stitching seams as boundaries, and panoramic pixels are extracted according to the fusion weights.

[0150] In one implementation, the logic for coordinating the stitching seam and the fusion weights is as follows: Non-overlapping areas of adjacent workstations are directly filled with image pixels from a single camera; overlapping areas are weighted and fused using fusion weights to determine the stitching seam as the path with the lowest traversal cost within the overlapping area. Non-overlapping areas on either side of the stitching seam are assigned to the corresponding workstation camera. The overlapping zone containing the stitching seam is smoothly transitioned using fusion weights. The stitching seam serves as a boundary reference between overlapping and non-overlapping areas, and panoramic pixels within the overlapping area are generated through weighted aggregation using fusion weights.

[0151] The identifier, image source, and target coordinates of each workstation unit are stored in an index table, and then combined with the extracted panoramic pixels to form a dynamic panoramic image. The dynamic panoramic image includes panoramic pixels and the identifier, image source, and target coordinates of the workstation units associated with those pixels. Thus, the dynamic panoramic image can both display a panoramic view and, through the index table, determine the workstation unit and target coordinates of each abnormal target.

[0152] In one implementation, the dynamic attributes of the dynamic panoramic image include real-time updates of the image content over time slices and dynamic changes in viewing angle. Real-time updates of the image content over time slices mean that a panoramic pixel update is completed each time a new set of time slices is generated, with the update frequency consistent with the video capture frame rate. Dynamic changes in viewing angle mean that the inspection terminal can adjust the display viewing angle and scaling ratio of the dynamic panoramic image through rotation and zoom operations to view detailed images of different areas of the site. The real-time update of the dynamic panoramic image adopts an incremental update mechanism, updating pixels only in workstation units with dynamic structures, while retaining the original pixel values ​​in static areas without dynamic structures.

[0153] Example 7: Reference Figure 2 The anomaly attribution and redirection section scans the dynamic panoramic image by workstation unit to locate anomalies and determine their types. These anomaly types include cable anomalies, space occupancy anomalies, and facility anomalies. During scanning, the workstation unit corresponding to each dynamic panoramic image area is determined based on an index table, and anomalies are identified within that area.

[0154] In one implementation, abnormal target detection employs a deep learning target detection algorithm, using the YOLO network as the basic detection framework. The training dataset includes samples of cables dragging on the ground, foreign objects occupying space, and damaged facilities in a charging station scenario. The detection input is an image of each workstation unit area from a dynamic panoramic view. The rule for determining cable anomalies is that the vertical height of the cable's centerline is less than... Meters, or the center line of the gun line crosses the physical boundary of the adjacent workstation unit; the rule for judging abnormal occupancy is that a non-charging vehicle, debris, or other target occupies the parking space within the workstation unit, and the target stays for more than [time period missing]. The facility anomaly detection rule is based on the morphological changes of fixed structures such as charging terminals and anti-collision posts, including displacement, damage, and missing parts, where the structural features differ from the background template by more than a set threshold. The input for anomaly detection consists of local images and dynamic structure extraction results corresponding to each workstation unit. Abnormal targets are first detected in the local images and then mapped to the dynamic panoramic image for location and display.

[0155] If the anomaly type is cable anomaly, the cable perpendicular point or boundary crossing point of the anomaly target is used as the target coordinates and bound to the workstation unit to which its fixed end belongs. The cable perpendicular point is determined in the local image by the pixel position at the lowest position along the vertical axis of the gun line, and the corresponding target coordinates are obtained through a planar mapping table.

[0156]

[0157] in, Indicates camera The corresponding pixel position of the cable perpendicular point Indicates camera The center line of the gun line in the acquired image. express pixel position on Indicates pixel position The coordinates of the image along its vertical axis.

[0158]

[0159] in, This refers to the camera used to determine the perpendicularity of the cable. This refers to a set of cameras capable of observing the center line of the gun. Indicates camera The fusion weight corresponds to the pixel position at the cable's perpendicular point. When multiple cameras have the same fusion weight, the camera that is bound to the workstation unit to which the fixed end belongs is selected.

[0160]

[0161] in, This indicates the target coordinates corresponding to the perpendicular point of the cable. Indicates camera The corresponding planar mapping table, Indicates camera The pixel position of the corresponding cable perpendicular point.

[0162] In one implementation, the image coordinate system The positive axis direction is vertically downward, corresponding to the top of the image. Minimum coordinate value, bottom The coordinate value is the largest, therefore The point with the largest coordinates is the lowest point in the vertical direction of the image, used to determine the cable perpendicular point. For cameras with a tilt angle, a tilt angle correction method is used to adjust the calculation of the cable perpendicular point. First, the vertical coordinates of the image are converted into the actual vertical direction based on the camera's tilt angle, and then the lowest point in the vertical direction is obtained as the cable perpendicular point.

[0163]

[0164] in, Represents the vertical pixel coordinates of the image. Indicates the camera's tilt angle. Indicates the camera installation height. This represents the corrected actual vertical coordinates.

[0165] In one implementation, the ground projection coordinate error of suspended targets such as cable plumb points is compensated by a height correction method. First, the height of the cable plumb point above the ground is estimated by using the tilt angle of the gun line centerline and the height information of the fixed end. Then, the projection result of the plane mapping table is corrected according to the camera's pitch angle and focal length.

[0166]

[0167] in, This indicates the target coordinates corresponding to the corrected cable perpendicular point. This indicates the target coordinates corresponding to the cable's perpendicular point before correction. Indicates the height of the cable's perpendicular point from the ground. Indicates the camera's tilt angle. This represents the horizontal offset direction vector.

[0168] The boundary crossing point is determined by the intersection of the gun line centerline and the physical boundary of the workstation unit. When there are multiple intersection points between the gun line centerline and the physical boundary, the intersection point closest to the fixed end is selected as the boundary crossing point.

[0169]

[0170] in, This indicates the target coordinates corresponding to the boundary crossing point. This represents the centerline of the gun line obtained through the plane mapping table. Indicates workstation unit The physical boundary express The intersection of the two points, This indicates the target coordinates corresponding to the fixed end. Indicates the intersection point Target coordinates corresponding to the fixed end The distance between them. When there are no boundary crossing points, the target coordinates corresponding to the perpendicular point of the cable are used.

[0171] In one implementation, the boundary crossing point calculation uses the intersection of the physical projection line segment and the physical boundary. First, the pixel coordinates of the gun line centerline are converted into physical coordinates on the ground plane through a plane mapping table to obtain the gun line projection line segment in the physical space. Then, the intersection of the projection line segment and the physical boundary of the workstation unit is calculated, and the intersection point closest to the fixed end is selected as the boundary crossing point.

[0172] In case of cable abnormality, the abnormality binding unit queries the index table based on the target coordinates corresponding to the fixed end to determine the workstation unit to which the fixed end belongs, and binds the target coordinates corresponding to the cable's vertical point or boundary crossing point to the workstation unit to which the fixed end belongs.

[0173] If the anomaly type is a space occupancy anomaly or a facility anomaly, the centroid of the anomaly target is obtained and used as the target coordinates to bind to the workstation unit where the centroid is located. The centroid is determined by aggregating the coordinates of each pixel position within the anomaly target contour using a planar mapping table.

[0174]

[0175] in, This indicates the target coordinates corresponding to the center of gravity. This indicates the number of pixels within the abnormal target contour that are included in the calculation. Indicates the first The coordinates of each pixel position are obtained through a planar mapping table. When Greater than hour, As the target coordinates; when equal At that time, no corresponding abnormal target is generated.

[0176] After binding abnormal targets such as cable abnormalities, occupation abnormalities, or facility abnormalities, the abnormal binding unit extracts the identifier, abnormality type, and target coordinates of the bound workstation unit.

[0177] The observation direction is determined by analyzing the reverse extension line of the line connecting the target coordinates and the camera coordinates, and the identifier, anomaly type, target coordinates, and observation direction are combined to form a jump instruction. The camera used to analyze the observation direction is preferentially selected as the one capable of observing the anomalous target and with the highest fusion weight; when multiple cameras have the same fusion weight, the camera closest to the target coordinates is selected.

[0178]

[0179] in, This indicates the camera used to analyze the orientation of the observation. This refers to a set of cameras capable of observing unusual targets. This indicates an abnormal target at the camera. Acquire pixel coordinates from the image. Indicates camera In pixel coordinates The corresponding fusion weight.

[0180] In one implementation, the design logic for the observation orientation is to align the virtual camera of the inspection terminal with the abnormal target along the camera's observation direction. The observation point is located on the line connecting the camera coordinates and the target coordinates, and is located on the side closer to the camera coordinates relative to the target coordinates. The observation direction points from the observation point to the target coordinates and is consistent with the actual observation direction of the camera.

[0181]

[0182] in, This represents the direction vector of the reverse extension line. Indicates the target coordinates. Indicates camera Corresponding camera coordinates This represents the distance between the target coordinates and the camera coordinates. This indicates a positive number used to avoid a denominator of zero. The backward extension line is determined by the target coordinates along the side closest to the camera coordinates, with the direction reference pointing from the camera coordinates to the target coordinates.

[0183]

[0184] in, This indicates the observation point determined based on the backward extension line. This represents a preset positive number. This represents the direction vector of the backward extension line. The observation point is located on the line connecting the camera coordinates and the target coordinates, and is positioned closer to the camera coordinates relative to the target coordinates. The observation orientation is... point to The direction is the directional reference for the rotating panoramic field of view angle of the inspection terminal.

[0185]

[0186] in, This represents the direction vector used to center the image region corresponding to the target coordinates in the image. Indicates the target coordinates. This indicates the observation point determined based on the backward extension line. This indicates a positive number used to avoid a denominator of zero. (By...) , and Together, we determined the orientation for observation.

[0187] In one implementation, a preset positive number is used in the observation point offset calculation. Pick This is used to control the offset distance between the observation point and the target coordinates. When the observation point crosses the camera coordinates, the midpoint of the line segment between the target coordinates and the camera coordinates is taken as the observation point. The observation orientation is represented in Euler angle form, including two rotational components: yaw and pitch, in the panoramic coordinate system. The positive direction of the axis is the yaw angle. Degree benchmark, The positive direction of the axis is The reference point is the horizontal direction as the pitch angle. The baseline is upward, indicating a positive value.

[0188] The jump instruction includes at least the identifier of the workstation unit, the anomaly type, the target coordinates, and the observation orientation. The identifier of the workstation unit is used to determine the workstation unit to which the anomaly target belongs; the anomaly type is used to indicate the category of the inspection anomaly; the target coordinates are used to determine the spatial position of the anomaly target in the dynamic panoramic image; and the observation orientation is used to cause the inspection terminal to rotate the panoramic field of view angle according to the target coordinates and the reverse extension line.

[0189] In one implementation, the jump instruction uses a structured data format, including the identifier of the workstation unit, the exception type, and the target coordinates. Quantity and The system contains six fields: yaw angle component, observation orientation component, and pitch angle component. These fields are encapsulated and transmitted in JSON format. After parsing the jump command field, the inspection terminal executes the corresponding view rotation and target centering operations.

[0190] Example 8: The scheduling output unit transmits the dynamic panoramic image to the inspection terminal for 3D rendering. The inspection terminal displays the image based on the panoramic pixels in the dynamic panoramic image and establishes a jumpable spatial association based on the workstation unit identifier, image source, and target coordinates in the index table.

[0191] In one implementation, the 3D site model of the charging station is constructed based on the actual size of the site. A lightweight 3D modeling method is used to generate 3D models of fixed structures such as the site ground, charging terminals, crash barriers, and parking lines. The size and spatial position of the model are aligned with the site coordinate system. Texture mapping between the 2D panoramic pixels and the 3D model uses UV mapping, applying dynamic panoramic images as texture maps to the ground plane and fixed structure surfaces of the 3D model, with the mapped coordinates corresponding one-to-one with the site coordinate system. 3D rendering is implemented using the OpenGL graphics interface. By loading the 3D site model and dynamic panoramic textures, 3D visualization rendering of the site scene is achieved. The inspection terminal can browse the 3D scene from different perspectives by adjusting the position and orientation of the virtual camera.

[0192] In one implementation, the jumpable spatial association adopts a click-triggered method. When the user clicks on the abnormal target icon in the dynamic panoramic image, the inspection terminal automatically jumps to the magnified detail of the corresponding abnormal target. The target screen is a real-time video stream captured by the corresponding camera, with the center of the screen aligned with the location of the abnormal target.

[0193] In response to the generated jump command, the scheduling output unit sends the jump command to the inspection terminal. The inspection terminal reads the observation orientation from the jump command, rotates the panoramic field of view according to the observation orientation, and places the image area corresponding to the target coordinates in the center of the screen.

[0194] In one implementation, the initial viewing angle of the panoramic field of view is a top-down view directly above the station, and the field of view angle is [value missing]. The rotation reference coordinate system is the panoramic coordinate system, and the rotation operation is performed around the vertical and horizontal axes, supporting horizontal rotation. Degree rotation and vertical positive and negative Tilting adjustment. The inspection terminal adopts a touch-screen tablet hardware form factor, is compatible with the Android operating system, receives dynamic panoramic image data and jump commands via wireless LAN, and has a rendering refresh rate of no less than [missing information]. frame.

[0195] The inspection terminal displays an identifier, anomaly type, and target coordinates next to the image area corresponding to the target coordinates to pinpoint the abnormal target. For cable anomalies, the inspection terminal displays the identifier of the workstation unit to which the fixed end belongs and the target coordinates corresponding to the cable's perpendicular point or boundary crossing point; for occupancy anomalies or facility anomalies, the inspection terminal displays the identifier of the workstation unit where the center of gravity is located and the target coordinates corresponding to the center of gravity.

[0196] Through the above methods, the inspection monitoring and management module can identify inspection anomalies in the dynamic panoramic image to determine the abnormal target and obtain the target coordinates. The anomaly binding unit can bind the abnormal target to the corresponding workstation unit and target coordinates and generate a jump instruction. The scheduling output unit can output the dynamic panoramic image and jump instruction to the inspection terminal to lock the abnormal target.

[0197] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A new energy charging station inspection and management system based on multi-view feature fusion, applied to new energy charging stations, wherein the new energy charging stations are equipped with charging stations, inspection terminals, and cameras, characterized in that, include: The inspection monitoring and management module is used to identify inspection anomalies in the dynamic panoramic image to determine the abnormal targets and obtain the target coordinates, so as to carry out the inspection and maintenance management of the new energy charging station. It includes an anomaly binding unit and a scheduling output unit. The inspection monitoring and management module is configured to identify inspection anomalies in the dynamic panoramic image by executing the following steps, in order to perform inspection and maintenance management of the new energy charging station, including: Establish a station coordinate system and register the charging station as a station unit, and lock the camera to generate an image-to-physical coordinate mapping relationship; The camera's multiple video streams are acquired in real time to generate an image set arranged in time slices; Within the workstation unit, a fixed structure and a dynamic structure are decomposed, and the center line of the gun line, including the fixed end and the vehicle-side charging end, is extracted from the dynamic structure. The fusion weights of the cameras are calculated based on cross-workstation features to aggregate panoramic pixels and determine the target coordinates; Based on the panoramic pixels, the stitching seams are constrained by the boundary of the workstation unit and the center line of the gun line to generate the dynamic panoramic image. Identify the inspection anomaly in the dynamic panoramic image to determine the anomaly target; An abnormal binding unit is used to bind the abnormal target to the corresponding workstation unit defined by the charging workstation and the target coordinates, and generate a jump instruction; The scheduling output unit is used to output the dynamic panoramic image and the jump command to the inspection terminal to lock the abnormal target.

2. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 1, characterized in that, The inspection monitoring and management module is configured to identify inspection anomalies in the dynamic panoramic image by performing the following steps, in order to perform inspection and maintenance management of the new energy charging station, and further includes: The exception binding unit binds the exception target to the corresponding workstation unit and the target coordinates to generate the jump instruction; The scheduling output unit outputs the dynamic panoramic image and the jump command to the inspection terminal.

3. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 2, characterized in that, Establish a site coordinate system, register the charging station as the station unit, and lock the camera to generate an image-to-physical coordinate mapping relationship, including: The station coordinate system is established based on the physical structure of the charging station; Each of the charging stations is defined as a station unit, and the physical boundaries and fixed structures of the station units are registered. Obtain the device parameters of the camera and bind them to the corresponding covered workstation unit; Extract the pixel positions of the structural anchor points of the fixed structure and correlate the pixel positions of the structural anchor points with physical points in the station coordinate system; A planar mapping table for images to ground coordinates in a panoramic coordinate system, a panoramic mapping table for images to panoramic coordinates, and an index table for associating the workstation unit and the structural anchor point are generated for the camera, serving as the mapping relationship.

4. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 3, characterized in that, Real-time acquisition of multiple video streams from the camera to generate an image set arranged by time slices, including: Frame sequences are extracted from the camera to generate time slices with errors within a preset time threshold; Perform lens distortion correction on the video frames within the time slice; The index table is used to crop the coverage area corresponding to the workstation unit to obtain a partial image; The brightness of the local image is normalized according to the preset reference brightness of the camera; The image blur, structural redundancy, and occlusion ratio of the local image are obtained and written into the quality tag; The partial image, the workstation unit, and the quality marker are combined to form the image set.

5. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 4, characterized in that, Within the workstation unit, a fixed structure and a dynamic structure are decomposed, and the center line of the charging gun, including the fixed end and the vehicle-side charging end, is extracted from the dynamic structure, including: Project the image set onto the local coordinate system of the corresponding workstation unit; The overlapping textures in the local coordinate system are erased according to the preset background template to extract the dynamic structure; Continuous edges and strip-shaped regions are extracted from the dynamic structure to form candidate line segments; Determine whether the candidate line segment passes through the pile-side gun seat, extends to the vehicle side, and is parallel to the extension direction of the fixed structure; Based on the judgment result, the center line of the gun wire is extracted from the candidate line segment, and the fixed end, the vehicle-side charging end, and the direction of the center line of the gun wire are determined.

6. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 5, characterized in that, Calculating the fusion weights of the cameras based on cross-workstation features to aggregate panoramic pixels and determine the target coordinates includes: The pixel distance from the fixed end and the vehicle-side charging end to the center line of the gun line is combined to resolve the endpoint fit. Based on the angle between the centerline direction and the extension direction of the fixed structure, the analytical direction deviation is determined. Topological feature values ​​are constructed based on the endpoint fit and the directional deviation. The fusion weights are constructed by combining the topological feature values, the structural repetition in the quality markers, and the image blur. The image features and pixels from each viewpoint are aggregated using the fusion weights to render the panoramic pixels; The pixel coordinates of the inspected object at each viewpoint are reverse mapped and aggregated using the fusion weights to obtain the target coordinates.

7. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 6, characterized in that, Based on the panoramic pixels, the dynamic panoramic image is generated by using the path constraints of the workstation unit boundary and the center line of the gun line to create the stitching seam, including: The workstation unit is deployed to the panoramic coordinate system according to the preset site orientation; The cost of crossing the gun line, the dynamic structure, and the fixed structure is analyzed to determine the path splicing process. Select a path that is distributed along the physical boundary and whose crossing cost meets the set criteria as the determining seam; The panoramic pixels of the corresponding area are extracted according to the determined stitching seam to generate the dynamic panoramic image; The identifier of the workstation unit, the image source, and the target coordinates are stored in the index table, and together with the extracted panoramic pixels, they form the dynamic panoramic image.

8. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 7, characterized in that, The inspection anomaly is identified in the dynamic panoramic image, and the anomaly binding unit binds the anomaly target to the corresponding workstation unit and the target coordinates to generate the jump instruction, including: The abnormal target is located and the type of abnormality is determined by scanning the workstation unit in the dynamic panoramic image. If the anomaly type is a cable anomaly, the cable perpendicular point or boundary crossing point of the anomaly target is used as the target coordinates and bound to the workstation unit to which its fixed end belongs. If the anomaly type is a occupancy anomaly or a facility anomaly, obtain the center of gravity of the anomaly target and bind it as the target coordinate to the workstation unit where the center of gravity is located; Extract the identifier of the bound workstation unit, the anomaly type, and the target coordinates; The reverse extension line of the line connecting the target coordinates and the camera coordinates is used as the observation direction, and the identifier, the anomaly type, the target coordinates and the observation direction are combined to form the jump command.

9. The new energy charging station inspection and management system based on multi-view feature fusion according to claim 8, characterized in that, The scheduling output unit outputs the dynamic panoramic image and the jump command to the inspection terminal, including: The dynamic panoramic image is transmitted to the inspection terminal for 3D rendering. In response to generating the jump instruction, the jump instruction is sent to the inspection terminal; The inspection terminal rotates the panoramic field of view angle according to the observation orientation in the jump command; The image region corresponding to the target coordinates is placed in the center of the screen for display. The identifier, the anomaly type, and the target coordinates are displayed together next to the image area to locate the anomaly target.

Images