Isolator position posture visual identification method and system

Abstract

The application relates to the technical field of visual identification, and discloses a disconnecting switch position and posture visual identification method and system. The method comprises the following steps: acquiring a continuous image sequence of a disconnecting switch action process and identifying a switch moving arm, and constructing an initial two-dimensional space coordinate sequence; performing motion state prediction and coordinate correction on an abnormal coordinate position to obtain a corrected two-dimensional space coordinate sequence; performing space coordinate conversion and trajectory interpolation to obtain a continuous three-dimensional motion trajectory; calculating an action posture angle according to a starting trajectory position and a terminal trajectory position to determine an angle to position state; and performing trajectory fluctuation analysis and stability checking on the continuous three-dimensional motion trajectory to obtain a posture interpretation result. The method can solve the technical problem of low accuracy of disconnecting switch action to position posture interpretation in a complex outdoor environment.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, and in particular to a method and system for visual recognition of the position and posture of a disconnector switch. Background Technology

[0002] Currently, disconnect switches are widely used in power systems such as substations to achieve reliable isolation between electrical equipment and live lines. Whether their opening and closing actions reach the specified positions directly affects equipment maintenance and operational status assessment. As substations develop towards unmanned operation and intelligent inspection, acquiring images of disconnect switch actions through monitoring equipment and automatically interpreting the switch position and attitude using image recognition technology is gradually becoming an important method for disconnect switch status detection.

[0003] In one existing technology, images of the disconnector switch in the open or closed state are captured by a camera device. The operation of the disconnector switch is determined based on the outline and position of the switch boom, or its matching relationship with a preset template. Another approach involves target detection and position tracking of the switch boom in continuous video, determining the disconnector switch's operational status based on the movement or termination position in the two-dimensional image. However, in outdoor substation environments, changes in lighting, metal reflections, equipment obstructions, and video frame drops can easily cause short-term loss or drift in the target position of the switch boom, resulting in jumps and discontinuities in the two-dimensional position extracted from continuous images. Furthermore, relying solely on the two-dimensional termination position is insufficient to fully reflect the rotation angle of the switch boom in real space and the stability of the entire operation process, easily leading to misinterpretations when the boom angle is insufficient, the movement is jittery, or abnormalities occur midway.

[0004] Therefore, existing technologies suffer from the technical problem of low accuracy in judging the position of disconnect switches in complex outdoor environments. Summary of the Invention

[0005] This invention provides a visual recognition method and system for the position and posture of disconnect switches, in order to solve the technical problem of low accuracy in judging the position and posture of disconnect switches in complex outdoor environments in the prior art.

[0006] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a visual recognition method for the position and posture of a disconnector switch, comprising: A continuous image sequence corresponding to the action process of the disconnecting switch is obtained, and the switch boom in the continuous image sequence is identified to obtain a two-dimensional bounding box sequence. An initial two-dimensional spatial coordinate sequence is constructed according to the image time sequence based on the bounding box positions in the two-dimensional bounding box sequence. The initial two-dimensional spatial coordinate sequence is subjected to coordinate continuity analysis to determine the abnormal coordinate positions. The preceding valid coordinates corresponding to the abnormal coordinate positions are used as state references to predict the motion state of the abnormal coordinate positions and obtain predicted coordinates. The initial two-dimensional spatial coordinate sequence is then corrected based on the predicted coordinates to obtain the corrected two-dimensional spatial coordinate sequence. Based on the pre-established pixel depth correspondence, the corrected two-dimensional spatial coordinate sequence is transformed to obtain a three-dimensional spatial coordinate sequence. The three-dimensional spatial coordinate sequence is then subjected to trajectory interpolation to obtain a continuous three-dimensional motion trajectory. The starting trajectory position is determined based on the continuous three-dimensional motion trajectory. Using the starting trajectory position as the attitude reference, the spatial angle of the ending trajectory position of the continuous three-dimensional motion trajectory is calculated to obtain the action posture angle. The angle position is determined based on the comparison between the action posture angle and the preset angle threshold. The continuous three-dimensional motion trajectory is analyzed for trajectory fluctuation to obtain trajectory fluctuation data. The stability of the angle position is verified based on the trajectory fluctuation data to obtain the attitude judgment result of the disconnecting switch.

[0007] In a second aspect, the present invention provides a visual recognition system for the position and posture of a disconnector switch, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0008] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0009] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention identifies the target of the boom in a continuous image sequence and constructs an initial two-dimensional spatial coordinate sequence according to the image time sequence. Coordinates that exceed the possible range of mechanical movement are filtered out, and the local motion state is estimated and the corresponding position is predicted. Thus, the impact of missing coordinates and non-physical position jumps on the continuity of the two-dimensional trajectory can be reduced, while retaining the real trajectory changes within the possible range of mechanical movement, providing a continuous position data basis for subsequent three-dimensional motion analysis.

[0010] (2) This invention uses a pre-established pixel depth correspondence to sequentially perform pixel coordinate correction and coordinate system transformation on the corrected two-dimensional spatial coordinates to obtain a three-dimensional spatial coordinate sequence; further, it performs trajectory fitting based on the sampling interval between the three-dimensional coordinates to obtain a continuous three-dimensional motion trajectory. Through the above operations, the entire spatial motion process of the disconnector boom from the starting position to the ending position can be continuously restored, providing a complete three-dimensional trajectory basis for accurate analysis of the action posture angle and action stability.

[0011] (3) This invention uses the starting position of the continuous three-dimensional motion trajectory as the attitude reference, performs displacement calculation and action plane projection on the ending trajectory position, and calculates the action attitude angle in combination with the preset action direction of the disconnecting switch. The angle arrival state is determined according to the action attitude angle and the preset angle threshold. Furthermore, the discreteness of the continuous three-dimensional motion trajectory is quantified to obtain trajectory fluctuation data, and the angle arrival state is verified according to the trajectory stability. Through the above operations, the angle arrival status of the ending position and the motion stability of the entire action process are used together as the attitude judgment basis, avoiding misjudgment caused by relying solely on the static endpoint position. It can also identify abnormal states where the attitude angle meets the arrival condition but the direction deviation changes significantly during the action process, and there are obvious trajectory fluctuations. This solves the technical problem of low accuracy in judging the action arrival attitude of disconnecting switches in complex outdoor environments. Attached Figure Description

[0012] Figure 1 This is a schematic flowchart of the visual recognition method for the position and posture of a disconnecting switch provided in the first embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] Reference Figure 1 The first embodiment of the present invention provides a visual recognition method for the position and posture of a disconnector switch, comprising the following steps: S1, acquire a continuous image sequence corresponding to the action process of the disconnecting switch, perform target recognition on the switch boom in the continuous image sequence to obtain a two-dimensional bounding box sequence, and construct an initial two-dimensional spatial coordinate sequence according to the image time sequence based on the bounding box positions in the two-dimensional bounding box sequence. S2, perform coordinate continuity analysis on the initial two-dimensional spatial coordinate sequence to determine the abnormal coordinate position, use the preceding valid coordinates corresponding to the abnormal coordinate position as the state reference, predict the motion state of the abnormal coordinate position to obtain the predicted coordinates, and perform coordinate correction on the initial two-dimensional spatial coordinate sequence according to the predicted coordinates to obtain the corrected two-dimensional spatial coordinate sequence. S3, perform spatial coordinate transformation on the corrected two-dimensional spatial coordinate sequence according to the pre-established pixel depth correspondence to obtain a three-dimensional spatial coordinate sequence, and perform trajectory interpolation on the three-dimensional spatial coordinate sequence to obtain a continuous three-dimensional motion trajectory. S4. Determine the starting trajectory position based on the continuous three-dimensional motion trajectory. Using the starting trajectory position as the attitude reference, calculate the spatial angle between the ending trajectory position of the continuous three-dimensional motion trajectory to obtain the action posture angle. Determine the angle in place based on the comparison between the action posture angle and the preset angle threshold. S5, perform trajectory fluctuation analysis on the continuous three-dimensional motion trajectory to obtain trajectory fluctuation data, and perform stability verification on the angle positioning state based on the trajectory fluctuation data to obtain the attitude judgment result of the disconnecting switch.

[0015] In step S1, a continuous image sequence corresponding to the operation of the disconnecting switch is acquired. Specifically, a camera is fixedly installed in the operating area of ​​the disconnecting switch, ensuring that the effective movement range of the switch arm from the open position to the closed position is within the imaging area of ​​the camera. When the disconnecting switch begins to perform an opening or closing action, continuous image acquisition is initiated and continues until the end of the current action. The image frame identifier and acquisition timestamp are recorded for each acquired image in chronological order to form the continuous image sequence. The installation position of the camera, image resolution, and acquisition frame rate remain unchanged under the same online recognition configuration. When the installation position of the camera or the acquisition configuration changes, the calibration corresponding to that configuration is re-performed.

[0016] In one implementation, target recognition is performed on the switch boom in the continuous image sequence to obtain a two-dimensional bounding box sequence. An initial two-dimensional spatial coordinate sequence is then constructed based on the bounding box positions in the two-dimensional bounding box sequence according to the image temporal sequence, including: The continuous image sequence is subjected to boom visual feature extraction processing to obtain boom feature data; Based on the boom feature data, candidate region localization processing is performed to obtain a set of candidate bounding boxes; The candidate bounding box set is subjected to bounding box filtering processing to obtain a two-dimensional bounding box sequence; The two-dimensional bounding box sequence is subjected to geometric center localization processing to obtain a center coordinate sequence; The center coordinate sequence is time-marked according to the image time sequence of the continuous image sequence to obtain the initial two-dimensional spatial coordinate sequence.

[0017] It should be noted that when performing boom visual feature extraction processing on the continuous image sequence, images are read frame by frame according to the image time sequence. Each image is scaled to 640×640 pixels before being input into the YOLOv5s target detection network. The YOLOv5s target detection network includes a backbone feature extraction network, a feature fusion network, and a detection head. The backbone feature extraction network consists of convolutional layers and a CSP structure, used to extract the edges, rod contours, and local surface texture responses of the boom from the input images. The feature fusion network fuses features at different scales, outputting feature maps at three scales. The feature maps at these three scales and their corresponding image frame identifiers are combined to form boom feature data, which is used for subsequent candidate region localization processing.

[0018] Next, candidate region localization processing is performed based on the boom feature data. The candidate region localization processing uses the detection head of the YOLOv5s network, which outputs rectangular regions that may contain the boom, target category, and target category confidence score based on the boom feature maps at three scales, and forms a candidate bounding box set by combining all rectangular regions.

[0019] The target detection network was trained offline before online recognition. Training samples consisted of images of the same type of disconnector switch captured from the same camera installation angle as the online monitoring system, covering the open, intermediate, and closed positions. The average grayscale value of the images (30-220) represented different lighting conditions, and the area of ​​the switch boom obscured (0%-40% of the actual bounding box area) represented unobstructed and partially obstructed conditions. Metal reflections and different background samples were also included. The switch booms in the training images were labeled with target categories and actual bounding boxes. Training samples and independent validation samples were divided into training and validation samples at an 8:2 action batch ratio, with adjacent images of the same action not simultaneously assigned to training and validation samples. The SGD optimizer was used during training, with an initial learning rate of 0.01, a batch size of 16, a maximum training epoch of 200, and binary cross-entropy loss for classification and target confidence loss, while CIoU loss was used for bounding box regression. Training stopped when the total loss corresponding to the validation samples did not decrease for 20 consecutive epochs, and the model parameters with the best validation performance were saved. The model parameters remain fixed during the online recognition phase after training and are used only for boom visual feature extraction and candidate region localization.

[0020] When performing bounding box filtering on the candidate bounding box set, firstly, low-confidence candidate bounding boxes that do not meet the requirements for boom recognition are deleted based on the target confidence level. Then, the spatial overlap of the remaining candidate bounding boxes is calculated. In cases where multiple bounding boxes point to the same boom region, the candidate bounding boxes with higher target confidence are retained. Non-maximum suppression is then performed based on the intersection-union ratio (IUU) between the candidate bounding boxes to remove duplicate candidate boxes that highly overlap with the retained boxes, thus ensuring that the same boom in the current image corresponds to a single valid bounding box.

[0021] The target confidence screening parameters and the cross-union ratio (CUP) judgment parameters used for non-maximum suppression are determined based on the aforementioned independent validation samples in the following manner: On the independent validation samples, the target confidence threshold is iterated within the range of 0.3 to 0.9 with a step size of 0.05, and the CUP threshold is iterated within the range of 0.3 to 0.7 with a step size of 0.05. For each set of parameter combinations, the accuracy of boom detection is calculated, and the set of parameters with the highest accuracy among those that meet the boom detection requirements is selected as the target confidence screening parameters and CUP judgment parameters. The determined parameter combinations are available for online recognition under the corresponding camera equipment and installation viewpoint; when the camera equipment, installation viewpoint, or target detection network is retrained, the corresponding parameters are re-determined in the above manner.

[0022] To ensure that the 2D bounding box sequence corresponds to the same physical switch boom, when processing subsequent images according to image time sequence, the valid bounding boxes obtained from the current image after filtering are positionally associated with the switch boom bounding boxes in the previous valid image. When multiple valid candidate boxes exist, a bounding box continuous with the previous switch boom position is selected based on the spatial overlap relationship and target confidence, and the same target label is maintained along the continuous image sequence. If a valid switch boom bounding box is not obtained in the current image due to complete occlusion or severe visual interference, a bounding box from another equipment area is not used as a substitute; instead, a missing bounding box marker is retained at the current image position. Valid bounding boxes and their corresponding missing states are recorded sequentially according to image time sequence to obtain the 2D bounding box sequence.

[0023] When performing geometric center localization on the two-dimensional bounding box sequence, the top-left corner of the image is used as the origin of the two-dimensional pixel coordinates. A horizontal coordinate axis is established along the horizontal direction of the image, and a vertical coordinate axis is established along the vertical direction of the image, with the coordinate unit being pixels. For each valid bounding box in the two-dimensional bounding box sequence, the horizontal and vertical pixel coordinates corresponding to its top-left and bottom-right corners are read. The midpoint of the two horizontal boundary coordinates is taken as the center horizontal coordinate, and the midpoint of the two vertical boundary coordinates is taken as the center vertical coordinate, thereby determining the geometric center position of the boom bounding box in the current image. All valid geometric center positions are recorded according to the arrangement order of the two-dimensional bounding box sequence to obtain the center coordinate sequence.

[0024] It should be noted that the geometric center position is used as a two-dimensional visual tracking reference position for the boom in continuous images. It represents the position benchmark of the current effective bounding box of the boom in the image plane, and does not represent the position of the boom's mechanical axis. Subsequent coordinate continuity analysis, depth matching, and 3D trajectory construction all use the two-dimensional visual tracking reference position, ensuring consistent position definitions across different processing steps. For images marked as having missing bounding boxes, geometric center coordinates are not artificially generated. Instead, the missing coordinate markers are retained at the same sequence positions in the center coordinate sequence, enabling subsequent coordinate continuity analysis to distinguish between normally detected coordinates and image positions where no effective observations were obtained.

[0025] When performing time-series marking processing on the center coordinate sequence based on the image time sequence of the continuous image sequence, the frame number and acquisition timestamp corresponding to each image in the continuous image sequence are read, and a one-to-one correspondence is established between the effective geometric center coordinates and the frame number and acquisition timestamp of their source images. For images with missing coordinate markers, the corresponding frame number and acquisition timestamp are also retained, so that the missing coordinate positions still occupy the time position in the actual action process. The data nodes are arranged in order of acquisition timestamps from first to last to obtain the initial two-dimensional spatial coordinate sequence.

[0026] Each data node in the initial two-dimensional spatial coordinate sequence includes at least an image frame identifier, a timestamp, a boom target identifier, two-dimensional center coordinates, and a coordinate validity status. Valid data nodes record the corresponding horizontal and vertical pixel coordinates, while missing data nodes record the coordinate missing status without assigning fictitious position values. Therefore, during subsequent coordinate continuity analysis, the spatial changes between adjacent valid coordinates can be calculated according to the actual image time sequence. Simultaneously, coordinate positions without valid observation results can be directly identified, and further coordinate correction can be performed by subsequent motion state prediction steps.

[0027] In step S2, coordinate continuity analysis is performed on the initial two-dimensional spatial coordinate sequence to determine the locations of abnormal coordinates, including: Extract adjacent coordinate points from the initial two-dimensional spatial coordinate sequence according to the image time sequence; The spatial distance between the adjacent coordinate points is calculated to obtain the distance between adjacent coordinates; The distance between adjacent coordinates is compared with a preset distance threshold to obtain the coordinate continuity determination result; Based on the coordinate continuity determination result, the initial two-dimensional spatial coordinate sequence is filtered for coordinate positions to determine abnormal coordinate positions.

[0028] Specifically, when extracting adjacent coordinate points from the initial two-dimensional spatial coordinate sequence according to the image time sequence, adjacent data nodes are read sequentially according to the image frame identifier and timestamp carried by each data node, in order of the timestamps from first to last. If both adjacent data nodes have valid two-dimensional center coordinates, the horizontal and vertical pixel coordinates corresponding to the preceding data node are combined with those corresponding to the following data node to form a set of adjacent coordinate points, and the preceding and following timestamps corresponding to this set of coordinate points are recorded. If the current data node carries a missing coordinate marker, the position of this data node in the original image time sequence is retained and used as the coordinate position to be filtered, without pre-filling the position using valid preceding and following coordinates.

[0029] When calculating the spatial distance between adjacent coordinate points, the changes in the horizontal and vertical coordinates of the subsequent coordinate point relative to the preceding coordinate point are calculated in the same two-dimensional image coordinate system. The Euclidean distance between two adjacent valid coordinate points is determined based on the coordinate changes in both directions. The unit of the adjacent coordinate distance is pixels, which describes the degree of two-dimensional positional change of the geometric center corresponding to the boom between adjacent image times. The obtained Euclidean distances are recorded according to the time sequence corresponding to the adjacent coordinate points, and each adjacent coordinate distance is associated with the data node where the subsequent coordinate point is located to obtain the adjacent coordinate distance used for continuity judgment.

[0030] It should be noted that the preset distance threshold is determined as follows: After the camera equipment and disconnector are installed, the disconnector is driven to complete no less than 30 normal opening and closing actions, corresponding continuous image sequences are acquired and target recognition is performed, the pixel Euclidean distance between adjacent valid coordinate points is calculated, and the mean and standard deviation of adjacent coordinate distances under normal operating conditions are statistically analyzed. The mean plus three times the standard deviation is used as the preset distance threshold. The preset distance threshold is used to filter out discontinuous coordinate jumps that exceed the normal mechanical movement range. When the acquisition frame rate of the camera equipment, the installation distance between the camera equipment and the disconnector, the installation angle of the camera equipment, or the model of the disconnector changes, the corresponding preset distance threshold is re-determined in the above manner.

[0031] When comparing the distance between adjacent coordinates with a preset distance threshold, the distance between each adjacent coordinate is read sequentially according to the image time sequence and compared with the preset distance threshold. When the distance between adjacent coordinates does not reach the preset distance threshold, it indicates that the two-dimensional position change of the subsequent coordinate point relative to the previous coordinate point does not exceed the continuous change range allowed by the current mechanical movement and visual measurement conditions of the disconnecting switch, and the corresponding subsequent data node is marked as a coordinate continuous state. When the distance between adjacent coordinates reaches the preset distance threshold, it indicates that the current coordinate change has exceeded the reasonable two-dimensional displacement range that can be formed between adjacent acquisition times, and the corresponding subsequent data node is marked as a coordinate abnormal state, thereby obtaining the coordinate continuity determination result.

[0032] It should be noted that the aforementioned initial two-dimensional spatial coordinate sequence already retains the coordinate missing markers formed when a valid boom boundary box was not obtained. Therefore, the coordinate continuity analysis handles both coordinate missing and coordinate jump anomalies. For data nodes with valid two-dimensional coordinates, coordinate drift caused by false detection, strong reflection, or partial occlusion is identified by comparing the distance between adjacent coordinates with a preset distance threshold. For data nodes carrying coordinate missing markers, distance calculations are not performed manually on non-existent two-dimensional coordinates; instead, they are directly identified as locations where coordinate continuity is interrupted. This avoids assigning zero values ​​or other fictitious coordinates to missing coordinates before participating in distance calculations, which could lead to incorrect judgments of abnormal locations.

[0033] When filtering the initial two-dimensional spatial coordinate sequence based on the coordinate continuity determination result, the validity status of the coordinates of each data node and the corresponding coordinate continuity determination result are checked according to the original image time sequence. Data nodes carrying coordinate missing markers are identified as first-type abnormal coordinate positions, data nodes with valid coordinates but whose corresponding adjacent coordinate distances reach a preset distance threshold are identified as second-type abnormal coordinate positions, and the remaining data nodes with valid coordinates and normal continuity determination are identified as valid coordinate positions. The first-type and second-type abnormal coordinate positions are recorded according to the original image frame identifier and timestamp to obtain the abnormal coordinate positions.

[0034] The abnormal coordinate positions retain at least the corresponding image frame identifier, timestamp, and abnormal type. The first type of abnormal coordinate position corresponds to the coordinate missing state, and the second type of abnormal coordinate position corresponds to the coordinate discontinuity state that exceeds the preset distance threshold, which can be used for subsequent motion state prediction and coordinate correction.

[0035] In one implementation, using the preceding valid coordinates corresponding to the abnormal coordinate position as a state reference, motion state prediction is performed on the abnormal coordinate position to obtain predicted coordinates. The initial two-dimensional spatial coordinate sequence is then corrected based on the predicted coordinates to obtain a corrected two-dimensional spatial coordinate sequence, including: Based on the abnormal coordinate positions, extract the preceding valid coordinate data from the initial two-dimensional spatial coordinate sequence; The preceding valid coordinate data is subjected to motion state estimation processing to obtain motion state data; Based on the motion state data, perform current position prediction processing to obtain predicted coordinates; The abnormal coordinate positions are updated based on the predicted coordinates to obtain corrected coordinates. The initial two-dimensional spatial coordinate sequence is reconstructed based on the corrected coordinates to obtain the corrected two-dimensional spatial coordinate sequence.

[0036] Specifically, when extracting preceding valid coordinate data from the initial two-dimensional spatial coordinate sequence based on the abnormal coordinate location, the image frame identifier and timestamp corresponding to the abnormal coordinate location are used as indexes to read the valid two-dimensional coordinate nodes along the image time sequence, while maintaining the correlation between the horizontal pixel coordinates, vertical pixel coordinates, and corresponding timestamps of each valid coordinate node. At least two preceding valid coordinates that are closest to the current abnormal location and are temporally consecutive are extracted first, and the correlation between the horizontal pixel coordinates, vertical pixel coordinates, and timestamps of each valid coordinate is maintained, so that adjacent valid coordinates can be used to determine the positional state and two-dimensional motion velocity before the anomaly occurred.

[0037] When performing motion state estimation processing on the preceding valid coordinate data, the lateral position, longitudinal position, lateral velocity, and longitudinal velocity of the boom in the two-dimensional image plane are used as motion state variables. The actual time interval is determined based on the timestamps of adjacent preceding valid coordinates, and the lateral velocity is determined based on the change in lateral coordinate and the actual time interval, and the longitudinal velocity is determined based on the change in longitudinal coordinate and the actual time interval, thereby forming the initial motion state characterizing the two-dimensional position and local motion trend of the boom before the anomaly occurs.

[0038] In one implementation, the initial motion state is continuously updated using Kalman state estimation. Each normal valid coordinate is used as the current position observation. Based on a local uniform motion model, the prior state at the current valid moment is calculated according to the posterior motion state of the previous valid moment. The prior state is then corrected using the observation residual between the current valid coordinate and the prior position. As valid image frames are input sequentially, the motion state data corresponding to the most recent moment before the abnormal coordinate position is obtained. The motion state data includes at least the current lateral position, longitudinal position, lateral motion velocity, longitudinal motion velocity, and the corresponding state moment.

[0039] It should be noted that the local uniform motion model is used to describe the continuous change of the two-dimensional projection position of the switch boom within adjacent short image acquisition periods, and does not mean that the switch boom moves in a straight line in space during the complete opening and closing process. The disconnector switch boom can actually form an arc trajectory along the mechanical axis. This step only uses the two-dimensional motion trend observed in the period adjacent to the abnormal position to predict the short-term missing position. The complete spatial motion trajectory will still be restored through three-dimensional coordinate transformation and trajectory interpolation.

[0040] When using Kalman state estimation, the process noise parameters and measurement noise parameters are determined as follows: After the camera equipment and disconnector are installed, the disconnector is driven to complete no less than 20 normal opening and closing actions, and the corresponding continuous image sequences are acquired and target recognition is performed to obtain a two-dimensional coordinate sequence under normal operating conditions. The coordinate sequences of each action are input into a Kalman filter for smoothing estimation, and the residuals between the observed values ​​and the estimated values ​​at each time point are calculated. The variance of the residuals is used as the measurement noise parameter. The deviations between the estimated values ​​and the predicted values ​​based on the uniform motion model at each time point are calculated, and the variance of the deviations is used as the process noise parameter. The average value of the parameters obtained from multiple normal actions is taken as the final parameter value used in the online recognition stage. When the installation status of the camera equipment, the acquisition configuration, or the model of the disconnector changes, the corresponding process noise parameters and measurement noise parameters are re-determined according to the above method.

[0041] When performing current position prediction processing based on the motion state data, the timestamp corresponding to the current abnormal coordinate position is read, and the time difference between this timestamp and the corresponding state time in the motion state data is calculated. Based on this time difference, the aforementioned lateral and longitudinal motion velocities are advanced over time. The predicted lateral coordinate is obtained by adding the lateral position in the motion state data to the displacement formed by the lateral motion velocity within the time difference, and the predicted longitudinal coordinate is obtained by adding the longitudinal position to the displacement formed by the longitudinal motion velocity within the time difference. Thus, even if the actual sampling time of consecutive images fluctuates, the predicted position is still calculated based on the actual timestamp difference, rather than assuming that all adjacent images have exactly the same fixed time interval.

[0042] For situations with multiple consecutive anomalous coordinate locations, the motion state corresponding to the last valid observation location is used as the initial state. State predictions are then performed sequentially based on the timestamps of each anomalous location. The predicted state generated at the previous anomalous location can be used to advance the time to the next anomalous moment, but the corresponding predicted coordinates retain their correction flags and are not reused as valid observations in the observation update. This continues until valid coordinates that meet the coordinate continuity requirement are subsequently obtained, at which point the observation update is performed using those valid coordinates. This approach avoids the erroneous compression of prediction errors caused by repeatedly inputting predicted values ​​as actual measurements during periods of continuous missing values.

[0043] When updating the abnormal coordinate positions based on the predicted coordinates, processing is performed according to the abnormality type carried by the abnormal coordinate positions. For coordinate missing type, since there are no valid two-dimensional observation coordinates at the current image time, the predicted horizontal and vertical coordinates corresponding to the current time are directly written into the abnormal coordinate positions to form corresponding corrected coordinates; for coordinate drift type, since the current observation coordinates have already been determined in the aforementioned coordinate continuity analysis to not meet the requirements of normal mechanical motion continuity, the drifted coordinates are not used as the current position, and the predicted coordinates corresponding to the current time are used to replace the drifted coordinates to obtain corrected coordinates.

[0044] It should be noted that the coordinate correction is always performed in the aforementioned two-dimensional image pixel coordinate system. The corrected coordinates still represent the horizontal and vertical pixel positions of the geometric center of the boom detection area in the image plane. This step only completes the abnormal two-dimensional observation positions based on the temporal motion state.

[0045] When reconstructing the initial two-dimensional spatial coordinate sequence based on the corrected coordinates, the initial two-dimensional spatial coordinate sequence is read sequentially according to the original image frame order. For data nodes with normal coordinate states, the original two-dimensional center coordinates remain unchanged; for abnormal data nodes, the corresponding corrected coordinates are written to the original abnormal coordinate positions, and the original image frame identifier and timestamp are retained. The coordinate source state of the current data node is further recorded to distinguish between the original valid coordinates and the predicted corrected coordinates. All data nodes are reorganized according to the chronological order of the original continuous image sequence to obtain the corrected two-dimensional spatial coordinate sequence.

[0046] The corrected two-dimensional spatial coordinate sequence maintains the same time index relationship as the original continuous image sequence. Each image moment corresponds to a defined two-dimensional coordinate node, and each node includes at least an image frame identifier, a timestamp, horizontal pixel coordinates, vertical pixel coordinates, and coordinate source status. Therefore, it can both fill in the two-dimensional coordinate gaps caused by occlusion and eliminate abnormal coordinate jumps caused by reflections or misidentification, while maintaining the pixel positions and time order required for subsequent pixel depth matching, providing continuous two-dimensional position input for further conversion to obtain a three-dimensional spatial coordinate sequence.

[0047] The above coordinate correction process is executed sequentially according to the timestamps corresponding to each abnormal coordinate position until valid observation coordinates that meet the coordinate continuity requirements are obtained again. Then, the observation update of the Kalman state is restored using the valid observation coordinates.

[0048] In step S3, the corrected two-dimensional spatial coordinate sequence is transformed into a three-dimensional spatial coordinate sequence based on the pre-established pixel depth correspondence, including: The corrected two-dimensional spatial coordinate sequence is subjected to pixel coordinate correction processing based on the pre-calibrated camera intrinsic parameters to obtain the corrected pixel coordinate sequence; The depth matching process is performed on the corrected pixel coordinate sequence according to the pre-established pixel depth correspondence to obtain a depth value sequence; Using the depth numerical sequence as a depth constraint, the corrected pixel coordinate sequence is subjected to spatial back projection processing to obtain a three-dimensional camera coordinate sequence; The coordinate system transformation is performed on the three-dimensional camera coordinate sequence based on the pre-calibrated camera extrinsic parameters to obtain a three-dimensional spatial coordinate sequence.

[0049] It should be noted that before recognizing the action of the disconnecting switch, the camera equipment used to acquire continuous image sequences is calibrated, and a pixel-depth correspondence is established while keeping the camera equipment's installation position fixed. This embodiment uses a fixed-installation monocular camera. The camera parameters obtained from the calibration include at least the focal length, principal point position, and lens distortion parameters, used to correct the distortion of the original pixel coordinates and establish an ideal pinhole imaging model. The camera extrinsic parameters describe the rotation and translation relationship of the camera coordinate system relative to the substation's reference coordinate system.

[0050] The pixel depth correspondence remains constant for the same type of disconnect switch, camera installation position, and imaging parameters. Furthermore, the geometric center reference point used for target recognition during switch boom operation is established under the condition that it moves within a pre-calibrated action plane under mechanical structural constraints, and is only used within the effective movement area corresponding to that action plane. The geometric center reference point is defined as a fixed physical point on the boom corresponding to the center pixel position of the boom detection bounding box. This fixed physical point is selected based on the disconnect switch model's boom centroid during system initialization and maintains the same physical correspondence in all subsequent processing steps. When the camera installation position, imaging parameters, disconnect switch model, or action plane changes, the camera and action plane calibration are re-executed, and the corresponding pixel depth correspondence is established.

[0051] Specifically, during the calibration phase, the boom shaft center and the normal motion plane are determined based on the disconnector's mechanical drawings or on-site measurements. At least three known spatial coordinate points within this motion plane are selected to determine its spatial position in the substation's reference coordinate system. For the correction pixel position within the effective motion area, a spatial line of sight is established from the camera's optical center through that pixel position based on the camera's intrinsic parameters. Then, the spatial line of sight is converted to the substation's reference coordinate system based on the camera's extrinsic parameters, and the intersection of the spatial line of sight and the calibrated motion plane is calculated. The coordinate value along the camera's optical axis after converting this intersection point to the camera coordinate system is used as the corresponding depth value. The correspondence between the correction pixel coordinates and depth values ​​is pre-calculated and saved according to the image area covered by the normal opening and closing movements. For correction pixel positions located between adjacent calibrated pixel positions, interpolation is performed based on the depth values ​​corresponding to nearby positions. This pixel depth correspondence is only used for online depth matching when the aforementioned fixed camera equipment and fixed motion plane conditions remain unchanged.

[0052] When performing pixel coordinate correction processing on the corrected two-dimensional spatial coordinate sequence based on pre-calibrated camera intrinsic parameters, the horizontal and vertical pixel coordinates of each two-dimensional coordinate node are read in chronological order. The pixel position deviation caused by lens imaging is corrected based on the imaging parameters obtained from camera calibration, transforming the original image coordinates to the pixel coordinates corresponding to the ideal pinhole imaging model. The corrected horizontal and vertical pixel coordinates and corresponding timestamps are recorded in chronological order of the original two-dimensional spatial coordinate sequence to obtain the corrected pixel coordinate sequence.

[0053] It should be noted that the two-dimensional spatial coordinates obtained in the aforementioned coordinate correction step still belong to the original image coordinate system of the camera device, while the pixel depth correspondence is established based on the pixel positions after camera calibration. Therefore, this step first performs pixel coordinate correction to ensure that the pixel position to be queried and the pixel depth correspondence use the same coordinate reference. The pixel coordinate correction is only used to eliminate positional deviations caused by the imaging parameters of the camera device, and does not change the image timing between coordinate nodes, nor does it infer the spatial depth of the boom during this process.

[0054] When performing depth matching processing on the corrected pixel coordinate sequence according to the pixel depth correspondence, each corrected pixel coordinate is used as a position index to retrieve the corresponding depth value from the pixel depth correspondence. For cases where the corrected pixel coordinate corresponds to an established pixel position, the depth value recorded at that position is read. For cases where the corrected pixel coordinate is located between adjacent calibration positions and within the aforementioned effective motion area, interpolation is performed based on neighboring calibration positions and their depth values ​​to determine the depth value corresponding to the current corrected pixel position. The depth values ​​are arranged according to the original time sequence of the corrected pixel coordinate sequence, ensuring a one-to-one correspondence between each depth value and its source pixel coordinate and timestamp, resulting in a depth value sequence.

[0055] The depth value represents the spatial distance relative to the camera coordinate system along the optical axis of the camera device, corresponding to the current 2D visual tracking reference position. It uses a length unit consistent with the subsequent 3D spatial coordinates. By maintaining the correspondence between the corrected pixel coordinates, depth value, and timestamp, mismatches between the depth data and the 2D position at other image times can be avoided, thus ensuring that each 2D trajectory node has a corresponding spatial depth constraint.

[0056] Using the depth numerical sequence as a depth constraint, when performing spatial backprojection processing on the corrected pixel coordinate sequence, the corrected pixel coordinates and corresponding depth values ​​are read according to the same timestamp, and the backprojection relationship from the two-dimensional pixel position to the three-dimensional camera coordinates is established using camera intrinsic parameters. Specifically, the corresponding three-dimensional camera coordinates can be determined according to the following formula: ; Where u and v are the corrected horizontal and vertical pixel coordinates of the current node, respectively, in pixels; and These are the horizontal and vertical equivalent focal lengths in the camera's intrinsic parameters, respectively, in pixels; and These are the horizontal and vertical principal point coordinates in the camera's intrinsic parameters, respectively, in pixels; Z is the depth value corresponding to the current corrected pixel coordinates. , and These represent the horizontal, vertical, and depth coordinates of the spatial point corresponding to the current 2D visual tracking reference position in the camera coordinate system.

[0057] Following the aforementioned back-projection relationship, all data nodes in the corrected pixel coordinate sequence are processed sequentially, while retaining the timestamps and original image frame identifiers corresponding to each three-dimensional coordinate node, thus obtaining the three-dimensional camera coordinate sequence. Therefore, the switch boom movement position in the aforementioned two-dimensional image plane is converted into a three-dimensional spatial position based on the optical center of the camera device, while maintaining the temporal sequence consistent with the original isolating switch operation process.

[0058] When performing coordinate system transformation on the 3D camera coordinate sequence based on pre-calibrated camera extrinsic parameters, the rotation parameters and translation parameters representing the attitude relationship of the camera coordinate system relative to the substation reference coordinate system are read from the camera extrinsic parameters, and the coordinate transformation is performed on each 3D camera coordinate node according to the following formula: ; in, This indicates the three-dimensional coordinates of the current boom center node in the camera coordinate system; represents the rotation matrix from the camera coordinate system to the substation reference coordinate system; t represents the translation vector of the position of the origin of the camera coordinate system in the substation reference coordinate system; This represents the three-dimensional spatial coordinates after the coordinate transformation. The rotation matrix and translation vector are both obtained from the extrinsic parameter calibration after the camera equipment is installed, and the correspondence remains unchanged while the camera equipment remains in its installed state.

[0059] The coordinate system transformation of all 3D camera coordinate nodes is performed sequentially according to the image time sequence. The resulting 3D spatial coordinates are then associated with the corresponding timestamps, image frame identifiers, and the aforementioned coordinate source states to obtain a 3D spatial coordinate sequence. Each coordinate node in the 3D spatial coordinate sequence is represented using a unified substation reference coordinate system, enabling direct comparison of the switch boom positions at different image times within the same physical space. This provides a unified spatial coordinate basis for subsequent calculation of sampling intervals in time sequence, execution of 3D trajectory interpolation, and formation of continuous 3D motion trajectories.

[0060] In one implementation, trajectory interpolation processing is performed on the three-dimensional spatial coordinate sequence to obtain a continuous three-dimensional motion trajectory, including: The sampling interval is obtained by calculating the time interval between adjacent coordinate points in the three-dimensional spatial coordinate sequence in chronological order. The sampling interval is compared with a preset time threshold to obtain the trajectory interpolation determination result. If the trajectory interpolation determination result indicates that the sampling interval has reached the preset time threshold, then the interpolation node sequence is determined according to the three-dimensional spatial coordinates within the corresponding time interval; The interpolation node sequence is subjected to trajectory fitting processing to obtain trajectory fitting data; Based on the trajectory fitting data, coordinate resampling processing is performed to obtain the completed coordinate sequence; The completed coordinate sequence is processed by trajectory connection according to the time sequence to obtain a continuous three-dimensional motion trajectory.

[0061] Specifically, when calculating the time interval between adjacent coordinate points in the three-dimensional spatial coordinate sequence according to time sequence, the timestamps carried by each three-dimensional spatial coordinate node are read, and adjacent coordinate nodes are determined sequentially according to the order of the timestamps. The sampling interval between the corresponding adjacent coordinate nodes is obtained by subtracting the timestamp of the previous coordinate node from the timestamp of the next coordinate node. All sampling intervals are recorded according to the original time sequence of the three-dimensional spatial coordinate sequence, and each sampling interval is associated with the corresponding two preceding and following three-dimensional spatial coordinate nodes.

[0062] The preset time threshold is determined as follows: After the camera equipment and disconnector are installed, the disconnector is driven to complete no less than 30 normal opening and closing actions. Within the action interval with no frame loss and normal image acquisition, the sampling interval of adjacent valid three-dimensional coordinate nodes is counted, and the mean and standard deviation of all normal sampling intervals are calculated. The mean plus three times the standard deviation is used as the preset time threshold. The preset time threshold is used to distinguish between normal connection intervals and interpolation intervals that require coordinate completion.

[0063] When comparing the sampling interval with a preset time threshold, the sampling intervals are read sequentially according to time. When the sampling interval does not reach the preset time threshold, the corresponding adjacent three-dimensional coordinate nodes are determined as normal connection intervals, and their original three-dimensional spatial coordinates are retained. When the sampling interval reaches the preset time threshold, the time interval between the corresponding two three-dimensional coordinate nodes is determined as the interpolation interval, and the start time, end time, and corresponding three-dimensional coordinate nodes of the time interval are recorded to obtain the trajectory interpolation determination result.

[0064] It should be noted that subsequent trajectory interpolation is only performed on time intervals where the sampling interval reaches a preset time threshold. Original 3D spatial coordinates with normal sampling intervals are not refitted or smoothed. This process avoids misinterpreting real local velocity changes, mechanical vibrations, or motion deviations during the disconnector's operation as noise and smoothing them out, thus preserving actual motion information for subsequent trajectory fluctuation analysis.

[0065] If the trajectory interpolation determination result indicates that the sampling interval reaches a preset time threshold, then the interpolation node sequence is determined based on the three-dimensional spatial coordinates within the corresponding time interval. Specifically, the effective three-dimensional spatial coordinates at both ends of the interval to be interpolated are used as interval boundary nodes, and effective three-dimensional coordinate nodes adjacent to the interval to be interpolated are extracted according to the time order of the original three-dimensional spatial coordinate sequence. The timestamp, horizontal spatial coordinate, vertical spatial coordinate, and height coordinate of each node are kept correlated and arranged in ascending order of timestamp to form the interpolation node sequence. The interpolation node sequence is used to characterize the true three-dimensional motion trend of the adjacent positions in the time interval to be filled.

[0066] When performing trajectory fitting on the interpolation node sequence, the timestamp is used as a unified independent variable, and cubic spline fitting is performed on the horizontal coordinate, vertical coordinate and height coordinate respectively, so that the piecewise functions corresponding to each coordinate component maintain the continuity of position, first-order change relationship and second-order change relationship at adjacent nodes, and the trajectory fitting data describing the change of three-dimensional spatial position with time in the interpolation interval is obtained.

[0067] It should be noted that all three spatial coordinate components use the same set of timestamps as fitting variables. Therefore, the horizontal, vertical, and height coordinates obtained at any subsequent target time point correspond to the same action time, avoiding time misalignment caused by processing the three coordinate directions separately. The purpose of trajectory fitting is to restore the continuous positional relationship within the time interval lacking effective sampling points, rather than changing the true three-dimensional coordinates at both ends of the interval to be interpolated.

[0068] When performing coordinate resampling processing based on the trajectory fitting data, the target time position to be filled is determined between the start and end times of the interpolation interval. The target time position is determined according to the time resolution under normal acquisition conditions of a continuous image sequence, ensuring that the filled coordinate sequence has a consistent time scale with the aforementioned image acquisition process. Each target time position is sequentially input into the trajectory fitting data corresponding to the horizontal, vertical, and height directions. The three spatial coordinate components corresponding to that moment are calculated, and the obtained horizontal, vertical, and height coordinates are combined with the corresponding timestamp to form the filled three-dimensional coordinate node.

[0069] For existing valid 3D coordinate nodes at both ends of the interpolation interval, the original coordinate values ​​are directly retained without using the fitting results for overwriting. New complete coordinates are generated only for target time positions in the original 3D spatial coordinate sequence that lack valid spatial nodes, and the newly generated coordinate nodes are marked as interpolation coordinates. The obtained interpolation coordinate nodes are arranged in order of target time position from first to last to obtain the complete coordinate sequence.

[0070] When performing trajectory connection processing on the completed coordinate sequence in chronological order, the valid coordinate nodes in the original three-dimensional spatial coordinate sequence and the completed coordinate nodes generated in each interpolation interval are merged and sorted according to their timestamps. For normal connection intervals, the corresponding trajectory segments are maintained according to the original valid three-dimensional spatial coordinate nodes; for interpolation intervals, the original coordinates at both ends of the interval and the internal completed coordinates are connected according to the spatial position change relationship described by the trajectory fitting data. After processing all time intervals in sequence, a continuous three-dimensional motion trajectory covering the disconnecting switch operation process is formed.

[0071] Each trajectory node in the continuous three-dimensional motion trajectory includes at least a timestamp, three-dimensional spatial coordinates, and a coordinate source identifier. The coordinate source identifier is used to distinguish between the original three-dimensional coordinates obtained by converting the actual image position and the interpolated three-dimensional coordinates generated by trajectory fitting. The continuous three-dimensional motion trajectory still uses the aforementioned unified substation reference coordinate system and maintains the temporal sequence of the disconnecting switch operation process for subsequent determination of the starting trajectory position, ending trajectory position, and execution of trajectory fluctuation analysis.

[0072] In step S4, the starting trajectory position is determined based on the continuous three-dimensional motion trajectory. Using the starting trajectory position as the attitude reference, the spatial angle of the ending trajectory position of the continuous three-dimensional motion trajectory is calculated to obtain the action posture angle. The angle arrival state is determined based on the comparison between the action posture angle and a preset angle threshold, including: Extract the first trajectory node of the continuous three-dimensional motion trajectory to obtain the starting trajectory position; Using the starting trajectory position as the position reference, the displacement of the ending trajectory position of the continuous three-dimensional motion trajectory is calculated to obtain the trajectory displacement vector; The trajectory displacement vector is subjected to motion plane projection processing to obtain the motion projection vector; The angle between the projection vectors of the action and the action posture angle is calculated based on the preset action direction of the disconnecting switch. The angle of the action posture is compared with the preset angle threshold to determine the angle in place.

[0073] Specifically, when extracting the first trajectory node of the continuous three-dimensional motion trajectory, the first valid three-dimensional trajectory node in the current disconnecting switch operation process is determined as the starting trajectory position according to the time sequence corresponding to the trajectory nodes, and the last valid three-dimensional trajectory node of the continuous three-dimensional motion trajectory is read as the ending trajectory position. Both the starting trajectory position and the ending trajectory position are represented using the aforementioned unified substation reference coordinate system.

[0074] Using the initial trajectory position as a position reference, the spatial coordinate components corresponding to the initial trajectory position are subtracted from the three spatial coordinate components of the final trajectory position to obtain the trajectory displacement vector pointing from the initial trajectory position to the final trajectory position. This trajectory displacement vector characterizes the overall spatial displacement direction of the boom relative to its initial position after this action.

[0075] When performing motion plane projection processing on the trajectory displacement vector, during the calibration stage, the boom shaft center position is determined based on the mechanical drawings or on-site measurement results of the corresponding model disconnector switch, and the corresponding motion plane is determined based on the shaft center and the boom spatial position under normal opening and closing states. During online identification, the trajectory displacement vector is projected onto the motion plane, and the displacement component perpendicular to the motion plane is removed to obtain the motion projection vector. The preset motion direction is also pre-calibrated within the motion plane based on the normal positioning motion. The shaft center is used to establish a spatial reference for the motion plane and the preset motion direction, and the linear displacement between the starting trajectory position and the ending trajectory position is not interpreted as the actual rotation angle of the boom around the mechanical shaft.

[0076] When calculating the angle between the action projection vector and the preset action direction of the disconnecting switch, the corresponding preset action direction is read in the action plane, and the spatial angle between the action projection vector and the preset action direction is calculated according to the following formula: ; Where θ is the attitude interpretation angle corresponding to the current action process; The motion projection vector is the vector that points from the starting trajectory position to the ending trajectory position and is then projected onto the motion plane. This refers to the preset operating direction corresponding to the current model of disconnector switch. The attitude judgment angle is used to characterize the angular relationship between the overall spatial motion direction formed by this action and the preset reference direction.

[0077] In this embodiment, the visual angle calculated according to the angle between the displacement projection direction and the preset action direction is used as the action posture angle to determine whether the termination posture of this action is consistent with the visual posture corresponding to the normal positioning action. The action posture angle is not the actual mechanical rotation angle generated by the boom around the mechanical axis. The center of the mechanical axis is used for the calibration of the aforementioned action plane and the preset action direction. In the online recognition stage, the action posture angle is still calculated according to the correspondence between the starting trajectory position, the ending trajectory position, the action projection vector, and the preset action direction.

[0078] It should be noted that the preset angle threshold is determined as follows: After the camera equipment and disconnector are installed, the disconnector is driven to complete no less than 30 opening and closing actions confirmed by mechanical indication. Following the same starting trajectory position, ending trajectory position, shaft center position, action plane, preset action direction, and included angle calculation rules, the action posture angle corresponding to each normal action is calculated. The average of the calculated angles is taken as the preset angle threshold, and three times the standard deviation of the calculated angles is taken as the allowable angle deviation range. During the online identification phase, when the angle difference between the action posture angle and the preset angle threshold is within the allowable angle deviation range, the angle arrival condition is determined to be met. For changes in the mechanical structure, camera equipment installation location, or disconnector model, the corresponding preset angle threshold and allowable angle deviation range are re-determined according to the above method.

[0079] The motion posture angle is compared with the preset angle threshold. In one implementation, the angle difference between the motion posture angle and the preset angle threshold is calculated. ; Where Δθ is the angle difference; θ is the current action posture angle; As a preset angle threshold, the two angles use the same angle unit. The angle difference is compared with the allowable angle deviation obtained from the normal positioning action calibration. When the angle difference is within the allowable angle deviation range, it is determined that the current disconnecting switch meets the angle positioning condition; otherwise, it is determined that the angle is not in position, and the angle positioning state is obtained.

[0080] In step S5, trajectory fluctuation analysis is performed on the continuous three-dimensional motion trajectory to obtain trajectory fluctuation data. Based on the trajectory fluctuation data, the stability of the angle positioning state is verified to obtain the attitude judgment result of the disconnecting switch, including: The continuous three-dimensional motion trajectory is sampled according to a preset sampling interval to obtain a trajectory sampling sequence; The trajectory sampling sequence is subjected to motion direction deviation analysis relative to a preset action direction to obtain trajectory deviation data; The trajectory deviation data is subjected to discreteness quantization processing to obtain trajectory fluctuation data; The trajectory fluctuation data is compared with a preset fluctuation threshold to obtain the trajectory stability determination result. If the angle is in position, indicating that the action posture angle has reached the preset angle threshold, and the trajectory stability determination result indicates that the trajectory fluctuation data has not reached the preset fluctuation threshold, then the disconnect switch is determined to be in position, and the posture judgment result is obtained.

[0081] Specifically, when sampling the trajectory according to the preset sampling interval, the coordinate source identifier of each trajectory node is read simultaneously. The original 3D coordinate nodes obtained by transforming the actual image position are used as valid observation nodes for trajectory fluctuation analysis; the interpolated nodes generated by trajectory fitting are used to maintain the temporal and spatial connectivity of the continuous 3D motion trajectory, and the weight of the corresponding completed interval in trajectory fluctuation statistics is not increased due to an increase in the number of interpolated nodes. When calculating trajectory fluctuations, trajectory nodes that reflect the actual observation state are used as the main statistical data.

[0082] When performing motion direction deviation analysis on the trajectory sampling sequence, the aforementioned motion plane and preset motion direction are used as the analysis benchmark to determine the lateral deviation component of each trajectory sampling point relative to the preset motion direction. These lateral deviation components are then arranged according to the sampling time sequence to obtain trajectory deviation data. This trajectory deviation data serves as the input for subsequent discreteness quantization, characterizing the change in lateral deviation over time during the motion process.

[0083] When performing discrete quantization on the trajectory deviation data, if the number of valid trajectory sampling points involved in the calculation is not less than 2, the sample variance of the deviation in each direction relative to the average directional deviation is calculated: ; in, This represents trajectory fluctuation data; Indicates the first The directional deviation corresponding to each trajectory sampling point; This represents the average value of the deviation in each direction; This indicates the number of valid trajectory sampling points involved in the calculation. The larger the trajectory fluctuation data, the more significant the deviation of the boom from its normal direction of motion during operation; the smaller the trajectory fluctuation data, the more stable the spatial motion state of the boom.

[0084] The preset fluctuation threshold is determined as follows: after the camera equipment and disconnector are installed, the same trajectory sampling, direction deviation analysis and variance calculation method as in the online identification stage is used to process no less than 30 opening and closing action trajectories that are confirmed to be in normal position by mechanical indication, calculate the trajectory fluctuation data corresponding to each normal action, and take the mean of each trajectory fluctuation data plus 3 times the standard deviation as the preset fluctuation threshold.

[0085] The trajectory fluctuation data is compared with a preset fluctuation threshold. When the trajectory fluctuation data does not reach the preset fluctuation threshold, it is determined that the current disconnector operation meets the trajectory stability requirements; when the trajectory fluctuation data reaches the preset fluctuation threshold, it is determined that the current operation has trajectory fluctuations exceeding the normal range, and a trajectory stability determination result is obtained.

[0086] Furthermore, the trajectory stability determination result is jointly judged with the aforementioned angle arrival state. If the action posture angle reaches a preset angle threshold and the trajectory fluctuation data does not reach a preset fluctuation threshold, it indicates that the switch boom has both reached the specified posture position and maintained the required motion stability during the action, and the disconnect switch is determined to be in the action arrival state. If the action posture angle does not reach the preset angle threshold, it is determined to be in the action arrival state; if the action posture angle reaches the preset angle threshold but the trajectory fluctuation data reaches the preset fluctuation threshold, it is determined to be in the angle arrival state but the trajectory stability is abnormal, and it is not determined to be the final action arrival state, thereby obtaining the attitude judgment result of the disconnect switch.

[0087] In summary, this invention discloses a visual recognition method for the position and posture of a disconnector switch, which solves the technical problem of low accuracy in judging the position and posture of a disconnector switch in complex outdoor environments.

[0088] A second embodiment of the present invention provides a visual recognition system for the position and posture of a disconnector switch, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0089] It should be noted that the visual recognition system for the position and attitude of a disconnector provided in this embodiment of the invention is used to execute all the process steps of the visual recognition method for the position and attitude of a disconnector in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0090] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0091] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A visual recognition method for the position and attitude of a disconnector switch, characterized in that, include: A continuous image sequence corresponding to the action process of the disconnecting switch is obtained, and the switch boom in the continuous image sequence is identified to obtain a two-dimensional bounding box sequence. An initial two-dimensional spatial coordinate sequence is constructed according to the image time sequence based on the bounding box positions in the two-dimensional bounding box sequence. The initial two-dimensional spatial coordinate sequence is subjected to coordinate continuity analysis to obtain a coordinate continuity determination result. Based on the coordinate continuity determination result, the initial two-dimensional spatial coordinate sequence is filtered for coordinate positions. Data nodes carrying coordinate missing markers are identified as first-type abnormal coordinate positions, data nodes with valid coordinates but whose corresponding adjacent coordinate distances reach a preset distance threshold are identified as second-type abnormal coordinate positions, and the remaining data nodes with valid coordinates and normal continuity are identified as valid coordinate positions. Each first-type abnormal coordinate position and second-type abnormal coordinate position is recorded according to the original image frame identifier and timestamp to obtain the abnormal coordinate positions. Using the preceding valid coordinates corresponding to the abnormal coordinate position as the state reference, the motion state of the abnormal coordinate position is predicted to obtain the predicted coordinates. The initial two-dimensional spatial coordinate sequence is then corrected based on the predicted coordinates to obtain the corrected two-dimensional spatial coordinate sequence. Based on the pre-established pixel depth correspondence, the corrected two-dimensional spatial coordinate sequence is transformed to obtain a three-dimensional spatial coordinate sequence. The three-dimensional spatial coordinate sequence is then subjected to trajectory interpolation to obtain a continuous three-dimensional motion trajectory. The starting trajectory position is determined based on the continuous three-dimensional motion trajectory. Using the starting trajectory position as the attitude reference, the spatial angle of the ending trajectory position of the continuous three-dimensional motion trajectory is calculated to obtain the action posture angle. The angle position is determined based on the comparison between the action posture angle and the preset angle threshold. The continuous three-dimensional motion trajectory is analyzed for trajectory fluctuation to obtain trajectory fluctuation data. The stability of the angle position is verified based on the trajectory fluctuation data to obtain the attitude judgment result of the disconnect switch. The determination condition of the stability verification is: the action posture angle reaches the preset angle threshold and the trajectory fluctuation data does not reach the preset fluctuation threshold. When both conditions are met, the disconnect switch is determined to be in the action position.

2. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The step of performing target recognition on the boom in the continuous image sequence to obtain a two-dimensional bounding box sequence, and constructing an initial two-dimensional spatial coordinate sequence according to the image time sequence based on the bounding box positions in the two-dimensional bounding box sequence, includes: The continuous image sequence is subjected to boom visual feature extraction processing to obtain boom feature data; Based on the boom feature data, candidate region localization processing is performed to obtain a set of candidate bounding boxes; The candidate bounding box set is subjected to bounding box filtering processing to obtain a two-dimensional bounding box sequence; The two-dimensional bounding box sequence is subjected to geometric center localization processing to obtain a center coordinate sequence; The center coordinate sequence is time-marked according to the image time sequence of the continuous image sequence to obtain the initial two-dimensional spatial coordinate sequence.

3. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The step of performing coordinate continuity analysis on the initial two-dimensional spatial coordinate sequence to obtain a coordinate continuity determination result includes: Extract adjacent coordinate points from the initial two-dimensional spatial coordinate sequence according to the image time sequence; The spatial distance between the adjacent coordinate points is calculated to obtain the distance between adjacent coordinates; The distance between adjacent coordinates is compared with a preset distance threshold to obtain the coordinate continuity determination result.

4. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The process involves using the preceding valid coordinates corresponding to the abnormal coordinate position as a state reference to predict the motion state of the abnormal coordinate position, obtaining predicted coordinates, and then correcting the initial two-dimensional spatial coordinate sequence based on the predicted coordinates to obtain a corrected two-dimensional spatial coordinate sequence, including: Based on the abnormal coordinate positions, extract the preceding valid coordinate data from the initial two-dimensional spatial coordinate sequence; The preceding valid coordinate data is subjected to motion state estimation processing to obtain motion state data; Based on the motion state data, perform current position prediction processing to obtain predicted coordinates; The abnormal coordinate positions are updated based on the predicted coordinates to obtain corrected coordinates. The initial two-dimensional spatial coordinate sequence is reconstructed based on the corrected coordinates to obtain the corrected two-dimensional spatial coordinate sequence.

5. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The step of transforming the corrected two-dimensional spatial coordinate sequence into a three-dimensional spatial coordinate sequence based on a pre-established pixel depth correspondence includes: The corrected two-dimensional spatial coordinate sequence is subjected to pixel coordinate correction processing based on the pre-calibrated camera intrinsic parameters to obtain the corrected pixel coordinate sequence; The depth matching process is performed on the corrected pixel coordinate sequence according to the pre-established pixel depth correspondence to obtain a depth value sequence; Using the depth numerical sequence as a depth constraint, the corrected pixel coordinate sequence is subjected to spatial back projection processing to obtain a three-dimensional camera coordinate sequence; The coordinate system transformation is performed on the three-dimensional camera coordinate sequence based on the pre-calibrated camera extrinsic parameters to obtain a three-dimensional spatial coordinate sequence.

6. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The process of interpolating the three-dimensional spatial coordinate sequence to obtain a continuous three-dimensional motion trajectory includes: The sampling interval is obtained by calculating the time interval between adjacent coordinate points in the three-dimensional spatial coordinate sequence in chronological order. The sampling interval is compared with a preset time threshold to obtain the trajectory interpolation determination result. If the trajectory interpolation determination result indicates that the sampling interval has reached the preset time threshold, then the interpolation node sequence is determined according to the three-dimensional spatial coordinates within the corresponding time interval; The interpolation node sequence is subjected to trajectory fitting processing to obtain trajectory fitting data; Based on the trajectory fitting data, coordinate resampling processing is performed to obtain the completed coordinate sequence; The completed coordinate sequence is processed by trajectory connection according to the time sequence to obtain a continuous three-dimensional motion trajectory.

7. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The step of determining the starting trajectory position based on the continuous three-dimensional motion trajectory, using the starting trajectory position as the attitude reference, calculating the spatial angle of the ending trajectory position of the continuous three-dimensional motion trajectory to obtain the action posture angle, and determining the angle arrival state based on the comparison relationship between the action posture angle and a preset angle threshold includes: Extract the first trajectory node of the continuous three-dimensional motion trajectory to obtain the starting trajectory position; Using the starting trajectory position as the position reference, the displacement of the ending trajectory position of the continuous three-dimensional motion trajectory is calculated to obtain the trajectory displacement vector; The trajectory displacement vector is subjected to motion plane projection processing to obtain the motion projection vector; The angle between the projection vectors of the action and the action posture angle is calculated based on the preset action direction of the disconnecting switch. The angle of the action posture is compared with the preset angle threshold to determine the angle in place.

8. The visual recognition method for the position and attitude of a disconnecting switch according to claim 1, characterized in that, The process of performing trajectory fluctuation analysis on the continuous three-dimensional motion trajectory to obtain trajectory fluctuation data, and then performing stability verification on the angle positioning state based on the trajectory fluctuation data to obtain the attitude judgment result of the disconnecting switch includes: The continuous three-dimensional motion trajectory is sampled according to a preset sampling interval to obtain a trajectory sampling sequence; The trajectory sampling sequence is subjected to motion direction deviation analysis relative to a preset action direction to obtain trajectory deviation data; The trajectory deviation data is subjected to discreteness quantization processing to obtain trajectory fluctuation data; The trajectory fluctuation data is compared with a preset fluctuation threshold to obtain the trajectory stability determination result. If the angle is in position, indicating that the action posture angle has reached the preset angle threshold, and the trajectory stability determination result indicates that the trajectory fluctuation data has not reached the preset fluctuation threshold, then the disconnect switch is determined to be in position, and the posture judgment result is obtained.

9. A visual recognition system for the position and posture of a disconnector switch, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1 to 8.

Images