Isolation switch state confirmation method and device based on multi-modal timing state reasoning, computer device, readable storage medium and program product

Abstract

The application relates to a disconnecting switch state confirmation method and device based on multi-modal timing state reasoning, computer equipment, a readable storage medium and a program product, relates to the technical field of power monitoring, and can improve the disconnecting switch state confirmation efficiency. The method comprises the following steps: acquiring a multi-modal action feature sequence of a disconnecting switch in an action process; determining an action evolution state based on an action state transition relationship, multi-modal action features in the multi-modal action feature sequence and an action state of the disconnecting switch corresponding to a previous moment; verifying the action evolution state based on a preset state constraint optimization model to obtain a target evolution state of each moment after verification; combining the target evolution state of each moment after verification in the action process in time sequence to obtain a target evolution sequence; and determining a target action state of the disconnecting switch according to the target evolution sequence.

Description

Technical Field

[0001] This application relates to the field of power monitoring technology, and in particular to a method, apparatus, computer equipment, readable storage medium, and program product for confirming the status of disconnecting switches based on multimodal time-series state reasoning. Background Technology

[0002] Disconnect switches are crucial primary equipment in power systems, and their accurate determination of open / closed status directly impacts grid operation and equipment safety. Currently, disconnect switch status identification primarily employs single methods such as auxiliary contacts, video monitoring, or drive motor current. However, auxiliary contacts are prone to mechanical jamming or false positioning, video recognition is easily interfered with by harsh environmental factors such as strong light and obstructions, and motor current lacks direct status feedback information such as contact position; some multi-sensor solutions also rely on simple result-level superposition or voting judgments, which are insufficient to adapt to the complex operating conditions of the equipment.

[0003] Improving the accuracy and reliability of determining the operating status of disconnect switches under complex and abnormal operating conditions is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, readable storage medium, and program product for confirming the state of disconnecting switches based on multimodal temporal state reasoning, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for confirming the state of a disconnector switch based on multimodal temporal state reasoning, including:

[0006] Obtain the multimodal action feature sequence of the disconnecting switch during its operation; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features;

[0007] Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, the action evolution state at each moment is determined.

[0008] The action evolution state at each moment is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0009] The target evolution states verified at each moment during the action process are combined in chronological order to obtain the target evolution sequence;

[0010] The target operating state of the disconnecting switch is determined based on the target evolution sequence.

[0011] In one embodiment, determining the action evolution state at each moment based on a preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment includes:

[0012] The visual features, drive motor current features, and auxiliary contact state features at each moment are mapped to a unified state inference space, and the multimodal action features in the unified state inference space are obtained based on the mapping results.

[0013] Based on the operating state of the disconnecting switch at the previous moment, the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features at each moment are determined; wherein, the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features are different in different operating states of the disconnecting switch.

[0014] Based on the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features at each time step, the multimodal action features in the unified state inference space are fused to obtain joint state features.

[0015] Based on the joint state characteristics, the operation state of the disconnector at the previous moment, and the operation state transition relationship, the operation evolution state at each moment is determined.

[0016] In one embodiment, the step of verifying the action evolution state at each time step based on a preset state constraint optimization model to obtain the verified target evolution state at each time step includes:

[0017] Based on the multimodal action characteristics, the visual state constraints, current characteristic constraints, and auxiliary contact state constraints of the disconnecting switch during the action process are determined.

[0018] Obtain the state stability constraints and the action timing constraints corresponding to the action state transition relationship;

[0019] Based on the visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, the action evolution state at each moment is jointly constrained and solved to obtain the verified target evolution state at each moment.

[0020] In one embodiment, the joint constraint solution for the action evolution state at each moment, based on the visual state constraint, current characteristic constraint, auxiliary contact state constraint, action timing constraint, and state stability constraint, yields the verified target evolution state at each moment, including:

[0021] If the auxiliary contact state feature corresponding to the action evolution state indicates that the contact has been switched, and the contact gap feature in the corresponding visual feature is greater than the preset gap threshold, then the target evolution state is determined to be a false positioning abnormal state.

[0022] If the current characteristic of the drive motor corresponding to the action evolution state shows an increasing trend in the drive motor current, and the speed of the disconnect switch corresponding to the visual characteristic shows a decreasing trend, then the target evolution state is determined to be a mechanism jamming abnormal state.

[0023] In one embodiment, determining the target operating state of the disconnector switch based on the target evolution sequence includes:

[0024] The target evolution sequence is input into a preset temporal state reasoning model. The temporal state reasoning model is used to extract evolution features from the target evolution sequence. The extracted evolution features are then subjected to probability mapping processing to calculate the state probability distribution vector of the disconnect switch corresponding to different action state categories.

[0025] Based on the state probability distribution vector of the disconnecting switch corresponding to different action state categories, the target action state and the confidence level of the target action state are determined.

[0026] In one embodiment, obtaining the multimodal action feature sequence of the disconnecting switch during its operation includes:

[0027] The edge-mounted visual state perception terminal is used to perform continuous target detection on the contact area of ​​the disconnecting switch. Based on the contact position features, movement speed features, contact gap features and key frame action tags obtained from the target detection, a visual feature sequence is determined.

[0028] Based on the pre-acquired drive motor current waveform data of the disconnecting switch, a current characteristic sequence is determined, and based on the pre-acquired auxiliary contact state change data, an auxiliary contact state characteristic sequence is determined.

[0029] For the visual feature sequence, the current feature sequence, and the auxiliary contact state feature sequence, time scale alignment processing is performed based on a preset time-domain sliding window to determine the multimodal action feature sequence.

[0030] Secondly, this application also provides a disconnector switch state confirmation device based on multimodal timing state reasoning, comprising:

[0031] The feature acquisition module is used to acquire the multimodal action feature sequence of the disconnecting switch during the operation process; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features;

[0032] The action prediction module is used to determine the action evolution state at each moment based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment.

[0033] The state verification module is used to verify the action evolution state at each moment based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0034] The prediction sequence determination module is used to combine the verified target evolution states at each moment in the action process in chronological order to obtain the target evolution sequence;

[0035] The action state determination module is used to determine the target action state of the disconnecting switch based on the target evolution sequence.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Obtain the multimodal action feature sequence of the disconnecting switch during its operation; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features;

[0038] Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, the action evolution state at each moment is determined.

[0039] The action evolution state at each moment is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0040] The target evolution states verified at each moment during the action process are combined in chronological order to obtain the target evolution sequence;

[0041] The target operating state of the disconnecting switch is determined based on the target evolution sequence.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0043] Obtain the multimodal action feature sequence of the disconnecting switch during its operation; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features;

[0044] Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, the action evolution state at each moment is determined.

[0045] The action evolution state at each moment is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0046] The target evolution states verified at each moment during the action process are combined in chronological order to obtain the target evolution sequence;

[0047] The target operating state of the disconnecting switch is determined based on the target evolution sequence.

[0048] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0049] Obtain the multimodal action feature sequence of the disconnecting switch during its operation; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features;

[0050] Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, the action evolution state at each moment is determined.

[0051] The action evolution state at each moment is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0052] The target evolution states verified at each moment during the action process are combined in chronological order to obtain the target evolution sequence;

[0053] The target operating state of the disconnecting switch is determined based on the target evolution sequence.

[0054] The aforementioned method, apparatus, computer equipment, readable storage medium, and program product for confirming the state of a disconnector switch based on multimodal temporal state reasoning acquire a multimodal action feature sequence of the disconnector switch during its operation. This multimodal action feature sequence includes multimodal action features at multiple time points, including at least visual features, drive motor current features, and auxiliary contact state features. Based on a preset action state transition relationship, the multimodal action features at each time point in the multimodal action feature sequence, and the corresponding action state of the disconnector switch at the previous time point, the action evolution state at each time point is determined. A preset state constraint optimization model is used to verify the action evolution state at each time point, resulting in a verified target evolution state. The state constraint optimization model is determined based on the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship. The verified target evolution states at each time point during the operation are combined in chronological order to obtain a target evolution sequence. Based on the target evolution sequence, the target action state of the disconnector switch is determined. In this application, a multimodal motion feature sequence containing visual features, drive motor current features, and auxiliary contact state features is obtained, breaking through the monitoring limitations of traditional single sensors. Then, the motion evolution state is determined based on the motion state transition relationship, current features, and previous state. A state constraint optimization model that integrates multimodal state constraints and motion timing constraints is used to dynamically verify the motion, realizing joint cross-verification of physical feature consistency and mechanical motion timing evolution law. Finally, the verified states are combined into a target evolution sequence according to time to determine the final motion state. This effectively filters out cross-stage misjudgments caused by local abnormal noise or single signal distortion, and accurately and stably identifies the true motion state of the disconnector switch under complex abnormal working conditions. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating a disconnector switch state confirmation method based on multimodal timing state reasoning in one embodiment.

[0057] Figure 2 This is a system architecture diagram of a disconnector state confirmation method based on multimodal temporal state reasoning applied in one embodiment;

[0058] Figure 3 This is a flowchart illustrating the disconnector switch state confirmation method based on multimodal timing state reasoning in another embodiment;

[0059] Figure 4 This is a structural block diagram of a disconnector state confirmation device based on multimodal timing state reasoning in one embodiment;

[0060] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] The disconnector switch status confirmation method based on multimodal time-series state reasoning proposed in this invention is not applicable to a specific model or single structure of switchgear, but is widely applicable to disconnector switch monitoring scenarios of various voltage levels in power systems, substations, switching stations and distribution networks.

[0063] Specifically, this method relies on the operation and maintenance architecture of digital and intelligent high-voltage switchgear. Its physical deployment environment primarily involves the switchyard of substations, where disconnecting switches, as key mechanical switching devices used to disconnect off-load currents and create clear disconnection points, require frequent opening and closing operations during daily operation. To achieve accurate confirmation of their status, this method operates within an intelligent monitoring system comprised of a multi-source data acquisition network and distributed computing nodes. At the hardware level, this system typically includes industrial vision sensing devices (such as edge smart cameras, contact monitoring cameras, etc.) deployed within the line-of-sight range of the disconnector switch contacts to capture the geometric motion trajectory and spatial contact state of the contacts during operation. Simultaneously, the system can continuously acquire current waveform signals reflecting changes in work done by the motor during mechanical transmission by measuring the load current of the drive motor in real time through a current transformer connected to the disconnector switch drive motor control circuit. Furthermore, the system can connect to auxiliary switches or auxiliary contact circuits within the disconnector switch mechanism box to acquire electrical switching signals characterizing the logical position switching of the transmission linkage. To ensure consistency in the benchmark for subsequent spatiotemporal comparison of multi-dimensional features, the acquisition operation of the motor load current waveform signal and the acquisition operation of the auxiliary contact switching signal are configured to maintain strict synchronous sampling.

[0064] At the software and network computing level, this application environment is jointly constructed by edge computing nodes and a station control layer state analysis platform. Edge devices are mainly responsible for local preprocessing and structured feature extraction of visual images, while data alignment, temporal state transition reasoning, physical consistency constraint solving, and final state decision-making and confidence assessment run in a central processing unit or server cluster with time-series data processing capabilities. The aim is to provide highly reliable dynamic monitoring and fault early warning support for the entire process of disconnector operation for the safe operation of the power grid through multimodal fusion of electrical signals, mechanical switching quantities, and visual images.

[0065] In one exemplary embodiment, such as Figure 1 As shown, a method for confirming the status of a disconnector switch based on multimodal temporal state reasoning is provided. Taking the application of this method to a server (such as a station control layer state analysis platform) in the above embodiment as an example, the method includes the following steps S101 to S105. Wherein:

[0066] Step S101: Obtain the multimodal action feature sequence of the disconnecting switch during its operation. The multimodal action feature sequence includes multimodal action features at multiple time points, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact status features.

[0067] Among them, disconnecting switches can be key switching devices in power systems used for isolating power sources, switching operations, and connecting or disconnecting low-current circuits.

[0068] A multimodal motion feature sequence can be a set of sequentially arranged data extracted from sensing sources of different dimensions over a continuous time span. It is used to comprehensively characterize the mechanical and electrical state changes of equipment from standstill, startup to final stabilization. The multimodal motion features include at least visual features extracted from continuous images or video streams to reflect the true kinematic state of the contacts, drive motor current features collected from the equipment drive circuit to reflect the changes in the mechanical load of the mechanism, and auxiliary contact state features obtained from the mechanism transmission link to indicate the logical position of the mechanism. These features can be acquired based on image acquisition devices, current transformers, and status monitoring nodes deployed on site. For example, visual features can be the movement speed or spatial gap of the contacts, drive motor current features can be the average current or current fluctuation during the action, and auxiliary contact state features can be the on / off level signal of the contacts.

[0069] Specifically, the multimodal action feature sequence of the disconnecting switch during the operation process is obtained. This includes acquiring sensing data uploaded by different sensing terminals within the time period when the disconnecting switch enters the opening or closing operation, and extracting multimodal action features at each moment, including visual features, drive motor current features, and auxiliary contact status features, from these sensing data. Then, the multimodal action features at multiple moments are combined and reconstructed according to the chronological evolution relationship to generate a multimodal action feature sequence covering the entire operation process, which serves as the basic input for subsequent process-based dynamic analysis.

[0070] Optionally, the multimodal motion feature sequence can also be extended to include vibration signal features that reflect the changes in vibration energy when the mechanism does mechanical work, or infrared temperature features that reflect the heating of contact engagement friction. By timestamping and splicing these additional modal features with visual, current, and contact features, a more three-dimensional multimodal motion feature sequence is formed.

[0071] Step S102: Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the isolation switch corresponding to the previous moment, determine the action evolution state at each moment.

[0072] Among them, the preset action state transition relationship can be a logical mapping rule that characterizes the evolution law of the action state between adjacent moments during the mechanical action flow of the disconnector switch. It is used to constrain the reasonable evolution path of the state over time and prevent non-physical misjudgment of the action stage due to occasional fluctuations in characteristic data.

[0073] Action evolution state can be the staged behavior of the disconnecting switch at a specific point in time during mechanical operation, jointly exhibited by its transmission mechanism, linkage mechanism and contact system. It is used to discretize the entire continuous mechanical action process into quantifiable temporal behavior nodes.

[0074] Specifically, after obtaining the multimodal action feature sequence, based on the time series, for any target time, the multimodal action features corresponding to that time are extracted, and the action state of the disconnecting switch that was determined in the immediate preceding time is obtained. Subsequently, the extracted multimodal action features of that time and the action state of the previous time are used as joint input parameters and input into the preset action state transition relationship. The action state transition relationship is used to perform correlation reasoning on the input multimodal action features and the historical state constraints of the previous time, thereby calculating the output action evolution state corresponding to the current time. By iterating through each time step, the action evolution state of each time step in the entire action process is determined sequentially.

[0075] Optionally, when determining the action evolution state at each moment, the action evolution state at the current moment can not only depend on the action state at the previous moment and the multimodal action features at the current moment, but can also selectively combine the multimodal action features of a preset number of future moments after the current moment for reverse temporal smoothing deduction. Through this bidirectional state transition relationship that combines forward state transition prediction and backward future feature verification, the transition probability value of the current moment in different candidate action states can be calculated, and the candidate action state with the highest probability value can be taken as the final determined action evolution state at that moment, thereby effectively enhancing the overall fault tolerance capability of the state transition logic in dealing with sudden interference.

[0076] Step S103: Verify the action evolution state at each moment based on the preset state constraint optimization model to obtain the verified target evolution state at each moment. The state constraint optimization model is determined based on the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship.

[0077] The preset state constraint optimization model can be a comprehensive evaluation framework that integrates multi-dimensional physical boundary conditions and temporal logic rules. It is used to perform consistency filtering and correction on the initially inferred large-scale state evolution sequence. It can be pre-constructed based on the mechanical motion characteristics of the disconnector and the safe operation characteristics of the electrical parameters.

[0078] The state constraints corresponding to the multimodal action features can refer to the logical combination conditions that the sensing data should meet in a specific action stage. For example, they can be the extreme value constraint boundary of the contact gap in the visual image, the fluctuation envelope range of the drive motor current, or the level matching logic of the auxiliary contacts. The action state transition relationship corresponding to the action timing constraints can refer to the logical boundaries that characterize the sequence of actions required for the disconnecting switch to complete a specific mechanical action.

[0079] Specifically, after initially determining the action evolution state at each moment, the multimodal action features corresponding to the current moment and the attached action state transition relationship are further extracted, and the corresponding state constraint terms and action temporal constraint terms are determined accordingly. Subsequently, the initially deduced action evolution state is placed in the state constraint optimization model jointly constructed by the above constraint terms for joint solution and cross-comparison. It is determined whether the current inferred state simultaneously satisfies the spatial rationality of physical feature expression, the rationality of time span, and the mutual verification of multi-source perception data. The action evolution state that successfully passes the joint constraint comparison conditions of all dimensions is confirmed and output as the verified target evolution state at that moment.

[0080] Optionally, in the verification process using the state-constrained optimization model, a multi-objective cost function can be constructed. Here, state constraints and action sequence constraints are transformed into independent penalty factors within the cost function. When a minor conflict arises between the initially inferred action evolution state and a constraint at a certain moment, the model calculates and increases the weight of the corresponding penalty factor to derive the comprehensive cost value of that state. If the comprehensive cost value exceeds a preset tolerance boundary, the initial state is rejected, and a state correction mechanism is activated. The model then searches the candidate state pool at the current moment for the solution that minimizes the multi-dimensional joint comprehensive cost value, and uses this solution as the final target evolution state. This provides a dynamic state optimization approach with strong fault tolerance.

[0081] Step S104: Combine the verified target evolution states at each moment during the action process in chronological order to obtain the target evolution sequence.

[0082] The target evolution sequence can be a time-series data chain or state matrix formed by concatenating state data at discrete time nodes in the order of physical occurrence. It is used to macroscopically and completely reconstruct the mechanical evolution trajectory of the disconnector switch during the operation cycle. For example, it can be an ordered set of a series of structured evolution states from the initial stage of the action to the final stage of the action.

[0083] Specifically, after completing the state inference and physical constraint verification at all discrete time points during the action process, the verified target evolution states distributed at different time points are extracted. Then, using the physical time of the action as the sorting benchmark, these originally isolated local time point states are orderly spliced ​​and structurally reorganized to generate a target evolution sequence that can reflect the smooth transition of states and completely cover the action process.

[0084] Optionally, during the process of combining the target evolution sequence in chronological order, a temporal noise reduction or key node sampling mechanism can be introduced. When the high-frequency sampling characteristics of the underlying sensors result in a large number of continuous and repetitive redundant steady-state stages within the concatenated sequence, adjacent and identical redundant evolution states can be merged and information compressed based on the state dwell time or state transition inflection point. This extracts the key evolution state nodes that truly represent the substantial action switching of the equipment's mechanical structure, thereby forming a more concise and feature-rich target evolution sequence without disrupting the core action evolution timeline, thus reducing the computational overhead of subsequent global inference calculations.

[0085] Step S105: Determine the target action state of the disconnecting switch based on the target evolution sequence.

[0086] Among them, the target action state can be a qualitative conclusion given based on the action evolution trajectory after the disconnecting switch has gone through a complete or incomplete physical control cycle. These states can be derived from the sequence analysis results of the whole cycle. For example, the specific categories they cover can be the normal closing position, the normal opening position, or the abnormal stagnation state that fails to complete the full stroke due to mechanical obstruction.

[0087] Specifically, after constructing the target evolution sequence of the covered equipment operation, by analyzing the sequential logic of state node changes in the continuous sequence, the duration of actions in each stage, and the steady-state performance at the end of the sequence's final convergence, this digitized evolution trajectory is mapped to a specific macroscopic physical action category, thereby ultimately determining the target action state of the disconnector switch qualitatively.

[0088] Optionally, when performing state determination based on the target evolution sequence, an ideal timing state template flow for standard opening and closing operations can be pre-established (e.g., it must go through startup, transmission, contact switching to final position in sequence). Then, the similarity of the evolution trajectory of the currently acquired target evolution sequence and these standard template flows is calculated. If the evolution path of the current sequence is highly consistent with a certain ideal template, the normal category corresponding to the template is directly determined as the target action state. For distorted sequences that cannot be matched with the standard template, they can be classified and characterized as specific abnormal target action states based on the key nodes where the sequence is interrupted or abnormally persists for a long time. This provides a lightweight and highly interpretable global qualitative alternative for upper-layer applications.

[0089] In this embodiment, by acquiring a multimodal motion feature sequence including visual features, drive motor current features, and auxiliary contact state features, the limitations of traditional single-sensor monitoring are broken. Then, based on the motion state transition relationship, current features, and previous state, the motion evolution state is determined, and a state constraint optimization model that integrates multimodal state constraints and motion timing constraints is used to dynamically verify it, realizing joint cross-verification of physical feature consistency and mechanical motion timing evolution law. Finally, the verified states are combined into a target evolution sequence according to time to determine the final motion state, effectively filtering out cross-stage misjudgments caused by local abnormal noise or single signal distortion, and accurately and stably identifying the true motion state of the disconnector switch under complex abnormal working conditions.

[0090] In one embodiment, obtaining the multimodal action feature sequence of the disconnecting switch during its operation includes:

[0091] The edge-mounted visual state perception terminal performs continuous target detection on the contact area of ​​the disconnector switch. Based on the contact position features, movement speed features, contact gap features, and keyframe action tags obtained from the target detection, a visual feature sequence is determined. Based on the pre-acquired drive motor current waveform data of the disconnector switch, a current feature sequence is determined. Based on the pre-acquired auxiliary contact state change data, an auxiliary contact state feature sequence is determined. For the visual feature sequence, current feature sequence, and auxiliary contact state feature sequence, time scale alignment processing is performed based on a preset time domain sliding window to determine a multimodal action feature sequence.

[0092] Among them, the edge visual state perception terminal can be an intelligent hardware device deployed at the physical site of the disconnect switch, with localized computing power and front-end image processing capabilities. It is used to directly capture targets and perform preliminary structured analysis on the video stream at the edge side close to the data source.

[0093] A time-domain sliding window is a fixed-length data extraction interval that is continuously shifted over time when processing time series data. It is used to confine data packets with different sampling frequencies within the same local time field of view for centralized processing.

[0094] Specifically, the edge-mounted visual state perception terminal performs continuous target detection on the contact area of ​​the disconnector switch. Based on the contact position features, movement speed features, contact gap features, and keyframe action tags obtained from the target detection, a visual feature sequence that changes over time is integrated and determined. Simultaneously, based on the pre-acquired drive motor current waveform data of the disconnector switch, a current feature sequence is determined and extracted, and based on the pre-acquired auxiliary contact state change data, an auxiliary contact state feature sequence is determined. Subsequently, for the aforementioned independent visual feature sequence, current feature sequence, and auxiliary contact state feature sequence, time scale alignment processing is performed based on a preset time-domain sliding window, thereby fusing and assembling multiple single-modal feature sequences from a time dimension to determine a multimodal action feature sequence.

[0095] For example, the edge-side visual state perception terminal can autonomously complete image recognition, motion tracking, and key point extraction of the contact area at the edge, and only report the extracted structured visual motion features and key frame motion labels to the back-end server in real time, rather than a large amount of raw continuous video stream. When faced with the fusion of multi-source heterogeneous data such as visual images (low-frequency two-dimensional data), drive motor current signals (high-frequency one-dimensional waveform data), and auxiliary contact status (discrete switching data), a unified timestamp mechanism can be used to construct a cross-modal time-domain sliding window. Within the micro-time scale of this sliding time window, a specific modality (e.g., the timestamp of each frame of visual image) is used as the reference time anchor point. The instantaneous values ​​of current features and auxiliary contact status values ​​within the range of the smallest time deviation before and after the anchor point are collected and strongly correlated and paired, thereby establishing the motion correlation relationship of multi-modal data and forming a strictly aligned temporal feature matrix.

[0096] In one specific embodiment, multimodal motion features are continuously acquired within the complete motion cycle T to form a multimodal motion feature sequence of the motion process. :

[0097]

[0098] in, Indicates the modal type; Representing modes At any moment Extracted multimodal action features.

[0099] Multimodal action characteristics include current characteristics Auxiliary contact characteristics Visual features :

[0100]

[0101] in, Average current; This is the peak current; This refers to the current fluctuation. The duration of the action.

[0102]

[0103] in, For auxiliary contact status; This refers to the moment of contact action; This is the contact delay time.

[0104]

[0105] in, For visual estimation of location features; Characteristics of motion speed; Characteristics of the contact gap; For keyframe action tags.

[0106] In this embodiment, by using an edge-end visual perception terminal to extract structured visual feature sequences at the front end, and performing strict time-scale alignment processing on heterogeneous feature sequences such as vision, current, and contacts based on a time-domain sliding window, the network communication bandwidth pressure is effectively alleviated and the timing misalignment deviation of heterogeneous data is eliminated from the source.

[0107] In one embodiment, the action evolution state at each moment is determined based on a preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, including:

[0108] The visual features, drive motor current features, and auxiliary contact state features at each moment are mapped to a unified state inference space. Based on the mapping results, multimodal action features in the unified state inference space are obtained. According to the action state of the disconnector switch at the previous moment, the confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features at each moment are determined. The confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features differ depending on the action state of the disconnector switch. Based on the confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features at each moment, the multimodal action features in the unified state inference space are fused to obtain joint state features. Based on the joint state features, the action state of the disconnector switch at the previous moment, and the action state transition relationship, the action evolution state at each moment is determined.

[0109] The unified state inference space can be a dimensionless, standardized numerical dimension constructed through a specific mapping function to eliminate the dimensional and numerical differences between different physical sensing quantities (such as amperes, millimeters, high and low levels, etc.). It is used to enable heterogeneous feature data to be computed and fused fairly on the same mathematical scale.

[0110] Confidence weights can be the weights of importance assigned to each feature at the current inference time. They are used to characterize the relative reliability or contribution of a certain modal data to the state determination conclusion at a specific physical action stage. These weights can be dynamically allocated based on the mechanical action state of the device at the previous moment. For example, the current data is given a higher weight when the mechanism just starts to move, while the contact data is given a higher weight when the contact is about to switch.

[0111] Joint state features can refer to a comprehensive feature descriptor formed after weighted summation and numerical smoothing, which is used as input parameters to drive further reasoning of state evolution.

[0112] Specifically, the visual features, drive motor current features, and auxiliary contact state features at each moment are mapped into a unified state inference space. Based on the mapping results, multimodal action features in the unified state inference space are obtained. Simultaneously, based on the action state of the disconnector switch at the previous moment, the confidence weights corresponding to the above features at each current moment are determined. Since the physical dominant characteristics exhibited by the disconnector switch differ in different mechanical action stages, the confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features will also be dynamically adjusted as the action state changes. Subsequently, based on these dynamically allocated confidence weights, the multimodal action features in the unified state inference space are deeply fused to calculate the joint state features specific to that moment. Finally, based on the joint state features, the action state of the disconnector switch at the previous moment, and the preset action state transition relationship, the action evolution state at that moment is inferred and determined.

[0113] For example, the preset state transition relationship (state transition model) is defined as:

[0114]

[0115] in: This is the current state; This refers to the state at the previous moment; This is the state transition function.

[0116] For example, when performing the above fusion operation and state determination, the joint state features can be obtained based on the following feature fusion formula. .

[0117]

[0118] in, represent Temporal mode Characteristic quantities (visual, electrical, or contact) This is a feature mapping function used to perform the transformation to a unified space; That is, the mode dynamically determined based on the action state at the previous moment. The confidence weight at the current moment. After multimodal weighted summation, then... The normalization function eliminates numerical abrupt changes and generates stable values. Then, substitute it into the following state transition model to complete the evolution state of the current node. The solution and deduction.

[0119]

[0120] in, The current action state; This refers to the action state at the previous moment; This is the joint state feature vector at the current moment; This is the action state transition function.

[0121] In specific application scenarios, when the previous state indicates that the current stage is in the start-up stage, since the mechanism has not yet undergone significant displacement but the motor has been energized, the system will prioritize the current characteristics and assign the highest weight to the current characteristics. When the linkage movement stage is in progress, the contacts produce obvious spatial displacement, and the visual motion characteristics will be prioritized and assigned the highest weight to the visual characteristics. When the contact switching stage is in progress, the electrical contact logic changes, and the auxiliary contact characteristics will be prioritized for weighting.

[0122] In this embodiment, by mapping heterogeneous features to a unified state reasoning space and introducing dynamic confidence weight allocation closely related to the previous action state, adaptive measurement and targeted extraction of information value of multimodal perception data under different mechanical work stages are realized. This overcomes the defect of traditional fixed weight fusion methods that are prone to introducing modal noise in non-sensitive stages, and improves the depth of multidimensional cross data fusion and the anti-interference ability of state reasoning.

[0123] In one embodiment, the action evolution state at each time step is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each time step, including:

[0124] Based on multimodal action characteristics, the visual state constraints, current characteristic constraints, and auxiliary contact state constraints of the disconnecting switch during the action process are determined; the state stability constraints and the action timing constraints corresponding to the action state transition relationship are obtained; based on the visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, the action evolution state at each moment is jointly constrained and solved to obtain the verified target evolution state at each moment.

[0125] Among them, the visual state constraint can be a spatial geometric boundary condition set to ensure that the inferred state matches the actual physical displacement. It is used to limit the reasonable range of the contact gap or movement speed at a specific stage and can be extracted based on the relative position pixel features detected by the image edge.

[0126] The current characteristic constraint can be an electrical signal envelope standard set to limit abnormal deviations in the mechanical work assessment. It is used to measure whether the fluctuation of the current load current is consistent with the expected force performance of the mechanism transmission and can be generated based on waveform data collected at high frequency by the current transformer.

[0127] The auxiliary contact state constraint can be a Boolean logic verification rule constructed to verify the accuracy of electrical switching logic. It is used to determine whether the on / off signal of the underlying hardware switch matches the current macroscopic action state.

[0128] The action state transition relationship corresponds to the action timing constraint term, which can specifically correspond to the underlying physical representation of the disconnector switch at different action stages to impose timing restrictions on the evolution state. For example, it can be refined into a static state, a starting state, a mechanism transmission state, a linkage motion state, a contact switching state, a position state, or a stable state.

[0129] State stability constraints can refer to time-domain smoothness indices introduced to suppress state jumps caused by high-frequency sampling. They are used to determine whether state switching behavior conforms to the laws of mechanical inertia.

[0130] Specifically, when performing state verification at the current moment, based on the acquired multimodal action features, visual state constraints, current characteristic constraints, and auxiliary contact state constraints are determined to characterize the disconnector switch during the action process. At the same time, state stability constraints to maintain state continuity and action timing constraints derived from the action state transition relationship are further acquired. Subsequently, based on the multidimensional constraint set including visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, a comprehensive joint constraint solution operation is performed on the action evolution state at each moment initially generated to obtain the verified target evolution state.

[0131] For example, the above process of solving the joint constraints can be specifically implemented based on the following pre-constructed multimodal state constraint optimization formula:

[0132]

[0133] in, This is the final, verified target evolution state obtained from the solution; , , , , These correspond to visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, respectively. , , , to These are the penalty weight coefficients dynamically configured for each constraint. In a specific embodiment, when a candidate action evolution state is substituted into the model, if the nominal mechanism is in place but the visual constraint is not... The data indicates that there is still a significant gap. The corresponding output penalty value will increase sharply; the model evaluates the global penalty value of the candidate state by weighted summation of the deviations of each constraint; and the operation that satisfies the joint constraint solution conditions is to perform the operation in the candidate state space at the current time. The minimum value operation is performed to find the state solution that minimizes the sum of the entire cost function, and this solution is output as the optimal target evolution state.

[0134] For example, the action timing constraints are defined as the sequence of the stationary state, the starting phase, the mechanism transmission phase, the linkage motion phase, the contact switching phase, the positioning phase, and the stable phase. The sequence can be expressed by the following formula:

[0135]

[0136] Wherein, S0 represents the stationary state, S1 represents the starting stage, S2 represents the mechanism transmission stage, S3 represents the linkage movement stage, S4 represents the contact switching stage, S5 represents the positioning stage, and S6 represents the stabilizing stage.

[0137] Furthermore, the action timing constraint term is defined by the following formula, which is used to constrain the action state of the disconnecting switch to meet the normal action evolution sequence.

[0138]

[0139] In this embodiment, by jointly solving the constraint terms covering multiple dimensions such as vision, current, contact, timing and stability, the hidden logical conflicts that cannot be detected by a single mode are exposed under the mathematical framework of multi-objective cost optimization, thereby minimizing local sensor distortion or false signal interference and improving the physical credibility of the final inferred state.

[0140] In one embodiment, based on visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, a joint constraint solution is performed on the action evolution state at each time step to obtain the verified target evolution state at each time step, including:

[0141] If the auxiliary contact state characteristic corresponding to the action evolution state indicates that the contact has been switched, and the contact gap characteristic in the corresponding visual feature is greater than the preset gap threshold, then the target evolution state is determined to be a false positioning abnormal state; if the drive motor current characteristic corresponding to the action evolution state indicates that the drive motor current shows an increasing trend, and the movement speed of the disconnect switch shows a decreasing trend in the corresponding visual feature, then the target evolution state is determined to be a mechanism jamming abnormal state.

[0142] The preset gap threshold can be a small spatial tolerance index pre-calibrated based on the physical structure dimensions of the disconnector contact and the safe insulation distance. It is used as a quantitative critical benchmark for judging whether the moving and stationary contacts have made substantial physical contact.

[0143] A false engagement anomaly can be caused by a disengaged transmission linkage or a loose auxiliary switch linkage mechanism, resulting in a dangerous physical condition where the electrical signal feedback indicates that the closing operation has been completed, but the main contacts have not actually made reliable contact. A mechanism jamming anomaly can refer to a fault scenario in mechanical transmission where poor lubrication, obstruction by foreign objects, or deformation of components cause the motor drive to be hindered, resulting in the moving contact remaining stationary.

[0144] Specifically, during the execution of the verification model, on the one hand, the auxiliary contact state features and contact gap features in the visual features at the current inference time are extracted. If the auxiliary contact state features indicate that the contact has logically switched actions, but the synchronous visual features indicate that the contact gap features are still greater than the preset gap threshold, it means that there is a serious deviation between the electrical feedback signal and the physical space image. Therefore, it is determined that the current joint constraint solution conditions are not met, and the current normal action evolution state inference is rejected. The action evolution state is directly determined as a false positioning abnormal state. On the other hand, the drive motor current features and the motion speed represented by the visual features at this moment are extracted. If the drive motor current features indicate that the work load of the drive motor shows an abnormal upward trend, and at the same time, the visual features indicate that the motion speed of the isolating switch moving contact shows an unreasonable downward trend, it means that the motor is continuously overloaded but the mechanical transmission is stagnant or blocked. Therefore, it is also determined that the joint constraint solution conditions are not met, and the action evolution state is determined as a mechanism jamming abnormal state.

[0145] For example, in the actual constraint solving operation of the above-mentioned abnormal state judgment, digital judgment can be performed based on the specific physical consistency verification formula in the abnormal state constraint model.

[0146] For false arrival anomalies, use conditional expressions. Logical comparison, in which This indicates that the level signal output by the auxiliary contact has flipped at the current moment, meaning the electrical indication is that the circuit breaker has been closed. Characterizing the gap distance between moving and stationary contacts extracted from the visual modality. Still greater than the set micro-gap threshold This means that the visual cue indicates a physical incompleteness; when this Boolean expression is true, a false completion state is immediately triggered. Similarly, for mechanical jamming anomalies, a trend condition expression can be used. The verification, that is, determining whether the following conditions are met simultaneously: when the current driving motor current collected by the current transformer is detected... The slope of the change remains positive (current increases), while the instantaneous velocity of the moving contact is extracted by the visual operator. The slope of the change is negative or approaches zero (speed decreases). When the trend condition expression is satisfied, it indicates that the conversion of mechanical kinetic energy is severely hindered. At this time, the model forcibly blocks it and characterizes it as an abnormal state of mechanical jamming.

[0147] In this embodiment, by concretizing the physical deviation logic across modes (such as the contact being activated but not visually closed, or the current increasing but the speed decreasing) into the constraint logic of the joint constraint solution process, the major power grid operation accidents caused by single sensor failures (such as the monitoring backend mistakenly believing that the equipment has been closed due to the deception caused by the failure of the auxiliary switch linkage rod) in the traditional monitoring scheme are effectively overcome.

[0148] In one embodiment, determining the target action state of the disconnector switch based on the target evolution sequence includes:

[0149] The target evolution sequence is input into a preset temporal state inference model. The temporal state inference model is used to extract evolution features from the target evolution sequence and perform probability mapping on the extracted evolution features to calculate the state probability distribution vectors of different action state categories corresponding to the disconnect switch. Based on the state probability distribution vectors of different action state categories corresponding to the disconnect switch, the target action state and the confidence level of the target action state are determined.

[0150] Among them, the temporal state reasoning model can be a machine learning architecture capable of processing sequential data with sequential dependencies. It is used to mine the context and recognize patterns of discrete state node sequences from a macroscopic time span. This model can be pre-trained based on a Long Short-Term Memory (LSTM) network or a Transformer architecture.

[0151] Evolutionary features can refer to the high-dimensional latent feature vectors extracted by the model when reasoning about the sequence. They are used to characterize the global dynamic evolution of the entire action process of the disconnector switch in the hidden space.

[0152] The state probability distribution vector can be a one-dimensional array composed of multiple probability components, which is used to simultaneously display the probability set of the current operation sequence corresponding to all candidate action state categories such as stationary, starting, transmission, and stable; the target action state and the judgment confidence can be the final qualitative global state conclusion and its corresponding reliability evaluation score.

[0153] Specifically, the target evolution sequence obtained by combining the previous steps in chronological order is input into a preset temporal state inference model. The temporal state inference model performs deep evolutionary feature extraction on the target evolution sequence in the time dimension to capture the implicit temporal dependencies between action stages. Subsequently, the evolutionary features extracted from the hidden layer of the model are subjected to probability mapping processing to transform the data in the feature space into data in the probability space, thereby calculating the state probability distribution vector of the disconnect switch corresponding to different action state categories. Finally, based on the state probability distribution vector of the disconnect switch corresponding to different action state categories, the final target action state is selected and determined, and the judgment confidence of the target action state is output simultaneously.

[0154] For example, in the specific low-level computation of decision-making using a temporal state reasoning model, a feature sequence containing the complete action cycle can be input as the target evolution sequence into a specific inference function, and the computation can be performed based on the following formula:

[0155]

[0156] in, That is, the characteristics of each joint state The mathematical expression for extracting evolutionary features from the composed sequence; the extracted evolutionary features are multiplied by the output weight matrix. And add a bias term Then, enter to The activation function performs probability mapping to output a normalized state probability distribution vector. When determining the final result, an optimization formula can be used. The action category corresponding to the highest probability component in the state probability distribution vector is directly determined as the target action state, and this highest probability value is used as the target action state. Extracted as the confidence level for this state determination .

[0157] In this embodiment, by inputting the target evolution sequence into the time-series state reasoning model for evolution feature extraction and probability mapping, and determining the final target action state and judgment confidence based on the state probability distribution vector, a global extrapolation from the micro-state at a local time point to the macro-state of the entire operation cycle is realized. This not only accurately outputs the final physical state of the equipment, but also provides a reliability assessment index of the state in the form of a quantitative score. This provides highly valuable confidence support for the risk assessment and operation and maintenance scheduling of the substation back-end system under complex operating conditions.

[0158] In one embodiment, the disconnector state confirmation method based on multimodal temporal state reasoning of this application can be applied to, for example, Figure 2The system shown consists of sensing devices (motor current acquisition device, auxiliary contact acquisition device, and edge-end visual state perception terminal), a state analysis platform, and an application layer. The motor current acquisition device continuously collects raw current waveform data of the disconnector switch drive motor to determine drive current characteristics, including motor starting characteristics, operating current characteristics, and load change characteristics. The auxiliary contact acquisition device continuously collects auxiliary contact state change data, such as open / closed position status, mechanism action status, and auxiliary signal acquisition. The edge-end visual state perception terminal, installed at the observation position of the disconnector switch contacts, performs real-time video acquisition of the disconnector switch's operation process and performs target detection, motion tracking, and key point extraction locally. The edge-end visual state perception terminal includes an image acquisition module, a target detection module, a motion tracking module, a key point extraction module, and a feature upload module. Specifically: the image acquisition module acquires video of the disconnector switch contact action; the target detection module identifies the contact area and the action target; the motion tracking module continuously tracks the contact movement trajectory; the key point extraction module extracts contact edges, contact areas, and key action feature points; and the feature upload module uploads structured visual motion features and keyframe information. The edge-side visual state perception terminal does not directly determine the final state, but only completes visual motion perception and structured feature extraction. In some optional embodiments, the system also has an expansion interface (reserved), including but not limited to vibration sensors, acoustic sensors, stroke sensors, and infrared sensors, to form richer and more versatile multimodal motion feature sequences to improve the accuracy of state confirmation during the action process. Based on the method described in this application, this application also provides an intelligent camera (visual sensor) applied to the method, which has a built-in opening and closing recognition algorithm to identify the contact position, linkage movement, and action process in real time to achieve high-precision recognition: real-time processing at the edge reduces network transmission pressure and improves response speed to achieve edge intelligent analysis; it has strong light suppression, low light enhancement, anti-shake, dustproof and waterproof capabilities to adapt to complex working conditions; it operates in a wide temperature range, is resistant to electromagnetic interference, and has a robust structure to meet the on-site operation requirements of substations; it supports multi-angle installation and is compatible with different models of disconnect switches and on-site installation conditions.

[0159] In one specific embodiment, the following is executed in the system: Figure 3The method described achieves accurate state confirmation through synchronous sensing of multi-source heterogeneous signals, action state chain modeling, and multimodal temporal reasoning. In one embodiment, data access and management are responsible for accessing current data, contact data, and visual feature data. Multi-source data processing and feature extraction are performed, including current feature extraction (including average current, peak current, current fluctuation, action duration, etc.), contact feature extraction (including contact state, action time, delay time, etc.), and visual feature processing (including position features, motion speed features, gap / contact depth features, key rotation action labels, etc.). Action state modeling and reasoning involve constructing an action state chain (S0~S6) according to the time sequence, with the device sequentially experiencing the stages of S0 (stationary), S1 (starting), S2 (mechanical transmission), S3 (linkage movement), S4 (contact switching), S5 (positioning), and S6 (stabilization). Furthermore, state reasoning and analysis are completed through multimodal state correlation analysis, state transition reasoning, temporal consistency analysis, and constraint optimization. Abnormal state identification: The system identifies various abnormalities based on characteristics, including "false positioning identification" where the virtual contact has switched but there is still a visual gap; "jamming identification" where the current continues to rise but the movement speed decreases; "failure to operate identification" where the current initiation characteristics are obvious but there are no subsequent action characteristics; and "contact malfunction identification" where the contact operates but there are no corresponding action characteristics in the current and vision. Result output and storage: The system uniformly stores results, records events, and manages models and parameters.

[0160] Specifically, the intelligent camera installed on-site (edge-end visual state perception terminal) continuously detects targets in the contact area, extracting contact position features, linkage speed features, and contact gap features in real time. Simultaneously, the motor current acquisition device acquires three-phase current waveform data during the start-up and operation of the drive motor; the auxiliary contact acquisition device records the opening and closing positions and mechanism action status signals. To eliminate time delay differences between different sensors, the state analysis platform uses a preset time-domain sliding window to strictly align the time scale of these three data streams, thereby generating a multimodal action feature sequence containing visual, current, and contact information at each moment.

[0161] As the closing action proceeds, the system analyzes the evolution of the state frame by frame according to the preset action state chain. At a specific moment, the system maps the currently extracted visual, current, and contact features into the state inference space. Crucially, the system dynamically assigns confidence weights based on the action state of the previous moment. For example, when the system confirms that the previous moment was in the startup phase, it can assign higher weights to current features; while after entering the transmission phase, the weight of visual features can be increased. Based on these dynamic weights, the system fuses multimodal features into joint state features and, combined with the action state transition relationships, preliminarily infers the current action evolution state.

[0162] The system further retrieves visual state constraints, current characteristic constraints, auxiliary contact state constraints, and action timing and stability constraints for joint constraint solving. In a specific embodiment, when the switch reaches the critical node where the linkage motion transitions to contact switching, the motor encounters significant resistance due to rust caused by long-term lack of maintenance. At this time, the drive motor current characteristic shows a continuous upward trend (the motor attempts to output greater torque), but the visual characteristics show a significant downward trend or even stagnation in the linkage motion speed of the isolating switch. The system can verify and lock the target evolution state at this moment as a mechanism jamming abnormal state. In addition, if the system finds that the auxiliary contact has switched, but the visual characteristics show that the contact gap is still greater than the preset threshold, the target evolution state is determined to be a false positioning; if the drive motor current characteristic is obvious but other action characteristics indicate no action, the target evolution state is determined to be a refusal to operate abnormal state; if the auxiliary contact state characteristic indicates action, but other characteristics indicate no action, the target evolution state is determined to be a contact maloperation abnormal state.

[0163] After the entire action process is completed, the system concatenates the verified target evolution states at each moment within the action process in chronological order to form a complete target evolution sequence. This sequence is then input into a pre-trained temporal state inference model for deep evolution feature extraction and probability mapping processing. The model can calculate the state probability distribution vector corresponding to different state categories for the current operation, store the intermediate data in the above state confirmation process, generate event logs, and manage the parameters and model of the above application model. The judgment result is then transferred to the application layer, where the system can implement visual analysis after action state confirmation, such as real-time state monitoring, action process curve display, multimodal feature visualization, historical record query and playback, and report generation. In a specific embodiment, when an abnormal action is confirmed, the application layer can issue an alarm through an alarm management control, such as audible and visual alarms, SMS or software push notifications, email notifications, or voice broadcasts. In an optional embodiment, external business personnel can call the visual analysis to achieve report generation, equipment health assessment, operation and maintenance decision support, system integration, etc.

[0164] In this embodiment, dynamic analysis of the entire process of disconnecting switch operation is achieved. Traditional solutions mostly only perform static result judgments on "open" or "closed" positions, which cannot reflect the continuous mechanical behavior during the disconnecting switch operation. This invention constructs an action state chain to achieve dynamic analysis of the entire process of action stages, including start-up, transmission, linkage movement, contact switching, and stabilization, thereby improving the equipment's action process analysis capabilities. It also breaks through the bottleneck of asynchronous heterogeneous data, significantly improving the accuracy and robustness of state determination. Existing technologies often rely on a single sensor or use simple threshold concurrent logic, which cannot handle the asynchronous timestamp problem between visual images (low-frequency two-dimensional) and electrical signals (high-frequency one-dimensional). This application provides a time-domain sliding window alignment technique based on high-frequency sampling, achieving strong anchoring of cross-modal data at the micro-timescale. It also introduces a dynamic weight allocation mechanism, enabling the system to automatically adjust the confidence weights of each modality's data according to different stages of the disconnector's operation (e.g., startup, acceleration, contact engagement), effectively overcoming the shortcomings of traditional fixed-weight algorithms prone to false alarms under complex operating conditions. Furthermore, it establishes a rational reasoning mechanism based on physical consistency verification, significantly reducing the false alarm rate. This application abandons the threshold judgment logic of a single sensor, constructing a cross-validation model between visual kinematic features (displacement / velocity) and electrical features (current energy / work). When complex environments (e.g., sudden changes in illumination, foreign object obstruction, auxiliary contact jitter) cause distortion of single-modal data, the system utilizes the consistency residuals of multi-source data for logical compensation and reasoning. This "multi-party verification" decision-making model effectively identifies and filters out interference signals from various non-genuine faults, ensuring the uniqueness and accuracy of the judgment conclusion. It also enhances the ability to identify complex abnormal states. By establishing a multimodal state consistency analysis and action timing constraint model, this application enables the identification of complex abnormal states such as mechanism jamming, motion stagnation, refusal to separate or engage, auxiliary contact malfunctions, and false positioning, thereby improving equipment anomaly analysis capabilities. Furthermore, it improves system real-time performance and scalability. This application adopts a combination of edge-end visual perception and unified backend state reasoning to achieve rapid perception and multimodal joint analysis of the action process. Simultaneously, the system possesses excellent scalability, allowing for the subsequent integration of other sensors such as vibration, infrared, and acoustic sensors to continuously expand its multimodal state analysis capabilities.

[0165] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0166] Based on the same inventive concept, this application also provides a device for confirming the state of a disconnector switch based on multimodal timing state reasoning, used to implement the aforementioned method for confirming the state of a disconnector switch based on multimodal timing state reasoning. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the device for confirming the state of a disconnector switch based on multimodal timing state reasoning provided below can be found in the limitations of the method for confirming the state of a disconnector switch based on multimodal timing state reasoning described above, and will not be repeated here.

[0167] In one exemplary embodiment, such as Figure 4 As shown, a disconnector switch state confirmation device based on multimodal temporal state reasoning is provided, comprising: a feature acquisition module 410, an action prediction module 420, a state verification module 430, a prediction sequence determination module 440, and an action state determination module 450, wherein:

[0168] The feature acquisition module 410 is used to acquire the multimodal action feature sequence of the disconnecting switch during the operation process; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features and auxiliary contact status features.

[0169] The action prediction module 420 is used to determine the action evolution state at each moment based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment.

[0170] The state verification module 430 is used to verify the action evolution state at each moment based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action timing constraint terms corresponding to the action state transition relationship.

[0171] The prediction sequence determination module 440 is used to combine the verified target evolution states at each moment in the action process in chronological order to obtain the target evolution sequence.

[0172] The action state determination module 450 is used to determine the target action state of the disconnecting switch based on the target evolution sequence.

[0173] In one embodiment, the feature acquisition module 410 is further configured to:

[0174] The contact area of ​​the disconnector switch is continuously detected using an edge-mounted visual state perception terminal. Based on the contact position features, movement speed features, contact gap features, and keyframe action tags obtained from the target detection, a visual feature sequence is determined. Based on the pre-acquired drive motor current waveform data of the disconnector switch, a current feature sequence is determined. Based on the pre-acquired auxiliary contact state change data, an auxiliary contact state feature sequence is determined. For the visual feature sequence, the current feature sequence, and the auxiliary contact state feature sequence, time scale alignment processing is performed based on a preset time-domain sliding window to determine the multimodal action feature sequence.

[0175] In one embodiment, the action prediction module 420 is further configured to:

[0176] The visual features, drive motor current features, and auxiliary contact state features at each time step are mapped to a unified state inference space. Multimodal action features in the unified state inference space are obtained based on the mapping results. The confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features at each time step are determined based on the action state of the disconnector switch at the previous time step. The confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features differ depending on the action state of the disconnector switch. Based on the confidence weights corresponding to the visual features, drive motor current features, and auxiliary contact state features at each time step, the multimodal action features in the unified state inference space are fused to obtain joint state features. The action evolution state at each time step is determined based on the joint state features, the action state of the disconnector switch at the previous time step, and the action state transition relationship.

[0177] In one embodiment, the status verification module 430 is further configured to:

[0178] Based on the multimodal action characteristics, the visual state constraints, current characteristic constraints, and auxiliary contact state constraints of the disconnecting switch during the action process are determined; the state stability constraints and the action timing constraints corresponding to the action state transition relationship are obtained; based on the visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, the action evolution state at each moment is jointly constrained and solved to obtain the verified target evolution state at each moment.

[0179] In one embodiment, the status verification module 430 is further configured to:

[0180] If the auxiliary contact state feature corresponding to the action evolution state indicates that the contact has been switched, and the contact gap feature in the corresponding visual feature is greater than a preset gap threshold, then the target evolution state is determined to be a false positioning abnormal state; if the drive motor current feature corresponding to the action evolution state indicates that the drive motor current shows an increasing trend, and the movement speed of the disconnect switch shows a decreasing trend in the corresponding visual feature, then the target evolution state is determined to be a mechanism jamming abnormal state.

[0181] In one embodiment, the action state determination module 450 is further configured to:

[0182] The target evolution sequence is input into a preset temporal state reasoning model. Evolutionary features are extracted from the target evolution sequence through the temporal state reasoning model, and the extracted evolutionary features are subjected to probability mapping processing to calculate the state probability distribution vector of the disconnect switch corresponding to different action state categories. Based on the state probability distribution vector of the disconnect switch corresponding to different action state categories, the target action state and the determination confidence of the target action state are determined.

[0183] Each module in the aforementioned disconnector state confirmation device based on multimodal timing state reasoning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0184] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for confirming the state of an isolating switch based on multimodal timing state reasoning.

[0185] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0186] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0187] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0188] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0189] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0190] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0191] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0192] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A disconnector state confirmation method based on multi-modal temporal state reasoning, characterized in that, The method includes: Obtain the multimodal action feature sequence of the disconnecting switch during its operation; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features; Based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, the action evolution state at each moment is determined. The action evolution state at each moment is verified based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action temporal constraint terms corresponding to the action state transition relationship. The target evolution states verified at each moment during the action process are combined in chronological order to obtain the target evolution sequence; The target operating state of the disconnecting switch is determined based on the target evolution sequence.

2. The method of claim 1, wherein, The determination of the action evolution state at each moment, based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment, includes: The visual features, drive motor current features, and auxiliary contact state features at each moment are mapped to a unified state inference space, and the multimodal action features in the unified state inference space are obtained based on the mapping results. Based on the operating state of the disconnecting switch at the previous moment, the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features at each moment are determined; wherein, the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features are different in different operating states of the disconnecting switch. Based on the confidence weights corresponding to the visual features, the drive motor current features, and the auxiliary contact state features at each time step, the multimodal action features in the unified state inference space are fused to obtain joint state features. Based on the joint state characteristics, the operation state of the disconnector at the previous moment, and the operation state transition relationship, the operation evolution state at each moment is determined.

3. The method of claim 1, wherein, The optimization model based on preset state constraints verifies the action evolution state at each moment to obtain the verified target evolution state at each moment, including: Based on the multimodal action characteristics, the visual state constraints, current characteristic constraints, and auxiliary contact state constraints of the disconnecting switch during the action process are determined. Obtain the state stability constraints and the action timing constraints corresponding to the action state transition relationship; Based on the visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, the action evolution state at each moment is jointly constrained and solved to obtain the verified target evolution state at each moment.

4. The method of claim 3, wherein, The method involves jointly solving for the action evolution state at each moment based on the visual state constraints, current characteristic constraints, auxiliary contact state constraints, action timing constraints, and state stability constraints, to obtain the verified target evolution state at each moment, including: If the auxiliary contact state feature corresponding to the action evolution state indicates that the contact has been switched, and the contact gap feature in the corresponding visual feature is greater than the preset gap threshold, then the target evolution state is determined to be a false positioning abnormal state. If the current characteristic of the drive motor corresponding to the action evolution state shows an increasing trend in the drive motor current, and the speed of the disconnect switch corresponding to the visual characteristic shows a decreasing trend, then the target evolution state is determined to be a mechanism jamming abnormal state.

5. The method of claim 1, wherein, Determining the target action state of the disconnector switch based on the target evolution sequence includes: The target evolution sequence is input into a preset temporal state reasoning model. The temporal state reasoning model is used to extract evolution features from the target evolution sequence. The extracted evolution features are then subjected to probability mapping processing to calculate the state probability distribution vector of the disconnect switch corresponding to different action state categories. Based on the state probability distribution vector of the disconnecting switch corresponding to different action state categories, the target action state and the confidence level of the target action state are determined.

6. The method according to any one of claims 1 to 5, characterized in that, The acquisition of the multimodal action feature sequence of the disconnecting switch during its operation includes: The edge-mounted visual state perception terminal is used to perform continuous target detection on the contact area of ​​the disconnecting switch. Based on the contact position features, movement speed features, contact gap features and key frame action tags obtained from the target detection, a visual feature sequence is determined. Based on the pre-acquired drive motor current waveform data of the disconnecting switch, a current characteristic sequence is determined, and based on the pre-acquired auxiliary contact state change data, an auxiliary contact state characteristic sequence is determined. For the visual feature sequence, the current feature sequence, and the auxiliary contact state feature sequence, time scale alignment processing is performed based on a preset time-domain sliding window to determine the multimodal action feature sequence.

7. A disconnector status confirmation device based on multi-modal temporal state reasoning, characterized in that The device includes: The feature acquisition module is used to acquire the multimodal action feature sequence of the disconnecting switch during the operation process; wherein, the multimodal action feature sequence includes multimodal action features at multiple times, and the multimodal action features include at least visual features, drive motor current features, and auxiliary contact state features; The action prediction module is used to determine the action evolution state at each moment based on the preset action state transition relationship, the multimodal action features at each moment in the multimodal action feature sequence, and the action state of the disconnecting switch corresponding to the previous moment. The state verification module is used to verify the action evolution state at each moment based on a preset state constraint optimization model to obtain the verified target evolution state at each moment; wherein, the state constraint optimization model is determined according to the state constraint terms corresponding to the multimodal action features and the action timing constraint terms corresponding to the action state transition relationship. The prediction sequence determination module is used to combine the verified target evolution states at each moment in the action process in chronological order to obtain the target evolution sequence; The action state determination module is used to determine the target action state of the disconnecting switch based on the target evolution sequence. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images