A monitoring device management subsystem human-computer interaction method and system

Abstract

The application discloses a kind of monitoring equipment management subsystem man-machine interaction method and system, comprising: emergency plan is decomposed into atomic action block containing time offset and physical constraint, and it is shown as draggable card according to offset order. After operator modifies offset or drags order, constraint solver detects the consistency of modified parameter and constraint. If it is passed, update card and reorganize plan and send execution engine;If it is not passed, lock card, highlight frame and pop-up window show conflict and replaceable offset value list. Operator selects alternative value and repeats detection until it is passed, and then unlock. The application effectively solves the problems of high modification cost, feedback lag and lack of explainable conflict repair mechanism in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of video surveillance and artificial intelligence technology, and in particular to a human-computer interaction method and system for a monitoring equipment management subsystem. Background Technology

[0002] With the deep application of artificial intelligence technology in the security monitoring field, modern monitoring equipment management subsystems have generally integrated intelligent event detection and automatic response capabilities. When the system detects an evolving security event (such as abnormal crowd gathering, early signs of fire, or unauthorized intrusion), it can automatically generate a structured emergency plan. This plan typically includes: a list of recommended cameras to be activated and their switching order, the security personnel or emergency response teams to be notified, preset PTZ (pan-tilt-zoom) action sequences (such as rotation angle and preset position activation), recording tagging, and storage strategies. Theoretically, this function can significantly reduce operator decision-making and operation time in emergency situations, improving emergency response efficiency.

[0003] However, practical application research has revealed a clear and unresolved technical problem in the human-computer interaction aspect of existing AI-generated emergency plans: there is a structural contradiction between the editable granularity of the plan and the operator's need for rapid intervention, and the modified plan lacks a feasibility verification mechanism with spatiotemporal constraints.

[0004] Specifically, existing systems offer two extreme approaches to AI-generated emergency plans, neither of which can meet the real-world interaction needs of operators in emergency scenarios.

[0005] The first approach: The system only generates a coarse-grained summary of the contingency plan, such as listing macro-level steps like invoking the cameras at the east, south, and west gates to notify the security captain to rotate the PTZ to preset position 2. While the operator can edit this text list (e.g., deleting the west gate or adding the north gate), the lack of specific parameters for each action (such as the camera's exact preset position number, the precise angle and speed of PTZ rotation, and the time interval between actions) forces the operator to manually fill in numerous missing details. This results in modifications taking over 30 seconds in emergency situations, making it slower than manually operating from scratch. Essentially, this approach completely shifts the editing burden to the operator, failing to meet the timeliness requirements of emergency scenarios.

[0006] The second method involves the system generating a highly specific sequence of actions. For example, after 2 seconds, the pan-tilt unit of camera #1043 will rotate horizontally by 45° at a speed of 30° / s; after 5 seconds, preset position 3 will be invoked, and the main screen will be switched to #1043; after 8 seconds, message ID_ALERT_02 will be sent to the security terminal. This sequence cannot be directly edited—operators cannot modify any parameter through simple interface operations (such as dragging, selecting, or replacing); they can only accept this black-box sequence or completely abandon it and operate manually. When the operator wants to make simple adjustments (such as changing the east gate camera to the west gate camera and delaying the PTZ rotation speed by 10 seconds to avoid image blur), the system does not support such local modifications, forcing the operator to terminate the automated plan and revert to fully manual mode.

[0007] More critically, the existing system completely lacks the ability to verify the feasibility of modified contingency plans under spatiotemporal constraints. When an operator manually changes a camera call command from camera A to camera B, the system does not automatically check: whether camera B's current field of view covers the target event area, whether camera B's PTZ rotation range can reach the required angle within the specified time, or whether the modified action sequence conflicts with subsequent actions (such as multi-camera linkage or personnel notification timing). For example, if an operator changes the call command from the east gate camera to the west gate camera, but the west gate camera's pan / tilt unit is unavailable due to maintenance, or the west gate camera's preset position has not yet been configured, the contingency plan will fail at this step without any system notification. Similarly, if an operator changes the delay of a PTZ action from 5 seconds to 15 seconds, it may cause a blind window in the subsequent capture of critical footage, resulting in missing event records. Such situations where modifications cause contingency plans to partially or completely fail in actual execution occur frequently in the existing system. Operators often only discover the problem after execution fails, seriously delaying emergency response.

[0008] In summary, existing monitoring equipment management subsystems face the following pressing technical challenges in the interactive layer of AI-generated emergency plans: how to allow operators to quickly modify any key parameters in the AI-generated plan at extremely low cost (e.g., with a single click, drag-and-drop, or keyword replacement), while simultaneously verifying the feasibility of the modified plan under physical equipment constraints and temporal logic constraints in real time, and providing clear conflict identification and alternative suggestions for infeasible modifications. Currently, no human-computer interaction method can simultaneously meet the dual requirements of low-burden, rapid intervention and real-time constraint feasibility verification, which has become a key bottleneck restricting the practical effectiveness of intelligent monitoring systems in emergency response. Summary of the Invention

[0009] To address the above problems, this invention provides a human-computer interaction method and system for a monitoring equipment management subsystem.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] On one hand, this invention discloses a human-computer interaction method for a monitoring equipment management subsystem, comprising:

[0012] Step 1: In response to the security incident detection signal, decompose the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints.

[0013] Step 2: Display the multiple atomic action blocks in a draggable card list in ascending order of their respective execution time offsets. Each card corresponding to an atomic action block contains a text label and an execution time offset value edit box.

[0014] Step 3: Respond to the operator's modification operation on the execution time offset value edit box of any atomic action block card or the drag and sort operation on the card, and replace the original time constraint parameters with the modified new time constraint parameters to generate the modified atomic action block;

[0015] Step 4: Call the constraint solver to perform a consistency check between the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true or false value.

[0016] Step 5: When the consistency detection outputs a true value, update the display parameters of the card according to the modified atomic action block, and reassemble all atomic action blocks into a complete emergency plan before sending it to the execution engine;

[0017] Step 6: When the consistency detection outputs a false value, lock the editing state of the card and highlight the border of the card. At the same time, pop up a modal dialog box to display the conflict constraint name, conflict device identifier and alternative execution time offset value list that caused the false value.

[0018] Step 7: The responding operator confirms the selection of a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency check outputs a true value and the card is unlocked.

[0019] Further: Step 1 includes:

[0020] The emergency response plan output by the multimodal generation model is parsed. The emergency response plan is a JSON structure with timestamp tags. Each action instruction in the JSON structure is extracted in ascending order of timestamp. Each action instruction includes at least an action type field, a target device field, and an offset field.

[0021] Each extracted action instruction creates an atomic action block. The data structure of the atomic action block includes an action type identifier field, a target device identifier field, an execution time offset field, and a pointer to a set of physical feasibility constraints that points to a pre-stored device parameter database.

[0022] Based on the target device field in the action instruction, query the pre-stored device parameter database to obtain the target device's maximum rotation speed parameter, preset position existence flag, and minimum time interval threshold between it and adjacent atomic action blocks. Encapsulate the maximum rotation speed parameter, preset position existence flag, and minimum time interval threshold into a set of physical feasibility constraints, and store the pointer of this set into the corresponding atomic action block.

[0023] Further: Step 2 includes:

[0024] Based on the order of execution time offsets carried by each atomic action block from smallest to largest, each atomic action block is mapped to an independent draggable card. The upper left corner of the draggable card displays a text label indicating the action type of the atomic action block, and the upper right corner displays a text label indicating the target device of the atomic action block.

[0025] A numerical step control component is embedded in the middle area of ​​the draggable card as an execution time offset value editing box. The numerical step control component allows the operator to click the up arrow to increase the value or click the down arrow to decrease the value, with a minimum adjustment step of 1 second.

[0026] Arrange all draggable cards vertically in order in the main interactive panel of the graphical user interface. Adjacent cards are separated by a gray dividing line with a width of 1 pixel. Each draggable card can be dragged up and down by holding down the left mouse button to change its position in the main interactive panel.

[0027] Furthermore, step 3 includes:

[0028] When it is detected that an operator modifies the value of the execution time offset in any atomic action block card, the new value is captured, the new value is used as the new time constraint parameter, and the new time constraint parameter is written to the execution time offset field of the atomic action block card to overwrite the original time constraint parameter, and the modified atomic action block is generated.

[0029] When it is detected that the operator performs a drag-and-drop sorting operation on any atomic action block card by holding down the left mouse button and moving it up and down and then releasing it, the new sorting number corresponding to the release position of the atomic action block card in the list of draggable cards is used to calculate the new time offset value relative to the previous atomic action block and the next atomic action block implied by the new sorting number. The new time offset value is used as the modified new time constraint parameter to replace the original time constraint parameter and generate the modified atomic action block.

[0030] The modified new time constraint parameters in the modified atomic action block are synchronously updated to the execution time offset value edit box of the atomic action block card, and the other fields in the modified atomic action block are kept at the values ​​before the original time constraint parameters were overwritten.

[0031] Furthermore, step 4 includes:

[0032] Extract the modified time constraint parameters from the modified atomic action block. The modified time constraint parameters include the new execution time offset value and the new sorting position of the modified atomic action block in the sequence of all atomic action blocks. At the same time, retrieve the set of physical feasibility constraints from the modified atomic action block.

[0033] The constraint solver is invoked to perform three consistency checks in sequence. The first check is to determine whether the new execution time offset value is less than or equal to the limit response time allowed by the maximum rotation speed parameter of the device associated with the target device identifier corresponding to the modified atomic action block. The second check is to determine whether the preset bit existence flag associated with the target device identifier is true. The third check is to determine whether the difference in execution time offset between the previous atomic action block and the next atomic action block adjacent to the new sort position is greater than or equal to the shortest time interval threshold in the physical feasibility constraint set.

[0034] When all three consistency checks pass, the constraint solver outputs a logical true value; when any one of the three consistency checks fails, the constraint solver outputs a logical false value and records the conflicting constraint name and conflicting device identifier corresponding to the failed check.

[0035] Furthermore: Step 5 includes:

[0036] The modified time constraint parameters carried by the modified atomic action block are rendered to the execution time offset value editing box of the corresponding card. At the same time, the border color of the card is restored from orange in the pending state to the default gray, and the drag function of the card is restored to the available state.

[0037] Based on the current order of all atomic action blocks in the draggable card list and the latest execution time offset value of each atomic action block, serialize all atomic action blocks into a JSON object with complete time sequence. Each element in the JSON object includes an action type identifier, a target device identifier, and an updated execution time offset.

[0038] The JSON object, representing the complete emergency plan, is sent to the execution engine via the internal process communication interface of the monitoring equipment management subsystem. Upon receiving the JSON object, the execution engine returns a confirmation signal indicating receipt.

[0039] Furthermore: Step 6 includes:

[0040] Disable the execution time offset value edit box of the modified atomic action block card corresponding to the output false value, temporarily turn off the drag sorting function of the card, and change the border color of the card from the default gray to a flashing red border for highlighting.

[0041] A non-modal dialog box pops up in the central area of ​​the graphical user interface. The title bar of the non-modal dialog box displays a consistency check failure message. The main area of ​​the non-modal dialog box displays the name of the conflicting constraint that caused the false value, the identifier of the conflicting device, and a list of three candidate execution time offset values ​​generated in real time by the constraint solver based on the current device state database.

[0042] At the bottom of the non-modal dialog box, a confirmation button and a cancel button are displayed. When the operator clicks the confirmation button, the candidate execution time offset value selected by the operator in the list is output as the alternative execution time offset value. When the operator clicks the cancel button, the non-modal dialog box is closed and no changes are made to the modified atomic action block card.

[0043] Furthermore: Step 7 includes:

[0044] In response to the operator clicking the confirmation button in the modal dialog box and selecting a target value from the list of alternative execution time offset values, the target value is captured and written as a new modified time constraint parameter into the execution time offset field of the modified atomic action block card;

[0045] Using the modified atomic action block card carrying the new modified time constraint parameters as input, steps 3, 4 and 5 are called sequentially. This step is repeated when a false value is output again in step 4, and jumps to this step when a true value is output in step 4.

[0046] After step 4 outputs a true value and step 5 is completed, the execution time offset value edit box of the modified atomic action block card is switched from disabled to enabled, the drag-and-drop sorting function of the card is switched from disabled to enabled, and the border color of the card is restored from the flashing red border to the default gray, thus unlocking the card.

[0047] On the other hand, this invention discloses a human-computer interaction system for a monitoring equipment management subsystem, comprising:

[0048] The contingency plan atomization module responds to security event detection signals by decomposing the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints.

[0049] The timing visualization mapping module displays the multiple atomic action blocks in an ascending order of their respective execution time offsets in the form of a draggable card list. Each card corresponding to an atomic action block contains a text label and an execution time offset value editing box.

[0050] The interactive editing mapping module responds to the operator's modification of the execution time offset value editing box of any atomic action block card or the drag-and-drop sorting operation of the card, and generates a modified atomic action block by replacing the original time constraint parameters with the modified new time constraint parameters.

[0051] The constraint consistency verification module calls the constraint solver to perform consistency checks on the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true value or a false value.

[0052] When the consistency check outputs a true value, the contingency plan confirmation and release module updates the display parameters of the card according to the modified atomic action blocks, and reassembles all atomic action blocks into a complete contingency plan before sending it to the execution engine.

[0053] The conflict feedback locking module locks the editing state of the card and highlights the border of the card when the consistency detection outputs a false value. At the same time, a modal dialog box pops up to display the name of the conflict constraint that caused the false value, the identifier of the conflicting device, and a list of alternative execution time offset values.

[0054] The constraint repair convergence module responds to the operator's confirmation operation of selecting a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency detection outputs a true value and the card's locked state is released.

[0055] The technological advancements achieved by this invention compared to existing technologies are as follows:

[0056] This invention achieves a unified approach to low-cost editing and real-time constraint verification of AI-generated emergency plans by constructing an interactive emergency plan reconstruction mechanism based on atomic action blocks. At the operational level, it supports lightweight interactive methods such as single clicks, drag-and-drop, and numerical adjustments to modify key parameters. At the system level, a constraint consistency verification module instantly verifies the modified time constraints and equipment physical constraints, automatically locating the source of conflict and associated equipment when conflicts are detected, and generating a candidate set of alternative execution time offsets. This effectively solves the problems of high cost of plan modification, delayed feedback, and lack of explainable conflict resolution mechanisms in existing technologies, significantly improving the usability and real-time response efficiency of emergency plans. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0058] In the attached diagram:

[0059] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0060] The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0061] Example 1

[0062] like Figure 1 As shown, the present invention discloses a human-computer interaction method for a monitoring equipment management subsystem, including:

[0063] Step 1: In response to the security incident detection signal, decompose the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints.

[0064] Step 2: Display the multiple atomic action blocks in a draggable card list in ascending order of their respective execution time offsets. Each card corresponding to an atomic action block contains a text label and an execution time offset value edit box.

[0065] Step 3: Respond to the operator's modification operation on the execution time offset value edit box of any atomic action block card or the drag and sort operation on the card, and replace the original time constraint parameters with the modified new time constraint parameters to generate the modified atomic action block;

[0066] Step 4: Call the constraint solver to perform a consistency check between the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true or false value.

[0067] Step 5: When the consistency detection outputs a true value, update the display parameters of the card according to the modified atomic action block, and reassemble all atomic action blocks into a complete emergency plan before sending it to the execution engine;

[0068] Step 6: When the consistency detection outputs a false value, lock the editing state of the card and highlight the border of the card. At the same time, pop up a modal dialog box to display the conflict constraint name, conflict device identifier and alternative execution time offset value list that caused the false value.

[0069] Step 7: The responding operator confirms the selection of a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency check outputs a true value and the card is unlocked.

[0070] Step 1 aims to transform the overall emergency response plan output by the multimodal generation model from a holistic semantic-level task description into a set of atomic action blocks that can be independently edited, verified, and scheduled. This process is not a simple data splitting, but a structured reconstruction of the plan's internal action logic, device constraints, and time execution relationships. This ensures that subsequent human-computer interaction modifications directly affect the smallest control granularity, while guaranteeing that the subsequent constraint solver can perform real-time consistency checks on each modification. When the monitoring equipment management subsystem receives a security event detection signal, the event scheduling module immediately triggers the multimodal generation model to output the corresponding emergency response plan. The input to the multimodal generation model includes not only the target behavior features in the video stream, but also historical linkage strategies, device online status, the current monitoring topology, and a predefined emergency rule base. Therefore, its output is not plain text, but a structured JSON object with timestamps.

[0071] In the JSON structure, each action command corresponds to a monitoring linkage behavior to be executed, such as calling a specific camera, switching the main view, triggering pan-tilt rotation, sending a notification message, or enabling recording markers. Each action command includes at least the following fields:

[0072] The action type field is used to identify whether the current action is a camera switching action, PTZ control action, notification action, or recording action;

[0073] The target device field identifies the specific device number associated with the current action;

[0074] The offset field indicates the time offset of the action relative to the start time of the emergency plan.

[0075] After receiving the JSON structure, the system's contingency plan parsing module iterates through all action instructions in ascending timestamp order. This ascending timestamp parsing method is used because actions in the emergency plan have inherent time dependencies. If a subsequent action executes before a preceding action, recording might begin before the monitoring perspective has been switched, or personnel dispatch might be executed before a notification has been sent, thus disrupting the linkage logic. After sorting by time, the system creates a corresponding atomic action block for each action instruction. In this embodiment, an atomic action block represents the smallest indivisible execution unit; essentially, it is a data object with complete execution semantics and constraint information. Each atomic action block includes:

[0076] Action type identifier field;

[0077] Target device identification field;

[0078] Execution time offset field;

[0079] Pointer to the set of physical feasibility constraints.

[0080] The action type identifier field is used to maintain the human-computer readability of the subsequent interface layer. For example, the interface can directly display text labels such as "Call Camera", "PTZ Rotation", and "Notify Security Group". The target device identifier field is used to establish a unique mapping relationship between the action and the actual monitoring device. The execution time offset field is used for subsequent drag-and-drop sorting and time editing operations. The physical feasibility constraint set pointer is used to establish the association between the current action and the underlying device capability model.

[0081] After creating the atomic action block, the system accesses the pre-stored device parameter database based on the target device field. This database is pre-maintained by the monitoring device management subsystem and stores the capability constraint parameters for each monitoring device. Different types of devices have different physical constraints; therefore, the system needs to dynamically extract the corresponding constraint set based on the target device type. When the target device is a PTZ camera, the system reads the following parameters:

[0082] Maximum rotational speed parameters of the equipment;

[0083] Preset existence flag;

[0084] Shortest time interval threshold.

[0085] The maximum rotational speed parameter is used to characterize the maximum angular velocity of rotation that the camera pan-tilt unit is allowed to reach per unit time. For example, the maximum horizontal rotational speed of a certain PTZ camera is... If the current action requires the camera to complete a 90° horizontal rotation within 2 seconds, then the theoretical time required is... ,in, This represents the target rotation angle. When the calculated theoretical time is greater than the execution window allowed for the current action, the subsequent constraint solver can determine that the action is not executable.

[0086] The preset bit existence flag indicates whether the preset bit invoked by the current action has been configured in the device. Due to issues such as missing preset bit numbers, accidental deletion of preset bits, or incomplete device initialization in many monitoring systems, the system must establish preset bit validity associations in advance. If the target preset bit does not exist, the atomic action block will be directly judged as a conflicting action in the subsequent consistency check phase. The shortest time interval threshold constrains the minimum allowable time difference between adjacent actions. For example, when the same PTZ camera performs two consecutive large-angle rotation actions, time for mechanical deceleration and stabilization needs to be reserved. If the time interval between two actions is too short, it may lead to image jitter, incomplete rotation, or subsequent actions overwriting previous actions. Therefore, the system establishes the following constraint relationship: ,in, Indicates the time interval between adjacent actions. This indicates the minimum allowed action interval threshold for the device.

[0087] The system encapsulates the maximum rotational speed parameter of the aforementioned equipment, the existence flag of preset positions, and the minimum time interval threshold into a set of physical feasibility constraints. This set is not directly copied and stored within the atomic action block; instead, it is associated with the equipment parameter database using pointer references. The reason for using pointer references is that equipment states have dynamic changing characteristics. For example, a camera may enter maintenance mode after the emergency plan is generated, or a preset position may be reconfigured. If a static parameter copying method is used, subsequent consistency checks cannot obtain the latest equipment state. By using the constraint set pointer method, the constraint solver can access the current equipment state database in real time during consistency checks, thereby ensuring that the modified plan is always verified based on the latest equipment capabilities.

[0088] At this point, the system completes the structured decomposition from the overall emergency response plan to a plurality of atomic action blocks, and each atomic action block is simultaneously provided with: independent editability; independent time scheduling capability; independent constraint verification capability; and independent device association capability. Therefore, subsequent operators do not need to regenerate the entire plan, and only need to make partial modifications to a single atomic action block, so that the system can automatically complete the corresponding constraint detection and linkage logic reconstruction, thereby realizing low-burden and high-efficiency human-machine collaborative emergency intervention.

[0089] The purpose of step 2 is to further convert the plurality of atomic action blocks that have completed structured decomposition in step 1 into human-machine interaction objects that can be directly perceived, directly modified and directly sorted by the operator. This step is not a simple data visualization display, but establishes a one-to-one mapping relationship between the data structure of the atomic action blocks and the interactive graphic interface components, so that every click, drag or time adjustment by a subsequent operator can directly act on the execution time offset field inside the corresponding atomic action block, while maintaining the temporal logic continuity of the entire emergency response plan.

[0090] After step 1 is completed, a plurality of atomic action block sequences have been formed inside the system, and each atomic action block at least includes an action type identifier field, a target device identifier field, an execution time offset field, and a pointer to a set of physical feasibility constraint conditions. Since subsequent human-machine interaction modifications are mainly carried out around time sequence and action scheduling, in step 2, it is first necessary to perform time serialization sorting on all atomic action blocks according to the execution time offset field, and the system calls the interface scheduling module to read the execution time offset values of all atomic action blocks, and establishes a time sorting sequence:

[0091]

[0092] wherein, represents the th atomic action block, satisfying:

[0093]

[0094] wherein, represents the execution time offset of the corresponding atomic action block, represents the total number of action instructions.

[0095] The reason for adopting the ascending sorting mode is that the operator's cognitive logic for the emergency response plan naturally follows the direction of the time flow. If the action sequence is chaotic, it is difficult for the operator to quickly determine the position of a certain action in the entire linkage process, thereby increasing the cognitive burden. After the time ascending sorting, the operator can directly observe the complete execution link from the initial stage of event occurrence to the end stage of linkage.

[0096] After sorting, the system maps each atomic action block to an independent draggable card. In essence, this mapping establishes a binding relationship between the internal fields of the atomic action block and visual components in the graphical user interface. Each draggable card corresponds to a unique atomic action block, thus ensuring a one-to-one correspondence between cards and atomic action blocks.

[0097]

[0098] in, Indicates the first Zhang can drag the card.

[0099] To ensure operators can quickly identify the meaning of actions, the system displays functional zones within the card's internal area. In the upper left corner of the draggable card, the system displays a text label corresponding to the action type identifier field, for example:

[0100] "Access the camera";

[0101] “PTZ rotation”;

[0102] "Notify the security team";

[0103] “Video marker”.

[0104] The action type identifier is displayed in a fixed position because operators usually focus on "what the current action is" rather than specific parameters in emergency situations. Therefore, the upper left corner is used as the visual starting area, which is conducive to forming a rapid action recognition.

[0105] In the upper right corner of the draggable card, the system displays a text label corresponding to the target device identifier field, for example:

[0106] “CAM_1043”;

[0107] “PTZ_EAST_GATE”;

[0108] “SECURITY_GROUP_A”.

[0109] The target device identifier and the action type identifier form a left-right correspondence structure, allowing operators to directly determine "which device is performing what action" without opening the detailed attribute window.

[0110] Since one of the core objectives of step 2 is to reduce the cost of modifying time parameters, the system embeds an execution time offset value edit box in the central area of ​​the card. This edit box is not a regular text input box, but a numerical step control component. The reason for using a numerical step control component is that regular text input is prone to illegal input, such as character input errors, incorrect time format, or incorrect units. The numerical step control component can limit the input range to discrete time increments, thereby ensuring that the input result always meets the valid time format. The numerical step control component includes: a display area for the current execution time offset; a value increase control button; and a value decrease control button.

[0111] When the operator clicks the up arrow, the system increases the current time offset by a fixed step size:

[0112]

[0113] When the operator clicks the down arrow, the system reduces the current time offset by a fixed step size:

[0114]

[0115] in, This indicates the minimum time adjustment step size. In this embodiment, the minimum adjustment step size is set to 1 second, i.e. .

[0116] The reason for using 1 second as the minimum adjustment granularity is that most of the action execution windows in monitoring emergency linkage are within the range of seconds. If the granularity is too large, it is impossible to complete fine timing adjustments; if the granularity is too small, the operator will need to make a lot of repeated clicks in emergency situations, which will increase the interaction burden. Therefore, the 1-second granularity can balance adjustment accuracy and interaction efficiency.

[0117] The numerical step control component maintains a real-time binding relationship with the execution time offset field within the atomic action block. When the displayed value changes, the system synchronously updates the cached value of the time field in the corresponding atomic action block. However, this modification remains pending confirmation until the consistency check is completed in step 4. After a single card is constructed, the system arranges all draggable cards vertically in ascending chronological order on the main interactive panel of the graphical user interface. The reason for using a vertical list structure, rather than a grid or tree structure, is that emergency plans are essentially time-driven task chains. Vertical arrangement strengthens the visual logic of downward time progression, allowing operators to understand the entire linkage process in a way similar to reading a timeline. Adjacent cards are separated by a 1-pixel-wide gray dividing line. The gray dividing line is not decorative but serves to reduce visual coupling and prevent visual fusion when multiple action cards are displayed at high density. Since emergency plans may include dozens of action nodes simultaneously, appropriate boundary isolation can improve the operator's ability to focus on individual action blocks.

[0118] Each draggable card supports left-click dragging. When the system detects that an operator is holding down the left mouse button on a card, the card immediately enters drag mode. Once in drag mode, the system records: the original arrangement position; the current mouse vertical coordinate; and the target insertion position. As the mouse moves up and down, the system calculates the current card's candidate sorting position in the list in real time and dynamically adjusts the display order of other cards. Let the original sequence number of the currently dragged card be... The target number after dragging is The system then according to Determine the direction of change in the sequence of actions.

[0119] Once the drag is complete and the left mouse button is released, the system uses the new arrangement as the base input for subsequent time constraint recalculation. At this point, although the operator appears to be performing drag-and-drop sorting, the underlying system has actually changed the temporal order between atomic action blocks. Therefore, step 2 not only completes the graphical display of atomic action blocks but also establishes a time series visualization mechanism; a rapid action semantic recognition mechanism; a second-level time editing mechanism; a drag-and-drop order adjustment mechanism; and a real-time binding mechanism between interface components and atomic action block fields. Through this structure, the operator does not need to enter a complex configuration interface or re-edit the entire plan. They can intervene in the AI-generated emergency plan in real time at low cost and high efficiency through only local interactions at the card level, providing a direct input basis for constraint consistency detection and automatic correction in subsequent steps.

[0120] The purpose of step 3 is to convert the operator's interactive behavior in the graphical user interface into a structured update result of the time constraint parameters within the corresponding atomic action block in real time. This step is a key transformation layer in the entire human-computer interaction chain. Its essence is not simply to modify a time value, but to establish a dynamic mapping relationship between interface operation behavior and the execution time logic of the plan. This allows the operator to directly intervene in the time scheduling structure of the AI-generated plan with minimal interaction cost, while keeping the other execution semantics of the atomic action block unchanged. Therefore, at the beginning of step 3, each draggable card in the graphical user interface actually corresponds to a data object with complete execution semantics. The data object includes at least: an action type identifier field; a target device identifier field; an execution time offset field; and a pointer to the set of physical feasibility constraints.

[0121] Step 3 does not modify the action type identifier field or the target device identifier field; instead, it only allows operators to make local adjustments around the execution time offset field. This avoids operators having to reconfigure complex action parameters in emergency situations, allowing for linkage strategy corrections through only minor adjustments at the time level. The system supports two modification methods in step 3: editing the execution time offset value and dragging and dropping cards for sorting.

[0122] Although the two methods have different interface interaction methods, their underlying goal is the same: to regenerate new time constraint parameters. When the operator modifies the execution time offset value edit box in any atomic action block card, the interface event listening module detects the value change event in real time. Since the execution time offset value edit box in step 2 uses a numerical step control component, the system can ensure that the input result is always a valid time value, preventing illegal characters or format errors from entering the subsequent constraint solving process. When the operator clicks the up arrow, the system increases the current time offset by one time step. When the operator clicks the down arrow The system will reduce the current time offset by one time step. ,in: This indicates the original execution time offset; This indicates the new execution time offset after modification.

[0123] Upon detecting a change in the time value, the system immediately triggers the parameter capture process. The interface caching module reads the latest value from the current value edit box and marks it as the modified new time constraint parameter. In this embodiment, the new time constraint parameter is not simply a time value, but a core input variable used for subsequent consistency checks. Essentially, it represents the new execution time of the current action within the entire linkage chain. The system then writes this new time constraint parameter into the execution time offset field of the corresponding atomic action block to overwrite the original time constraint parameter.

[0124]

[0125] in: Indicates the first A single atomic action block; Indicates the original execution time offset; This represents the updated execution time offset.

[0126] After the overlay is completed, the system generates a modified atomic action block. In this embodiment, the modified atomic action block does not create a new action object, but rather replaces the new state object formed by the time constraint parameters while retaining the original action execution semantics. Therefore, the content that remains unchanged before and after the modification includes: the action type identifier field; the target device identifier field; and the pointer to the physical feasibility constraint set. Only the execution time offset field changes. The reason for adopting this partial overlay method is that in emergency scenarios, operators usually only want to adjust the execution timing, not reconfigure the device logic. For example, advancing the camera call action by 3 seconds; delaying the PTZ action by 5 seconds; and advancing the notification action. These operations are all modifications at the time scheduling layer, not at the action semantic layer.

[0127] In addition to direct numerical editing, step 3 also supports drag-and-drop sorting modification. When the system detects that the operator holds down the left mouse button on any draggable card, the card enters drag mode. The interface control module begins to continuously record the vertical movement trajectory of the mouse. As the mouse moves up and down, the system calculates the candidate insertion position of the current card in the list of draggable cards in real time and updates the temporary arrangement order in the list synchronously.

[0128] When the operator releases the left mouse button, the system records the new sequence number corresponding to the end position of the drag. , This indicates the new sequence number after dragging. Since the entire card list is essentially sorted in ascending order by execution time offset, dragging and sorting is actually equivalent to redefining the execution position of the action on the timeline. The system then reads: the execution time offset of the previous atomic action block; and the execution time offset of the next atomic action block, denoted as:

[0129]

[0130] The system calculates a new time insertion interval based on the current drag-and-release position.

[0131] To ensure the continuity of action time, the system generates a new time offset that satisfies the following relationship:

[0132]

[0133] in: This represents the new execution time offset generated after dragging.

[0134] In practical implementation, the system can use intermediate interpolation to generate new time values:

[0135]

[0136] The reason for using intermediate interpolation is that drag-and-drop sorting is essentially a relative time order adjustment, rather than precise second-level input. By automatically generating intermediate time values, the system can avoid operators manually entering time parameters again after dragging, thereby reducing interaction complexity. For example:

[0137] The original action sequence is:

[0138] 5 seconds;

[0139] 10 seconds;

[0140] 15 seconds.

[0141] If the operator drags the 15-second action between the 5-second and 10-second actions, the system will automatically generate:

[0142]

[0143] The system then performs integerization according to preset time granularity rules:

[0144]

[0145] This generates new time constraint parameters.

[0146] After generating the new time value, the system also writes it to the execution time offset field of the corresponding atomic action block to overwrite the original time constraint parameters and generates the modified atomic action block. Regardless of whether numerical editing or drag-and-drop sorting is used, the system needs to synchronize the interface update. Therefore, the system refreshes the new execution time offset value in the modified atomic action block to the execution time offset value edit box in the corresponding card in real time, allowing the operator to immediately observe the current modification result. At the same time, the following fields in the atomic action block remain unchanged: action type identifier field; target device identifier field; physical feasibility constraint set pointer; device association relationship; action semantic description.

[0147] Therefore, step 3 actually completes the following key processes: transforming operator interaction behavior into structured time parameter modifications; synchronously mapping interface layer modifications to the atomic action block data structure; completing local time scheduling reconstruction without changing the action semantics; implicitly generating new time logic relationships through drag-and-drop behavior; and maintaining the stability of the association between atomic action blocks and the set of device constraints. After step 3, the system has obtained modified atomic action blocks carrying new time constraint parameters. Whether the modification result truly satisfies the device physical capability constraints and linkage time logic constraints will be checked for consistency in the subsequent step 4 through the constraint solver.

[0148] The purpose of step 4 is to perform real-time feasibility verification on the modified atomic action block generated in step 3, thereby determining whether the time adjustment operation performed by the operator still meets the physical execution capabilities of the monitoring equipment and the time linkage logic of the entire emergency plan. This step is essentially a constraint closed-loop verification process between human-computer interaction modification and the actual execution capabilities of the equipment. Its purpose is not to restrict operator modification, but to immediately determine whether the modification can be actually executed in the real monitoring environment after the operator completes low-cost intervention. Therefore, at the beginning of step 4, the system already has atomic action objects carrying new time constraint parameters.

[0149] The modified atomic action block should include at least:

[0150] Action type identifier field;

[0151] Target device identification field;

[0152] Modified execution time offset field;

[0153] pointer to the set of physical feasibility constraints;

[0154] Modified sorting position parameters.

[0155] After step 4 begins, the system first extracts the modified time constraint parameters from the modified atomic action block. In this embodiment, the modified time constraint parameters are not a single time value, but include two key components:

[0156] New execution time offset value;

[0157] The new sorting position of the modified atomic action block in the entire sequence of atomic action blocks.

[0158] The new execution time offset value describes the new execution time of the action within the entire emergency plan, while the new sorting position describes the sequential relationship of the action in the time chain. The system then accesses the device parameter database established in step 1 based on the pointer to the set of physical feasibility constraints in the modified atomic action block, and retrieves the constraint set for the corresponding device. The set of physical feasibility constraints includes at least:

[0159] Maximum rotational speed parameters of the equipment;

[0160] Preset existence flag;

[0161] Shortest time interval threshold.

[0162] Since the device status may change in real time, the constraint reading in step 4 adopts a dynamic retrieval method instead of a fixed cache method. This ensures that the consistency check is always performed based on the current device status. For example, a camera may enter maintenance mode after step 1, or a preset position may be deleted again. At this time, the system can still identify that the device is currently unavailable in real time. After the parameter extraction is completed, the system calls the constraint solver to perform the consistency check.

[0163] The constraint solver in this embodiment is essentially a rule inference module based on time logic and device capability constraints. Its inputs include:

[0164] Modified time constraint parameters;

[0165] Set of physical feasibility constraints;

[0166] The temporal relationship between adjacent atomic action blocks.

[0167] Its output includes only two states:

[0168] Logical truth value;

[0169] Logical false value.

[0170] A logical truth value indicates that the current modification can be executed by the real device;

[0171] A logical false value indicates that the current modification violates at least one device constraint or time logic constraint.

[0172] The constraint solver performs three consistency checks in a fixed order. The first consistency check determines whether the modified execution time meets the maximum rotation speed constraint of the equipment. Since many emergency actions involve the rotation of the PTZ camera, even if the operator executes the action in advance, the system must verify whether the equipment can complete the physical rotation within that time window. The system reads the maximum rotation speed parameter of the target equipment. At the same time, it reads the target rotation angle required by the current action. The theoretical minimum execution time required for the constraint solver to complete this action:

[0173]

[0174] in: This indicates the theoretical response time required for the device to complete the current action.

[0175] The system then determines:

[0176]

[0177] If the above relationship is satisfied, it means that the current modified execution time is sufficient to support the device in completing the actual physical action;

[0178] If this condition is not met, it means that although the operator has completed the time modification, the device is unable to complete the action on time at the physical level.

[0179] For example:

[0180] The maximum rotation speed of a certain PTZ camera is The current action requires completing a rotation. .

[0181] The theoretical shortest response time is:

[0182]

[0183] If the operator adjusts the action to be completed within 2 seconds, the system immediately determines that the modification violates the maximum rotation speed constraint of the equipment. After completing the first check, the constraint solver performs the second consistency check. The second consistency check is used to determine whether the preset position corresponding to the target equipment actually exists. Since many actions in the emergency plan rely on the camera preset position call, even if the timing logic is correct, if the target preset position does not exist, the action still cannot be executed. The system reads the existence flag of the preset position corresponding to the target equipment. ,in ,when This indicates that the target preset bit exists; when This indicates that the target preset bit does not exist.

[0184] The constraint solver directly determines If the condition is met, it means the device preset position is available; if it is not met, it means the device configuration corresponding to the current action is incomplete.

[0185] For example, if an operator changes the original "East Gate Camera Preset Position 2" to "West Gate Camera Preset Position 5", but the West Gate camera has not yet been configured with Preset Position 5, the system will immediately determine that the action is not feasible.

[0186] After completing the second check, the constraint solver continues with the third consistency check. This third consistency check verifies whether the modified action ordering violates the minimum time interval threshold between adjacent actions. Since the drag-and-drop sorting operation in step 3 changes the temporal order of actions, even if a single action is executable, it may still cause a time conflict in the entire action chain. The system reads the execution time offset of the previous atomic action block. The execution time offset of the next atomic action block Current modified action execution time offset Simultaneously, read the shortest time interval threshold from the set of physical feasibility constraints. The constraint solver calculates the following respectively:

[0187] Time difference between the current action and the previous action:

[0188]

[0189] Time difference between the current action and the next action:

[0190]

[0191] Subsequent judgment:

[0192]

[0193] as well as:

[0194]

[0195] Whether they are true at the same time.

[0196] If both conditions are met, it indicates that there is sufficient device execution buffer time between the current action and the adjacent action. If either condition is not met, it indicates that there is time overlap, action overriding, or linkage conflict between actions. For example:

[0197] A camera performs two consecutive large-angle rotation movements; the minimum time interval threshold is: If the operator drags and sorts the data, resulting in only a 2-second interval between two actions, the system will immediately determine that the current modification violates the time interval constraint.

[0198] After completing the three consistency checks, the constraint solver makes a comprehensive judgment on the results, and the following conditions are met simultaneously:

[0199] The maximum rotational speed constraint of the equipment is met;

[0200] The existence constraint of the preset bit holds;

[0201] The shortest time interval constraint holds.

[0202] The constraint solver then outputs the logical truth value: ,in, This indicates the consistency check result. The logical truth value means that the modified atomic action block satisfies both the physical execution capability of the device and the overall time linkage logic, thus allowing it to enter the subsequent contingency plan update process.

[0203] If any one of the three checks fails, the constraint solver outputs a logical false value: Simultaneously, the system records the constraint source information that caused the detection failure. The recorded information includes: the name of the conflicting constraint; and the identifier of the conflicting device. For example:

[0204] "Maximum rotational speed constraint conflict";

[0205] "There is no conflict with the preset position";

[0206] "Action time interval conflict".

[0207] And the corresponding device number:

[0208] “PTZ_EAST_GATE”;

[0209] “CAM_1043”.

[0210] This conflict information will be used in subsequent step 6 to generate conflict alerts, alternative timing suggestions, and highlighted content on the interface. Therefore, step 4 effectively establishes a complete real-time constraint verification closed loop, enabling every low-cost modification by the operator to be immediately compared with the actual equipment capability model for consistency, thereby avoiding the lag defect in traditional systems where problems are only discovered after modification and execution failure.

[0211] The purpose of step 5 is to formally write the modified result, which has passed the constraint verification, back to the graphical user interface and the emergency plan execution chain after the consistency check outputs a logical truth value in step 4. This achieves synchronous linkage between restoring the state of the modification result confirmation interface and regenerating the complete emergency plan. This step essentially belongs to the effective modification submission stage in the entire interactive loop. Its purpose is to ensure that only modification results that meet the physical capability constraints and time logic constraints can truly enter the execution engine. Therefore, at the beginning of step 5, a modified atomic action block that satisfies all constraints already exists within the system. Let the modified atomic action block that has passed the consistency check be... The corresponding new execution time offset is The constraint solver outputs in step 4. At this point, the system enters the modification confirmation state.

[0212] Since the time modification in step 3 is a modification awaiting verification, the corresponding draggable card remains in a pending confirmation state until the consistency check is completed. In this state, the card's border color is set to orange to indicate to the operator that the current modification has not yet officially taken effect. Therefore, after step 5 begins, the graphical interface rendering module first reads the modified time constraint parameters from the modified atomic action block and officially renders these parameters to the execution time offset value edit box in the corresponding card. In this embodiment, official rendering means that the system no longer uses temporary cached values ​​but instead writes the time parameters that have passed constraint verification into the official display state of the current interface. After completing the time parameter rendering, the system synchronously restores the card's default display state.

[0213] Since the modification has passed the consistency check, the corresponding card no longer needs to maintain the pending confirmation prompt state. The system restores the border color of the card from orange (pending confirmation state) to the default gray. The purpose of restoring the border color is not just a visual change, but to convey to the operator that the current modification has been verified by the system.

[0214] In emergency scenarios, operators typically need to rapidly modify multiple action nodes. Without a status feedback mechanism, it is difficult for operators to determine which modifications have taken effect and which are still under verification. Therefore, this embodiment establishes an explicit confirmation mechanism through color-coded status transitions.

[0215] Orange indicates a pending verification status;

[0216] Gray indicates that the verification has passed;

[0217] Red indicates a conflict-locked state.

[0218] Simultaneously, the system also needs to restore the card's interactivity. During the consistency check in step 4, the system may temporarily freeze some of the card's operational capabilities to prevent the operator from repeatedly modifying it during verification. Therefore, after the check result outputs a logical truth value, the system re-enables the card's drag-and-drop and time-editing functions. At this point, the card re-enters an interactive state, allowing the operator to continue performing subsequent modifications. After restoring the state of a single card, the system begins to rebuild the complete contingency plan. In this embodiment, the rebuild does not involve re-invoking the multimodal generation model, but rather re-serializing the entire time action chain based on the latest state of all current atomic action blocks. The system first reads all atomic action blocks from the current list of draggable cards:

[0219]

[0220] in:

[0221] Each atomic action block may have been modified by the operator;

[0222] Each atomic action block already carries the latest execution time offset.

[0223] The system then regenerates the time series based on the current card list order:

[0224]

[0225] Since the drag-and-drop sorting operation in step 3 may change the order of actions, the time series at this point is different from the original plan sequence in step 1.

[0226] The system then reserializes all atomic action blocks into complete JSON objects. In this embodiment, serialization means converting the system's internal data objects back into a standard structured data format that the execution engine can recognize. Each element in the serialized JSON object includes at least:

[0227] Action type identifier;

[0228] Target device identification;

[0229] Updated execution time offset.

[0230] For example:

[0231] An action node can be represented as:

[0232] Action type: PTZ rotation;

[0233] Target device: CAM_1043;

[0234] Execution time offset: 8 seconds.

[0235] Since all actions have been rearranged according to the latest time sequence, the regenerated JSON object essentially forms a new, complete contingency plan. Let the complete contingency plan object be:

[0236]

[0237] in: This indicates the updated atomic action block.

[0238] The system then calls the internal process communication interface of the monitoring equipment management subsystem to send the regenerated complete emergency plan to the execution engine. The internal process communication interface in this embodiment is essentially a data transmission channel between internal modules of the system, used to realize data synchronization between the human-computer interaction module, constraint solving module, and execution scheduling module.

[0239] The reason for using an internal process communication interface instead of direct UI calls is that the execution engine needs to maintain independent action scheduling capabilities. This way, even if the graphical interface refreshes or experiences a partial anomaly, the execution engine can still continue execution according to the confirmed contingency plan. When the system sends the complete contingency plan, the content includes: the complete action sequence; the execution order in time; the target device mapping relationship; and the action scheduling parameters.

[0240] After receiving the complete emergency plan, the execution engine immediately executes the plan parsing and returns an acknowledgment signal. The acknowledgment status is set to... , This indicates that the execution engine has successfully received the current emergency plan. If no response signal is received, the system maintains the current interface state and re-attempts to send the plan. After the execution engine returns a confirmation signal, the system marks the current emergency plan as submitted for execution.

[0241] At this point, step 5 completes the following key processes: formally writing the modified results that have passed the consistency check back to the interface display layer; restoring the default interactive state of the card; regenerating the complete emergency plan based on the latest action sequence; establishing a new time execution chain; submitting the final executable action sequence to the execution engine; and completing the state loop from local operator modifications to overall system-wide coordinated execution.

[0242] Therefore, step 5 not only confirms the individual modification action, but also realizes the dynamic reconstruction of the entire emergency plan at the time logic level, enabling operators to make real-time local corrections to the AI-generated plan without interrupting the automated linkage process, and immediately put it into actual execution.

[0243] The purpose of step 6 is to isolate the "unexecutable modification result" from the system execution chain when the consistency check in step 4 outputs a false value. It then guides the operator to complete the error correction with minimal interaction cost through explicit interface feedback and alternative repair suggestions, rather than directly rolling back or discarding the modification result. This step constructs an interpretable feedback mechanism for conflict constraints. Essentially, it provides a structured expression of constraint solving failure results at the human-computer interaction level, enabling operators to quickly locate the source of the problem and complete local corrections.

[0244] In steps 1 to 3, the system has completed the atomized decomposition and visualization mapping of the emergency plan, as well as the operator's modification of the time constraint parameters. In step 4, the constraint solver performs a consistency check on the modified atomic action block and outputs a logical false value R=FALSE. Therefore, when step 6 is triggered, there is already a clearly existing modified atomic action block that violates the constraint conditions within the system. Let this conflicting atomic action block be... Its modified execution time offset is Furthermore, the constraint solver has recorded the corresponding set of failed constraints:

[0245]

[0246] Step 6 first performs a state freeze operation on the interface layer. Based on the conflict results returned by the constraint solver, the system locates the corresponding draggable cards and places them in an edit-locked state. This lock does not completely disable display, but rather restricts their key interactive behaviors, specifically including:

[0247] The execution time offset value edit box is disabled, prohibiting any numerical input and step adjustment;

[0248] The drag-and-drop sorting feature has been temporarily disabled to prevent the conflicting action from further altering the timeline structure if it is not fixed.

[0249] The card will remain displayed in the list, but will no longer be allowed to participate in new sorting calculations.

[0250] Through the above mechanism, the system changes the card from a modifiable state to a constrained frozen state, thereby preventing the error from being further propagated to the entire contingency plan chain.

[0251] At the same time, the system enhances the visual status of the card. Its border color changes from the default gray state in steps 2 to 5 to a flashing red border, with the flashing red indicating a clear semantic meaning:

[0252] This indicates that the current modification violates device constraints;

[0253] This indicates that the action cannot be executed under the current time configuration;

[0254] This indicates that immediate operator intervention is required for correction.

[0255] The purpose of this highlighting mechanism is to quickly focus attention on the conflict node in complex emergency plans where multiple cards exist simultaneously, without having to check the parameters one by one.

[0256] After freezing and highlighting the cards, a non-modal dialog box pops up in the central area of ​​the graphical user interface. This non-modal nature means it doesn't disrupt the overall interface flow; the operator can still view other cards, but the conflict card repair prompts remain visible. This achieves a localized strong prompt and a globally uninterrupted interaction mode. The dialog box title bar displays a consistency check failure message, indicating that the system is currently in conflict repair mode. The main area of ​​the dialog box uses a structured information display method, with three core types of information constituting the conflict explanation results:

[0257] Conflicting constraint names;

[0258] Conflicting equipment identification;

[0259] List of alternative execution time offset values.

[0260] The names of conflict constraints are derived from the failure records of the constraint solver in step 4, for example:

[0261] "Conflict in maximum rotational speed constraint of equipment";

[0262] "There are no constraint conflicts in the preset position";

[0263] "Shortest time interval constraint conflict".

[0264] Conflicting device identifiers are used to identify the specific device involved in the conflict, for example:

[0265] CAM_1043;

[0266] PTZ_EAST_GATE.

[0267] This information is used to prevent operators from misjudging the source of conflict in multi-device parallel contingency plans.

[0268] Most importantly, the list of alternative execution time offsets is not statically preset but generated in real time by the constraint solver based on the current device state database. When the system identifies a time configuration as infeasible, it calculates multiple feasible alternative solutions within the time-space range that satisfies the constraints. For example, when the original time constraint is... When the constraints are not met, the system searches for alternative values ​​within the time interval that meets the following conditions:

[0269]

[0270] And generate multiple candidate time offsets, for example, three candidate values:

[0271]

[0272] These candidate values ​​all satisfy:

[0273] Equipment physical execution capability constraints;

[0274] Constraints on the time interval between adjacent actions;

[0275] Preset availability constraints.

[0276] By providing multiple candidate values, the system transforms error correction from manual guessing into a deterministic choice problem within a constrained space, thereby significantly reducing operator repair costs.

[0277] At the bottom of the dialog box, the system provides two key interactive buttons:

[0278] Confirm button;

[0279] Cancel button.

[0280] When the operator clicks the confirmation button, the system writes the target time offset selected from the candidate list back to the atomic action block as the new modified time constraint parameter:

[0281]

[0282] in: This indicates the time value after the replacement correction.

[0283] After the write-back, the system re-sends the atomic action block into the step 3 process, re-triggering the consistency detection link, i.e., the closed-loop re-computation mechanism from step 3 to step 5.

[0284] If the operator clicks the cancel button, the system closes the non-modal dialog box and keeps the current atomic action block in a conflict-locked state, without making any modifications, thus allowing the operator to readjust or abandon the modifications later. Furthermore, in the canceled state, the system does not affect other verified atomic action blocks, nor does it interrupt the overall emergency plan display and editing process. Therefore, the overall mechanism of step 6 essentially constructs a closed-loop interactive system of conflict detection—interpretation output—candidate repair—user decision-making, with key features including:

[0285] Structure the results of consistency check failures;

[0286] Explicitly map the non-executable state to a highlighted lock on the interface;

[0287] Transform constraint failures into a set of alternative solutions;

[0288] Transform error repair from free search to selection within a constrained space;

[0289] Ensure the overall contingency plan editing process remains uninterrupted.

[0290] Through this mechanism, the system achieves rapid location and low-cost repair capabilities for local conflict actions in complex monitoring and emergency scenarios, while avoiding the problem of the entire contingency plan process being forced to terminate due to the failure of a single point of constraint.

[0291] The purpose of step 7 is to transform the "list of alternative execution time offset values" generated in step 6 into finally executable and valid time parameters, and to achieve gradual convergence of the conflict state and final release of the card lock state through closed-loop re-execution of steps 3 to 5. This step is essentially a constraint repair-driven recursive consistency convergence mechanism. Its function is to resolve conflicts by replacing time parameters without changing the action semantics and device mapping relationship, thereby allowing the atomic action block to re-enter the executable state.

[0292] In steps 1 through 5, the system has completed the generation, visualization mapping, time parameter modification, consistency check, and submission of valid results for atomic action blocks. In step 6, when the consistency check outputs a false value, the system has already generated a list of alternative execution time offset values ​​for the conflicting atomic action block. Therefore, at the beginning of step 7, there is already an atomic action block in a red locked conflict state within the system, and this action block carries multiple candidate repair time values.

[0293] Let the current conflicting atomic action block be Its current invalid execution time offset is Step 7 first responds to the operator's interaction in the modal dialog box. When the operator selects a candidate value from the list of alternative execution time offset values ​​and clicks the OK button, the system captures the selected value, assuming the selected candidate time value is... This value satisfies the feasibility boundary conditions defined by the constraint solver in step 6, meaning that the value is already inside the constraint space and therefore belongs to a locally feasible solution.

[0294] The system then writes the target value as the new modified time constraint parameter into the execution time offset field of the corresponding atomic action block, completing the time parameter replacement:

[0295]

[0296] At this point, the action type identifier field and target device identifier field of the atomic action block remain unchanged; only the time constraint parameters are updated, ensuring that the repair process does not disrupt the action semantic structure. After completing the time replacement, the system does not directly end the process but instead uses the modified atomic action block as input again, sequentially calling steps 3, 4, and 5 to form a closed-loop recursive verification mechanism. The significance of re-calling step 3 is to reintegrate the candidate time value selected by the operator into the standard time constraint update process, ensuring consistency with the interface display. The system synchronously updates the execution time offset value edit box in the card, maintaining a consistent mapping relationship between the interface layer and the data layer.

[0297] Then, in step 4, the constraint solver re-executes a consistency check on the updated atomic action blocks. The input for this check is:

[0298] Updated execution time offset ;

[0299] The latest sorted position of adjacent atomic action blocks;

[0300] The corresponding set of physical feasibility constraints.

[0301] The constraint solver then performs the three consistency checks again in sequence:

[0302] Maximum rotational speed constraint detection for equipment;

[0303] Preset bit existence detection;

[0304] Shortest time interval constraint detection.

[0305] Since the candidate values ​​provided in step 6 have already been generated within the constraint space, theoretically, the re-detection result will output a logically true value in most cases. However, if other atomic action blocks are concurrently modified during the execution of step 7, or if the global time structure changes, a logically false value may still be output. When step 4 outputs a false value again, the system does not terminate the process but triggers the conflict feedback mechanism of step 6 again and regenerates a new list of alternative execution time offset values. At this point, the system enters a multi-round iterative repair process until the constraint space converges.

[0306] When step 4 outputs a logical true value, the system proceeds to step 5. In step 5, the system first writes the modified result that has passed the consistency check back to the interface display layer, specifically as follows:

[0307] New execution time offset Render the value to the corresponding card's numeric editing box;

[0308] Restore the card border color from the conflicted flashing red to the default gray;

[0309] Restore the card's drag-and-drop functionality, allowing it to rejoin the overall time-series ordering structure.

[0310] After completing the interface status update, the system regenerates the complete emergency plan JSON structure based on the latest status of all current atomic action blocks and sends it to the execution engine, ensuring that the execution chain always runs based on the latest consistent results.

[0311] When step 5 is completed and step 4 outputs a true value, the system determines that the atomic action block has completed conflict resolution and entered a stable executable state. At this time, the system performs a final state recovery operation, completely unlocking the interaction permissions of the atomic action block, including:

[0312] The execution time offset value edit box has been restored from a disabled state to an enabled state;

[0313] The drag-and-drop sorting function has been turned on again.

[0314] The border color has been restored from flashing red to the default gray.

[0315] The significance of the above recovery operation lies in transforming the atomic action block from the conflict repair process state to the normal editable state, allowing it to return to the standard interaction structure defined in steps 2 to 3. Therefore, step 7 actually constructs a recursive repair closed loop based on candidate solution-driven mechanisms, the key mechanisms of which include:

[0316] Use the candidate time values ​​generated in step 6 as inputs to feasible solutions within the constraint space;

[0317] Local state recalculation is achieved by repeating steps 3 to 5;

[0318] Perform dynamic consistency verification in step 4;

[0319] Conflicts are gradually resolved through multiple rounds of iteration;

[0320] The card is unlocked once eventual consistency is achieved.

[0321] This mechanism enables the system to quickly repair and converge time conflicts in complex multi-device linkage scenarios, while ensuring that the repair process is always controlled by the constraint solver, avoiding arbitrary human adjustments that could lead to distortion of the plan structure or execution failure.

[0322] Example 2

[0323] This embodiment discloses a human-computer interaction system for a monitoring equipment management subsystem, including:

[0324] The contingency plan atomization module responds to security event detection signals by decomposing the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints.

[0325] The timing visualization mapping module displays the multiple atomic action blocks in an ascending order of their respective execution time offsets in the form of a draggable card list. Each card corresponding to an atomic action block contains a text label and an execution time offset value editing box.

[0326] The interactive editing mapping module responds to the operator's modification of the execution time offset value editing box of any atomic action block card or the drag-and-drop sorting operation of the card, and generates a modified atomic action block by replacing the original time constraint parameters with the modified new time constraint parameters.

[0327] The constraint consistency verification module calls the constraint solver to perform consistency checks on the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true value or a false value.

[0328] When the consistency check outputs a true value, the contingency plan confirmation and release module updates the display parameters of the card according to the modified atomic action blocks, and reassembles all atomic action blocks into a complete contingency plan before sending it to the execution engine.

[0329] The conflict feedback locking module locks the editing state of the card and highlights the border of the card when the consistency detection outputs a false value. At the same time, a modal dialog box pops up to display the name of the conflict constraint that caused the false value, the identifier of the conflicting device, and a list of alternative execution time offset values.

[0330] The constraint repair convergence module responds to the operator's confirmation operation of selecting a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency detection outputs a true value and the card's locked state is released.

[0331] The modules in Embodiment 2 are used to implement the functions in Embodiment 1. This embodiment can be implemented by a system including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement the human-computer interaction method of the monitoring equipment management subsystem according to Embodiment 1 of this application. The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. Their settings and functions are known in the art and will not be described in detail here.

[0332] In this application, the aforementioned memory can be any tangible medium that includes or stores a program that can be used or combined with an instruction execution system, system, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store required information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this application can be implemented using computer-readable / executable instructions that can be stored or otherwise retained by such a computer-readable medium.

[0333] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A human-computer interaction method for a monitoring equipment management subsystem, characterized in that, Includes the following steps: Step 1: In response to the security incident detection signal, decompose the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints. Step 2: Display the multiple atomic action blocks in a draggable card list in ascending order of their respective execution time offsets. Each card corresponding to an atomic action block contains a text label and an execution time offset value edit box. Step 3: Respond to the operator's modification operation on the execution time offset value edit box of any atomic action block card or the drag and sort operation on the card, and replace the original time constraint parameters with the modified new time constraint parameters to generate the modified atomic action block; Step 4: Call the constraint solver to perform a consistency check between the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true or false value. Step 5: When the consistency detection outputs a true value, update the display parameters of the card according to the modified atomic action block, and reassemble all atomic action blocks into a complete emergency plan before sending it to the execution engine; Step 6: When the consistency detection outputs a false value, lock the editing state of the card and highlight the border of the card. At the same time, pop up a modal dialog box to display the conflict constraint name, conflict device identifier and alternative execution time offset value list that caused the false value. Step 7: The responding operator confirms the selection of a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency check outputs a true value and the card is unlocked.

2. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 1 includes: The emergency response plan output by the multimodal generation model is parsed. The emergency response plan is a JSON structure with timestamp tags. Each action instruction in the JSON structure is extracted in ascending order of timestamp. Each action instruction includes at least an action type field, a target device field, and an offset field. Each extracted action instruction creates an atomic action block. The data structure of the atomic action block includes an action type identifier field, a target device identifier field, an execution time offset field, and a pointer to a set of physical feasibility constraints that points to a pre-stored device parameter database. Based on the target device field in the action instruction, query the pre-stored device parameter database to obtain the target device's maximum rotation speed parameter, preset position existence flag, and minimum time interval threshold between it and adjacent atomic action blocks. Encapsulate the maximum rotation speed parameter, preset position existence flag, and minimum time interval threshold into a set of physical feasibility constraints, and store the pointer of this set into the corresponding atomic action block.

3. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 2 includes: Based on the order of execution time offsets carried by each atomic action block from smallest to largest, each atomic action block is mapped to an independent draggable card. The upper left corner of the draggable card displays a text label indicating the action type of the atomic action block, and the upper right corner displays a text label indicating the target device of the atomic action block. A numerical step control component is embedded in the middle area of ​​the draggable card as an execution time offset value editing box. The numerical step control component allows the operator to click the up arrow to increase the value or click the down arrow to decrease the value, with a minimum adjustment step of 1 second. Arrange all draggable cards vertically in order in the main interactive panel of the graphical user interface. Adjacent cards are separated by a gray dividing line with a width of 1 pixel. Each draggable card can be dragged up and down by holding down the left mouse button to change its position in the main interactive panel.

4. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 3 includes: When it is detected that an operator modifies the value of the execution time offset in any atomic action block card, the new value is captured, the new value is used as the new time constraint parameter, and the new time constraint parameter is written to the execution time offset field of the atomic action block card to overwrite the original time constraint parameter, and the modified atomic action block is generated. When it is detected that the operator performs a drag-and-drop sorting operation on any atomic action block card by holding down the left mouse button and moving it up and down and then releasing it, the new sorting number corresponding to the release position of the atomic action block card in the list of draggable cards is used to calculate the new time offset value relative to the previous atomic action block and the next atomic action block implied by the new sorting number. The new time offset value is used as the modified new time constraint parameter to replace the original time constraint parameter and generate the modified atomic action block. The modified new time constraint parameters in the modified atomic action block are synchronously updated to the execution time offset value edit box of the atomic action block card, and the other fields in the modified atomic action block are kept at the values ​​before the original time constraint parameters were overwritten.

5. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 4 includes: Extract the modified time constraint parameters from the modified atomic action block. The modified time constraint parameters include the new execution time offset value and the new sorting position of the modified atomic action block in the sequence of all atomic action blocks. At the same time, retrieve the set of physical feasibility constraints from the modified atomic action block. The constraint solver is invoked to perform three consistency checks in sequence. The first check is to determine whether the new execution time offset value is less than or equal to the limit response time allowed by the maximum rotation speed parameter of the device associated with the target device identifier corresponding to the modified atomic action block. The second check is to determine whether the preset bit existence flag associated with the target device identifier is true. The third check is to determine whether the difference in execution time offset between the previous atomic action block and the next atomic action block adjacent to the new sort position is greater than or equal to the shortest time interval threshold in the physical feasibility constraint set. When all three consistency checks pass, the constraint solver outputs a logical true value; when any one of the three consistency checks fails, the constraint solver outputs a logical false value and records the name of the conflicting constraint and the identifier of the conflicting device corresponding to the failed check.

6. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 5 includes: Render the modified time constraint parameters carried by the modified atomic action block to the execution time offset value editing box of the corresponding card, restore the border color of the card from orange in the pending state to the default gray, and restore the drag function of the card to the available state. Based on the current order of all atomic action blocks in the draggable card list and the latest execution time offset value of each atomic action block, serialize all atomic action blocks into a JSON object with complete time sequence. Each element in the JSON object includes an action type identifier, a target device identifier, and an updated execution time offset. The JSON object, representing the complete emergency plan, is sent to the execution engine via the internal process communication interface of the monitoring equipment management subsystem. Upon receiving the JSON object, the execution engine returns a confirmation signal indicating receipt.

7. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 6 includes: Disable the execution time offset value edit box of the modified atomic action block card corresponding to the output false value, temporarily turn off the drag sorting function of the card, and change the border color of the card from the default gray to a flashing red border for highlighting. A non-modal dialog box pops up in the central area of ​​the graphical user interface. The title bar of the non-modal dialog box displays a consistency check failure message. The main area of ​​the non-modal dialog box displays the name of the conflicting constraint that caused the false value, the identifier of the conflicting device, and a list of three candidate execution time offset values ​​generated in real time by the constraint solver based on the current device state database. At the bottom of the non-modal dialog box, a confirmation button and a cancel button are displayed. When the operator clicks the confirmation button, the candidate execution time offset value selected by the operator in the list is output as the alternative execution time offset value. When the operator clicks the cancel button, the non-modal dialog box is closed and no changes are made to the modified atomic action block card.

8. The human-computer interaction method for a monitoring equipment management subsystem according to claim 1, characterized in that, Step 7 includes: In response to the operator clicking the confirmation button in the modal dialog box and selecting a target value from the list of alternative execution time offset values, the target value is captured and written as a new modified time constraint parameter into the execution time offset field of the modified atomic action block card; Using the modified atomic action block card carrying the new modified time constraint parameters as input, steps 3, 4 and 5 are called sequentially. This step is repeated when a false value is output again in step 4, and jumps to this step when a true value is output in step 4. After step 4 outputs a true value and step 5 is completed, the execution time offset value edit box of the modified atomic action block card is switched from disabled to enabled, the drag-and-drop sorting function of the card is switched from disabled to enabled, and the border color of the card is restored from the flashing red border to the default gray, thus unlocking the card.

9. A human-computer interaction system for a monitoring equipment management subsystem, characterized in that, include: The contingency plan atomization module responds to security event detection signals by decomposing the emergency plan output by the multimodal generation model into multiple atomic action blocks. Each atomic action block includes an action type identifier, a target device identifier, an execution time offset, and a set of physical feasibility constraints. The timing visualization mapping module displays the multiple atomic action blocks in an ascending order of their respective execution time offsets in the form of a draggable card list. Each card corresponding to an atomic action block contains a text label and an execution time offset value editing box. The interactive editing mapping module responds to the operator's modification of the execution time offset value editing box of any atomic action block card or the drag-and-drop sorting operation of the card, and generates a modified atomic action block by replacing the original time constraint parameters with the modified new time constraint parameters. The constraint consistency verification module calls the constraint solver to perform consistency checks on the modified time constraint parameters included in the modified atomic action block and the set of physical feasibility constraints. The consistency check outputs a true value or a false value. When the consistency check outputs a true value, the contingency plan confirmation and release module updates the display parameters of the card according to the modified atomic action blocks, and reassembles all atomic action blocks into a complete contingency plan before sending it to the execution engine. The conflict feedback locking module locks the editing state of the card and highlights the border of the card when the consistency detection outputs a false value. At the same time, a modal dialog box pops up to display the name of the conflict constraint that caused the false value, the identifier of the conflicting device, and a list of alternative execution time offset values. The constraint repair convergence module responds to the operator's confirmation operation of selecting a target value from the list of alternative execution time offset values, and repeats steps 3 to 5 with the target value as the new modified time constraint parameter until the consistency detection outputs a true value and the card's locked state is released.

Images