Interaction control system and method for smart urban security and protection and large screen

Abstract

The invention relates to an interactive control system and method for smart city security and large screen, and relates to the technical field of computers, and the system comprises a data collection module which is used for collecting heterogeneous security data and injecting spatio-temporal information to generate a standardized data packet; the data fusion module is used for receiving the data packet and carrying out lightweight structure processing on the video data to reduce the data volume; the large-screen display module is used for visually presenting the structured information; the interaction control module is used for capturing a touch operation on a large screen and performing permission verification, encryption and priority scheduling on an instruction; and the security equipment control module is used for receiving and translating the instruction and issuing the instruction to front-end equipment for execution. According to the method, the technical problems of difficulty in heterogeneous data fusion, low interaction efficiency and unsafe control are solved, and unified presentation of a global situation, efficient and visual interaction control and safe and reliable instruction execution are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to an interactive control system and method for smart urban security and large screens. BACKGROUND

[0002] With the advancement of smart urban construction, large-scale and multi-type security devices are widely deployed in urban management. However, these security devices usually come from different manufacturers, use different communication protocols and data formats, resulting in differences in the structure of the data they generate. This makes it a technical challenge to effectively fuse the data of multiple independent subsystems such as video monitoring, access control, and environmental sensing to form a unified security view.

[0003] In particular, in the aspect of video monitoring, a large number of front-end cameras continuously generate high-definition video streams. If all these raw video data are transmitted to the central server for analysis and processing, it will cause a huge bandwidth pressure on the network backbone, and also put high requirements on the computing resources of the central server.

[0004] In terms of human-computer interaction, the existing large screen display and control system usually relies on traditional menu or button operation. When security personnel need to intervene in a specific event or device, they often need to click several times or switch through complex interfaces to issue control instructions. This operation method is not efficient and intuitive in emergency situations.

[0005] In addition, in the process of issuing and executing instructions, the existing technical solutions often lack fine-grained control of operation permissions and unified instruction scheduling mechanism, making it difficult to ensure that high-priority emergency instructions are executed first. At the same time, the security of the instructions in the transmission process cannot be fully guaranteed. SUMMARY

[0006] The purpose of the present application is to provide an interactive control system and method for smart urban security and large screens, aiming to solve the problems of data fusion difficulty, low human-computer interaction efficiency and insufficient security of control link in existing security systems.

[0007] In a first aspect, the interactive control system for smart urban security and large screens provided by the present application adopts the following technical solution:

[0008] A data acquisition module for acquiring heterogeneous security data and injecting time and space information into the heterogeneous security data to generate standardized data packets;

[0009] A data fusion module for receiving the standardized data packets, cleaning the standardized data packets, and constructing five-dimensional structured information based on the cleaned data;

[0010] The large-screen display module is configured to visually present the five-dimensional structured information; the interactive control module is configured to capture a touch operation of the security personnel on the large-screen display module, and analyze the touch operation in a context-aware manner to generate a standardized device control instruction;

[0011] The security device control module is configured to receive the standardized device control instruction, translate the standardized device control instruction, generate a native instruction executable by a front-end device, and issue the native instruction.

[0012] The data fusion module specifically includes:

[0013] The data cleaning unit is configured to perform data quality assessment and filtering on the standardized data packet;

[0014] The five-dimensional information model construction unit is configured to extract or generate a device identity, a data type, an event time, an event description, and an event location from the valid data packet to construct the five-dimensional structured information.

[0015] The data fusion module further includes:

[0016] The video stream lightweight and structured processing unit is configured to, when the data type of the data packet is video, analyze the video stream by using a lightweight machine vision algorithm to identify abnormal behavior and generate a structured video data packet containing an event key frame, a feature vector, and an event label.

[0017] The interactive control module includes:

[0018] The instruction analysis unit is internally configured with an instruction mapping rule library, and is configured to, in combination with a gesture type of the touch operation and a user interface context corresponding to a touch point, match in the instruction mapping rule library to generate the standardized device control instruction.

[0019] The interactive control module further includes:

[0020] The permission management unit is configured to, before issuing the standardized device control instruction, perform role-based permission verification according to a role of a current operator.

[0021] The encryption unit is configured to encrypt the standardized device control instruction that passes the permission verification by using a symmetric key.

[0022] The interactive control module further includes:

[0023] The instruction flow scheduling component is internally provided with a priority queue, and is configured to, according to a preset priority value of the standardized device control instruction, schedule and dequeue the encrypted instruction.

[0024] The interface of the large-screen display module is divided into multiple functional partitions, and the functional partitions include a global situation area;

[0025] The large-screen display module is used for analyzing the event position in the five-dimensional structured information and visually presenting the security event on the electronic map of the global situation area.

[0026] The security device control module includes:

[0027] The multi-threaded task processing engine configured with a thread pool and a task queue is used to realize parallel processing of high-concurrency instructions;

[0028] Each working thread adopts a non-blocking I / O model when issuing instructions, and asynchronously processes device receipts through the registration of callback functions.

[0029] The data fusion module is deployed on an edge computing node;

[0030] The security device control module is distributed and deployed near the front-end device.

[0031] In the second aspect, the intelligent urban security and large-screen interactive control method provided by the present application adopts the following technical solution:

[0032] Collecting heterogeneous security data, and injecting space-time information into the heterogeneous security data to generate standardized data packets;

[0033] Receiving the standardized data packets, performing data cleaning on the standardized data packets, and constructing five-dimensional structured information based on the cleaned data;

[0034] Visually presenting the five-dimensional structured information;

[0035] Capturing the touch operation of the security personnel on the visual presentation interface, and analyzing the context of the touch operation to generate standardized device control instructions;

[0036] Receiving the standardized device control instructions, and translating the standardized device control instructions to generate native instructions executable by the front-end device and issuing.

[0037] In summary, the present application includes at least one of the following beneficial technical effects:

[0038] 1. This invention, by setting up a data fusion module, unifies heterogeneous security data from different sources and in different formats into five-dimensional structured information containing dimensions such as device identity, time, and location. Furthermore, by performing lightweight structural processing on video streams, core event information is carried in concise data packets. This solves the problem of data silos between different security subsystems, and while achieving a unified presentation of the global security situation, it significantly reduces the network bandwidth consumption of data transmission.

[0039] 2. This invention, through the context-aware parsing mechanism built into the interactive control module, can combine the touch gestures of security personnel and the business objects they touch on the large screen interface to accurately map intuitive physical operations into standardized device control commands. Compared with the traditional multi-step menu operation, this method simplifies the human-computer interaction process, shortens the emergency response time, and improves the efficiency of event handling in complex scenarios.

[0040] 3. This invention constructs a complete security control link from user operation to device execution by integrating an access control unit and an encryption unit into the interactive control module, as well as a priority scheduling mechanism before the security device control module issues instructions. This ensures that all control operations are subject to compliance verification and security encryption, preventing the risk of unauthorized operations and instruction interception. At the same time, it ensures that control instructions for emergency events can be processed with priority, thereby improving the security and reliability of the entire system. Attached Figure Description

[0041] Figure 1 This is a system architecture diagram of the present invention;

[0042] Figure 2 This is a flowchart of the method of the present invention;

[0043] Figure 3 This is a schematic diagram of the data fusion and structured processing flow of the present invention;

[0044] Figure 4 This is a schematic diagram of the control command generation and issuance process of the present invention.

[0045] Explanation of reference numerals in the attached diagram: 100, Data acquisition module; 200, Data fusion module; 300, Interactive control module; 400, Large screen display module; 500, Security equipment control module. Detailed Implementation

[0046] The following is in conjunction with the appendix Figure 1 - Appendix Figure 4 This application will be described in further detail below.

[0047] The interactive control system for smart city security and large screens, referring to Figure 1 ,include:

[0048] The data acquisition module 100, data fusion module 200, interactive control module 300, large screen display module 400, and security equipment control module 500 are all connected via a dual communication link of high-speed Ethernet and 5G private network for data interaction and command transmission.

[0049] The interactive control method for smart city security and large screens refers to... Figure 2 This includes the following steps:

[0050] S201. Perform initial configuration. In the early stage of system deployment, input basic information such as the identity, device type, network address and physical location coordinates of each front-end security device; set the abnormal judgment threshold of sensor data and video analysis behavior; and preset the default display layout scheme of the large screen.

[0051] S202. Collect and standardize security data. The data acquisition module obtains raw data streams from front-end security devices at a preset frequency, unifies them into JSON data format through the built-in protocol conversion unit, and injects a collection timestamp and device GPS coordinate label into each data item to form a standardized data packet.

[0052] S203. Data fusion and structuring: The data fusion module receives standardized data packets and first removes invalid data such as blurry images and duplicate false alarms through the data cleaning unit. Then, the valid data is organized into five-dimensional structured information containing device ID, data type, time, event, and location, and pushed to the large screen for display. For video data, this step also includes extracting key features from the video, identifying specific behaviors, and generating structured data packets containing keyframes, feature vectors, and event labels.

[0053] S204. Responding to interactive operations and issuing instructions: The system receives touch operations from security personnel on the large screen, and the interactive control module parses these operations into standardized device control instructions. After verifying operator permissions through the permission management unit, the instructions are encrypted using AES-256 and then sent to the corresponding security device control module.

[0054] S205. Execute automatic early warning and closed-loop archiving. When the data fusion module detects emergency events that meet preset conditions, such as perimeter intrusion or fire, the system automatically triggers the early warning mechanism, highlights a pop-up window on the large screen and broadcasts it via voice, and provides a one-click handling option. The entire process of the event, from triggering and personnel intervention to completion of handling, is recorded by the system, forming a traceable closed-loop archive.

[0055] The composition and operation of the data acquisition module 100 are described in detail. This data acquisition module 100 is the entry point for this system to interact with the front-end physical security world. Its core function is to enable the extensive access and data acquisition of security devices of different types and manufacturers deployed in the urban area.

[0056] To enable access to multi-source heterogeneous devices, the data acquisition module 100 adopts a modular design in its physical structure. The data acquisition module 100 may include a main control unit and one or more expandable interface units. The main control unit is responsible for the management and basic processing tasks of the internal data bus, while the interface units provide physical ports for connection to specific devices. This design allows the system to flexibly configure different interface units according to the needs of actual security scenarios to adapt to different types of front-end devices.

[0057] The interface units may include, but are not limited to, the following:

[0058] One type is the network interface unit, which provides a standard RJ45 Ethernet port or an SFP / SFP+ fiber optic interface. The RJ45 port is used to connect devices that conform to the Ethernet protocol, such as network HD cameras, network video recorders, and IP access controllers. The SFP / SFP+ fiber optic interface is used to connect devices that require long-distance, high-bandwidth transmission, such as high-altitude observation dome cameras deployed at urban high points or optical transceivers that aggregate remote video streams.

[0059] Another type is the serial communication interface unit, which provides serial ports such as RS-485 or RS-232. These ports are mainly used to connect devices that communicate using serial bus protocols, such as traditional pan-tilt control decoders, some models of perimeter protection equipment, or industrial-grade sensors.

[0060] Another type is a digital / analog input / output (I / O) interface unit. This unit provides multiple dry or wet contact input terminals and relay output terminals. The dry contact input is used to receive passive switching signals, such as alarm signals from infrared beam detectors, door magnetic sensors, or emergency buttons. The analog input can be used to connect sensors that output continuous voltage or current signals, such as temperature and humidity sensors. The relay output is used to control actuators such as audible and visual alarms and electric locks.

[0061] In addition, to connect to wireless security devices, the data acquisition module 100 can also be configured with a wireless communication interface unit. This wireless communication unit can integrate one or more wireless communication modules, such as Wi-Fi modules, Zigbee modules, LoRa modules, or 5G communication modules. Zigbee and LoRa modules are mainly used to connect to low-power, low-data-volume devices, such as smart fire smoke detectors, wireless door magnets, or environmental monitoring sensors. The 5G communication module provides a high-speed, low-latency wireless access channel for devices such as high-definition cameras deployed in areas without fixed network conditions.

[0062] Through the combination of one or more of the above interface units, the data acquisition module 100 can establish physical communication links with conventional and special security equipment in smart cities, providing a foundation for subsequent data acquisition and standardized processing. As for the electrical characteristics and connection methods of various interfaces, those skilled in the art can configure them according to the technical manual of the specific equipment, which are well-known technologies in the field and will not be described in detail here.

[0063] After the data acquisition module 100 establishes a physical communication link with the front-end device, its built-in protocol conversion unit begins to perform automatic protocol adaptation and data formatting. This process aims to solve the problem of inconsistent data communication protocols caused by different manufacturers and models of front-end devices, and is the basis for realizing subsequent data fusion of the system.

[0064] The protocol conversion unit can be implemented as a software collection containing multiple protocol parsing libraries, deployed within the main control unit of the data acquisition module 100. This protocol parsing library has pre-installed drivers or parsing scripts for various mainstream and industry-specific security communication protocols, such as ONVIF, RTSP, and GB / T28181 protocols for network video devices, as well as proprietary TCP / UDP protocols for other sensors. This protocol parsing library is extensible, allowing new protocol drivers to be added as needed to support future device types.

[0065] The specific process of automatic protocol adaptation and data formatting may include the following steps:

[0066] Protocol identification and driver loading: The protocol conversion unit matches the currently connected device with the device information entered during the system initialization configuration phase and loads the corresponding protocol processing driver from the protocol parsing library.

[0067] Session establishment and data acquisition: The loaded protocol processing driver actively establishes a session connection with the front-end device. Depending on the protocol, this process may include device authentication, capability set negotiation, and the establishment of a bitstream or data channel. After the session is established, the protocol processing driver receives the raw data stream from the front-end device according to the specifications of its specific protocol. .

[0068] Data parsing and content extraction, protocol processing driven by the received raw data stream The data is analyzed to extract the core data payload. For video data streams, the core data payload is the video frame data encoded by standards such as H.264 or H.265; for access control card swipe events, it is information such as card number and swipe result; for infrared detectors, it is the switch signal that triggers or resets the data.

[0069] The unified formatting and encapsulation process involves the protocol conversion unit encapsulating the core data payload extracted in the previous step into a unified JSON data object within the system. This formatting process can be handled by a conversion function. express:

[0070] ;

[0071] in: A unique identifier representing the current device; Represents the raw data stream received from the device; This represents a function that transforms raw data into a JSON object, and internally encapsulates the parsing logic of S303. This represents the generated JSON format data object.

[0072] generated It is a structured data object that contains fields used to carry the core data load. For example, for video data, The value can be a Base64 encoded video frame data string; for sensor data, The value can be a specific numerical value or a Boolean value. This step ensures that the data generated by the front-end device has a consistent, machine-readable structure before entering the next processing stage of the system, regardless of the source of the front-end device.

[0073] For message parsing methods of specific protocols such as ONVIF and RTSP, those skilled in the art can implement them based on their published technical standards. These are well-known technologies in the field and will not be elaborated here. The JSON format data object generated after processing by the protocol conversion unit will be submitted to the next processing unit for spatiotemporal information injection.

[0074] The data formatting is completed in the protocol conversion unit, generating a JSON format data object. Subsequently, the data acquisition module 100 continues to accurately inject spatiotemporal information into the data object. This step provides an indispensable data dimension for subsequent data fusion analysis, event tracing, and accurate positioning on the electronic map.

[0075] The spatiotemporal information injection process can be broken down into the following steps:

[0076] Timestamp injection: The main control unit of the data acquisition module 100 obtains its current system time and uses it as the timestamp for data acquisition. To ensure consistent time reference across all devices and modules in the security system, the system clock of the data acquisition module 100 periodically synchronizes with the central time server via a network time protocol, ensuring the accuracy and uniformity of timestamps across the entire network. The precision can reach the millisecond level, and it is then added to the data object as a new field. middle.

[0077] Standardized data packet generation, through the above steps, data objects Having successfully injected information in both temporal and spatial dimensions, the system will now combine all fields containing the original data payload and the newly added spatiotemporal labels to generate a standardized data packet with a complete structure. The generation process can be represented by the following formula:

[0078] ;

[0079] in: This represents the final standardized data packet; From the original The device's unique identifier is inherited in the process; It is the precise timestamp of the injection; These are the injected device GPS coordinates; From the original The load field inherited from the middle and carrying the core data.

[0080] Through precise injection of spatiotemporal information, each piece of data originating from the front-end device is given a unique spatiotemporal identity. After completing this step, the data acquisition module 100 ends its processing flow and generates standardized data packets. Data is transmitted to the data fusion module 200 via high-speed Ethernet or 5G private network for further in-depth processing.

[0081] The data fusion module 200 receives standardized data packets sent by the data acquisition module 100. It then begins to perform data fusion and structured processing.

[0082] The data fusion module 200 adopts an edge deployment mode, which can be deployed at computing nodes located at the edge of the communication network, such as edge servers in community computer rooms or smart gateways at street aggregation points. This deployment mode constitutes the local preprocessing and cloud collaborative architecture of the present invention. It performs preliminary data processing near the data source to reduce the amount of data transmitted to the central cloud platform and reduce the latency of data analysis. The data fusion module 200 has a built-in data cleaning unit, which is used to identify and filter the data for validity before it enters the subsequent processing stage.

[0083] The data cleaning process may specifically include the following steps:

[0084] Data reception and parsing: The input interface of the data fusion module 200 receives standardized data packets. And parse out the device identification identifier. Data load Fields such as...

[0085] Data quality assessment, data cleaning unit's assessment of data load Perform a quality assessment, which is multi-dimensional and employs different assessment strategies for different types of data.

[0086] For video data, the data cleaning unit calls the built-in image quality assessment algorithm to calculate the sharpness index of the current video frame. This algorithm can be based on the Laplacian operator, gradient function, or other algorithms with image texture complexity.

[0087] For sensor data, the data cleaning unit checks whether the values ​​are within the preset valid range. For example, if the temperature sensor reading exceeds its range, it is determined to be abnormal.

[0088] For event-based data, the data cleaning unit checks the integrity of its data packets to ensure that no key information fields are missing.

[0089] The data filtering and cleaning unit makes filtering decisions on data packets based on the evaluation results of the previous step. This decision-making process can be achieved through a comprehensive data quality evaluation function. and preset quality threshold To formally describe, functions The output is a comprehensive score, which represents the quality of the data packets.

[0090] A data packet is considered valid and proceeds to the next processing stage when the following conditions are met:

[0091] ;

[0092] in: Represents the data payload in a standardized data packet; This represents a comprehensive data quality assessment function, which integrates the calculation results of one or more assessment strategies described in S502; This represents the data quality validity threshold that is set in advance during system initialization configuration.

[0093] Data packets that do not meet this condition, such as video frames with too low resolution, will be judged as invalid data and discarded directly by the cleaning unit.

[0094] In addition, the data cleaning unit also includes a filtering mechanism to suppress duplicate false alarms. This mechanism is implemented by maintaining a short event window in memory for each device, which is set within a very short time window. Received continuously from the same device When identical event reports are received, the mechanism retains only the first event report, and subsequent duplicate reports will be filtered out and discarded.

[0095] For the specific implementation of the image quality assessment algorithm, those skilled in the art can design it based on publicly available computer vision literature. It is a well-known technology in the field and will not be elaborated here. After being processed by the data cleaning unit, the effective standardized data packets will be transmitted to the five-dimensional information model construction unit.

[0096] The effective and standardized data packets processed by the data cleaning unit are transmitted to the five-dimensional information model construction unit in the data fusion module 200. The function of this unit is to convert discrete and heterogeneous data packet streams into a unified and standardized security information model, providing standardized data input for upper-level large-screen display, interactive control and intelligent analysis.

[0097] The model building process maps each valid data packet into five-dimensional structured information. The five-dimensional structured information specifically includes five core dimensions: device identification, data type, event time, event description, and event location, which are used to fully describe the basic attributes of the security information unit.

[0098] The specific construction process may include the following steps:

[0099] Information dimension extraction: The model building unit extracts information from the valid and standardized data packets input. In the process, directly extract the already included device identity identifier. Data collection timestamp and device GPS coordinates .

[0100] The five-dimensional information model construction unit extracts and generates all dimensional information, combining them to construct a complete five-dimensional structured information object. The construction process can be represented by the following formula:

[0101] ;

[0102] in: This represents the final generated five-dimensional structured information object; From The device's unique identifier extracted from it; It is a defined data type; From Extracted timestamps In this model, this is taken as the time of event occurrence; This is the generated event description, whose value may be updated in subsequent video structuring and other analysis steps; From GPS coordinates extracted In this model, it represents the location where the event occurs.

[0103] Through this step, all data from diverse sources that have undergone data cleaning are unified within a standard information framework, generating five-dimensional structured information. As an atomic information unit, it provides the system with a high-level data view decoupled from specific devices and protocols. After construction, if the data type is video, the associated raw video stream will be submitted to the video stream structuring processing unit for in-depth analysis; if it is other types of data, then... The object will be directly pushed to the interactive control module 300 and the large screen display module 400.

[0104] For video data, after the initial construction of the five-dimensional information model is completed, the associated raw video stream data... The video stream lightweight and structured processing unit, which is fed into the data fusion module 200, is executed on the edge computing node. Its purpose is to extract high-value information from the video content, identify potential security events, and generate a lightweight structured data packet that can be quickly retrieved and analyzed by upper-layer applications without transmitting a large amount of the original video bitstream.

[0105] The processing procedure may specifically include the following steps:

[0106] Video target feature extraction: This processing unit employs one or more lightweight machine vision algorithms. For the input video stream Real-time analysis is performed. The algorithm is designed to operate efficiently with limited computing resources at edge nodes. Specifically, this can be achieved by using background modeling methods based on inter-frame difference or Gaussian mixture models to quickly segment moving foreground targets in the video frame; or by deploying a lightweight target detection model that has been pruned or quantized to directly identify and locate specific targets such as people or vehicles in the frame. Through these algorithms, the processing unit extracts the target's contour information, centroid coordinates, and bounding box from the video, and calculates the target's trajectory and velocity through continuous frame tracking. This extracted information collectively constitutes a feature vector. This process can be represented by the following formula:

[0107] ;

[0108] in: Represents the raw video stream data input; The above represents a lightweight feature extraction algorithm; This represents the extracted feature vector containing information such as target contour, trajectory, and velocity.

[0109] Abnormal behavior recognition and label generation: The processing unit will process the feature vector generated in the previous step. Input to behavior classification algorithm In this algorithm, the target's behavior is used to determine whether it belongs to a predefined abnormal event type. The behavior classification algorithm can be a rule-based decision engine or a small machine learning model specifically designed for behavior classification. For example, a running event can be classified by determining whether it belongs to a predefined abnormal event type. The algorithm identifies events by determining whether the recorded target speed consistently exceeds a preset speed threshold over a period of time; clustered events are identified by determining whether the number of detected targets within a certain area exceeds a preset number threshold and this state persists for a certain period of time. The algorithm's output is a specific event label. For example, when running or gathering, the process can be represented by the following formula:

[0110] ;

[0111] in: It is the input feature vector; Representative behavior classification algorithm; This represents the identified event label. If no abnormal behavior is identified, this label can be empty or marked as normal.

[0112] Structured data packet generation and information updating, when abnormal event tags are identified. At that time, the processing unit generates structured video data packets. The data packet contains three core parts:

[0113] A keyframe image captured at the moment the event was confirmed. Feature vectors used to describe the dynamic process of an event and event tags that identify the nature of the event. This combination of visuals, features, and tags preserves direct visual evidence of the event while providing machine-readable analytical data, all in much smaller quantity than the original video clips. This process can be represented as:

[0114] ;

[0115] in: This represents the final structured video data packet generated; Keyframe images representing the event.

[0116] At the same time, the processing unit will generate event tags This is used to update the five-dimensional structured information associated with the video stream that has been initially constructed beforehand. Event description in the object Field.

[0117] For the specific implementation of the target detection and tracking algorithm, those skilled in the art can develop it based on publicly available computer vision technical materials. This is common knowledge in the field and will not be elaborated upon here. After processing, the updated... The object along with the generated The data packet is pushed to both the interactive control module 300 and the large screen display module 400.

[0118] After receiving a touch operation performed by a security personnel on the large screen display module 400, the interactive control module 300 uses its built-in instruction parsing unit to perform context-aware parsing of the operation. The purpose of this parsing is to accurately convert the physical, ambiguous touch action into one or more unambiguous, standardized device control instructions, thereby realizing the understanding of the human-computer interaction intent.

[0119] The context-aware touch finger Specifically, this may include the following steps:

[0120] The original touch operation is captured in real time by the touch engine of the interaction control module 300, which captures the original touch operation generated by the touch layer of the large screen display module 400. The original touch operation It is a data structure that contains the screen coordinates of the touch point, the type of touch gesture, and the start and end states of the gesture.

[0121] User interface context acquisition, in capture Simultaneously, the instruction parsing unit queries the large-screen display module 400 for the user interface context associated with the touch coordinates. , the user interface context It refers to the interface element and its associated business data at the screen location when a touch operation occurs.

[0122] Command mapping and standardized command generation: The command parsing unit processes the captured raw touch operations according to a preset command mapping rule base. and the obtained user interface context Perform joint matching to generate standardized device control commands. This instruction mapping rule base defines the specific business meanings of different gestures in different user interface contexts.

[0123] The specific implementation is that when It is a double-tap gesture, and When the map icon is associated with the device ID CAM001, the instruction parsing unit generates an instruction that retrieves and displays the real-time video from the camera with ID CAM001 in full screen.

[0124] Another specific implementation is, when It is a two-finger zoom gesture, and When the associated device ID is CAM002, the instruction parsing unit generates an instruction that calls the list of surrounding cameras associated with the geographic location of CAM002 and displays it in a multi-screen format.

[0125] Another specific implementation is, when It is a drag gesture, and When a video window is used to support gimbal control, the instruction parsing unit converts the dragging direction and distance into parameters for rotating the camera's gimbal in real time.

[0126] The entire analytical process can be described by an analytical function. Formal description:

[0127] ;

[0128] in: This represents the final standardized device control command. This command is a structured object that contains fields such as command type, target device ID, and specific parameters. This represents the captured original touch operation; Represents user interface context information associated with touch operations; This represents the instruction parsing function, whose internal logic is the matching process of the instruction mapping rule base.

[0129] Through this context-aware parsing process, the intuitive operations of security personnel are accurately translated into machine-executable instructions, generating standardized equipment control commands. It is then passed to the permission management unit within the interactive control module 300 for further permission verification.

[0130] Standardized device control instructions are generated in the instruction parsing unit. Subsequently, the instruction is transmitted to the permission management unit within the interactive control module 300. This unit is the core executor of the system security policy, responsible for verifying the operation permission before the instruction is issued, and encrypting the verified instruction to ensure the compliance of the system operation and the security of the instruction transmission.

[0131] This process may specifically include the following steps:

[0132] Operator identity and role acquisition: The access control unit first obtains the unique user identifier of the operator currently logged into the system. This identity is typically obtained through authentication when an operator logs into the system and is bound to the current session. Subsequently, the access control unit utilizes this... Search the preset user role configuration library to retrieve the roles granted to the user. The roles can include, but are not limited to, system administrators, regional supervisors, and regular operators, with different roles having different sets of operating permissions.

[0133] Role-based access control allows the access management unit to process standardized device control commands. The unit performs parsing to extract the instruction type and target device ID contained within, and then executes the permission verification function. Determine the role of the current operator. To determine whether an operator has the authority to execute the specified instruction on the target device, the verification function queries a predefined permission rule table. This table details the types of operations that each role is allowed to perform and the range of devices that can be operated. For example, the rule might specify that a regular operator role can only view video from cameras within a designated area, while a system administrator role can perform all operations, including remotely restarting the device.

[0134] ;

[0135] in: This represents the validation result, with a value indicating either allow or deny. This represents the permission verification function; This refers to the current operator's role; It is the operation type parsed from the instruction; It is the target device ID parsed from the instruction.

[0136] like If the request is rejected, the instruction will be intercepted, and the system will return a message indicating that the operation is not authorized to the large screen display module 400.

[0137] Command encryption, if To allow it, the access control unit will send standardized device control commands. The instructions are passed to an encryption unit, which encrypts them to prevent unauthorized eavesdropping or tampering during network transmission. In this embodiment, the encryption unit employs an Advanced Encryption Standard (AES) algorithm, specifically a 256-bit key length. The encryption process transmits the entire instruction to an encryption unit. Serializing an object into a plaintext string Then, a symmetric key pre-installed between the interactive control module 300 and the security device control module 500 is used. The plaintext is encrypted to generate the encrypted instruction ciphertext. .

[0138] ;

[0139] in: This represents the generated encrypted instruction ciphertext; Represents the AES-256 encryption function; This represents the serialized plaintext instruction; This represents a pre-set symmetric key.

[0140] After encryption is completed, the generated encrypted command ciphertext It is passed to the process scheduling component, waiting to be sent to the front-end security equipment control module 500.

[0141] The encrypted command ciphertext generated after processing by the access control unit The instructions are sent to the instruction flow scheduling component in the interactive control module 300. This component is responsible for the unified management and orderly dispatch of instructions waiting to be issued. Especially when the system faces multiple concurrent operation requests or sudden emergency events, it ensures that critical instructions can receive the highest priority response and execution.

[0142] The specific process of instruction flow scheduling and priority handling may include the following steps:

[0143] Instruction enqueueing and priority marking; the instruction flow scheduling component receives encrypted instruction ciphertext. Before the instructions are encrypted, their original plaintext instructions are... The permission management unit has assigned a preset priority value based on its instruction type. This priority value As metadata, along with encrypted instructions ciphertext Both are sent to the scheduling component, which will then process this pair of data ( , As a scheduling unit, it is stored in the priority queue to await processing.

[0144] priority value The value is an integer, and the smaller the value, the higher the priority. The specific priority division scheme can be as follows:

[0145] (Highest Priority): Commands triggered by the highest level of safety-threatening events automatically detected by the system, such as fire signals or perimeter intrusion signals identified by the data fusion module, triggering alarms or evacuation commands.

[0146] : Emergency plan instructions assigned to security personnel via a one-click response button on the large screen, such as manually initiating area lockdown or activating fire-fighting facilities.

[0147] Active real-time control operations assigned to security personnel, such as adjusting the pan-tilt-zoom of cameras and conducting voice intercoms.

[0148] (Lowest Priority): Assigned to non-real-time query or configuration operations, such as retrieving historical recordings or modifying device parameters.

[0149] Priority-based dequeueing involves a scheduler within the instruction flow scheduling component continuously monitoring this priority queue. The queue's data structure ensures that regardless of the instruction enqueue order, each instruction possesses the lowest priority value. The scheduling unit is always located at the head of the queue. The scheduler retrieves the highest priority scheduling unit from the head of the queue and extracts the encrypted instruction ciphertext from it. .

[0150] Command routing and distribution: Before issuing a command, the scheduler needs to determine its target recipient, due to the encryption of the command. The content cannot be read directly; the scheduler will read it from the... The target device ID of the instruction is obtained from the metadata transmitted along with the instruction. Subsequently, the scheduler queries the internally maintained device routing table, which records the mapping relationship between each front-end device ID and the network address of its corresponding security device control module 500. Based on this mapping relationship, the scheduler encrypts the instruction ciphertext. Send the data via the network to the correct security equipment control module 500.

[0151] The system supports the execution of predefined emergency response plans for certain complex security incidents. An emergency response plan is essentially a set of instructions containing multiple ordered instructions. When the triggering condition is met, the interactive control module 300 loads the corresponding plan, and the instruction flow scheduling component is responsible for sending all the instructions in the plan into the priority queue in sequence or in parallel according to the preset priority, so as to ensure that multiple steps in the entire handling process can be coordinated and executed in an orderly manner by the system.

[0152] The large screen display module 400 is the core of the human-computer interaction interface provided by the present invention, and is responsible for integrating and visualizing various structured information from the data fusion module 200.

[0153] The large screen display module 400 divides the entire display interface into several logical functional zones. This functional zone design allows security personnel to simultaneously grasp the overall situation and focus on key details. The functional zones may include the overall situation zone, the key monitoring zone, the event warning zone, and the equipment status zone.

[0154] The large screen display module receives 400 pairs of five-dimensional structured information. and structured video data packets The specific process of visualization may include the following steps:

[0155] Data reception and distribution: The rendering engine of the large screen display module 400 receives the data stream from the data fusion module 200 and distributes it to the corresponding functional partitions for rendering according to the data content.

[0156] The visualization of the global situation area, which typically occupies the main part of the large screen, is set against a high-precision two-dimensional or three-dimensional electronic map. The rendering engine parses the five-dimensional structured information. Event location in And render the device icon at the corresponding location on the electronic map; the style of the icon is determined by the data type. Decisions can be made, for example, using camera graphics to represent video devices and door icons to represent access control devices, with the icon's color or status determined by the event description. Decision, for example, when The icon is green when it is normal, and when... When an abnormal event occurs, such as running or a fire alarm, the icon turns red and flashes brightly to visually represent the location and status of the event in a spatial dimension.

[0157] The visualization of key monitoring areas typically exists as a video matrix, displaying one or more critical real-time video feeds. By default, this area can cycle through preset important location videos. When security personnel click on a camera icon in the global situational awareness area, or when the system receives an emergency alert associated with a camera, this area automatically switches, displaying the corresponding real-time video stream or structured video data packet. Keyframe images in This is displayed here for security personnel to observe in detail.

[0158] The event alert area is visualized as a scrollable list or pop-up window. This area is displayed when the rendering engine receives an event alert. Event description When the status is abnormal, a new warning message entry will be added to the list, and this entry should at least display the event time. Event location and event description This list format provides security personnel with a clear, chronological log of event alerts.

[0159] The device status area is visualized, showcasing the overall operational health of the entire security system. The rendering engine periodically summarizes the data from all devices. The system provides statistics on the total number of online devices, the total number of offline devices, and daily / weekly statistics on various alarm events, which are then visualized in the area using digital dashboards, pie charts, or bar charts.

[0160] Through the aforementioned functional partitioning and information visualization mechanism, the system transforms abstract data into a graphical language closely integrated with security business scenarios, enabling security personnel to quickly understand the current security situation and respond accordingly. For the loading and rendering of electronic maps, those skilled in the art can use existing GIS engines, which are well-known technologies in the field and will not be elaborated upon here.

[0161] The large screen display module 400 is not only responsible for the static presentation of information, but its close integration with the interactive control module 300 also jointly constructs a scenario-based multi-dimensional interactive design. This design deeply associates the intuitive touch operation of security personnel with multi-dimensional information in different functional areas, thereby realizing intelligent interface linkage and information drilling in specific security scenarios.

[0162] The following two specific security scenarios illustrate this multi-dimensional interaction design.

[0163] Scenario 1: Rapid Event Response and Source Tracing

[0164] In this scenario, when the data fusion module 200 identifies an emergency event and generates corresponding five-dimensional structured information... Subsequently, the large-screen display module 400 displays the event location on the electronic map in the global situation area. The warning icon is highlighted and flashed in the display area.

[0165] Security personnel perform a single click on the alarm icon on the large screen. The interactive control module 300 captures this operation and parses it into a standardized instruction to view event details based on the user interface context (alarm icon in the global situation area).

[0166] In response to this command, the large screen display module 400 executes a set of linked operations:

[0167] First, the video window in the key monitoring area automatically switches to display the devices associated with the event. Real-time video stream, or direct playback of images containing event keyframes. Structured video data packets Secondly, the list of event warning zones automatically scrolls to locate the alarm message and highlights it. This process enables rapid information drilling down from spatial location to visual details and time records.

[0168] Scenario 2: Autonomous Situation Building

[0169] In this scenario, security personnel are observing the real-time video from a camera in a key monitoring area. To obtain information about the surrounding environment, the operator performs a two-finger zoom-in touch gesture on the video window.

[0170] The interactive control module 300 captures this gesture and parses it into a standardized command for constructing the surrounding situation based on the user interface context in which it occurs (the video window of the key monitoring area). This command includes the device ID associated with the current video window and its GPS coordinates. .

[0171] In response to this command, the interactive control module 300 performs a spatial query in the system's device information database, with the goal of retrieving all data related to the current device's geographical location. The distance is less than the preset radius For other camera devices, the query process can be represented as finding a set of devices that meet the following conditions. :

[0172] ;

[0173] in: This represents the set of nearby camera device IDs retrieved. and These represent the ID and GPS coordinates of other devices in the database, respectively. It is a function that calculates the geographical distance between two GPS coordinate points; It is a configurable radius for surrounding queries, such as 50 meters.

[0174] After the query is completed, the large screen display module 400 automatically switches the layout of the key monitoring area from a single screen to a multi-screen video matrix, and simultaneously retrieves and displays the video from each grid. The collection of real-time video streams from all devices enables an expansion of the perspective from single-point monitoring to area coverage, providing security personnel with temporary, event-centric 360-degree virtual patrol capabilities.

[0175] The security equipment control module 500 is the final execution end of the control link of this invention, responsible for accurately translating the instructions from the interactive control module 300 into operations on the front-end physical devices.

[0176] The security equipment control module 500 adopts a distributed deployment approach. This security equipment control module 500 is not a centralized single entity, but a group of software units or embedded firmware deployed at the network edge and adjacent to the front-end security equipment. Specifically, the security equipment control module 500 can be deployed in the same hardware entity as the data acquisition module 100, or run as a software service on the edge computing node where the data fusion module 200 is located. This distributed deployment approach shortens the physical path for control commands to reach the final device, providing a network foundation for achieving low-latency real-time control.

[0177] The specific process by which the security equipment control module 500 receives and executes control commands may include the following steps:

[0178] Encrypted command reception: The network monitoring service of the security equipment control module 500 receives the encrypted command ciphertext sent by the command flow scheduling component of the interactive control module 300. .

[0179] Command decryption: The decryption unit within the module uses a symmetric key pre-shared with the interactive control module 300. The received encrypted command ciphertext Decryption is performed to recover the original, serialized plaintext instructions. The decryption process is the inverse operation of the encryption process, and can be represented by the following formula:

[0180] ;

[0181] in: This represents the plaintext instruction string recovered after decryption. This represents the AES-256 decryption function; It is the received encrypted instruction ciphertext; It is a pre-set symmetric key, the same as that of the interactive control module 300.

[0182] After successful decryption, the decryption unit will return the plaintext string. Deserialization restores the device to a standardized device control command containing fields such as command type and target device ID. .

[0183] Instruction translation, standardized equipment control instructions This is a unified instruction format within the system, which cannot be directly recognized by specific front-end devices. Therefore, the instruction translation unit within the security equipment control module 500 needs to translate this standard instruction into the native control instructions of specific devices. This instruction translation unit internally maintains a device driver library, which contains control protocol drivers for different manufacturers and models of devices. The translation unit parses... The target device ID is retrieved, and the corresponding device driver is loaded. This driver is based on... The system generates native commands that conform to the device's proprietary protocol or industry standard protocol, based on the command type and parameters specified in the instructions. .

[0184] Command issuance and status feedback: The command translation unit will generate the original command. The command is sent to the target device via a physical communication interface connected to the front-end device, thereby executing specific operations, such as rotating the camera pan-tilt unit or controlling the access control relay to engage. After the command is issued, the security equipment control module 500 listens for execution feedback from the device. This feedback may be a confirmation response or a response containing an error code. The security equipment control module 500 encapsulates this execution result into a status feedback message. This feedback mechanism forms a closed loop for control operations, enabling upper-layer applications to know the final execution status of instructions.

[0185] For specific device SDK calling methods or private control protocol message formats, those skilled in the art can refer to the development documents provided by the corresponding device manufacturers for implementation, which are well-known technologies in the field and will not be elaborated here.

[0186] To cope with the large number of concurrent control requests that may occur in scenarios such as emergency response or multi-user collaborative operation, the security equipment control module 500 has the ability to process high-concurrency commands in parallel. This capability ensures that even in the event of a sudden increase in command traffic, the system can still process and issue each command in a timely manner, avoiding control delays caused by command backlog.

[0187] In this embodiment of the invention, the parallel processing capability is achieved through a multi-threaded task processing engine built into the security equipment control module 500. The core of this engine is a thread pool and a task queue, which work together to achieve parallel execution of instructions.

[0188] The specific processing procedure may include the following steps:

[0189] Task encapsulation and queuing: Once the decryption unit of the security equipment control module 500 completes the decryption of a command and restores the standardized equipment control command... Subsequently, the instruction will not be executed immediately, but will be encapsulated into an independent executable task unit. Subsequently, the task unit was placed in the first-in-first-out task queue. In the middle, waiting for the engine to schedule.

[0190] Thread pool scheduling and task execution: The multi-threaded task processing engine pre-creates and maintains a fixed number of threads when the module starts. A thread pool of worker threads. These worker threads are in a ready state and continuously monitor the task queue. Once a new task unit appears in the queue An idle worker thread will retrieve the task from the queue and begin executing it independently. This execution process is the instruction translation, instruction issuance, and status feedback steps. Since multiple worker threads can retrieve tasks from the queue and execute them simultaneously, parallel processing of multiple instructions is achieved.

[0191] The number N worker threads in the thread pool This is a configurable parameter, and its optimal value is usually related to the hardware capabilities of the edge computing node where the module is deployed. For specific configuration methods, please refer to the following formula:

[0192] ;

[0193] in: This refers to the number of worker threads in the thread pool. It is the number of physical cores of the CPU in the edge computing node; It is an empirical coefficient, usually ranging from 1 to 2, used to balance CPU-intensive tasks and I / O-intensive tasks.

[0194] Non-blocking I / O and asynchronous feedback: To further improve processing efficiency, each worker thread uses a non-blocking I / O model when executing instructions. Specifically, when a worker thread sends a raw instruction... After being sent to the front-end device over the network, it does not wait for a response from the device. Instead, it immediately terminates the execution of the current task and returns itself to the thread pool to prepare for the next task. For device execution status acknowledgments, the system processes them asynchronously by registering a callback function. When the device acknowledgment arrives, the operating system triggers the callback function, which then completes the status feedback message. Encapsulation and return.

[0195] Through the aforementioned thread pool and non-blocking I / O mechanism, the security equipment control module 500 can fully utilize the computing resources of edge nodes and decouple instruction processing from network communication waiting time, thereby maintaining a low instruction execution latency even when facing high concurrency requests. For the specific programming implementation of the thread pool and non-blocking I / O model, those skilled in the art can use the APIs provided by modern operating systems or mature third-party libraries to complete it, which are well-known technologies in the field and will not be elaborated here.

Claims

1. An interaction control system of security and large screen in smart city, characterized in that, The method comprises the following steps: The data acquisition module is used for collecting heterogeneous security data and injecting space-time information into the heterogeneous security data to generate standardized data packets; The data fusion module is used for receiving the standardized data packets, performing data cleaning on the standardized data packets, and constructing five-dimensional structured information based on the cleaned data; The large-screen display module is used for visualizing the five-dimensional structured information; The interactive control module is used for capturing touch operations of security personnel on the large-screen display module, performing context-aware analysis on the touch operations to generate standardized device control instructions; 2.The intelligent city security and large screen interactive control system according to claim 1, characterized in that, The security device control module is used for receiving the standardized device control instructions, translating the standardized device control instructions, generating native instructions executable by front-end devices, and issuing the native instructions. The data fusion module specifically comprises: The data cleaning unit is used for performing data quality evaluation and filtering on the standardized data packets; 3.The intelligent city security and large screen interactive control system according to claim 2, characterized in that, The five-dimensional information model construction unit is used for extracting or generating device identity, data type, event time, event description, and event location from valid data packets to construct the five-dimensional structured information. The data fusion module further comprises: 4.The intelligent city security and large screen interactive control system of claim 1, wherein, The video stream lightweight and structured processing unit is used for analyzing video streams using lightweight machine vision algorithms to identify abnormal behaviors and generate structured video data packets containing event key frames, feature vectors, and event labels when the data type of the data packet is video. The interactive control module comprises: 5.The intelligent city security and large screen interactive control system according to claim 1, characterized in that, The instruction analysis unit is internally configured with an instruction mapping rule library, and is used for matching in the instruction mapping rule library in combination with the gesture type of the touch operation and the user interface context corresponding to the touch point to generate the standardized device control instructions. The interactive control module further comprises: The permission management unit is used for performing role-based permission verification according to the role of the current operator before issuing the standardized device control instructions; 6.The intelligent city security and large screen interactive control system according to claim 5, characterized in that, The encryption unit is used for encrypting the standardized device control instructions that pass the permission verification using a symmetric key. The interactive control module further comprises: 7.The intelligent city security and large screen interactive control system according to claim 1, characterized in that, The instruction flow scheduling component is internally provided with a priority queue, and is used for scheduling and dequeuing the encrypted instructions according to the preset priority values of the standardized device control instructions. The interface of the large-screen display module is divided into multiple functional partitions, and the functional partitions include a global situation area; 8.The intelligent city security and large screen interactive control system according to claim 1, characterized in that, The large-screen display module is used for analyzing the event location in the five-dimensional structured information and visualizing security events on the electronic map in the global situation area. The security device control module comprises: The multi-threaded task processing engine configured with a thread pool and a task queue is used for implementing parallel processing of high-concurrency instructions; 9.The intelligent city security and large screen interactive control system according to claim 1, characterized in that, Each working thread uses a non-blocking I / O model when issuing instructions and asynchronously processes device receipts through a registered callback function. The data fusion module is deployed on an edge computing node; 10. The interactive control method of the security and protection of the smart city and the large screen according to any one of claims 1-9, wherein the interactive control system of the security and protection of the smart city and the large screen according to any one of claims 1-9 is characterized in that, The security device control module is distributed and deployed adjacent to the front-end devices. The method comprises the following steps: Collect heterogeneous security data, and inject space-time information into the heterogeneous security data to generate standardized data packets; Receive the standardized data packets, perform data cleaning on the standardized data packets, and construct five-dimensional structured information based on the cleaned data; Visualize the five-dimensional structured information; Capture touch operations of security personnel on the visualized interface, and analyze the touch operations in a context-aware manner to generate standardized device control instructions; Receive the standardized device control instructions, and translate the standardized device control instructions to generate native instructions executable by front-end devices and issue the native instructions.

Images