An intelligent security and protection early warning method and system based on multi-modal perception fusion

Abstract

This invention relates to the field of security early warning technology, and particularly to an intelligent security early warning method and system based on multimodal perception fusion. The method includes: performing local early warning on raw monitoring data to obtain raw modal early warning values; sending the raw modal early warning values ​​and node transmission parameters to a security early warning terminal to obtain node positioning instructions; identifying the set of sensing nodes to be fused based on the node positioning instructions; constructing transmission paths for the set of sensing nodes to be fused based on the relay forwarding node set to obtain a current forwarding vector set; constructing a current forwarding matrix based on the current forwarding vector set; optimizing node transmission on the current forwarding matrix to obtain a target forwarding matrix; and generating a target transmission strategy based on the target forwarding matrix. This invention can improve the adaptive optimization capability of data transmission paths within the early warning area and reduce the transmission delay and data loss of early warning data.

Description

Technical Field

[0001] This invention relates to the field of security early warning technology, and in particular to an intelligent security early warning method and system based on multimodal perception fusion. Background Technology

[0002] In the field of industrial safety monitoring, with the widespread application of IoT technology, large-scale sensor nodes are deployed in target early warning areas such as chemical industrial parks and industrial parks to collect multimodal data such as temperature, gas concentration, and vibration in real time. How to aggregate large amounts of monitoring data from scattered sensing nodes to a central processing platform, while ensuring the timeliness and reliability of early warning information under limited communication resources, has become a core issue in the design of security early warning systems. The rational planning of data transmission paths directly affects the system's response speed and data integrity; therefore, a method is needed that can adaptively adjust transmission strategies in dynamic environments to cope with complex scenarios such as dense node distribution and channel interference.

[0003] Traditional methods typically employ fixed routing, pre-assigning a fixed data transmission path to each sensing node. All nodes then forward the raw monitoring data to the central server in a predetermined order. However, this static path lacks flexibility in the face of node failures, channel fluctuations, or localized data bursts, leading to congestion on some paths while others remain idle, resulting in increased transmission delays and data loss. Summary of the Invention

[0004] This invention provides an intelligent security early warning method based on multimodal perception fusion and a computer-readable storage medium. Its main purpose is to improve the adaptive optimization capability of data transmission paths within the early warning area and reduce the transmission delay and data loss of early warning data.

[0005] To achieve the above objectives, the present invention provides an intelligent security early warning method based on multimodal perception fusion, comprising: The target warning area is identified, and the set of modal sensing nodes in the target warning area is identified, wherein the set of modal sensing nodes includes multiple modal sensing nodes; Modal sensing nodes are extracted sequentially from the modal sensing node set, and the extracted modal sensing nodes are used for data monitoring to obtain raw monitoring data. Local early warning is generated from the raw monitoring data to obtain the raw modal early warning value; The original modal warning value and the preset node transmission parameters are sent to the pre-built security warning terminal to obtain the node positioning command; The node localization instructions corresponding to each modal sensing node are summarized to obtain the node localization instruction set, and the set of sensing nodes to be fused is determined based on the node localization instruction set; The relay forwarding node set in the target early warning area is identified, and the transmission path is constructed based on the relay forwarding node set to be fused sensing node set to obtain the current forwarding vector set; The current forwarding matrix is ​​constructed based on the current forwarding vector set, and node transmission optimization is performed on the current forwarding matrix to obtain the target forwarding matrix; Based on the target forwarding matrix, a target transmission strategy is generated to complete intelligent security early warning.

[0006] Optionally, the step of constructing the transmission path based on the relay forwarding node set to obtain the current forwarding vector set includes: For each sensing node in the set of sensing nodes to be fused, perform the following operations: The current forwarding count is obtained by randomly generating a forwarding count for the nodes to be fused based on the preset maximum forwarding count; Based on the current number of forwardings, a random node is selected from the set of relay forwarding nodes to obtain multiple current forwarding nodes; The current forwarding vector is obtained by constructing binary vectors of forwarding nodes based on multiple current forwarding nodes and a preset forwarding vector template; The current forwarding vector set is obtained by summing the current forwarding vectors corresponding to each sensing node to be fused.

[0007] Optionally, the step of optimizing node transmission in the current forwarding matrix to obtain the target forwarding matrix includes: For each current forwarding vector in the current forwarding matrix, perform the following operation: A signal transmission path is generated using the current forwarding vector, the sensing node to be fused corresponding to the current forwarding vector, and the security early warning terminal. Transmission characteristics analysis is performed on the signal transmission path to obtain the effective transmission amount and total transmission delay of the path; Identify the starting sensing node in the signal transmission path, perform data priority analysis on the starting sensing node, and obtain the sensing data priority; By summing up the effective transmission volume of the path, the total transmission delay of the path, and the priority of the sensed data for each current forwarding vector, we can obtain multiple effective transmission volumes of the path, multiple total transmission delays of the path, and multiple priorities of the sensed data. The current transmission fitness value is calculated based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data. The target forwarding matrix is ​​obtained by executing a preset optimization algorithm based on the current transmission fitness value.

[0008] Optionally, the step of performing transmission characteristic analysis on the signal transmission path to obtain the effective transmission amount and total transmission delay of the path includes: Node path segmentation is performed based on the signal transmission path to obtain multiple segmented transmission paths, wherein the segmented transmission paths include: a first transmission node and a second transmission node; For each of the multiple split transmission paths, perform the following operation: Data transmission simulation was performed on the segmented transmission path to obtain the path transmission delay, segmented path success rate, and segmented path loss rate. By summing up the path transmission delay, segmentation path success rate and segmentation path loss rate for each segmented transmission path, we can obtain multiple path transmission delays, multiple segmentation path success rates and multiple segmentation path loss rates. The data to be transmitted is queried along the signal transmission path to obtain the original amount of data to be transmitted and the original transmission frequency. The effective transmission volume of the path is calculated based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the amount of original transmitted data, and the original transmission frequency. The total path transmission delay is obtained by calculating the total delay based on the transmission delay of multiple paths.

[0009] Optionally, the step of simulating data transmission on the segmented transmission path to obtain the path transmission delay, segmented path success rate, and segmented path loss rate includes: Obtain the segmented transmission distance between the first transmission node and the second transmission node; The transmission delay of the segmented transmission path is calculated based on the segmented transmission distance to obtain the path transmission delay. Success rate analysis was performed on the segmented transmission path to obtain the segmented path success rate. The transmission loss rate of the split path is obtained by querying the transmission loss of the first and second transmission nodes in the split transmission path.

[0010] Optionally, the step of calculating the transmission delay of the segmented transmission path based on the segmented transmission distance to obtain the path transmission delay includes: The node transmission delay between the first and second transmission nodes in the segmented transmission path is obtained based on the segmented transmission distance. Queries the current amount of input data in the segmented transmission path; Based on the current input data volume, perform a data processing delay query on the first transmission node in the segmented transmission path to obtain the node processing delay. Calculate the path transmission delay based on the node transmission delay and node processing delay.

[0011] Optionally, the step of calculating the effective transmission amount of the path based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the original transmitted data volume, and the original transmission frequency includes: The effective transmission volume of the path is calculated using the following formula: ; in, Indicates the effective transmission volume of the path. Indicates the original transmission frequency. Indicates the original amount of data transmitted. This represents the number of segmentation path success rates out of multiple segmentation path success rates. This represents the success rate of multiple segmentation paths. Success rate of each segmentation path Represents the loss rate of multiple split paths. Individual path loss rate.

[0012] Optionally, the step of performing data priority analysis on the initial sensing node to obtain the sensing data priority includes: Obtain the node data type of the initial sensing node, and determine the current data importance based on the node data type in the pre-built type importance ranking table; Determine the initial modal warning value of the initial sensing node; The priority of the perceived data is obtained by weighting the current data importance and the initial modal warning value.

[0013] Optionally, the step of calculating the current transmission fitness value based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data includes: The set of sensing nodes to be fused is denoted as the target sensing node set, and the target sensing nodes are extracted sequentially from the target sensing node set. Based on the extracted target sensing nodes, the effective transmission volume of the target, the total transmission delay of the target, and the priority of the target data are identified in multiple paths, multiple paths, and multiple sensing data priorities, respectively. Obtain the range of iterative transmission volume, the range of total iterative delay, and the range of iterative priority; The target effective transmission volume, target total transmission delay, and target data priority are normalized using the iterative transmission volume range, iterative total delay range, and iterative priority range, respectively, to obtain the normalized effective transmission volume, normalized total transmission delay, and normalized data priority. The unit fitness value is calculated based on the normalized effective transmission volume, the normalized total transmission delay, and the normalized data priority. The unit fitness values ​​corresponding to each target perception node are summarized to obtain the unit fitness value set; The current transmission fitness value is obtained by summing the set of unit fitness values.

[0014] To achieve the above objectives, the present invention also provides an intelligent security early warning system based on multimodal perception fusion, comprising: The monitoring data acquisition module is used to identify the target warning area and the modal sensing node set in the target warning area. The modal sensing node set includes multiple modal sensing nodes. Modal sensing nodes are extracted sequentially from the modal sensing node set, and data monitoring is performed using the extracted modal sensing nodes to obtain the raw monitoring data. The local data early warning module is used to perform local early warning on the raw monitoring data, obtain the raw modal early warning value, and send the raw modal early warning value and preset node transmission parameters to the pre-built security early warning terminal to obtain the node positioning command; The transmission path optimization module is used to summarize the node positioning instructions corresponding to each modal sensing node to obtain the node positioning instruction set, identify the set of sensing nodes to be fused based on the node positioning instruction set, identify the relay forwarding node set in the target warning area, construct the transmission path for the set of sensing nodes to be fused based on the relay forwarding node set, and obtain the current forwarding vector set. The transmission strategy construction module is used to construct the current forwarding matrix based on the current forwarding vector set, optimize node transmission on the current forwarding matrix to obtain the target forwarding matrix, and generate the target transmission strategy based on the target forwarding matrix.

[0015] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: Memory, storing at least one instruction; The processor executes the instructions stored in the memory to implement the intelligent security early warning method based on multimodal perception fusion described above.

[0016] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned intelligent security early warning method based on multimodal perception fusion.

[0017] To address the problems described in the background section, this invention first analyzes and calculates the original modal warning values ​​locally at the modal sensing nodes. This step enables preliminary quantitative judgment of abnormal situations, reducing the transmission pressure and processing latency caused by uploading all original monitoring data to the central server. Furthermore, this solution sends the original modal warning values ​​and node transmission parameters to the security warning terminal and obtains node positioning instructions, enabling the central server to promptly initiate targeted data transmission path adjustments based on the warning status and transmission characteristics of each node. Then, a set of relay forwarding nodes randomly constructs the current forwarding vector for the sensing nodes to be fused, generating diverse candidate data transmission paths. This avoids premature convergence to locally poor path schemes in subsequent optimization processes. In addition, this solution optimizes node transmission in the current forwarding matrix. This step comprehensively considers the effective transmission volume, total transmission delay, and data priority of each signal transmission path, resulting in a target forwarding matrix that balances transmission efficiency and data importance globally. Therefore, this invention improves the adaptive optimization capability of data transmission paths within the warning area and reduces transmission latency and data loss of warning data. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an intelligent security early warning method based on multimodal perception fusion provided in an embodiment of the present invention. Figure 2 This is a functional block diagram of an intelligent security early warning system based on multimodal perception fusion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the intelligent security early warning method based on multimodal perception fusion, according to an embodiment of the present invention.

[0019] Explanation of reference numerals in the attached figures: 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0020] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] This application provides an intelligent security early warning method based on multimodal perception fusion. The executing entity of the intelligent security early warning method based on multimodal perception fusion includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the intelligent security early warning method based on multimodal perception fusion can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0023] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent security early warning method based on multimodal perception fusion according to an embodiment of the present invention. In this embodiment, the intelligent security early warning method based on multimodal perception fusion includes: S1. Identify the target warning area and identify the modal sensing node set in the target warning area, wherein the modal sensing node set includes multiple modal sensing nodes.

[0024] It is clear that the target early warning area refers to the area requiring security early warning. Example 1: A production area in a large chemical industrial park requires real-time monitoring and early warning of temperature, gas leaks, and abnormal vibrations. This production area is the target early warning area, and the temperature monitoring nodes and gas concentration monitoring nodes installed at various locations within the production area are modal sensing nodes. Example 2: Monitoring of personnel flow in an industrial park is required. Therefore, a large number of cameras cover the industrial park, which can be considered the target early warning area. The covered cameras are modal sensing nodes. The modal sensing node set refers to a collection of multiple modal sensing nodes. A modal sensing node refers to a sensor device capable of monitoring a portion of the target early warning area. For example, a modal sensing node is deployed in a tank area of ​​a chemical industrial park to monitor the temperature of the tank area. The aforementioned modal sensing node includes a data acquisition unit, a data transmission unit, and a data processing unit. The data acquisition unit refers to the hardware module that collects raw monitoring data within the target early warning area, such as a temperature sensor or a gas concentration sensor. The data transmission unit refers to the communication module that sends the data collected by the data acquisition unit (such as the original modal warning values ​​and node transmission parameters) to the security warning terminal. Optionally, the data transmission unit is a wireless transmission module with a frequency band of 2.4GHz. The data processing unit refers to an embedded processor used for local warning.

[0025] S2. Extract modal sensing nodes sequentially from the modal sensing node set, and use the extracted modal sensing nodes to perform data monitoring to obtain raw monitoring data.

[0026] Understandably, the raw monitoring data refers to the monitoring data obtained after data monitoring over a period of time. For example, if a modal sensing node monitors the gas concentration at a location, then the raw monitoring data of the modal sensing node is a collection of multiple gas concentrations collected over a period of time, and the amount of raw monitoring data collected in a single instance is 1024 bits.

[0027] S3. Perform local early warning on the raw monitoring data to obtain the raw modal early warning value.

[0028] It should be explained that the original modal warning value refers to the quantitative value of anomalies in the original monitoring data. The larger the original modal warning value, the higher the probability of an anomaly occurring at the monitoring location corresponding to the original monitoring data. The above-mentioned local warning of the original monitoring data refers to: using the data processing unit of the modal sensing node to analyze the original monitoring data, thereby calculating the original modal warning value.

[0029] Optionally, the specific steps for the data processing unit to analyze the raw monitoring data include: collecting normal monitoring data at the location corresponding to the modal sensing node, where normal monitoring data refers to monitoring data collected when no abnormalities occur at that location; then extracting data features from the normal monitoring data, which are time-series features, including but not limited to: mean, variance, peak value, and zero-crossing rate, and recording these data features as normal data features. Using the same extraction method, data features are also extracted from the raw monitoring data and recorded as raw data features. The feature similarity between the raw data features and the normal data features is calculated. The greater the feature similarity, the more similar the time-series features of the raw monitoring data are to the time-series features of the normal monitoring data, i.e., the smaller the raw modal warning value.

[0030] Furthermore, the cosine value between normal data features and original data features is used as the feature similarity, and the feature similarity and the original modality warning value are expressed as: ,in, Indicates feature similarity. This represents the original modal warning value.

[0031] Optionally, since feature similarity has limited accuracy in representing complex raw monitoring data—for example, a modal sensing node simultaneously collects three types of data: temperature, gas concentration, and vibration—meaning the raw monitoring data contains the specific values ​​of these three types of data over a period of time—this solution also introduces a neural network to calculate the aforementioned raw modal warning value. This process collects a large amount of normal and abnormal monitoring data, where abnormal monitoring data refers to data collected when abnormal situations such as gas leaks or fires occur at the corresponding locations. Then, normal and abnormal labels are used as labels for the normal and abnormal monitoring data, respectively. These normal and abnormal monitoring data are used to train the selected neural network. The trained neural network can output normal label probability values ​​and abnormal label probability values ​​based on the input raw monitoring data. The normal label probability value is the probability that the raw monitoring data belongs to the normal label, and the abnormal label probability value is the probability that the raw monitoring data belongs to the abnormal label. Finally, the abnormal label probability value is used as the raw modal warning value.

[0032] For example, this solution uses a one-dimensional convolutional neural network (1D-CNN) combined with fully connected layers as the neural network described above. The network structure of this neural network is as follows: The input layer receives three-channel monitoring data of length 128, where each channel of monitoring data corresponds to a type of monitoring data, such as the monitoring data corresponding to temperature. Then, it contains two convolutional layers. The first convolutional layer has a kernel size of 5 and outputs 32 feature maps. The second convolutional layer has a kernel size of 3 and outputs 64 feature maps. Each convolutional layer is followed by a ReLU activation function and a max pooling layer. After the two convolutional layers, two fully connected layers are connected. The first fully connected layer has 128 neurons, and the second fully connected layer has 2 neurons (corresponding to normal labels and abnormal labels). Finally, the probability values ​​corresponding to normal labels and abnormal labels are output through the Softmax function. The training parameters of the above neural network are set as follows: the loss function is cross-entropy loss, the optimizer is Adam, the learning rate is 0.001, the batch size is 32, the number of training rounds is 200, and the early stopping method is used for training. After training, the original monitoring data is input into the neural network, and the output value of the Softmax function of the abnormal label is taken as the original modal warning value.

[0033] S4. Send the original modal warning value and the preset node transmission parameters to the pre-built security warning terminal to obtain the node positioning command.

[0034] It is understood that the node transmission parameters refer to the transmission configuration information of the modal sensing node when uploading raw monitoring data. These parameters include the amount of raw monitoring data collected in a single instance and the frequency at which the raw monitoring data is collected. The security early warning terminal refers to the central server responsible for centrally receiving, processing, and analyzing the data uploaded by all modal sensing nodes. The node positioning command refers to the command generated by the security early warning terminal after receiving the raw modal early warning value and node transmission parameters corresponding to a certain modal sensing node. After receiving the raw modal early warning values ​​and node transmission parameters uploaded by each modal sensing node, the security early warning terminal will formulate a transmission strategy (i.e., the subsequent target forwarding matrix) for each modal sensing node's subsequent data transmission. Optionally, to ensure that the transmission strategy can be updated in a timely manner, the security early warning terminal can receive the raw modal early warning values ​​and node transmission parameters sent by each modal sensing node at regular polling intervals, thereby dynamically adjusting the data transmission path of each modal sensing node. The aforementioned polling interval can be set by relevant operators according to the accuracy of the early warning.

[0035] S5. Summarize the node positioning instructions corresponding to each modal sensing node to obtain the node positioning instruction set, and identify the set of sensing nodes to be fused based on the node positioning instruction set.

[0036] It is clear that the set of sensing nodes to be fused refers to a collection of multiple sensing nodes to be fused, wherein the sensing node to be fused refers to a modal sensing node that has received a node positioning command.

[0037] S6. Identify the relay forwarding node set in the target warning area, construct the transmission path based on the relay forwarding node set to be fused sensing node set, and obtain the current forwarding vector set.

[0038] Understandably, the relay forwarding node refers to an intermediate communication node within the target early warning area that can receive and forward data from other nodes (including relay forwarding nodes and modal sensing nodes). For example, in a chemical industrial park, a fixed wireless gateway deployed on the roof of a factory building or a temporarily deployed UAV relay station. The current forwarding vector set refers to a collection of multiple current forwarding vectors, where a current forwarding vector is a vector representing the data transmission path of a certain sensing node to be fused.

[0039] In detail, the step of constructing the transmission path based on the relay forwarding node set and the set of sensing nodes to be fused, to obtain the current forwarding vector set, includes: For each sensing node in the set of sensing nodes to be fused, perform the following operations: The current forwarding count is obtained by randomly generating a forwarding count for the nodes to be fused based on the preset maximum forwarding count; Based on the current number of forwardings, a random node is selected from the set of relay forwarding nodes to obtain multiple current forwarding nodes; The current forwarding vector is obtained by constructing binary vectors of forwarding nodes based on multiple current forwarding nodes and a preset forwarding vector template; The current forwarding vector set is obtained by summing the current forwarding vectors corresponding to each sensing node to be fused.

[0040] It should be explained that the maximum forwarding count refers to the maximum number of relay forwarding nodes that a single sensing node to be fused can use, which is manually set. This maximum forwarding count can be set according to the range of the target early warning area and the urgency of the early warning. For example, in a chemical industrial park, if the average distance between relay forwarding nodes is large, the maximum forwarding count can be set to 3 to avoid excessive latency due to too many hops. The current forwarding count refers to the forwarding count obtained after random forwarding count generation. The above-mentioned random forwarding count generation based on the preset maximum forwarding count refers to: using a uniformly distributed random forwarding count generation function, ranging from 0 to... An integer value is generated within the range of [value], which represents the current number of forwards. This indicates the maximum number of forwarding nodes. The current forwarding node refers to a specific relay forwarding node selected from the set of relay forwarding nodes. The random node selection mentioned above refers to randomly selecting a specified number (i.e., 0 to 10) from all relay forwarding nodes without replacement. The above steps of generating the random forwarding number and selecting random nodes are used to improve the diversity of multiple current forwarding nodes. Stronger diversity can prevent premature local optima when optimizing node transmission in the current forwarding matrix.

[0041] Furthermore, the forwarding vector template refers to a vector composed of a set of relay forwarding nodes, and the forwarding vector template has the following form: ,in, This represents the binary code corresponding to the forwarding vector template of a relay forwarding node with a code of 1. When the value is 1, it indicates that the relay node corresponding to this binary code is the current relay node. This represents the binary code of the relay forwarding node with code 'l' in the forwarding vector template. The specific method for constructing the binary vector of the forwarding node based on multiple current forwarding nodes and the preset forwarding vector template is as follows: the value of the vector position corresponding to the current forwarding node is recorded as the value 1. If the relay forwarding node corresponding to a certain vector position does not exist in multiple current forwarding nodes, the value at that vector position is recorded as the value 0. For example, if there are 5 relay forwarding nodes in the target warning area, numbered 1 to 5, the maximum forwarding quantity is 3, the current forwarding quantity generated by random forwarding quantity is 2, and the relay forwarding nodes numbered 2 and 4 are randomly selected as the current forwarding nodes, then the current forwarding vector is [0, 1, 0, 1, 0].

[0042] S7. Construct the current forwarding matrix based on the current forwarding vector set, optimize node transmission on the current forwarding matrix, and obtain the target forwarding matrix.

[0043] It is clear that the current forwarding matrix refers to a matrix composed of all current forwarding vectors in the current forwarding vector set. The matrix is ​​constructed by using the current forwarding vectors as rows, with each row corresponding to a current forwarding vector forming the current forwarding matrix. Since the current forwarding matrix contains the data transmission paths of each sensing node to be fused, and these data transmission paths are the signal transmission paths corresponding to the current forwarding vectors, these data transmission paths need to be optimized to minimize latency and transmission loss. Therefore, node transmission optimization of the current forwarding matrix is ​​required. The target forwarding matrix refers to the current forwarding matrix after node transmission optimization.

[0044] In detail, the step of optimizing node transmissions in the current forwarding matrix to obtain the target forwarding matrix includes: For each current forwarding vector in the current forwarding matrix, perform the following operation: A signal transmission path is generated using the current forwarding vector, the sensing node to be fused corresponding to the current forwarding vector, and the security early warning terminal. Transmission characteristics analysis is performed on the signal transmission path to obtain the effective transmission amount and total transmission delay of the path; Identify the starting sensing node in the signal transmission path, perform data priority analysis on the starting sensing node, and obtain the sensing data priority; By summing up the effective transmission volume of the path, the total transmission delay of the path, and the priority of the sensed data for each current forwarding vector, we can obtain multiple effective transmission volumes of the path, multiple total transmission delays of the path, and multiple priorities of the sensed data. The current transmission fitness value is calculated based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data. The target forwarding matrix is ​​obtained by executing a preset optimization algorithm based on the current transmission fitness value.

[0045] It should be explained that the signal transmission path refers to the transmission path of the original monitoring data from the sensing node to be fused to the security early warning terminal. The starting point and the ending point of the signal transmission path are the sensing node to be fused and the security early warning terminal, respectively, and it needs to pass through each current forwarding node corresponding to the current forwarding vector in between.

[0046] Optionally, the above signal transmission path generation method specifically includes: taking the sensing node to be fused as the starting node of the signal transmission path, then identifying the current forwarding node closest to the starting node in the current forwarding vector, and recording the current forwarding node as the second node. Further, identifying the current forwarding node closest to the second node in the current forwarding vector, and recording the current forwarding node as the third node. Repeating the above operation until all current forwarding nodes in the current forwarding vector have been identified, at which point the starting node, the second node, the third node, ... security early warning terminal constitute a complete signal transmission path.

[0047] Understandably, the effective transmission volume of the path refers to the amount of effective data that the signal transmission path can successfully transmit per unit time. The total transmission delay of the path refers to the total time required for the original monitoring data to travel from the starting sensing node, through all relay forwarding nodes, to the security early warning terminal. The starting sensing node refers to the starting node in the signal transmission path, i.e., the sensing node to be fused. The sensing data priority refers to a quantitative value of the importance and urgency of the original monitoring data. The higher the sensing data priority, the more important or urgent the original monitoring data that needs to be transmitted by the signal transmission path is.

[0048] It should be explained that the above optimization algorithm refers to an intelligent optimization algorithm that optimizes the current forwarding matrix based on the current transmission fitness value. The process of performing the following operations on each current forwarding vector in the current forwarding matrix to obtain the current transmission fitness value corresponding to each current forwarding matrix is ​​the initialization step of the genetic population and the calculation step of the fitness value of each genetic individual in the optimization algorithm.

[0049] Optionally, a genetic algorithm can be used as the optimization algorithm. First, the execution parameters of the genetic algorithm are set, including but not limited to: population size (e.g., 50), maximum number of iterations (e.g., 200), crossover probability (e.g., 0.8), and mutation probability (e.g., 0.1). After setting, the following operations are performed: First, the genetic population is initialized, which contains multiple genetic individuals. Each genetic individual corresponds to a different current forwarding matrix, that is, each genetic individual represents a transmission strategy. The above initialization method is the way to obtain the current forwarding matrix. Then, the fitness value corresponding to each genetic individual in the genetic population is calculated, which is the current transmission fitness value in this scheme. Next, iterative optimization is performed to obtain a new genetic population. The iterative optimization steps include selection, crossover, mutation, and update. These operations are all conventional techniques in genetic algorithms and will not be elaborated here. The above iterative optimization steps are repeated until the number of iterations reaches the maximum number of iterations. The current forwarding matrix corresponding to the genetic individual with the highest fitness at this time is taken as the target forwarding matrix.

[0050] In detail, the transmission characteristic analysis of the signal transmission path to obtain the effective transmission amount and total transmission delay of the path includes: Node path segmentation is performed based on the signal transmission path to obtain multiple segmented transmission paths, wherein the segmented transmission paths include: a first transmission node and a second transmission node; For each of the multiple split transmission paths, perform the following operation: Data transmission simulation was performed on the segmented transmission path to obtain the path transmission delay, segmented path success rate, and segmented path loss rate. By summing up the path transmission delay, segmentation path success rate and segmentation path loss rate for each segmented transmission path, we can obtain multiple path transmission delays, multiple segmentation path success rates and multiple segmentation path loss rates. The data to be transmitted is queried along the signal transmission path to obtain the original amount of data to be transmitted and the original transmission frequency. The effective transmission volume of the path is calculated based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the amount of original transmitted data, and the original transmission frequency. The total path transmission delay is obtained by calculating the total delay based on the transmission delay of multiple paths.

[0051] It should be explained that the segmented transmission path refers to a portion of the transmission path obtained after node path segmentation. The node path segmentation refers to dividing the signal transmission path into paths consisting of two transmission nodes. For example, if a signal transmission path starts from the sensing node to be fused, passes through the current forwarding nodes A2 and A3, and reaches the security early warning terminal D, then this signal transmission path can be segmented into the following three segmented transmission paths: A1 to A2, A2 to A3, and A3 to D. The first transmission node and the second transmission node refer to the starting node and the ending node of the segmented transmission path, respectively. For example, in A1 to A2, the first transmission node and the second transmission node are A1 and A2, respectively. The path transmission delay refers to the transmission delay in the segmented transmission path. The segmented path success rate refers to the probability that data is successfully transmitted from the first transmission node to the second transmission node on the segmented transmission path. The segmented path loss rate refers to the proportion of data lost on the segmented transmission path due to signal attenuation, collisions, bit errors, etc. The original transmitted data volume refers to the amount of data that needs to be transmitted along the signal transmission path, and the original transmission frequency refers to the frequency at which the sensing node to be fused collects the original monitoring data, i.e., the collection frequency in the node transmission parameters. The specific calculation method for the effective transmission volume of the above path will be given in subsequent embodiments. The above calculation of the total delay based on multiple path transmission delays means that the sum of the transmission delays of each path in the multiple path transmission delays is taken as the total path transmission delay.

[0052] In detail, the step of simulating data transmission on the segmented transmission path to obtain the path transmission delay, segmented path success rate, and segmented path loss rate includes: Obtain the segmented transmission distance between the first transmission node and the second transmission node; The transmission delay of the segmented transmission path is calculated based on the segmented transmission distance to obtain the path transmission delay. Success rate analysis was performed on the segmented transmission path to obtain the segmented path success rate. The transmission loss rate of the split path is obtained by querying the transmission loss of the first and second transmission nodes in the split transmission path.

[0053] It should be explained that the segmented transmission distance refers to the actual distance between the first transmission node and the second transmission node.

[0054] Optionally, the specific methods for analyzing the success rate of the segmented transmission path mentioned above include: statistically analyzing the proportion of successful transmissions of the segmented transmission path over a past period to the total number of transmissions from historical data. This proportion is the segmented path success rate. Historical data refers to data recorded during previous data transmissions. The transmission loss query mentioned above refers to the process of calculating the segmented path loss rate. The specific steps of the transmission loss query include: statistically analyzing the path loss rate of the segmented transmission path for each data transmission over a past period from historical data. This path loss rate has the same definition as the segmented path loss rate mentioned above. The specific statistical method for the loss rate of a single path is as follows: statistically analyzing the amount of data to be transmitted in a single data transmission, recorded as the initial data amount, and recording the amount of data when the data transmission is completed (after passing through the segmented transmission path), recorded as the arriving data amount. Calculating the absolute difference between the initial data amount and the arriving data amount, the ratio of this absolute difference to the initial data amount is the path loss rate, and taking the average of all path loss rates as the segmented path loss rate.

[0055] In detail, the calculation of transmission delay for the segmented transmission path based on the segmented transmission distance to obtain the path transmission delay includes: The node transmission delay between the first and second transmission nodes in the segmented transmission path is obtained based on the segmented transmission distance. Queries the current amount of input data in the segmented transmission path; Based on the current input data volume, perform a data processing delay query on the first transmission node in the segmented transmission path to obtain the node processing delay. Calculate the path transmission delay based on the node transmission delay and node processing delay.

[0056] It should be explained that the node transmission delay refers to the delay introduced by the spatial distance between the first and second transmission nodes. This node transmission delay is obtained by dividing the segmented transmission distance by the signal transmission speed in the medium (such as air) of the segmented transmission path; the resulting value is the node transmission delay. The current input data volume refers to the amount of data that the segmented transmission path needs to forward. The current input data volume of this segmented transmission path is the amount of original monitoring data after the previous adjacent segmented transmission path has completed its transmission. If the first transmission node of this segmented transmission path is the node to be fused (i.e., the starting sensing node), then the current input data volume is the amount of original monitoring data collected by the node to be fused. The node processing delay refers to the time required for the first transmission node to process the data corresponding to the current input data volume. For example, if the first transmission node is an ARM Cortex-M4 processor with a main frequency of 200MHz, and the current input data volume is 256 bytes, the clock cycles required for the processor to perform data encapsulation, protocol conversion, and cache write operations are approximately 5000, then the node processing delay is: Microseconds. The above path transmission delay can be expressed as the sum of the node transmission delay and the node processing delay.

[0057] Specifically, the calculation of the effective transmission volume of the path based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the original transmitted data volume, and the original transmission frequency includes: The effective transmission volume of the path is calculated using the following formula: ; in, Indicates the effective transmission volume of the path. Indicates the original transmission frequency. Indicates the original amount of data transmitted. This represents the number of segmentation path success rates out of multiple segmentation path success rates. This represents the success rate of multiple segmentation paths. Success rate of each segmentation path Represents the loss rate of multiple split paths. Individual path loss rate.

[0058] It needs to be explained that in the above formula for calculating the effective transmission volume of the path, The item represents the total amount of data that the signal transmission path theoretically needs to transmit per unit time. The term represents the product of the success rates of all segmented paths, that is, the overall probability that the original monitoring data successfully reaches the security early warning terminal from the initial sensing node through all relay forwarding nodes. The larger this term is, the higher the reliability of the entire signal transmission path. The larger. This term represents the proportion of raw monitoring data that is not lost during the entire transmission process. The larger this term is, the stronger the anti-interference capability of the entire signal transmission path and the more raw monitoring data can be transmitted. The larger.

[0059] In detail, the step of performing data priority analysis on the initial sensing node to obtain the sensing data priority includes: Obtain the node data type of the initial sensing node, and determine the current data importance based on the node data type in the pre-built type importance ranking table; Determine the initial modal warning value of the initial sensing node; The priority of the perceived data is obtained by weighting the current data importance and the initial modal warning value.

[0060] It should be explained that the node data type refers to the data type of the raw monitoring data collected by the initial sensing node. This data type refers to the physical category to which the raw monitoring data belongs, such as temperature data, gas concentration data, vibration data, image data, etc. Since different data types have different levels of importance for security early warning, this solution introduces a type importance grading table. This table is a mapping table based on the importance of different data types in security early warning. It includes multiple data types, and each data type corresponds to a data importance level. This data importance level refers to the quantitative level reflecting the importance of that data type in security early warning. Data importance includes five levels, from level 1 to level 5. The higher the level, the more important the corresponding data type. The data importance levels for different data types are set by relevant experts. For example, gas concentration data can be set to level 5, vibration data to level 3, and temperature data to level 2. The current data importance level refers to the data importance corresponding to the node data type. The initial modal warning value refers to the original modal warning value output by the data processing unit of the initial sensing node. The larger the initial modal warning value, the more urgent the original monitoring data collected by the initial sensing node is, that is, the higher the priority of the sensing data.

[0061] Optionally, the specific calculation formula for the weighted calculation using the current data importance and the initial modal warning value is as follows: ; in, The weighting coefficient represents the weight of the current data importance, as set manually. The weighting coefficient represents the manually set initial modal warning value. and This setting can be configured by the operator according to different actual situations, and the sum of the two values ​​is 1. Indicates the current importance of the data. and These represent the minimum and maximum values ​​of the current data importance for all node data types (each initial sensing node corresponds to one node data type here), used for... Normalize, Indicates the initial modal warning value. and These represent the minimum and maximum values ​​of the initial modal warning values ​​for all initial sensing nodes, respectively, used for... Normalize.

[0062] Specifically, the calculation of the current transmission fitness value based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data includes: The set of sensing nodes to be fused is denoted as the target sensing node set, and the target sensing nodes are extracted sequentially from the target sensing node set. Based on the extracted target sensing nodes, the effective transmission volume of the target, the total transmission delay of the target, and the priority of the target data are identified in multiple paths, multiple paths, and multiple sensing data priorities, respectively. Obtain the range of iterative transmission volume, the range of total iterative delay, and the range of iterative priority; The target effective transmission volume, target total transmission delay, and target data priority are normalized using the iterative transmission volume range, iterative total delay range, and iterative priority range, respectively, to obtain the normalized effective transmission volume, normalized total transmission delay, and normalized data priority. The unit fitness value is calculated based on the normalized effective transmission volume, the normalized total transmission delay, and the normalized data priority. The unit fitness values ​​corresponding to each target perception node are summarized to obtain the unit fitness value set; The current transmission fitness value is obtained by summing the set of unit fitness values.

[0063] It should be explained that the target effective transmission volume, target total transmission delay, and target data priority refer to the path effective transmission volume, path total transmission delay, and sensing data priority corresponding to the target sensing node, respectively. The iterative transmission volume range, iterative total delay range, and iterative priority range refer to the numerical ranges of the target effective transmission volume, target total transmission delay, and target data priority corresponding to all genetic individuals during the same iterative optimization process, respectively. The normalization mentioned above refers to minimum-maximum value normalization. The normalized effective transmission volume, normalized total transmission delay, and normalized data priority refer to the target effective transmission volume, target total transmission delay, and target data priority after normalization, respectively.

[0064] Optionally, the formula for calculating the above unit fitness value is as follows: ; in, Represents the unit fitness value. This indicates the normalized data priority, which is used to classify data. The higher the priority of the normalized data, the more important the raw monitoring data transmitted by this signal transmission path, and the more important the signal transmission path corresponding to this unit fitness value. and These represent the weighting coefficients for the normalized effective transmission volume and the normalized total transmission delay, respectively. These are manually set based on different emphases, and their sum is 1. Indicates normalized effective transmission volume. This represents the normalized total transmission delay. The larger the fitness of the above units, the more important the corresponding signal transmission path is.

[0065] S8. Generate target transmission strategy based on target forwarding matrix to complete intelligent security early warning.

[0066] Understandably, the target transmission strategy refers to the strategy for each sensing node to be fused to upload its own original monitoring data. The target transmission strategy includes the transmission path when each sensing node to be fused uploads data. This transmission path can be extracted from the target forwarding matrix, and the extraction method is the same as the extraction method of the signal transmission path mentioned above. Each sensing node to be fused will upload the collected original monitoring data to the security early warning center according to the received transmission path.

[0067] To address the problems described in the background section, this invention first analyzes and calculates the original modal warning values ​​locally at the modal sensing nodes. This step enables preliminary quantitative judgment of abnormal situations, reducing the transmission pressure and processing latency caused by uploading all original monitoring data to the central server. Furthermore, this solution sends the original modal warning values ​​and node transmission parameters to the security warning terminal and obtains node positioning instructions, enabling the central server to promptly initiate targeted data transmission path adjustments based on the warning status and transmission characteristics of each node. Then, a set of relay forwarding nodes randomly constructs the current forwarding vector for the sensing nodes to be fused, generating diverse candidate data transmission paths. This avoids premature convergence to locally poor path schemes in subsequent optimization processes. In addition, this solution optimizes node transmission in the current forwarding matrix. This step comprehensively considers the effective transmission volume, total transmission delay, and data priority of each signal transmission path, resulting in a target forwarding matrix that balances transmission efficiency and data importance globally. Therefore, this invention improves the adaptive optimization capability of data transmission paths within the warning area and reduces transmission latency and data loss of warning data.

[0068] like Figure 2 The diagram shown is a functional block diagram of an intelligent security early warning system based on multimodal perception fusion provided in an embodiment of the present invention.

[0069] The intelligent security early warning system 100 based on multimodal perception fusion described in this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent security early warning system 100 based on multimodal perception fusion may include a monitoring data acquisition module 101, a local data early warning module 102, a transmission path optimization module 103, and a transmission strategy construction module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device. The monitoring data acquisition module 101 is used to identify the target warning area and the modal sensing node set in the target warning area. The modal sensing node set includes multiple modal sensing nodes. Modal sensing nodes are extracted sequentially from the modal sensing node set, and the extracted modal sensing nodes are used for data monitoring to obtain the original monitoring data. The local data early warning module 102 is used to perform local early warning on the original monitoring data, obtain the original modal early warning value, and send the original modal early warning value and preset node transmission parameters to the pre-built security early warning terminal to obtain the node positioning command; The transmission path optimization module 103 is used to summarize the node positioning instructions corresponding to each modal sensing node to obtain a node positioning instruction set, identify the set of sensing nodes to be fused based on the node positioning instruction set, identify the relay forwarding node set in the target early warning area, and construct a transmission path for the set of sensing nodes to be fused based on the relay forwarding node set to obtain the current forwarding vector set. The transmission strategy construction module 104 is used to construct a current forwarding matrix based on the current forwarding vector set, optimize node transmission on the current forwarding matrix to obtain a target forwarding matrix, and generate a target transmission strategy based on the target forwarding matrix.

[0070] In detail, the modules in the intelligent security early warning system 100 based on multimodal perception fusion described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method uses the same technical means as the intelligent security early warning method based on multimodal perception fusion described in the article, and can produce the same technical effect, so it will not be repeated here.

[0071] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements an intelligent security early warning method based on multimodal perception fusion, according to an embodiment of the present invention.

[0072] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a smart security early warning method program based on multimodal perception fusion.

[0073] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an intelligent security early warning method program based on multimodal perception fusion, but also to temporarily store data that has been output or will be output.

[0074] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., intelligent security early warning method programs based on multimodal perception fusion) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0075] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0076] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0077] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0078] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0079] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0080] The intelligent security early warning method program based on multimodal perception fusion stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0081] The target warning area is identified, and the set of modal sensing nodes in the target warning area is identified, wherein the set of modal sensing nodes includes multiple modal sensing nodes;

[0082] Modal sensing nodes are extracted sequentially from the modal sensing node set, and the extracted modal sensing nodes are used for data monitoring to obtain raw monitoring data.

[0083] Local early warning is generated from the raw monitoring data to obtain the raw modal early warning value; The original modal warning value and the preset node transmission parameters are sent to the pre-built security warning terminal to obtain the node positioning command; The node localization instructions corresponding to each modal sensing node are summarized to obtain the node localization instruction set, and the set of sensing nodes to be fused is determined based on the node localization instruction set; The relay forwarding node set in the target early warning area is identified, and the transmission path is constructed based on the relay forwarding node set to be fused sensing node set to obtain the current forwarding vector set; The current forwarding matrix is ​​constructed based on the current forwarding vector set, and node transmission optimization is performed on the current forwarding matrix to obtain the target forwarding matrix; Based on the target forwarding matrix, a target transmission strategy is generated to complete intelligent security early warning.

[0084] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0085] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0086] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following: The target warning area is identified, and the set of modal sensing nodes in the target warning area is identified, wherein the set of modal sensing nodes includes multiple modal sensing nodes; Modal sensing nodes are extracted sequentially from the modal sensing node set, and the extracted modal sensing nodes are used for data monitoring to obtain raw monitoring data. Local early warning is generated from the raw monitoring data to obtain the raw modal early warning value; The original modal warning value and the preset node transmission parameters are sent to the pre-built security warning terminal to obtain the node positioning command; The node localization instructions corresponding to each modal sensing node are summarized to obtain the node localization instruction set, and the set of sensing nodes to be fused is determined based on the node localization instruction set; The relay forwarding node set in the target early warning area is identified, and the transmission path is constructed based on the relay forwarding node set to be fused sensing node set to obtain the current forwarding vector set; The current forwarding matrix is ​​constructed based on the current forwarding vector set, and node transmission optimization is performed on the current forwarding matrix to obtain the target forwarding matrix; Based on the target forwarding matrix, a target transmission strategy is generated to complete intelligent security early warning.

[0087] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0088] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0090] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A smart security early warning method based on multimodal perception fusion, characterized in that, The method includes: The target warning area is identified, and the set of modal sensing nodes in the target warning area is identified, wherein the set of modal sensing nodes includes multiple modal sensing nodes; Modal sensing nodes are extracted sequentially from the modal sensing node set, and the extracted modal sensing nodes are used for data monitoring to obtain raw monitoring data. Local early warning is generated from the raw monitoring data to obtain the raw modal early warning value; The original modal warning value and the preset node transmission parameters are sent to the pre-built security warning terminal to obtain the node positioning command; The node localization instructions corresponding to each modal sensing node are summarized to obtain the node localization instruction set, and the set of sensing nodes to be fused is determined based on the node localization instruction set; The relay forwarding node set in the target early warning area is identified, and the transmission path is constructed based on the relay forwarding node set to be fused sensing node set to obtain the current forwarding vector set; The current forwarding matrix is ​​constructed based on the current forwarding vector set, and node transmission optimization is performed on the current forwarding matrix to obtain the target forwarding matrix; Based on the target forwarding matrix, a target transmission strategy is generated to complete intelligent security early warning.

2. The intelligent security early warning method based on multimodal perception fusion as described in claim 1, characterized in that, The step of constructing transmission paths based on the relay forwarding node set and the set of sensing nodes to be fused, to obtain the current forwarding vector set, includes: For each sensing node in the set of sensing nodes to be fused, perform the following operations: The current forwarding count is obtained by randomly generating a forwarding count for the nodes to be fused based on the preset maximum forwarding count; Based on the current number of forwardings, a random node is selected from the set of relay forwarding nodes to obtain multiple current forwarding nodes; The current forwarding vector is obtained by constructing binary vectors of forwarding nodes based on multiple current forwarding nodes and a preset forwarding vector template; The current forwarding vector set is obtained by summing the current forwarding vectors corresponding to each sensing node to be fused.

3. The intelligent security early warning method based on multimodal perception fusion as described in claim 2, characterized in that, The step of optimizing node transmission in the current forwarding matrix to obtain the target forwarding matrix includes: For each current forwarding vector in the current forwarding matrix, perform the following operation: A signal transmission path is generated using the current forwarding vector, the sensing node to be fused corresponding to the current forwarding vector, and the security early warning terminal. Transmission characteristics analysis is performed on the signal transmission path to obtain the effective transmission amount and total transmission delay of the path; Identify the starting sensing node in the signal transmission path, perform data priority analysis on the starting sensing node, and obtain the sensing data priority; By summing up the effective transmission volume of the path, the total transmission delay of the path, and the priority of the sensed data for each current forwarding vector, we can obtain multiple effective transmission volumes of the path, multiple total transmission delays of the path, and multiple priorities of the sensed data. The current transmission fitness value is calculated based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data. The target forwarding matrix is ​​obtained by executing a preset optimization algorithm based on the current transmission fitness value.

4. The intelligent security early warning method based on multimodal perception fusion as described in claim 3, characterized in that, The transmission characteristic analysis of the signal transmission path to obtain the effective transmission amount and total transmission delay of the path includes: Node path segmentation is performed based on the signal transmission path to obtain multiple segmented transmission paths, wherein the segmented transmission paths include: a first transmission node and a second transmission node; For each of the multiple split transmission paths, perform the following operation: Data transmission simulation was performed on the segmented transmission path to obtain the path transmission delay, segmented path success rate, and segmented path loss rate. By summing up the path transmission delay, segmentation path success rate and segmentation path loss rate for each segmented transmission path, we can obtain multiple path transmission delays, multiple segmentation path success rates and multiple segmentation path loss rates. The data to be transmitted is queried along the signal transmission path to obtain the original amount of data to be transmitted and the original transmission frequency. The effective transmission volume of the path is calculated based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the amount of original transmitted data, and the original transmission frequency. The total path transmission delay is obtained by calculating the total delay based on the transmission delay of multiple paths.

5. The intelligent security early warning method based on multimodal perception fusion as described in claim 4, characterized in that, The data transmission simulation of the segmented transmission path, to obtain the path transmission delay, segmented path success rate, and segmented path loss rate, includes: Obtain the segmented transmission distance between the first transmission node and the second transmission node; The transmission delay of the segmented transmission path is calculated based on the segmented transmission distance to obtain the path transmission delay. Success rate analysis was performed on the segmented transmission path to obtain the segmented path success rate. The transmission loss rate of the split path is obtained by querying the transmission loss of the first and second transmission nodes in the split transmission path.

6. The intelligent security early warning method based on multimodal perception fusion as described in claim 5, characterized in that, The calculation of transmission delay for the segmented transmission path based on the segmented transmission distance to obtain the path transmission delay includes: The node transmission delay between the first and second transmission nodes in the segmented transmission path is obtained based on the segmented transmission distance. Queries the current amount of input data in the segmented transmission path; Based on the current input data volume, perform a data processing delay query on the first transmission node in the segmented transmission path to obtain the node processing delay. Calculate the path transmission delay based on the node transmission delay and node processing delay.

7. The intelligent security early warning method based on multimodal perception fusion as described in claim 6, characterized in that, The calculation of the effective transmission volume of the path based on the success rate of multiple segmented paths, the loss rate of multiple segmented paths, the original transmitted data volume, and the original transmission frequency includes: The effective transmission volume of the path is calculated using the following formula: ； in, Indicates the effective transmission volume of the path. Indicates the original transmission frequency. Indicates the original amount of data transmitted. This represents the number of segmentation path success rates out of multiple segmentation path success rates. This represents the success rate of multiple segmentation paths. Success rate of each segmentation path Represents the loss rate of multiple split paths. Individual path loss rate.

8. The intelligent security early warning method based on multimodal perception fusion as described in claim 7, characterized in that, The step of performing data priority analysis on the initial sensing node to obtain the sensing data priority includes: Obtain the node data type of the initial sensing node, and determine the current data importance based on the node data type in the pre-built type importance ranking table; Determine the initial modal warning value of the initial sensing node; The priority of the perceived data is obtained by weighting the current data importance and the initial modal warning value.

9. The intelligent security early warning method based on multimodal perception fusion as described in claim 8, characterized in that, The calculation of the current transmission fitness value based on the effective transmission volume of multiple paths, the total transmission delay of multiple paths, and the priority of multiple sensing data includes: The set of sensing nodes to be fused is denoted as the target sensing node set, and the target sensing nodes are extracted sequentially from the target sensing node set. Based on the extracted target sensing nodes, the effective transmission volume of the target, the total transmission delay of the target, and the priority of the target data are identified in multiple paths, multiple paths, and multiple sensing data priorities, respectively. Obtain the range of iterative transmission volume, the range of total iterative delay, and the range of iterative priority; The target effective transmission volume, target total transmission delay, and target data priority are normalized using the iterative transmission volume range, iterative total delay range, and iterative priority range, respectively, to obtain the normalized effective transmission volume, normalized total transmission delay, and normalized data priority. The unit fitness value is calculated based on the normalized effective transmission volume, the normalized total transmission delay, and the normalized data priority. The unit fitness values ​​corresponding to each target perception node are summarized to obtain the unit fitness value set; The current transmission fitness value is obtained by summing the set of unit fitness values.

10. An intelligent security early warning system based on multimodal perception fusion, characterized in that, The system includes: The monitoring data acquisition module is used to identify the target warning area and the modal sensing node set in the target warning area. The modal sensing node set includes multiple modal sensing nodes. Modal sensing nodes are extracted sequentially from the modal sensing node set, and data monitoring is performed using the extracted modal sensing nodes to obtain the raw monitoring data. The local data early warning module is used to perform local early warning on the raw monitoring data, obtain the raw modal early warning value, and send the raw modal early warning value and preset node transmission parameters to the pre-built security early warning terminal to obtain the node positioning command; The transmission path optimization module is used to summarize the node positioning instructions corresponding to each modal sensing node to obtain the node positioning instruction set, identify the set of sensing nodes to be fused based on the node positioning instruction set, identify the relay forwarding node set in the target warning area, construct the transmission path for the set of sensing nodes to be fused based on the relay forwarding node set, and obtain the current forwarding vector set. The transmission strategy construction module is used to construct the current forwarding matrix based on the current forwarding vector set, optimize node transmission on the current forwarding matrix to obtain the target forwarding matrix, and generate the target transmission strategy based on the target forwarding matrix.

Images