Multi-modal security dynamic regulation and control system and method based on alliance block chain technology

Abstract

The invention discloses a multi-modal security dynamic regulation and control system based on an alliance block chain technology, and the system comprises a multi-modal data collection layer which is used for collecting multi-modal data and carrying out the preliminary preprocessing of the multi-modal data; the data processing and analysis layer is used for receiving the data transmitted by the multi-modal data acquisition layer and carrying out feature extraction, correlation analysis and safety evaluation; the data processing and analysis layer is used for recording the security evaluation index, the early warning information and the regulation and control strategy data generated by the data processing and analysis layer to a block chain, and realizing automatic execution and dynamic adjustment of a regulation and control strategy by using an intelligent contract; and the security regulation and control execution layer is used for carrying out real-time regulation and control on each component and equipment of the modal data acquisition layer and feeding back a regulation and control result to the data processing and analysis layer, and the system realizes real-time monitoring, dynamic evaluation and accurate regulation and control on the security of the multi-modal system by utilizing the characteristics of the alliance block chain in the use process, so that the security of the multi-modal system is improved. And the safety and the reliability of the multi-mode system are improved.

Description

Technical Field

[0001] The present invention relates to the fields of information security and blockchain technology. Specifically, it relates to a multi-modal security dynamic regulation system and method based on consortium blockchain technology. Background Art

[0002] With the rapid development of information technology, multi-modal systems have been widely used in various fields, such as smart homes. The multi-modal system integrates various different types of data and information, including images, voices, sensor data, etc. By comprehensively analyzing and processing these multi-modal data, more intelligent and efficient functions are realized.

[0003] However, the existing multi-modal systems of smart homes have the following problems:

[0004] (1) The diversity and complexity of multi-modal data increase the risks of data leakage and tampering. Different types of data have different security requirements and characteristics, and traditional security protection means are difficult to comprehensively and effectively protect them.

[0005] (2) The multi-modal system is usually in a dynamically changing environment. The state of the system, the behavior of users, and external environmental factors are all constantly changing, which requires the security protection mechanism to be able to adjust and adapt in real time and dynamically.

[0006] In summary, most of the existing security regulation methods are designed based on single-modal data or static rules, and cannot fully consider the associations between multi-modal data and the dynamic change characteristics of the system, making it difficult to meet the growing needs of multi-modal systems.

[0007] Therefore, it is an urgent problem to be solved by the present invention to provide a multi-modal security dynamic regulation system and method based on consortium blockchain technology that can, during use, utilize the characteristics of consortium blockchain to achieve real-time monitoring, dynamic evaluation, and precise regulation of the security of multi-modal systems, and improve the security and reliability of multi-modal systems. Summary of the Invention

[0008] Aiming at the above technical problems, the purpose of the present invention is to overcome the problem that most of the existing security regulation methods in the prior art are designed based on single-modal data or static rules, and cannot fully consider the associations between multi-modal data and the dynamic change characteristics of the system, making it difficult to meet the growing needs of multi-modal systems. Thus, a multi-modal security dynamic regulation system and method based on consortium blockchain technology is provided that can, during use, utilize the characteristics of consortium blockchain to achieve real-time monitoring, dynamic evaluation, and precise regulation of the security of multi-modal systems, and improve the security and reliability of multi-modal systems.

[0009] In order to achieve the above objectives, the present invention provides a multi-modal secure dynamic control system based on alliance blockchain technology, comprising:

[0010] Multimodal data acquisition layer, used to collect multimodal data and perform preliminary preprocessing;

[0011] The data processing and analysis layer is used to receive data from the multimodal data collection layer and use machine learning and deep learning algorithms to perform feature extraction, correlation analysis, and security assessment;

[0012] The alliance blockchain layer is a alliance blockchain network composed of multiple participating nodes, which is used to record the security assessment indicators, early warning information and control strategy data generated by the data processing and analysis layer on the blockchain, and use smart contracts to realize the automatic execution and dynamic adjustment of control strategies;

[0013] The security control execution layer is used to perform real-time control on the various components and devices of the modal data acquisition layer according to the control strategy recorded by the alliance blockchain layer, and feed back the control results to the data processing and analysis layer.

[0014] Preferably, the multimodal data collected by the multimodal data collection layer includes image data, voice data, environmental parameter data and device status data.

[0015] Preferably, the data processing and analysis layer constructs a multimodal data association model to mine the intrinsic connections and potential risks between data of different modalities.

[0016] Preferably, the alliance blockchain layer uses a consensus algorithm to verify and confirm transactions to ensure the consistency and credibility of data.

[0017] Preferably, the alliance blockchain layer is an alliance blockchain network composed of device manufacturers, service providers and user representatives of the smart home system.

[0018] A multimodal secure dynamic control method based on alliance blockchain technology, comprising the following steps:

[0019] S1. Multimodal data acquisition and preprocessing: Utilize various sensors and devices in the multimodal data acquisition layer to collect multimodal data in real time and perform preprocessing;

[0020] S2. Multimodal data correlation analysis and security assessment: In the data processing and analysis layer, multimodal fusion algorithms are used to fuse and correlate data, build security assessment models, evaluate the security status of the system and generate early warning information;

[0021] S3. Data Upload to the Blockchain and Consensus Reaching: Package data such as security assessment metrics, warning information, and preliminary control strategies into transactions, send them to the consortium blockchain layer, and achieve data consistency through a consensus algorithm and record it on the blockchain;

[0022] S4. Smart Contract Execution and Generation of Control Strategies: Utilize smart contracts on the consortium blockchain layer to automatically generate or adjust security control strategies based on security assessment metrics and historical data;

[0023] S5. Execution and Feedback of Security Control: The security control execution layer receives control strategies and performs real-time control on the multimodal data collected by the multimodal data acquisition layer, and feeds back the control results to the data processing and analysis layer to dynamically optimize the security assessment model and control strategies.

[0024] Preferably, in step S1, using various sensors and devices of the multimodal data acquisition layer to collect multimodal data in real time and perform preprocessing includes: performing median filtering on the data collected by the multimodal data acquisition layer to remove noise, and using a normalization method to unify the data to the same numerical range for subsequent analysis.

[0025] Preferably, in step S3, the consensus algorithm includes: PBFT algorithm or PoS algorithm.

[0026] According to the above technical solution, the beneficial effects of the multimodal security dynamic control system and method provided by the present invention when in use are as follows:

[0027] (1) Enhance data security: Utilize the tamper-proof and traceable characteristics of the consortium blockchain to ensure the integrity and authenticity of multimodal data, effectively prevent data leakage and tampering, and improve the overall security of the system.

[0028] (2) Achieve dynamic control: Through real-time monitoring and analysis of multimodal data, combined with the automated execution ability of smart contracts, it is possible to adjust security control strategies in real time according to the dynamic changes of the system and adapt to complex and changeable security environments.

[0029] (3) Multimodal fusion analysis: Adopt multimodal fusion algorithms to mine the correlation relationships between different modal data, and be able to more comprehensively and accurately evaluate the security status of the system and discover potential security risks.

[0030] (4) Improve system reliability: Through a closed-loop feedback mechanism, continuously optimize the security assessment model and control strategies, improve the self-repair and self-adaptive capabilities of the system, and ensure the reliable operation of the multimodal system.

[0031] Other features and advantages of the present invention will be described in detail in the following specific implementation section; and the parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific implementation, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the drawings:

[0033] Figure 1 is a working principle block diagram of a multi-modal security dynamic regulation system based on consortium blockchain technology provided in a preferred implementation manner of the present invention;

[0034] Figure 2 is a flowchart of a multi-modal security dynamic regulation method based on consortium blockchain technology provided in a preferred implementation manner of the present invention.

[0035] DESCRIPTION OF THE REFERENCE NUMERALS

[0036] 1. Multi-modal data acquisition layer; 2. Data processing and analysis layer; 3. Consortium blockchain layer; 4. Security regulation execution layer. SPECIFIC IMPLEMENTATION

[0037] The following will describe in detail the specific implementation of the present invention with reference to the drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0038] As Figure 1-2 shown, a multi-modal security dynamic regulation system based on consortium blockchain technology provided by the present invention includes:

[0039] Multi-modal data acquisition layer 1: It includes various types of sensors and data acquisition devices, which are used to collect multi-modal data, such as cameras to collect image data, microphones to collect voice data, and various sensors to collect environmental parameters and device status data, etc. The collected raw data is subjected to preliminary format conversion and preprocessing for subsequent transmission and processing;

[0040] Data processing and analysis layer 2: It is used to receive the data transmitted from the multi-modal data acquisition layer, and uses machine learning and deep learning algorithms to extract and analyze the features of the data, construct a multi-modal data association model, mine the internal connections and potential risks between different modal data, monitor the running state and security status of the system in real time, and generate security evaluation indicators and warning information;

[0041] Consortium blockchain layer 3: A consortium blockchain network is composed of multiple participating nodes. Through a consensus mechanism, data consistency and credibility are achieved among nodes, which is used to record data such as security assessment metrics, early warning information, and control strategies generated by the data processing and analysis layer onto the blockchain, ensuring data immutability and traceability. Smart contracts are used to automate the execution and dynamic adjustment of security control strategies.

[0042] In the above solution, the multiple participating nodes refer to device manufacturers, service providers, and user representatives of the smart home system, etc.

[0043] Security control execution layer 4: According to the control strategies recorded by the consortium blockchain layer, it is used to perform real-time control on each component and device of the multi-modal data acquisition layer 1, including operations such as access control, data encryption, and device status adjustment, to ensure the secure and stable operation of the system.

[0044] In the above solution, in the smart home scenario, high-definition cameras are installed to collect indoor image data, microphones are used to collect voice commands and ambient sounds, and temperature and humidity sensors, light sensors, etc. are used to collect environmental parameters. Deep learning models such as convolutional neural networks (CNNs) are used to process image data, and recurrent neural networks (RNNs) are used to process voice data to extract features and classify the data. Association rule mining algorithms are used to mine the association relationships between different modal data, such as analyzing the relationship between the activity status of people in the image and environmental parameters. A security assessment model based on machine learning is constructed to evaluate the security level of the system according to data features and association relationships, and different security thresholds are set. When the assessment metrics exceed the thresholds, early warnings are triggered. A consortium blockchain network is formed by device manufacturers, service providers, and user representatives of the smart home system. The Practical Byzantine Fault Tolerance (PBFT) consensus algorithm is adopted to ensure data consistency among nodes and fast confirmation of transactions. Smart contracts are written to define the generation rules and execution conditions of security control strategies. For example, when an abnormal person is detected entering, the door lock is automatically triggered to lock and the alarm mechanism is activated. According to the control strategies transmitted from the consortium blockchain layer, operation instructions are sent to each smart home device through a smart gateway, such as controlling the light switch, adjusting the air conditioner temperature, and locking the doors and windows. The execution status of the devices and the feedback data of the system are monitored in real time and fed back to the data processing and analysis layer for subsequent evaluation and optimization.

[0045] This embodiment also provides a multi-modal security dynamic control based on consortium blockchain technology, including the following steps:

[0046] S1. Multimodal Data Collection and Preprocessing: Set the time interval for data collection. For example, collect environmental parameter data every 5 seconds and image and voice data every 10 seconds. Perform median filtering on the collected data to remove noise, and use the normalization method to unify the data into the same numerical range for subsequent analysis.

[0047] S2. Multimodal Data Association Analysis and Security Assessment: Use the feature fusion algorithm to splice and weight-fuse the feature vectors of different modalities to obtain a comprehensive feature representation. Train a security assessment model based on historical data, and use the random forest algorithm for security level classification. Classify the system security status into four levels: safe, low risk, medium risk, and high risk. When an abnormal data pattern or association relationship is detected, such as the appearance of an image of a stranger and abnormal environmental sounds, trigger a warning message.

[0048] S3. Data Uplink and Consensus Reaching: Package the security assessment results, warning messages, and preliminary control suggestions into blockchain transactions. The transactions contain information such as timestamps, data sources, and data summaries. The consortium blockchain nodes verify the transactions, check the integrity and legality of the data, and record the transactions on the blockchain after reaching a consensus through the PBFT algorithm.

[0049] S4. Smart Contract Execution and Control Strategy Generation: The smart contract automatically generates specific control strategies based on the security assessment results and historical data recorded on the blockchain. For example, when the system is in a high-risk state, the smart contract generates instructions to turn off all non-essential devices and strengthen access control. The smart contract can adjust the control strategy in real time according to the dynamic changes of the system, such as making personalized settings according to different time periods and user habits.

[0050] S5. Security Control Execution and Feedback: The security control execution layer 4 converts the control strategies generated by the smart contract into instructions recognizable by the devices and sends them to each smart home device for execution through the network. After the devices execute the control operations, they feedback the execution results and the current status to the data processing and analysis layer. The data processing and analysis layer updates the security assessment model and control strategies according to the feedback information to achieve dynamic optimization.

[0051] It can be seen from this that the present invention utilizes the immutable and traceable characteristics of the consortium blockchain to ensure the integrity and authenticity of multimodal data, effectively prevent data leakage and tampering, and improve the overall security of the system. By real-time monitoring and analyzing multimodal data and combining the automated execution ability of smart contracts, it is possible to adjust the security control strategy in real time according to the dynamic changes of the system and adapt to the complex and changeable security environment. Moreover, by using the multimodal fusion algorithm to mine the correlation relationship between different modal data, it is possible to more comprehensively and accurately evaluate the security state of the system and discover potential security risks. Furthermore, through the closed-loop feedback mechanism, the security assessment model and control strategy are continuously optimized to improve the self-repair and self-adaptation ability of the system and ensure the reliable operation of the multimodal system.

[0052] In summary, the multimodal security dynamic regulation system and method based on the consortium blockchain technology provided by the present invention overcome the problem that most of the existing full-regulation methods are designed based on single-modal data or static rules, and cannot fully consider the correlation between multimodal data and the dynamic change characteristics of the system, making it difficult to meet the growing problems of multimodal systems.

[0053] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solutions of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0054] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not separately describe various possible combination methods.

[0055] In addition, any combination can be made between different embodiments of the present invention as long as it does not violate the idea of the present invention, and it should also be regarded as the content disclosed by the present invention.

Claims

1. A multi-modal security dynamic regulation system based on consortium blockchain technology, characterized in that, include: Multimodal data acquisition layer (1), used to collect multimodal data and perform preliminary preprocessing; The data processing and analysis layer (2) is used to receive data from the multimodal data collection layer (1) and use machine learning and deep learning algorithms to perform feature extraction, correlation analysis, and security assessment; The alliance blockchain layer (3) is an alliance blockchain network composed of multiple participating nodes, which is used to record the security assessment indicators, early warning information and control strategy data generated by the data processing and analysis layer (2) on the blockchain, and use smart contracts to realize the automatic execution and dynamic adjustment of the control strategy; The security control execution layer (4) is used to perform real-time control on the various components and devices of the modal data acquisition layer (1) according to the control strategy recorded by the alliance blockchain layer (3), and feed back the control results to the data processing and analysis layer (2).

2. The multimodal security dynamic regulation system based on the consortium blockchain technology according to claim 1, wherein The multimodal data collected by the multimodal data collection layer (1) includes image data, voice data, environmental parameter data and device status data.

3. The multimodal security dynamic regulation system based on the consortium blockchain technology according to claim 1, wherein The data processing and analysis layer (2) constructs a multimodal data association model to mine the intrinsic connections and potential risks between data of different modalities.

4. The multimodal security dynamic regulation system based on the consortium blockchain technology according to claim 1, wherein The alliance blockchain layer uses a consensus algorithm to verify and confirm transactions to ensure the consistency and credibility of data.

5. The multimodal security dynamic regulation system based on the consortium blockchain technology according to claim 1, wherein The alliance blockchain layer (3) is an alliance blockchain network composed of equipment manufacturers, service providers and user representatives of the smart home system.

6. The multimodal security dynamic regulation method based on the federated blockchain technology according to claims 1-5, characterized in that, The following steps are involved: S1. Multimodal data collection and preprocessing: Using various sensors and devices in the multimodal data collection layer (1), multimodal data is collected in real time and preprocessed; S2. Multimodal data correlation analysis and security assessment: In the data processing and analysis layer (2), multimodal fusion algorithms are used to fuse and correlate data, build a security assessment model, evaluate the security status of the system and generate warning information; S3, data on-chain and consensus reached: Data such as security assessment indicators, early warning information, and preliminary control strategies are packaged into transactions and sent to the alliance blockchain layer (3). Data consistency is achieved through a consensus algorithm and recorded on the blockchain. S4, smart contract execution and control strategy generation: using the smart contract on the alliance blockchain layer (3), automatically generate or adjust security control strategies based on security assessment indicators and historical data; S5. Security control execution and feedback: The security control execution layer receives (4) the control strategy and performs real-time control on the multimodal data collected by the multimodal data collection layer (1), and feeds back the control results to the data processing and analysis layer (2) to dynamically optimize the security assessment model and control strategy.

7. The multimodal security dynamic regulation method based on the federated blockchain technology according to claim 6, wherein In the step S1, various sensors and devices of the multimodal data acquisition layer (1) are used to collect multimodal data in real time and perform preprocessing, including: performing median filtering on the data collected by the multimodal data acquisition layer (1) to remove noise, and using a normalization method to unify the data into the same numerical range to facilitate subsequent analysis.

8. The multimodal security dynamic regulation method based on the federated blockchain technology according to claim 6, wherein In the step S3, the common algorithm includes: PBFT algorithm or PoS algorithm.

Images