Monitoring video distributed storage platform based on encryption level

Abstract

The invention relates to the technical field of distributed storage platforms, in particular to a monitoring video distributed storage platform based on encryption level, which comprises a transmission unit, a management unit, a resource pool unit and a security unit, and is characterized in that the output end of the transmission unit is in communication connection with the input end of the resource pool unit; the output end of the management unit is in communication connection with the input end of the resource pool unit, the output end of the resource pool unit is in communication connection with the input end of the transmission unit, and the output end of the security unit is in communication connection with the input end of the resource pool unit; the video file can be encrypted in a hierarchical encryption mode, the outflow direction of the monitoring video file in the resource pool unit can be tracked through the full-link tracking technology, whether the monitoring video file accidentally flows out or not is judged, protection on the video file is enhanced, and the safety of the monitoring video file is improved. And the problem of loss caused by video file outflow is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed storage platforms, and in particular to a surveillance video distributed storage platform based on encryption levels. Background Art

[0002] A distributed storage platform is a data storage technology that stores data on multiple computers or servers. By connecting these devices to form a virtual storage device, the reliability, scalability and performance of the data can be improved. Each computer or server can be regarded as a storage node. They communicate and collaborate with each other through network connections to achieve distributed storage and management of data.

[0003] For example, a video large-capacity storage platform and method with authorization announcement number CN202211645749.2 and publication date 20230523 belongs to the field of data storage technology and includes the following steps: Step S1: The vehicle-mounted front camera and rear camera respectively collect the target video in front of and behind the vehicle in real time, and transmit the collected target video to the driving recorder. In the present invention, the deep convolutional neural network technology is used to identify the objects appearing in the target video and whether their behavior is abnormal. If it is determined to be abnormal, on the one hand, the abnormal target video is stored in an external storage card and marked for subsequent verification. On the other hand, the abnormal target video is sent to the owner, so that the owner far away in other cities can also understand the vehicle condition in time, and forward the vehicle condition content to the traffic management department in time, so that the owner far away in other cities can also quickly solve the vehicle condition problem, because the on-site content can be sent to the traffic management department in time, thereby further improving the problem solving efficiency.

[0004] Although the video storage platforms mentioned above and in the prior art can store video files, they have poor management of video files and poor protection of videos, which can easily lead to accidental leakage of videos and their use by criminals, causing great losses to users. In view of this, it is urgent to design a distributed storage platform for surveillance videos at an encryption level to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a distributed storage platform for surveillance videos based on encryption levels to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: A distributed storage platform for surveillance videos based on encryption levels, comprising a transmission unit, a management unit, a resource pool unit and a security unit, wherein the output end of the transmission unit is communicatively connected to the input end of the resource pool unit, the output end of the management unit is communicatively connected to the input end of the resource pool unit, the output end of the resource pool unit is communicatively connected to the input end of the transmission unit, and the output end of the security unit is communicatively connected to the input end of the resource pool; The transmission unit is used to receive the surveillance video file transmitted from the outside or transmit the surveillance video file to the external device; The management unit is used to manage the surveillance video files in the resource pool unit; The resource pool unit is used to receive the monitoring video file transmitted by the transmission unit; The security unit is used to protect the surveillance video files in the resource pool unit; The security unit includes a log module, an encryption module, a tracking module, an alarm module, a firewall module, and a monitoring module. The log module is used to record the calls and storage records of the monitoring video files in the resource pool unit. The encryption module encrypts the monitoring video files in the resource pool unit by hierarchical encryption. The tracking module is used to track the outflow direction of the monitoring video files in the resource pool unit. The monitoring module is used to monitor the stored monitoring video files in the resource pool unit. The alarm module sends an alarm command to an external computer according to the monitoring results of the monitoring module. The firewall module is used to establish a network virus protection barrier on the storage platform. The log module uses Python log recording, the encryption module uses EPON hierarchical encryption technology, the firewall module uses one of 360 antivirus software and Huorong antivirus software, the tracking module is established based on the full-link tracking technology, the alarm module sends an alarm command to the external computer by sending SMS, voice, and pop-up windows, and the monitoring module monitors the data in the resource pool unit through the DAM tool.

[0007] It should be noted that full-link tracing is a monitoring and analysis technology that tracks the entire process of a request from the client to the server, and then to the response. By recording relevant information of the request in each component, such as time consumption, status, error information, etc., and associating this information with the tracing ID, a complete request link diagram is constructed, breaking data silos, achieving end-to-end monitoring, and quickly locating and solving problems.

[0008] DAM tools monitor, capture, and log database events in near real time and issue alerts when policies are violated, helping administrators detect attacks and provide evidence of data breaches‌.

[0009] ‌EPON hierarchical encryption technology‌ is a technology that implements data encryption in Ethernet Passive Optical Network (EPON) systems to ensure the security of data transmission. EPON technology combines the advantages of Ethernet and PON technologies, including low cost, high bandwidth, strong scalability, and compatibility with existing Ethernet‌1.

[0010] Implementation of EPON Hierarchical Encryption Technology ‌Scrambling-based encryption method‌: Provides security policies through key updates, but has lower security.

[0011] ‌Advanced Encryption Standard (AES): Used for encryption of downstream data streams to ensure that the possibility of information leakage is reduced. AES is a symmetric key method that uses the same key for encryption and decryption and requires a secure channel to transmit the key‌2.

[0012] ‌Public key encryption algorithm‌: uses public key encryption and private key decryption, and does not require the key to be transmitted over the network, thus improving security‌2.

[0013] ‌‌Python Logging‌: Configure loggers, handlers, formatters, and filters, customize log formats and filters, and improve the readability and usefulness of logs.

[0014] In another embodiment provided by the present invention, the transmission unit includes a receiving module, an output module, an identification module and a processing module. The receiving module and the output module are both composed of a plurality of network data interfaces. The data receiving end is connected to an external monitoring device through an optical fiber and a wireless network. The identification module has a video integrity judgment function and a video size judgment function. The video integrity judgment function judges the integrity of the video through a video judgment tool. The video size judgment function judges the video file according to a comparison judgment formula, and the comparison judgment formula is as follows:

[0015] , where if S>1, the video file received by the receiving unit needs to be processed; if S≤1, the video file received by the receiving unit does not need to be processed; M is the memory size of the video file received by the receiving unit; N is the set memory size value; N=500M; the processing module uses a video repair tool to repair the defective video identified by the recognition module; the compression module is used to compress the video file that the recognition module determines to be larger.

[0016] It should be noted that the video integrity judgment function uses ffmpeg, which is a powerful multimedia framework that can be used to check the integrity of video files. The command line tool of ffmpeg can be used to detect whether there are errors in video files. The specific commands are as follows: bash CopyCode ffmpeg-iyour_file.mp4-verror-map0:1-fnull If the command produces output, the file has errors, otherwise the file is complete; The compression module uses WinRAR compression software.

[0017] In another embodiment provided by the present invention, the resource pool unit includes a classification module and a resource pool module. The classification module performs classification processing according to the monitoring device model corresponding to the monitoring video file. The resource pool module has an automatic partitioning function, and the resource pool module performs initial partitioning according to the classification results of the classification module, and then establishes sub-partitions in the corresponding partitions according to the time when the monitoring video file is transmitted to the platform.

[0018] It should be noted that the classification module uses MySQL Workbench, which is a powerful database design and management tool that supports a variety of database operations, including splitting databases. It provides an intuitive user interface that makes database splitting simple and efficient. Users can easily copy data from one database to another through the data import and export functions. In addition, MySQL Workbench also allows users to partition and split data tables through the table structure modification function, thereby achieving more fine-grained data management.

[0019] In another embodiment provided by the present invention, the management unit includes a search module, a retrieval module and a control module. The search module is established based on performance bottleneck analysis technology, and is used to search and judge the video files stored in the resource pool unit. The retrieval module deletes the video files in the resource pool unit according to the result of the search module. The control module is established based on deep learning technology, and the control module is used to adjust and summarize the video files in the resource pool unit.

[0020] It should be noted that the performance bottleneck analysis technology can identify the performance bottleneck of the resource pool unit. Through performance bottleneck analysis, key information such as the hot table, hot query, and hot index of the resource pool unit can be understood. If the read volume of a table in the resource pool unit is very large, it may mean that the table has data redundancy or insufficient index problems.

[0021] In the above technical solution, the present invention provides a distributed storage platform for surveillance videos based on encryption levels, which has the following beneficial effects: (1) The distributed storage platform designed by the present invention can encrypt video files by means of hierarchical encryption, and can also track the direction of outflow of monitoring video files in the resource pool unit by means of full-link tracking technology, and determine whether the monitoring video files have been accidentally outflowed, thereby strengthening the protection of video files and avoiding the occurrence of losses caused by the outflow of video files.

[0022] (2) The distributed storage platform designed by the present invention has an identification module with video integrity judgment functions and video size judgment functions. It can judge the video files transmitted by the transmission unit and perform repair processing to prevent video files with defects from being stored in the storage platform. At the same time, it can also compress large memory video files to prevent video files from having a large impact on the storage of the storage platform.

[0023] (3) In the distributed storage platform designed by the present invention, the management unit can search for redundant data in the resource pool unit through performance bottleneck analysis technology and process the redundant data. At the same time, it can also summarize and adjust the video files in the resource pool unit to ensure the orderliness of the video files in the resource pool unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0025] Figure 1 A storage platform block diagram is provided for an embodiment of a distributed storage platform for surveillance videos based on encryption levels of the present invention.

[0026] Figure 2 A schematic diagram of a resource pool unit provided for an embodiment of a distributed storage platform for surveillance videos based on encryption levels of the present invention.

[0027] Figure 3 A schematic diagram of a security unit provided for an embodiment of a distributed storage platform for surveillance videos based on encryption levels of the present invention.

[0028] Figure 4 A schematic diagram of a management unit provided for an embodiment of a distributed storage platform for surveillance videos based on encryption levels of the present invention.

[0029] Figure 5 A schematic diagram of a transmission unit provided for an embodiment of a distributed storage platform for surveillance videos based on encryption levels of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0031] like Figure 1-5 As shown, an embodiment of the present invention provides a distributed storage platform for surveillance videos based on encryption levels, including a transmission unit, a management unit, a resource pool unit and a security unit, wherein the output end of the transmission unit is communicatively connected to the input end of the resource pool unit, the output end of the management unit is communicatively connected to the input end of the resource pool unit, the output end of the resource pool unit is communicatively connected to the input end of the transmission unit, and the output end of the security unit is communicatively connected to the input end of the resource pool; The transmission unit is used to receive surveillance video files transmitted from the outside or transmit surveillance video files to external devices; The management unit is used to manage the surveillance video files in the resource pool unit; The resource pool unit is used to receive the surveillance video file transmitted by the transmission unit; The security unit is used to protect the surveillance video files in the resource pool unit; The security unit includes a log module, an encryption module, a tracking module, an alarm module, a firewall module, and a monitoring module. The log module is used to record the calls and storage records of the monitoring video files in the resource pool unit. The encryption module encrypts the monitoring video files in the resource pool unit by means of hierarchical encryption. The tracking module is used to track the outflow direction of the monitoring video files in the resource pool unit. The monitoring module is used to monitor the stored monitoring video files in the resource pool unit. The alarm module sends an alarm command to an external computer according to the monitoring results of the monitoring module. The firewall module is used to set up a network virus protection barrier on the storage platform.

[0032] Specifically, in this embodiment, a transmission unit, a management unit, a resource pool unit and a security unit are included, the output end of the transmission unit is communicatively connected to the input end of the resource pool unit, the output end of the management unit is communicatively connected to the input end of the resource pool unit, the output end of the resource pool unit is communicatively connected to the input end of the transmission unit, and the output end of the security unit is communicatively connected to the input end of the resource pool; The transmission unit is used to receive surveillance video files transmitted from the outside or transmit surveillance video files to external devices; The management unit is used to manage the surveillance video files in the resource pool unit; The resource pool unit is used to receive the surveillance video file transmitted by the transmission unit; The security unit is used to protect the surveillance video files in the resource pool unit; The security unit includes a log module, an encryption module, a tracking module, an alarm module, a firewall module, and a monitoring module. The log module is used to record the calls and storage records of the surveillance video files in the resource pool unit. The encryption module encrypts the surveillance video files in the resource pool unit by means of hierarchical encryption. The tracking module is used to track the outflow direction of the surveillance video files in the resource pool unit. The monitoring module is used to monitor the surveillance video files stored in the resource pool unit. The alarm module sends an alarm command to the external computer according to the monitoring results of the monitoring module. The firewall module is used to set up a network virus protection barrier on the storage platform. The log module uses Python log recording, the encryption module uses EPON hierarchical encryption technology, the firewall module uses one of 360 antivirus software and Huorong antivirus software, the tracking module is established based on the full-link tracking technology, the alarm module sends an alarm command to the external computer by means of SMS, voice, and pop-up windows, and the monitoring module monitors the data in the resource pool unit through the DAM tool.

[0033] It should be noted that full-link tracing is a monitoring and analysis technology that tracks the entire process of a request from the client to the server, and then to the response. By recording relevant information of the request in each component, such as time consumption, status, error information, etc., and associating this information with the tracing ID, a complete request link diagram is constructed, breaking data silos, achieving end-to-end monitoring, and quickly locating and solving problems.

[0034] DAM tools monitor, capture, and log database events in near real time and issue alerts when policies are violated, helping administrators detect attacks and provide evidence of data breaches‌.

[0035] ‌EPON hierarchical encryption technology‌ is a technology that implements data encryption in Ethernet Passive Optical Network (EPON) systems to ensure the security of data transmission. EPON technology combines the advantages of Ethernet and PON technologies, including low cost, high bandwidth, strong scalability, and compatibility with existing Ethernet‌1.

[0036] Implementation of EPON Hierarchical Encryption Technology ‌Scrambling-based encryption method‌: Provides security policies through key updates, but has lower security.

[0037] ‌Advanced Encryption Standard (AES): Used for encryption of downstream data streams to ensure that the possibility of information leakage is reduced. AES is a symmetric key method, which uses the same key for encryption and decryption and requires a secure channel to transmit the key‌2.

[0038] ‌Public key encryption algorithm‌: uses public key encryption and private key decryption, and does not require the key to be transmitted over the network, thus improving security‌2.

[0039] ‌‌Python Logging‌: Configure loggers, handlers, formatters, and filters, customize log formats and filters, and improve the readability and usefulness of logs.

[0040] In another embodiment provided by the present invention, the transmission unit includes a receiving module, an output module, an identification module and a processing module. The receiving module and the output module are both composed of multiple network data interfaces. The data receiving end is connected to the external monitoring device through an optical fiber and a wireless network. The identification module has a video integrity judgment function and a video size judgment function. The video integrity judgment function judges the integrity of the video through a video judgment tool. The video size judgment function judges the video file according to a comparison judgment formula, and the comparison judgment formula is as follows:

[0041] , where if S>1, the video file received by the receiving unit needs to be processed; if S≤1, the video file received by the receiving unit does not need to be processed; M is the memory size of the video file received by the receiving unit; N is the set memory size value, N=500M; the processing module uses a video repair tool to repair the defective video identified by the recognition module; the compression module is used to compress the video file that the recognition module determines to be larger.

[0042] It should be noted that the video integrity judgment function uses ffmpeg, which is a powerful multimedia framework that can be used to check the integrity of video files. The command line tool of ffmpeg can be used to detect whether there are errors in video files. The specific commands are as follows: bash CopyCode ffmpeg-iyour_file.mp4-verror-map0:1-fnull If the command produces output, the file has errors, otherwise the file is complete; The compression module uses WinRAR compression software.

[0043] In another embodiment provided by the present invention, the resource pool unit includes a classification module and a resource pool module. The classification module performs classification processing according to the monitoring device model corresponding to the monitoring video file. The resource pool module has an automatic partitioning function, and the resource pool module performs initial partitioning according to the classification results of the classification module, and then establishes sub-partitions in the corresponding partitions according to the time when the monitoring video file is transmitted to the platform.

[0044] It should be noted that the classification module uses MySQL Workbench, which is a powerful database design and management tool that supports a variety of database operations, including splitting databases. It provides an intuitive user interface that makes database splitting simple and efficient. Users can easily copy data from one database to another through the data import and export functions. In addition, MySQL Workbench also allows users to partition and split data tables through the table structure modification function, thereby achieving more fine-grained data management.

[0045] In another embodiment provided by the present invention, the management unit includes a search module, a retrieval module and a control module. The search module is established based on performance bottleneck analysis technology, and is used to search and judge the video files stored in the resource pool unit. The retrieval module deletes the video files in the resource pool unit according to the result of the search module. The control module is established based on deep learning technology, and the control module is used to adjust and summarize the video files in the resource pool unit.

[0046] It should be noted that the performance bottleneck analysis technology can identify the performance bottleneck of the resource pool unit. Through performance bottleneck analysis, key information such as the hot table, hot query, and hot index of the resource pool unit can be understood. If the read volume of a table in the resource pool unit is very large, it may mean that the table has data redundancy or insufficient index problems.

[0047] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A distributed storage platform for surveillance videos based on encryption levels, characterized in that: It includes a transmission unit, a management unit, a resource pool unit and a security unit, wherein the output end of the transmission unit is communicatively connected to the input end of the resource pool unit, the output end of the management unit is communicatively connected to the input end of the resource pool unit, the output end of the resource pool unit is communicatively connected to the input end of the transmission unit, and the output end of the security unit is communicatively connected to the input end of the resource pool; The transmission unit is used to receive the surveillance video file transmitted from the outside or transmit the surveillance video file to the external device; The management unit is used to manage the surveillance video files in the resource pool unit; The resource pool unit is used to receive the monitoring video file transmitted by the transmission unit; The security unit is used to protect the surveillance video files in the resource pool unit; The security unit includes a log module, an encryption module, a tracking module, an alarm module, a firewall module, and a monitoring module. The log module is used to record the calls and storage records of the monitoring video files in the resource pool unit. The encryption module encrypts the monitoring video files in the resource pool unit by hierarchical encryption. The tracking module is used to track the outflow direction of the monitoring video files in the resource pool unit. The monitoring module is used to monitor the monitoring video files stored in the resource pool unit. The alarm module sends an alarm command to an external computer according to the monitoring results of the monitoring module. The firewall module is used to set up a network virus protection barrier on the storage platform.

2. The distributed storage platform for surveillance video based on encryption level according to claim 1, characterized in that: The log module uses Python log recording, the encryption module uses EPON hierarchical encryption technology, and the firewall module uses one of 360 antivirus software and Huorong antivirus software.

3. The distributed storage platform for surveillance video based on encryption level according to claim 1, characterized in that: The tracking module is established based on the full-link tracking technology. The alarm module sends an alarm command to the external computer through text messages, voice, and pop-up windows. The monitoring module monitors the data in the resource pool unit through the DAM tool.

4. The distributed storage platform for surveillance video based on encryption level according to claim 1, characterized in that: The transmission unit includes a receiving module, an output module, an identification module and a processing module. The receiving module and the output module are both composed of multiple network data interfaces. The data receiving end is connected to external monitoring equipment through optical fiber and wireless network.

5. The distributed storage platform for surveillance video based on encryption level according to claim 4 is characterized in that: The identification module has a video integrity judgment function and a video size judgment function. The video integrity judgment function judges the integrity of the video through a video judgment tool.

6. The distributed storage platform for surveillance video based on encryption level according to claim 5, characterized in that: The video size judgment function judges the video file according to the comparison judgment formula, and the comparison judgment formula is as follows: ， Among them, if S>1, the video file received by the receiving unit needs to be processed, if S≤1, the video file received by the receiving unit does not need to be processed, M is the memory size of the video file received by the receiving unit, N is the set memory size value, N=500M.

7. The distributed storage platform for surveillance video based on encryption level according to claim 6, characterized in that: The processing module uses a video repair tool to repair the defective video identified by the recognition module, and the compression module is used to compress the video file determined to be larger by the recognition module.

8. The distributed storage platform for surveillance video based on encryption level according to claim 1, characterized in that: The resource pool unit includes a classification module and a resource pool module. The classification module performs classification processing according to the monitoring device model corresponding to the monitoring video file. The resource pool module has an automatic partitioning function, and the resource pool module performs initial partitioning based on the classification results of the classification module, and then establishes sub-partitions in the corresponding partitions based on the time when the monitoring video file is transmitted to the platform.

9. The distributed storage platform for surveillance video based on encryption level according to claim 1, characterized in that: The management unit includes a search module, a retrieval module and a control module. The search module is established based on performance bottleneck analysis technology and is used to search and judge the video files stored in the resource pool unit. The retrieval module deletes the video files in the resource pool unit according to the judgment result of the search module.

10. The distributed storage platform for surveillance video based on encryption level according to claim 9, characterized in that: The control module is established based on deep learning technology, and the control module is used to adjust and summarize the video files in the resource pool unit.

Images