Decentralized smart home individual privacy protection method and system

Abstract

The invention discloses a decentralized smart home individual privacy protection method and system, relates to the technical field of smart home, and solves the problem that the information collection and processing level is reduced by adjusting the network access authority of smart equipment in the prior art; however, due to the fact that strict authority management affects normal use of the intelligent household equipment, visitors cannot remotely control the equipment, and convenience of the intelligent household equipment is reduced. An original video collected by a smart home camera is obtained from a local database, audio and video frames in the original video are separated through an audio and video processing technology, and the audio and video frames are respectively uploaded to an encryption network unit; setting a password of an encryption network unit and a terminal password synthesized by audio and video frames based on a password strategy of time; judging whether the visitor is allowed to view the complete video; if yes, recording access log information of the visitor; if not, sending a reminding signal; according to the invention, the security of personal privacy protection is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of smart home and relates to blockchain technology, and specifically is a decentralized smart home personal privacy protection method and system. Background Art

[0002] In traditional centralized smart home networks, data and management are highly centralized. Once the central platform is attacked by hackers or the management actively leaks information, a large amount of user data will be leaked. The decentralized system reduces the risk of single point failure by dispersing data and control rights, greatly reducing the impact of hacker attacks or data leaks. The decentralized smart home personal privacy protection system can use advanced encryption technology to protect the data transmission and storage process, ensuring that the data is not stolen or tampered with during the transmission process. At the same time, because the data is stored in different nodes, even if a node is breached, all the data cannot be obtained.

[0003] The existing decentralized privacy protection strategy for smart furniture mainly focuses on distributing and storing personal data in different nodes to increase the difficulty of illegal data acquisition. However, the limitation of this method is that it fails to fully consider that among all types of furniture data, surveillance video data often contains the richest personal privacy information. Although the current decentralized data storage scheme has tried to store surveillance videos in a decentralized manner to reduce risks, once hackers successfully invade the nodes storing surveillance videos, they can still easily obtain and abuse a large amount of personal privacy data, which seriously weakens the actual effect of decentralized privacy protection.

[0004] The present invention provides a decentralized smart home personal privacy protection method and system to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a decentralized smart home personal privacy protection method and system, which are used to solve the technical problem that the mechanism of video storage through the existing decentralized personal privacy protection method has limitations, resulting in poor centralized privacy protection effect.

[0006] To achieve the above objectives, the first aspect of the present invention provides a decentralized smart home personal privacy protection method, comprising:

[0007] Obtaining the original video captured by the smart home camera from the local database, separating the audio and video frames in the original video through audio and video processing technology, and uploading them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit;

[0008] Time-based password policy settings encrypt the passwords of network units and the terminal passwords after the synthesis of audio and video frames;

[0009] Determine whether to allow the visitor to view the full video; if yes, record the visitor's access log information; if no, send a reminder signal.

[0010] Preferably, the separating the audio and video frames in the original video by using audio and video processing technology includes:

[0011] Retrieve the original video and back it up in the local database; use third-party software to extract the audio and video frames of the backed-up original video separately;

[0012] Number the videos of adjacent frames and group the numbers; divide the total number of frames by the maximum number of frames that the video encryption node can store to obtain the number of groups; retrieve a complete group and subtract the minimum number from the maximum number to obtain the maximum frame difference; determine whether the difference between the numbers of two randomly retrieved video frames is less than the maximum frame difference, if so, store them respectively in the video encryption nodes with a set correlation of A; if not, randomly disperse them in the video encryption nodes with other correlations;

[0013] Adjust the audio encryption node where the keyword is placed according to the matching degree of the keyword;

[0014] The encrypted network unit includes a number of encrypted nodes, which are numbered; the breadth-first search technology is used to determine whether the shortest path length between two nodes is lower than a set threshold length; if yes, the correlation between the two nodes is marked as A-; if no, it is determined whether the two nodes are in similar network positions; if yes, the correlation between the two nodes is marked as A-; if no, the cosine similarity method is used to count the number of common neighbors of the two nodes; it is determined whether the number of neighbors is lower than a set neighbor number threshold; if yes, the correlation between the two nodes is marked as A; if no, the correlation between the two nodes is marked as A-; wherein: the correlation size is: A>A->A-.

[0015] It should be noted that the original video is a complete video that has not been separated by audio and video processing technology.

[0016] The present invention uses audio and video processing technology to accurately separate the sound signal from the video frame; the separated audio and video frames are uploaded to two different dedicated encryption network units respectively to ensure the security of data during transmission and storage.

[0017] Preferably, the step of performing time synchronization on the audio and video frames comprises:

[0018] Retrieving the separately extracted audio and video frames for encoding respectively, and extracting the timestamp information attached to the corresponding audio and video frames;

[0019] Determine whether the extracted audio and video timestamps are consistent; if yes, complete the audio and video time synchronization; if not, set the standard time axis and calibrate the timestamps of the audio and video frames according to the standard time axis.

[0020] It should be noted that the rules for setting the standard timeline include: using the audio timestamp as the basis, using the video frame timestamp as the basis, and using the timestamp set based on actual work experience as the basis.

[0021] The present invention combines audio and video frames in a correct order and time point and ensures synchronization between the audio and video frames, which is beneficial for rapid and accurate integration in the later stage.

[0022] Preferably, the time-based password policy setting encrypts the password of the network unit and the terminal password synthesized from the audio and video frames, including:

[0023] Set a time period, customize the time period to several time periods, determine the start time and end time of each time period; set the encrypted network unit dynamic password and the terminal dynamic password for the corresponding time period within the time period; the dynamic password includes: start time + end time + set letters + special symbols.

[0024] It should be noted that custom setting of several time periods ensures that there is no overlap between the time periods and covers the entire time period; it is stipulated that the dynamic passwords of the two encryption network units and the terminal are inconsistent.

[0025] The present invention effectively prevents malicious access by setting different passwords for visitors in different time periods, thereby further enhancing the security of personal privacy protection.

[0026] Preferably, the determining whether to allow the visitor to view the complete video includes:

[0027] Determine whether the dynamic password input by the visitor to the encryption network unit is correct; if yes, integrate the video frame and the corresponding audio one by one to form a complete video according to the calibrated timestamp information; if no, continue to determine;

[0028] Determine whether the visitor has entered the correct terminal dynamic password; if yes, the visitor is allowed to view the full video; if no, the visitor is denied to view the full video.

[0029] It should be noted that when integrating into a complete video, the frame rates of audio and video are different, which can be solved by changing the playback frame rate of audio or video or adding additional frames.

[0030] The present invention greatly improves the security of the system by respectively inputting the passwords of the corresponding sound confidentiality unit and the video frame confidentiality unit. This double verification mechanism means that even if the password of one of the units is cracked, the data of the other unit remains safe. Only after the visitor correctly inputs the passwords of the two units at the same time, the system will automatically accurately match and integrate the sound with the corresponding video frame according to the timestamp information, thereby restoring the complete video content. By requiring the visitor to enter the correct terminal password before allowing the visitor to view the complete video, the terminal password serves as the last line of defense for accessing the complete video, which is conducive to further enhancing the security protection level of the data.

[0031] Preferably, the recording of visitor access log information includes:

[0032] Obtain historical access log information from the database, wherein the historical access log information includes: the visitor's IP address, access time, and password input;

[0033] Determine whether the visitor's IP address exists in the historical access log information; if so, continue to determine and record the corresponding visitor's access log information to the database; if not, notify the user's mobile phone bound to the corresponding camera.

[0034] It should be noted that when a visitor hides his IP address, the corresponding visitor is denied access to the encrypted network unit.

[0035] The present invention provides strong support for subsequent security audit and troubleshooting by recording access time, visitor information and password input conditions.

[0036] To achieve the above-mentioned object, the second aspect of the present invention provides a decentralized smart home personal privacy protection system, including: a data acquisition module, a data encryption module, and a terminal access module;

[0037] Data acquisition module: obtains the original video collected by the smart home camera from the local database, separates the audio and video frames in the original video through audio and video processing technology, and uploads them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit;

[0038] Data encryption module: The password of the encrypted network unit and the terminal password after the synthesis of audio and video frames are set based on the time password policy;

[0039] Terminal access module: determines whether the visitor is allowed to view the complete video; if yes, records the visitor's access log information; if no, sends a reminder signal.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention requires that when a visitor wishes to view these audio and video, the visitor must correctly enter the passwords of the corresponding sound confidentiality unit and the video frame confidentiality unit respectively. This double verification mechanism greatly improves the security of personal privacy protection. Even if the password of one of the network units is cracked, the data of the other network unit remains safe. When the visitor correctly enters the passwords of the two units at the same time, the system will automatically match and integrate the sound with the corresponding video frame according to the timestamp information, thereby restoring the complete video content. During the whole process, the system also has detailed log recording and monitoring functions, which can track and record each access attempt, including access time, visitor information and password input, providing strong support for subsequent security audits and troubleshooting. After the visitor views, the visitor's access IP is recorded. If the access IP does not appear in the historical access log, a reminder signal is issued, which is conducive to the users of smart homes to promptly understand the viewing status of the camera video.

[0042] 2. The present invention ensures the security of data during transmission and storage by adopting dynamic password encryption technology; strict access control is achieved by setting a multiple password authentication method, requiring visitors to enter a terminal password before allowing them to view the complete video. The terminal password serves as the last line of defense for accessing the complete video, further enhancing the security protection level of the data; if the visitor successfully enters the correct terminal password, the viewing permission of the complete video is unlocked, and a complete audio-visual experience is obtained; on the contrary, if the visitor fails to enter the correct terminal password, his request to view the complete video will be rejected, thereby protecting the user's privacy data from being illegally obtained or abused. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of the steps of personal privacy protection of the present invention;

[0045] Figure 2 It is a schematic diagram of the process of data collection of the present invention;

[0046] Figure 3 A schematic diagram of the process of data encryption and access of the present invention;

[0047] Figure 4 This is a schematic diagram of the relationship between the modules included in the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 The first aspect of the present invention provides a decentralized smart home personal privacy protection method, including:

[0050] Obtaining the original video captured by the smart home camera from the local database, separating the audio and video frames in the original video through audio and video processing technology, and uploading them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit;

[0051] Time-based password policy settings encrypt the passwords of network units and the terminal passwords after the synthesis of audio and video frames;

[0052] Determine whether to allow the visitor to view the full video; if yes, record the visitor's access log information; if no, send a reminder signal.

[0053] See also Figure 2 , the data collection process, obtains the original video collected by the smart home camera from the local database, and backs up the original video in the local database; uses third-party software to extract the audio and video frames of the backed-up original video separately; retrieves the original video, and backs up the original video in the local database; uses third-party software to extract the audio and video frames of the backed-up original video separately;

[0054] Number the videos of adjacent frames and group the numbers; divide the total number of frames by the maximum number of frames that the video encryption node can store to obtain the number of groups; retrieve a complete group and subtract the minimum number from the maximum number to obtain the maximum frame difference; determine whether the difference between the numbers of two randomly retrieved video frames is less than the maximum frame difference, if so, store them respectively in the video encryption nodes with a set correlation of A; if not, randomly disperse them in the video encryption nodes with other correlations;

[0055] Adjust the audio encryption node where the keyword is placed according to the matching degree of the keyword;

[0056] The encrypted network unit includes a number of encrypted nodes, and the encrypted nodes are numbered; the breadth-first search technology is used to determine whether the shortest path length between two nodes is less than a set threshold length; if yes, the correlation between the two nodes is marked as A-; if no, it is determined whether the two nodes are in similar network positions; if yes, the correlation between the two nodes is marked as A-; if no, the number of common neighbors of the two nodes is counted by the cosine similarity method; it is determined whether the number of neighbors is less than a set neighbor number threshold; if yes, the correlation between the two nodes is marked as A; if no, the correlation between the two nodes is marked as A-; wherein: the correlation size is: A>A->A-;

[0057] Encode the separately extracted audio and video frames respectively, and extract the timestamp information attached to the corresponding audio and video frames;

[0058] Determine whether the extracted audio and video timestamps are consistent; if yes, complete the audio and video time synchronization; if not, set a standard timeline and calibrate the timestamps of the audio and video frames according to the standard timeline; compress the audio and video frames separately, upload the compressed audio to the audio encryption network unit, and upload the compressed video frames to the video encryption network unit.

[0059] For example, the original video captured by the smart home camera is obtained from the local database, and the original video is backed up in the local database; the audio and video frames of the backed-up original video are separated by FFmpeg software;

[0060] After the video is separated, a total of 1052 video frames are obtained. The video frames are numbered. The maximum number of frames that can be stored in a video encryption node is 125, which is obtained from the database. The total number of frames is divided by the maximum number of frames that can be stored in the video encryption node to obtain 9 groups, where the maximum frame difference is 124; the two randomly retrieved video frames are numbered 3 and 631, 631-3=628>124, and the video frames numbered 3 and 631 are randomly dispersed in the video encryption nodes with a correlation of A--;

[0061] The keywords in the audio are vectorized, and the keyword matching value is calculated by the formula cos(θ)=(A·B) / (|A|×|B|); the higher the keyword matching value, the corresponding keywords are placed in the audio encryption nodes with lower relevance; where A and B represent the vector representation of keyword 1 and the vector representation of keyword 2, respectively, · represents the dot product of the vectors, |A| and |B| represent the modulus of vectors A and B, respectively; the value range of cos(θ) is between [-1,1];

[0062] It should be noted that the closer the cos(θ) value is to 1, the higher the matching degree of the keywords corresponding to the two vectors is; the closer the cos(θ) value is to -1, the lower the matching degree of the keywords corresponding to the two vectors is; when the cos(θ) value is 0, it means that the matching degree of the keywords corresponding to the two vectors is 0;

[0063] Exemplarily, by setting a local path index, it is determined whether the number of paths of length n between two nodes exceeds the set local path index; if yes, the two nodes are in similar network positions; if no, the two nodes are in non-similar network positions;

[0064] Extract the timestamp information of audio and video; the timestamp format is: YYYY-MM-DDTHH:MM:SS; YYYY represents year, MM represents month, DD represents date, T is the separator of date and time, HH represents hour, MM represents minute, SS represents second; there is a two-minute video, the extracted audio timestamp information is 20241119T14:18:01-20241119T14:19:60; the extracted video frame timestamp information is 20241119T14:18:01-20241119T14:19:60;

[0065] If the extracted audio and video timestamps are consistent, the audio and video time synchronization is completed; the audio and video frames are compressed separately; the compressed audio is uploaded to the audio encryption network unit, and the compressed video frames are uploaded to the video encryption network unit.

[0066] See also Figure 3 , the process of data encryption and access, set the time period, customize the time period to several time periods, determine the start time and end time of each time period; set the encryption network unit dynamic password and the terminal dynamic password for the corresponding time period in the time period; where the dynamic password includes: start time + end time + set letters + special symbols;

[0067] Determine whether the dynamic password input by the visitor to the encryption network unit is correct; if yes, integrate the video frame and the corresponding audio one by one to form a complete video according to the calibrated timestamp information; if no, continue to determine;

[0068] Determine whether the visitor has entered the correct terminal dynamic password; if yes, the visitor is allowed to view the full video; if no, the visitor is denied to view the full video;

[0069] Obtain historical access log information from the database, wherein the historical access log information includes: the visitor's IP address, access time, and password input;

[0070] Determine whether the visitor's IP address exists in the historical access log information; if so, continue to determine and record the corresponding visitor's access log information to the database; if not, send a reminder signal to notify the user's mobile phone bound to the corresponding camera.

[0071] For example, set the time period to 24 hours, determine 4 hours as a time period, the first time period starts from 00:00:01 and ends at 03:59:59, and the time period is divided into 6 time periods in sequence;

[0072] Set the dynamic password of the encrypted network unit in the first time period to: 000001035959JM@; the terminal dynamic password is: 000001035959ZD@;

[0073] Set the dynamic password of the encrypted network unit in the second time period to: 040001075959JM@; the terminal dynamic password is: 040001075959ZD@;

[0074] The dynamic password of the encrypted network unit in the third time period is set to: 080001115959JM@; the dynamic password of the terminal is set to: 080001115959ZD@;

[0075] The dynamic password of the encrypted network unit in the fourth time period is set to: 120001155959JM@; the dynamic password of the terminal is set to: 120001155959ZD@;

[0076] The dynamic password of the encrypted network unit in the fifth time period is set to: 160001195959JM@; the dynamic password of the terminal is set to: 160001195959ZD@;

[0077] The dynamic password of the encrypted network unit in the sixth time period is set to: 200001235959JM@; the dynamic password of the terminal is set to: 200001235959ZD@;

[0078] When the visitor visits within the corresponding time period, the dynamic password will also change;

[0079] There are three visitors, marked as A, B, and C, and their IP addresses are recorded in the database. If the dynamic passwords entered by visitors A and B into the encryption network unit are correct, the video frames and the corresponding audio are integrated one by one to form a complete video according to the calibrated timestamp information. Visitor C re-enters the dynamic password of the encryption network unit.

[0080] If visitor A enters the terminal dynamic password correctly, visitor A is allowed to view the full video;

[0081] If visitor B enters the terminal dynamic password incorrectly, visitor B re-enters the terminal dynamic password;

[0082] The IP addresses of three visitors are retrieved from the database and compared with the historical access log information. The IP addresses of visitors A and C exist in the historical access log information; therefore, there is no need to send a reminder signal;

[0083] If the IP address of visitor B does not exist in the historical access log information, a reminder signal is sent to notify the user's mobile phone bound to the corresponding camera.

[0084] See also Figure 4 , the second aspect of the present invention provides a decentralized smart home personal privacy protection system, including: a data acquisition module, a data encryption module, and a terminal access module;

[0085] Data acquisition module: obtains the original video collected by the smart home camera from the local database, separates the audio and video frames in the original video through audio and video processing technology, and uploads them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit;

[0086] Data encryption module: The password of the encrypted network unit and the terminal password after the synthesis of audio and video frames are set based on the time password policy;

[0087] Terminal access module: determines whether the visitor is allowed to view the complete video; if yes, records the visitor's access log information; if no, sends a reminder signal.

[0088] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0089] The working principle of the present invention is as follows: the present invention obtains the original video captured by the smart home camera from the local database, separates the audio and video frames in the original video through audio and video processing technology, and uploads them to the encryption network unit respectively; the password of the encryption network unit and the terminal password after the audio and video frames are synthesized are set based on the time password policy; only when the visitor enters the correct multiple dynamic passwords at the same time can the complete audio and video be viewed.

[0090] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A decentralized smart home personal privacy protection method, characterized in that: Obtaining the original video captured by the smart home camera from the local database, separating the audio and video frames in the original video through audio and video processing technology, and uploading them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit; Time-based password policy settings encrypt the passwords of network units and the terminal passwords after the synthesis of audio and video frames; Determine whether to allow the visitor to view the full video; if yes, record the visitor's access log information; if no, send a reminder signal.

2. A decentralized smart home personal privacy protection method according to claim 1, characterized in that: The method of separating the audio and video frames in the original video by using the audio and video processing technology includes: Retrieve the original video and back it up in the local database; use third-party software to extract the audio and video frames of the backed-up original video separately; Number the videos of adjacent frames and group the numbers; divide the total number of frames by the maximum number of frames that the video encryption node can store to obtain the number of groups; retrieve a complete group and subtract the minimum number from the maximum number to obtain the maximum frame difference; determine whether the difference between the numbers of two randomly retrieved video frames is less than the maximum frame difference, if so, store them respectively in the video encryption nodes with a set correlation of A; if not, randomly disperse them in the video encryption nodes with other correlations; Adjust the audio encryption node where the keyword is placed according to the matching degree of the keyword; The encrypted network unit includes a number of encrypted nodes, which are numbered; the breadth-first search technology is used to determine whether the shortest path length between two nodes is lower than a set threshold length; if yes, the correlation between the two nodes is marked as A-; if no, it is determined whether the two nodes are in similar network positions; if yes, the correlation between the two nodes is marked as A-; if no, the cosine similarity method is used to count the number of common neighbors of the two nodes; it is determined whether the number of neighbors is lower than a set neighbor number threshold; if yes, the correlation between the two nodes is marked as A; if no, the correlation between the two nodes is marked as A-; wherein: the correlation size is: A>A->A-.

3. A decentralized smart home personal privacy protection method according to claim 2, characterized in that: The step of performing time synchronization on the audio and video frames comprises: Retrieving the separately extracted audio and video frames for encoding respectively, and extracting the timestamp information attached to the corresponding audio and video frames; Determine whether the extracted audio and video timestamps are consistent; if yes, complete the audio and video time synchronization; if not, set the standard time axis and calibrate the timestamps of the audio and video frames according to the standard time axis.

4. According to claim 1, a decentralized smart home personal privacy protection method is characterized in that: The time-based password policy setting encrypts the password of the network unit and the terminal password after the audio and video frames are synthesized, including: Set a time period, customize the time period to several time periods, determine the start time and end time of each time period; set the encrypted network unit dynamic password and the terminal dynamic password for the corresponding time period within the time period; the dynamic password includes: start time + end time + set letters + special symbols.

5. A decentralized smart home personal privacy protection method according to claim 1, characterized in that: The determination of whether to allow the visitor to view the complete video includes: Determine whether the dynamic password input by the visitor to the encryption network unit is correct; if yes, integrate the video frame and the corresponding audio one by one to form a complete video according to the calibrated timestamp information; if no, continue to determine; Determine whether the visitor has entered the correct terminal dynamic password; if yes, the visitor is allowed to view the full video; if no, the visitor is denied to view the full video.

6. A decentralized smart home personal privacy protection method according to claim 1, characterized in that: The access log information of the visitor is recorded, including: Obtain historical access log information from the database, wherein the historical access log information includes: the visitor's IP address, access time, and password input; Determine whether the visitor's IP address exists in the historical access log information; if so, continue to determine and record the corresponding visitor's access log information to the database; if not, notify the user's mobile phone bound to the corresponding camera.

7. A decentralized smart home personal privacy protection system, applied to a decentralized smart home personal privacy protection method according to any one of claims 1 to 6, characterized in that: include: Data collection module, data encryption module, terminal access module; Data acquisition module: obtains the original video collected by the smart home camera from the local database, separates the audio and video frames in the original video through audio and video processing technology, and uploads them to the encryption network unit respectively; wherein the encryption network unit includes: an audio encryption network unit and a video encryption network unit; Data encryption module: The password of the encrypted network unit and the terminal password after the synthesis of audio and video frames are set based on the time password policy; Terminal access module: determines whether the visitor is allowed to view the complete video; if yes, records the visitor's access log information; if no, sends a reminder signal.

Images