A privacy violation detection method for intelligent voice products based on energy consumption
Through the matching of the energy consumption map of the smart voice product, the gap in the privacy infringement detection of smart voice product is solved, risk-free privacy protection is achieved, and consumer confidence is enhanced.
Patent Information
- Application Number
- CN202110332794.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-29
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-03-29
AI Technical Summary
The existing technology lacks effective detection methods for privacy violations of smart voice products, resulting in insufficient confidence in the privacy protection of smart voice products, and the existing methods may lead to further leakage of voice data during the detection process.
By obtaining the energy consumption map of smart voice products under different working conditions, analyzing their functional requirements, defining the working status of privacy violations, and detecting whether there is any invasion through energy consumption map matching, including detection in silent and noise environments, ensuring detection The process does not involve voice data backup analysis to avoid privacy leakage.
It provides an effective method for detecting privacy infringement of smart voice products, which enhances consumer confidence and fills the industry gap. The detection process is risk-free and can accurately determine whether the product has privacy infringement.
Smart Images

Figure CN113284516B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of personal information protection, and in particular, relates to a privacy infringement detection method for intelligent voice products. Background Art
[0002] The smart voice product referred to in this invention is a hardware product that has a voice recording function and can be used online. Smart voice products can free your hands and bring comfort and convenience. At the same time, consumers are increasingly worried about the privacy violations of smart voice products such as "secret recording" and "secret uploading".
[0003] For privacy protection, there are standards such as GDPR and CCPA internationally, and GB / T 35273-2020 in China. GDPR (General Data Protection Regulation) is a regulation on data protection and privacy for all EU individuals in EU law, involving the export of personal data outside Europe. Its main goal is to regain control of personal data and simplify unified regulations within the EU for international business. CCPA (California Consumer Privacy Act) is a California law passed in June 2018 that enables California residents to better control their personal information. GB / T35273-2020 "Information Security Technology Personal Information Security Specification" puts forward clear requirements for each stage of the life cycle of personal information. Unlike GDPR, GDPR is more explained from the perspective of subject and object, while the personal information security specification is more explained according to the operation process.
[0004] From a legislative perspective, the Cybersecurity Law of the People's Republic of China, which came into effect on June 1, 2017, is a law enacted by the People's Republic of China to strengthen cybersecurity management, and has some provisions for the protection of personal information. The Data Security Law of the People's Republic of China (Draft) includes some provisions for the protection of personal information data by security solutions. On October 21, 2020, China's highest legislative body issued the Personal Information Protection Law (Draft) for public comment. This is the first special law for personal information, which marks an important step for China to improve and deepen the protection of personal information.
[0005] Most of the personal privacy protection laws and regulations in countries and regions such as GDRP, CCPA, and GB / T 35273 follow the privacy protection framework standard ISO / IEC 29100, and there are some extensions of the ISO / IEC 27001 information security management system standard such as ISO27017 and ISO27018. These are basically carried out from the aspects of clear notification, authorized use, data desensitization, data transfer, etc. As for voice, the voiceprint of the data itself has the characteristics of personal biometrics, and many smart voice products must collect voice to ensure the availability of functions, so authorization and permission cannot eliminate users' concerns about privacy violations, and these standards do not have specific methods to detect whether there is privacy violation.
[0006] The Cybersecurity Law of the People's Republic of China and the Data Security Law of the People's Republic of China (draft) are both aimed at protecting data, of which personal information is only a small part. The Personal Information Protection Law, which targets personal privacy, is still in the draft stage and has not yet been implemented. Moreover, the provisions are mainly about supervision.
[0007] At present, the means of voice privacy protection mainly involve two directions. One is based on desensitizing voice data or identifying and processing sensitive information, such as the two Chinese patent applications with application number 2019110514220, invention name "A safe desensitization method for voice data release", and application number 2019111174870, invention name "A voice data processing method based on privacy protection". The other is to protect voice data from being illegally obtained, such as application number 2020101064580, invention name "A method and system for privacy protection of IoT devices". "A safe desensitization method for voice data release" and "A voice data processing method based on privacy protection" are both aimed at voice data, which is the step of recording voice or even uploading voice to a specific system for processing. The data security in this process is difficult to guarantee, but it may cause further leakage of personal privacy due to the processing of one more party. "A method and system for privacy protection of IoT devices" refers to the IoT terminal device sending data to the back-end system based on the user's voice activation signal. When the user tag information shows that the user does not allow his voice information to be saved, the back-end system device can discard the voice information. This patent aims to prevent the user's voice information from being illegally obtained. One problem with this method is how to ensure the security of the back-end system and whether it is equivalent to agreeing to upload the private voice data.
[0008] The industry's detection of privacy violations is basically based on apps, and there is no effective inspection method for smart voice products.
[0009] Currently, the main protection for voice information is to prevent the voice from being leaked by a third party or to protect the sensitive information in the voice information, rather than from the individual's own need for privacy protection.
[0010] The present invention is explained from the perspective of consumers, believing that both personal voice data and the voice data of the personal environment can contain personal sensitive information (biological characteristics or personal environment or location information). The protection of voice privacy is not only to protect the voice data from infringement by third parties, but also to protect it from infringement by all entities other than the individual himself and his authorized entities.
[0011] In view of the current lack of means and methods for detecting infringement of personal privacy by smart voice products, the present invention aims to provide an effective method for detecting infringement of personal privacy, so as to enhance consumers' confidence in smart voice products and fill the gap in this area of the industry. Summary of the invention
[0012] The purpose of the present invention is to provide a method for detecting privacy infringement of intelligent voice products.
[0013] The invention objective of the present invention is achieved through the following technical solution: a method for detecting privacy violation of intelligent voice products based on energy consumption, comprising the following steps:
[0014] 1) Obtain the energy consumption graph of the intelligent voice product to be tested in various working states;
[0015] 2) Analyze the requirements for voice recording and voice uploading of each function of the intelligent voice product to be tested, and define its working status that infringes on privacy, such as recording in standby mode;
[0016] 3) Matching the working state of step 2) with the energy consumption spectrum of step 1) to obtain the energy consumption spectrum corresponding to the privacy-invading working state;
[0017] 4) Define and set the state of the intelligent voice product to be tested according to step 2), such as setting it to standby state, and detect its energy consumption spectrum within the preset time. When the energy consumption spectrum corresponding to the privacy-violating working state in step 3) appears, it is referred to as the infringement spectrum below, and it is considered that the intelligent voice product to be tested has violated privacy.
[0018] The states in step 1) mainly include a standby state, a recording state, a network transmission state, and the coexistence of two or more of them.
[0019] Step 2) is carried out from two aspects. One is to detect whether the intelligent voice product "secretly records" and "secretly records" refers to recording when the function is not necessary. The other is to detect whether the intelligent voice product "secretly uploads" recording and "secretly uploads" refers to uploading when the function is not necessary.
[0020] As an improvement of the present invention, the detection in step 4) includes detection in a silent environment and detection in a noisy environment.
[0021] The noise environment includes environmental noise and multi-person interaction noise.
[0022] As a further improvement of the present invention, when the same infringement graph appears in a silent environment and a noisy environment, it is considered that the intelligent voice product to be detected has violated privacy.
[0023] Beneficial effects:
[0024] The method of the present invention measures the energy consumption of smart voice products from the perspective of consumers to detect whether there is any violation of user privacy, thus filling the gap in the detection of privacy violation of smart voice products. Moreover, the detection method of the present invention does not perform backup analysis on voice data during the entire detection process, so there is no risk of further leakage of voice data privacy. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic structural diagram of a specific embodiment of the detection system of the present invention. DETAILED DESCRIPTION
[0026] Taking Tmall Genie Cube 2 as an example, the energy consumption-based smart voice product privacy violation detection method of the present invention is further described in detail below.
[0027] Step (1): Connect the intelligent voice product to be tested (DUT), namely Tmall Genie Cube 2, to the energy consumption detection device, manually control it to work in various states, and record the energy consumption spectrum in each state, especially the standby state, recording state, network transmission state and the coexistence of two or more of them.
[0028] Specifically, turn off other alarms and device pairing functions of Tmall Genie Cube 2, connect the power supply to the energy detection device, and record its spectrum in standby state, spectrum in recording state, spectrum in recording and uploading state after successful awakening, etc. Suppose their spectrum numbers are G1, G2, G3, etc. The standby state here refers to the state when no operation is performed, the recording state refers to the state when only recording is performed, and recording and uploading refers to the state when recording and performing network transmission operations.
[0029] Step (2): Analyze the voice recording and voice uploading requirements of each function of the DUT and define its privacy-invading working status.
[0030] The privacy-invading work status mainly consists of two parts: operation and the status that is considered as a privacy violation under the operation. The above analysis work is mainly carried out from two aspects. One is to detect whether there is "secret recording". "Secret recording" here refers to recording when it is not necessary for the function. The second is to detect whether the recording is "secretly uploaded". "Secretly uploading" here refers to uploading when it is not necessary for the function.
[0031] The detection of "secret recording" mainly focuses on the following aspects: for DUTs that record by pressing buttons, recording when the buttons are not pressed is considered an infringement of personal privacy; for DUTs that record by switching buttons, recording when the function switch is not turned on is considered an infringement of personal privacy.
[0032] The detection of "secret upload" is mainly focused on the following aspects: for offline voice DUT, uploading voice data is regarded as an infringement of personal privacy; for online voice recognition DUT, uploading voice data when the device is not awake is regarded as an infringement of personal privacy.
[0033] Specifically for Tmall Genie Cube 2, analysis found that the product has a mute button, which turns off the microphone when pressed. At this time, recording should not be done, and recording is considered an infringement of personal privacy; when the product is not awake, voice recognition is not required, and uploading the recording at this time is also an infringement of customer privacy. The details are shown in Table 1.
[0034] Table 1
[0035] operate Status considered as privacy violation No Action Record and upload Press the Mute button Record, record and upload
[0036] Step (3): Match the working state of step (2) with the energy consumption spectrum of step (1) to obtain the energy consumption spectrum corresponding to the privacy-infringing working state, referred to as the infringement spectrum. The details are shown in Table 2.
[0037] Table 2
[0038]
[0039] It should be noted that for some devices, the working state of step (2) and the energy consumption spectrum of step (1) may not be matched simply and directly as shown in Table 2. For example, some devices with larger display screens are often in an active state (the display screen is bright) when detecting the recording state energy spectrum in step (1), but in a standby state (the display screen is black) when secretly recording. There is a difference in energy consumption here, so in this case, it may be necessary to perform some processing on the energy consumption spectrum, such as finding the difference.
[0040] Step (4): Connect Tmall Genie Cube 2 to the energy consumption monitoring device, set the status of Tmall Genie Cube 2 according to the definition of step (2), that is, no operation and pressing the mute button (set in sequence), set the test time to 72 hours or longer, and record the energy consumption graph output by the energy consumption monitoring device in real time. In order to eliminate the selectivity of the equipment to the environment and make the detection results more reliable, this method is tested in both silent and noisy environments. Noisy environments include using a computer to control the speaker to play environmental noise, multi-person interaction noise, etc. Table 3 shows the test results.
[0041] Table 3
[0042] environment operate result Quiet environment No Action O1-1 Quiet environment Press the Mute button O2-1 Noise environment No Action O1-2 Noise environment Press the Mute button O2-2
[0043] Step (5): Analyze the results of step (4). When the same infringement pattern appears in a silent environment and in a sound environment, it is considered that the intelligent voice product to be tested has violated privacy.
[0044] Comparing the results O1-1 and O1-2, when both of them show graph G3, it is judged that Tmall Genie Cube 2 has violated the privacy. Comparing the results O2-1 and O2-2, when both of them show G2 and / or G3, it is also judged that Tmall Genie Cube 2 has violated the privacy.
[0045] If no infringement graph appears in any of the four results, it is considered that Tmall Genie Cube 2 does not violate privacy.
[0046] For situations that are between the above two, it is considered that the confidence level of the results does not meet the expected judgment, and the accuracy of the conclusion can be improved by repeating the experiment.
[0047] In this embodiment, the determination method of step (5) is only to make the determination result more accurate. In fact, in a single environment, a conclusion can be drawn based on the results of multiple tests through multiple tests.
[0048] The detection method of the present invention does not perform backup analysis on the voice data during the entire detection process, and there is no risk of further leakage of the voice data privacy.
[0049] Figure 1The result schematic diagram of the detection system of the present invention is given. The test system includes a soundproof room 6, the DUT is placed in the center of the soundproof room 6, speakers 3 are respectively set at the four corners of the soundproof room 6, the energy detection device 5 and the sound card 4 are also respectively set in the soundproof room 6, the control computer 1 is set outside the soundproof room 6, the DUT is connected to the energy detection device 5, the energy detection device 5 is connected to the control computer 1, the control computer 1 is also connected to the sound card 4, and the sound card 4 is connected to four speakers 3. The sound card 4 and the speakers 3 are used to form a noise environment under the action of the control computer 1. The energy detection device 5 is used to collect the energy output data of the DUT and transmit it to the control computer 1 for display and analysis.
Claims
1. A privacy violation detection method for intelligent voice products based on energy consumption, characterized in that: The steps include: 1) Obtain the energy consumption graph of the intelligent voice product to be tested in various working states; 2) Analyze the requirements for voice recording and voice uploading of each function of the intelligent voice product to be tested, and define its privacy-infringing working status; 3) Match the working state of step 2) with the energy consumption spectrum of step 1) to obtain the energy consumption spectrum corresponding to the privacy-invading working state; 4) Define and set the status of the intelligent voice product to be tested according to step 2), and detect its energy consumption spectrum within the preset time. When the energy consumption spectrum corresponding to the privacy-infringing working state in step 3) appears, it is considered that the intelligent voice product to be tested has violated privacy; The detection in step 4) includes detection in a silent environment and detection in a noisy environment; When the same infringement pattern appears in a silent environment and a noisy environment, it is considered that the smart voice product to be tested has violated privacy.
2. The method for detecting privacy violation of intelligent voice products based on energy consumption according to claim 1, characterized in that: The states in step 1) include a standby state, a recording state, a network transmission state, and the coexistence of two or more of them.
3. The method for detecting privacy violation of intelligent voice products based on energy consumption according to claim 1, characterized in that: Step 2) is carried out from two aspects. One is to detect whether the smart voice product "secretly records" and "secretly records" refers to recording when the function is not necessary. The other is to detect whether the smart voice product "secretly uploads" recordings and "secretly uploads" refers to uploading when the function is not necessary.
4. The method for detecting privacy violation of intelligent voice products based on energy consumption according to claim 1, characterized in that: The noise environment includes environmental noise and multi-person interaction noise.
