A threat assessment method for an intelligent sensing device

Abstract

The application discloses a threat assessment method of intelligent sensing equipment, comprising: acquiring a to-be-processed video, wherein the to-be-processed video comprises a process in which a user unlocks an intelligent door lock through a PIN code; determining a starting frame and an ending frame in which the user unlocks the intelligent door lock according to an elbow bending angle in each video frame, and obtaining a video segment in which the user unlocks the intelligent door lock through the PIN code; extracting a key point motion track of the elbow of the user when unlocking the intelligent door lock through the PIN code from the video segment; mapping the key point motion track to a two-dimensional coordinate system, and calculating motion parameters of the key point motion track; outputting a PIN code inference result of the intelligent door lock in the to-be-processed video according to a click state of the user and relative positions of keys, and taking a one-time inference and a three-time inference success probability of the PIN code inference result as a security assessment result of the scene. According to the application, the relative positions of the keys pressed by the user can be inferred according to the elbow key point motion track, and a possible PIN code can be analyzed, so that the available scene of the threat assessment method is expanded.

Description

Technical Field

[0001] The present invention belongs to the technical field of video analysis, and in particular relates to a threat assessment method for an intelligent sensing device. Background Art

[0002] PIN codes are a common user authentication method for smart devices and are widely used in scenarios such as mobile phone settlement and payment and smart door locks. They can not only be used for identity authentication of electronic devices such as mobile phones and tablets, but are also used in services that are closer to people's property security, such as ATMs and smart door locks.

[0003] PIN codes typically consist of a fixed four or six digits, making them easier to remember and easier to enter. However, their widespread use also increases security risks. For example, with the increasing adoption of smart homes, more and more smart devices, such as smart home surveillance cameras, smart door locks, smart doorbells, and wireless routers and Bluetooth gateways with wireless sensing capabilities, are being deployed in homes and at entrances. Shoulder-surfing attacks using computer vision technology are becoming increasingly difficult to prevent. Video-assisted methods can provide attackers with side-channel information, allowing them to more easily infer PIN codes, posing a significant threat to users' property safety. Specifically, video-based side-channel attacks primarily involve surveillance cameras, particularly those installed in hallways, such as surveillance cameras and smart doorbells. Attackers can exploit the location of surveillance devices in hallways to record the user entering their smart door lock PIN and, using computer vision techniques, decipher the PIN sequence.

[0004] However, there is no effective method in the prior art to evaluate the security of the above scenarios, which is not conducive to users avoiding threats when using smart devices. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a threat assessment method for intelligent sensing devices. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a threat assessment method for an intelligent sensing device, comprising:

[0007] Obtain a video to be processed, where the video to be processed includes a process in which a user unlocks a smart door lock using a PIN code;

[0008] Determining the start frame and the end frame of the user unlocking the smart door lock according to the elbow bending angle in each video frame of the video to be processed, and obtaining a video clip of the user unlocking the smart door lock using the PIN code;

[0009] Extracting from the video clip the key point motion trajectory of the elbow when the user unlocks the smart door lock using the PIN code;

[0010] Mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system, and calculating the motion parameters of the key point motion trajectory;

[0011] Determining the user's click state and the relative positions of the keys in the smart door lock keypad interface based on the motion parameters;

[0012] Output the PIN code inference result of the smart door lock in the video to be processed based on the user's click status and the relative position of the button;

[0013] According to the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed, a security assessment result in the scenario where the user unlocks the smart door lock is obtained.

[0014] In one embodiment of the present invention, the step of determining the start frame and the end frame of the user unlocking the smart door lock based on the elbow bending angle in each video frame of the video to be processed, and obtaining a video clip of the user unlocking the smart door lock using a PIN code includes:

[0015] For the video to be processed, the pytorch-openpose framework is used to detect the key points of the elbow and the bending angle of the elbow joint in each video frame; the key points are predicted by the pytorch-openpose framework using Gaussian graphs;

[0016] When the bending angle of the elbow joint is less than a preset threshold, it indicates that the user starts the unlocking process at this video frame, and the video frame is determined as the starting frame;

[0017] When the bending angle of the elbow joint is greater than a preset threshold, it indicates that the user ends the unlocking process at this video frame, and the video frame is determined as the end frame;

[0018] All video frames between the start frame and the end frame are obtained to obtain a video clip of the user unlocking the smart door lock using the PIN code.

[0019] In one embodiment of the present invention, before the steps of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory, the method further includes:

[0020] The motion trajectory of the key points of the elbow is smoothed using a Savitzky-Golay filter.

[0021] In one embodiment of the present invention, the step of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory includes:

[0022] After mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system, the moving speed of the key point is calculated according to the key point motion trajectory.

[0023] In one embodiment of the present invention, the click state of the user in the video to be processed is determined by a change trend of the moving speed of the key point;

[0024] Among them, when the moving speed of the key point at a certain button in the smart door lock shows a trend of first decelerating and then accelerating, it means that the user has clicked the button.

[0025] In one embodiment of the present invention, the step of outputting a PIN code inference result of a smart door lock in a video to be processed based on the user's click state and the relative position of the button includes:

[0026] Record the position of the keys clicked by the user in sequence and add 1 to the score of the clicked key;

[0027] Generate a set of all possible PIN codes for the smart door lock in each video to be processed based on the scores and order of the buttons clicked;

[0028] After calculating the score of each PIN code in the PIN code set, the three groups of PIN codes with the highest scores are determined as the PIN code inference results of the smart door lock in the corresponding video to be processed.

[0029] In one embodiment of the present invention, the step of obtaining a security assessment result in a scenario where a user unlocks a smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes:

[0030] Based on the PIN code inference result of the video to be processed and the actual PIN code, the probability of successful inference once and the probability of successful inference three times are calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

[0031] In one embodiment of the present invention, the step of obtaining a security assessment result in a scenario where a user unlocks a smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes:

[0032] Calculating the Euclidean distance between the PIN code inference result and the actual PIN code of the video to be processed, and using the Euclidean distance as the security assessment result in the scenario where the user unlocks the smart door lock;

[0033] For each PIN code in the PIN code inference result, the Euclidean distance between the PIN code and the actual PIN code is calculated according to the following steps:

[0034] Compare each digit of the actual PIN code with the inferred PIN code; if the two are the same, the Euclidean distance is 0; otherwise, the Euclidean distance is the position difference between the two corresponding buttons in the smart door lock;

[0035] The sum of the Euclidean distances between each digit in the actual PIN code and each digit in the inferred PIN code is calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention provides a threat assessment method for intelligent sensing devices. In the scenario where a user unlocks a smart door lock, the method automatically determines the process from the start to the end of the user entering a PIN code based on the elbow bending angle, and analyzes the PIN code inference result through the movement trajectory of the elbow key point, thereby evaluating the security of the smart door lock in a side-channel attack manner. In addition, the present invention utilizes the movement of the user's elbow key point rather than the movement of the hand when cracking the PIN code. In this way, possible PIN codes can be analyzed even when the user's hand is blocked, further expanding the applicable scenarios of the security assessment method.

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a threat assessment method for an intelligent sensing device provided by an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of a threat assessment method for an intelligent sensing device provided by an embodiment of the present invention;

[0041] Figure 3 This is another schematic diagram of the threat assessment method for an intelligent sensing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0043] Figure 1 This is a flow chart of a threat assessment method for an intelligent sensing device provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of a threat assessment method for an intelligent sensing device provided by an embodiment of the present invention. Figure 3 FIG. 1 is another schematic diagram of a threat assessment method for an intelligent sensing device provided by an embodiment of the present invention. Figure 3 As shown. Figure 1-3As shown, the embodiment of the application provides a threat assessment method of an intelligent sensing device, comprising:

[0044] S1, acquiring a to-be-processed video, the to-be-processed video including a process in which a user unlocks an intelligent door lock through a PIN code;

[0045] S2, determining a starting frame and an ending frame in which the user unlocks the intelligent door lock according to an elbow bending angle in each video frame of the to-be-processed video, and obtaining a video segment in which the user unlocks the intelligent door lock through the PIN code;

[0046] S3, extracting a key point motion trajectory of the elbow of the user when unlocking the intelligent door lock through the PIN code from the video segment;

[0047] S4, mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating a motion parameter of the key point motion trajectory;

[0048] S5, determining a click state of the user and a relative position of a key in an intelligent door lock keyboard interface based on the motion parameter;

[0049] S6, outputting a PIN code inference result of the intelligent door lock in the to-be-processed video according to the click state of the user and the relative position of the key;

[0050] S7, obtaining a security assessment result in a scenario in which the user unlocks the intelligent door lock according to the PIN code inference result and an actual PIN code of the intelligent door lock in the to-be-processed video.

[0051] In the embodiment, the to-be-processed video refers to a video including a process in which the user unlocks the intelligent door lock through the PIN code. Specifically, the to-be-processed video can be captured by a monitoring camera or captured by a device with a monitoring function such as an intelligent doorbell. After the to-be-processed video is acquired, the human posture in the video is detected based on a pytorch-openpose framework to locate key points of the elbow joint of the user, and then the bending angle of the elbow joint, that is, the included angle between the upper arm and the lower arm of the user, is detected frame by frame for the to-be-processed video. When the elbow joint is detected to be bent to a certain degree, it can be considered that the user starts to unlock. Therefore, the starting frame and the ending frame in which the user unlocks the intelligent door lock through the PIN code can be determined, and the video segment in which the user unlocks the intelligent door lock can be obtained from the to-be-processed video, so that the key point motion trajectory of the elbow of the user can be fitted through the change of the key point position.

[0052] Furthermore, based on the motion trajectory of the elbow key points, the button clicked by the user during the current unlocking of the smart door lock is determined, and the score of the clicked button is calculated. The higher the score, the greater the probability that the button was clicked, so as to obtain the PIN code inference result of the smart door lock in the video to be processed, wherein the PIN code inference result includes the three most likely PIN codes. Finally, combined with the actual PIN code of the smart door lock in the video to be processed, the security assessment result in the scenario where the user unlocks the smart door lock is calculated. For example, this embodiment can use the probability of successfully unlocking the smart door lock using the PIN code inference result to assess the security of the scenario. The higher this probability, the lower the security of the smart door lock in the current scenario and the greater the possibility of being attacked by a side channel.

[0053] In step S2 above, the step of determining the start frame and end frame of the user unlocking the smart door lock based on the elbow bending angle in each video frame of the video to be processed, and obtaining a video clip of the user unlocking the smart door lock using the PIN code includes:

[0054] S201. For the video to be processed, use the pytorch-openpose framework to detect the key points of the elbow and the bending angle of the elbow joint in each video frame; the key points are predicted by the pytorch-openpose framework using the Gaussian graph;

[0055] S202: When the bending angle of the elbow joint is less than a preset threshold, it indicates that the user starts the unlocking process at the video frame, and the video frame is determined as the starting frame;

[0056] S203: When the bending angle of the elbow joint is greater than a preset threshold, it indicates that the user has ended the unlocking process at this video frame, and the video frame is determined as the end frame;

[0057] S204: Acquire all video frames between the start frame and the end frame to obtain a video clip of the user unlocking the smart door lock using the PIN code.

[0058] Specifically, the preset threshold can be set to 90°. When using the pytorch-openpose framework to perform frame-by-frame detection on the video to be processed, when the angle of the user's elbow joint bending in the video frame is less than 90°, it indicates that the user has begun to enter the PIN code to unlock. At this time, the corresponding video frame is used as the starting frame; and when the user completes the unlocking, the bending angle of the elbow joint will return to a state greater than 90°, at which time the corresponding end frame can be determined. Obviously, the starting frame, the end frame, and the video frames between the starting frame and the end frame constitute a video clip of the user unlocking the smart door lock using the PIN code. After performing frame-by-frame detection on the video to be processed, the user's entire unlocking process can be composed of all the first video frames obtained therefrom.

[0059] Optionally, before the step of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory, the method further includes:

[0060] The Savitzky-Golay filter is used to smooth the motion trajectory of the key points of the elbow.

[0061] It should be understood that because key points are used to estimate the elbow joint position based on independent video frames, there is a lack of continuity between each key point, which can lead to significant fluctuations in the estimated key point motion trajectory, hindering the subsequent inference of key position. To this end, this embodiment uses a Savitzky-Golay filter to smooth the motion trajectory of the elbow key point to eliminate fluctuations. This not only allows for more accurate inference of key position, but also helps improve the accuracy of threat assessment results.

[0062] Furthermore, in the above step S3, the step of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory includes:

[0063] After mapping the key point motion trajectory of the elbow to the two-dimensional coordinate system, the moving speed of the key point is calculated based on the key point motion trajectory.

[0064] In the above step S303, when the moving speed of the key point at the first button in the smart keyboard shows a trend of first decelerating and then accelerating, it means that the user clicks the first button.

[0065] During the research process, the inventor discovered that when a user clicks a certain button, the key point of the user's elbow will show a trend of first decelerating and then accelerating at the position of the button. Therefore, this embodiment can determine the first button clicked by the user in the process of unlocking the smart door lock by calculating the moving speed of the key point.

[0066] In the above step S6, the step of outputting the PIN code inference result of the smart door lock in the video to be processed according to the user's click status and the position of the button includes:

[0067] S601, record the positions of the keys clicked by the user in sequence, and add 1 to the score of the clicked key;

[0068] S602: Generate a set of all possible PIN codes for the smart door lock in each video to be processed based on the scores and order of the clicked buttons;

[0069] S603: After calculating the score of each PIN code in the PIN code set, determine the three groups of PIN codes with the highest scores as the PIN code inference results of the smart door lock in the video to be processed.

[0070] This embodiment uses a scoring mechanism to determine the PIN code inference result of the smart door lock in the video to be processed. Specifically, when the user's elbow key point passes the relative position of a certain digital key, the key is recorded and assigned a default score of 0, indicating that the user did not press the key. When the user's elbow key point shows a movement trend of first decelerating and then accelerating within the range of a certain key, the key is considered to have been pressed once and the key score is increased by 1. Obviously, a higher key score indicates a higher probability that the key has been pressed or pressed multiple times.

[0071] For example, if the actual PIN code of the smart door lock in the video to be processed is 125790, the user accelerates and decelerates three times at number key 1, accelerates and decelerates three times at number key 2, accelerates and decelerates once at number key 3, accelerates and decelerates four times at number key 5, accelerates and decelerates twice at number key 7, accelerates and decelerates once at number key 8, accelerates and decelerates twice at number key 9, and accelerates and decelerates three times at number key 0. Based on the order in which the user presses the number keys and the scores of the keys, the possible PIN codes can be inferred as: 125790, 123590, 125780, 123580, and the corresponding scores are: 17 points, 16 points, 16 points, and 15 points, respectively.

[0072] Therefore, the three PIN codes with the highest scores are determined as PIN code inference results, namely: 125790, 123590 and 125780.

[0073] Of course, in some other embodiments of the present application, more PIN codes may be selected as the final PIN code inference result, and the present application does not limit this.

[0074] Furthermore, in the above step S7, the step of obtaining a security assessment result in the scenario where the user unlocks the smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes:

[0075] Based on the PIN code inference result of the video to be processed and the actual PIN code, the probability of successful inference once and the probability of successful inference three times are calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

[0076] In this embodiment, the one-time inference success probability refers to the probability that the PIN code with the highest score in the PIN code inference result is the actual PIN code, and the three-time inference success probability refers to the probability that the three PIN codes in the input PIN code inference result contain the actual PIN code.

[0077] Specifically, assuming there are N unlocking video clips, among which P groups successfully unlock the smart door lock using the most likely PIN code, and Q groups successfully unlock the smart door lock using the top three most likely PIN codes. The probability of successful inference in one time is expressed as P / N, and the probability of successful inference in three times is expressed as: Q / N.

[0078] This embodiment obtains the threat assessment result by counting the probability of successful inference once or three times. It should be understood that the greater the probability of successful inference once or three times, the lower the security of the smart door lock.

[0079] Optionally, in the threat assessment method for an intelligent sensing device provided by the present invention, other methods may be used to obtain security assessment results.

[0080] For example, the step of obtaining a security assessment result in a scenario where a user unlocks a smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes:

[0081] Calculate the Euclidean distance between the PIN code inference result of the video to be processed and the actual PIN code, and use the Euclidean distance as the security assessment result in the scenario where the user unlocks the smart door lock;

[0082] For each PIN code in the PIN code inference result, the Euclidean distance between it and the actual PIN code is calculated according to the following steps:

[0083] Compare each digit of the actual PIN code with the inferred PIN code; if the two are the same, the Euclidean distance is 0; otherwise, the Euclidean distance is the position difference between the two corresponding buttons in the smart door lock;

[0084] The sum of the Euclidean distances between each digit in the actual PIN code and each digit in the inferred PIN code is calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

[0085] It should be noted that the position difference between any two numbers on the keyboard is the minimum number of times the keyboard needs to move from the inferred wrong number key to the actual correct number key. For example, the Euclidean distance between the number key 9 and the number key 1 is 4.

[0086] For example, if the actual PIN code is 125790 and the inferred PIN code is 123580, since the first two digits and the last digit of 125790 and 123580 are the same, the Euclidean distance between the first two digits and the last digit is 0. Furthermore, the third digit of the actual PIN code 125790 is 5, while the third digit of the inferred PIN code 123580 is 3, so the Euclidean distance is 2. The fourth digit of the actual PIN code 125790 is 7, while the fourth digit of 123580 is 5, so the Euclidean distance is 2. The fifth digit of the actual PIN code 125790 is 9, while the fifth digit of 123580 is 8, so the Euclidean distance is 1. In summary, the Euclidean distance between PIN codes 125790 and 123580 is 0+0+2+2+1+0=5.

[0087] Compared with the prior art, the present invention has the following beneficial effects:

[0088] The present invention provides a threat assessment method for intelligent sensing devices. In the scenario where a user unlocks a smart door lock, the method automatically determines the process from the start to the end of the user entering a PIN code based on the elbow bending angle, and analyzes the PIN code inference result through the movement trajectory of the elbow key point, thereby evaluating the security of the smart door lock in a side-channel attack manner. In addition, the present invention utilizes the movement of the user's elbow key point rather than the movement of the hand when cracking the PIN code. In this way, possible PIN codes can be analyzed even when the user's hand is blocked, further expanding the applicable scenarios of the security assessment method.

[0089] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0090] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0091] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims.

[0092] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A threat assessment method for an intelligent sensing device, characterized in that: include: Obtain a video to be processed, where the video to be processed includes a process in which a user unlocks a smart door lock using a PIN code; Determining the start frame and the end frame of the user unlocking the smart door lock according to the elbow bending angle in each video frame of the video to be processed, and obtaining a video clip of the user unlocking the smart door lock using the PIN code; Extracting from the video clip the key point motion trajectory of the elbow when the user unlocks the smart door lock using the PIN code; Mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system, and calculating the motion parameters of the key point motion trajectory; Determining the user's click state and the relative positions of the keys in the smart door lock keypad interface based on the motion parameters; Output the PIN code inference result of the smart door lock in the video to be processed based on the user's click status and the relative position of the button; According to the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed, a security assessment result in the scenario where the user unlocks the smart door lock is obtained.

2. The threat assessment method for an intelligent sensing device according to claim 1, characterized in that: The step of determining a start frame and an end frame of a user unlocking a smart door lock according to an elbow bending angle in each video frame of the video to be processed, and obtaining a video clip of the user unlocking the smart door lock using a PIN code, comprises: For the video to be processed, the pytorch-openpose framework is used to detect the key points of the elbow and the bending angle of the elbow joint in each video frame; the key points are predicted by the pytorch-openpose framework using Gaussian graphs; When the bending angle of the elbow joint is less than a preset threshold, it indicates that the user starts the unlocking process at this video frame, and the video frame is determined as the starting frame; When the bending angle of the elbow joint is greater than a preset threshold, it indicates that the user ends the unlocking process at this video frame, and the video frame is determined as the end frame; All video frames between the start frame and the end frame are obtained to obtain a video clip of the user unlocking the smart door lock using the PIN code.

3. The threat assessment method for an intelligent sensing device according to claim 1, characterized in that: Before the steps of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory, the method further includes: The motion trajectory of the key points of the elbow is smoothed using a Savitzky-Golay filter.

4. The threat assessment method for an intelligent sensing device according to claim 3, characterized in that: The steps of mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system and calculating the motion parameters of the key point motion trajectory include: After mapping the key point motion trajectory of the elbow to a two-dimensional coordinate system, the moving speed of the key point is calculated according to the key point motion trajectory.

5. The threat assessment method for an intelligent sensing device according to claim 4, characterized in that: The click state of the user in the video to be processed is determined by the change trend of the moving speed of the key point; Among them, when the moving speed of the key point at a certain button in the smart door lock shows a trend of first decelerating and then accelerating, it means that the user has clicked the button.

6. The threat assessment method for an intelligent sensing device according to claim 5, characterized in that: The step of outputting a PIN code inference result of the smart door lock in the video to be processed according to the user's click state and the relative position of the button includes: Record the position of the keys clicked by the user in sequence and add 1 to the score of the clicked key; Generate a set of all possible PIN codes for the smart door lock in each video to be processed based on the scores and order of the buttons clicked; After calculating the score of each PIN code in the PIN code set, the three groups of PIN codes with the highest scores are determined as the PIN code inference results of the smart door lock in the corresponding video to be processed.

7. The threat assessment method for an intelligent sensing device according to claim 6, characterized in that: The step of obtaining a security assessment result in a scenario where a user unlocks the smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes: Based on the PIN code inference result of the video to be processed and the actual PIN code, the probability of successful inference once and the probability of successful inference three times are calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

8. The threat assessment method for an intelligent sensing device according to claim 6, characterized in that: The step of obtaining a security assessment result in a scenario where a user unlocks the smart door lock based on the PIN code inference result and the actual PIN code of the smart door lock in the video to be processed includes: Calculating the Euclidean distance between the PIN code inference result and the actual PIN code of the video to be processed, and using the Euclidean distance as the security assessment result in the scenario where the user unlocks the smart door lock; For each PIN code in the PIN code inference result, the Euclidean distance between the PIN code and the actual PIN code is calculated according to the following steps: Compare each digit of the actual PIN code with the inferred PIN code; if the two are the same, the Euclidean distance is 0; otherwise, the Euclidean distance is the position difference between the two corresponding buttons in the smart door lock; The sum of the Euclidean distances between each digit in the actual PIN code and each digit in the inferred PIN code is calculated as the security assessment result in the scenario where the user unlocks the smart door lock.

Images