Electronic device operation mode switching method and apparatus, and electronic device and storage medium

Abstract

The present specification provides an operation mode switching method and device of an electronic device, and the electronic device and a storage medium, the method is applied to an electronic device with a biological information collection module, including: obtaining historical detection information obtained by the electronic device detecting a target object appearing in a network detection environment within a preset time period. Based on the historical detection information, a prediction result of determining the future behavior of the target object is obtained. In response to the current time being the time before the first preset time length of the future behavior occurrence time determined based on the prediction result, the operation mode is switched.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of network, and particularly relates to an operating mode switching method and device of an electronic device, and the electronic device and a storage medium. BACKGROUND

[0002] At present, electronic security devices such as intelligent door locks and community monitoring systems can monitor the whereabouts of strangers. Residents hope to protect the privacy and safety of their own or part of their neighbors' biological information, and the residents themselves also hope to avoid data leakage of their own devices. However, the general technology for identifying the identity of an object requires the electronic device to establish a communication connection with the target object to obtain relevant information of the target object, so as to determine the identity of the target object according to the relevant information. This way has the risk of leaking the privacy information of the electronic device.

[0003] At the same time, the existing image privacy protection method usually relies on the following process: image acquisition -> face detection / recognition -> if a face is recognized, trigger the protection mechanism (such as blurring, shielding). However, there is an unavoidable processing delay between the acquisition of the image containing the face from the image sensor and the completion of the face recognition by the algorithm and the sending of the instruction. During this delay period, the image containing the clear face information without protection has been obtained by the system and may be cached, transmitted or displayed, causing the risk of user privacy leakage. Therefore, this "first acquisition and recognition, then protection" lag mode is difficult to meet the application scenarios with high requirements for privacy security. SUMMARY

[0004] In order to overcome the problems in the related art, the present specification provides an operating mode switching method and device of an electronic device, and the electronic device and a storage medium.

[0005] According to a first aspect of an embodiment of the present specification, an operating mode switching method of an electronic device is provided, applied to an electronic device with a biological information acquisition module, and the method comprises:

[0006] obtaining historical detection information obtained by the electronic device detecting a target object appearing in a network detection environment within a preset time period;

[0007] obtaining a prediction result of a future behavior of the target object based on the historical detection information;

[0008] switching the operating mode in response to a time being a first preset time length before a time of the future behavior determined based on the prediction result.

[0009] According to a second aspect of an embodiment of the present specification, an operating mode switching device of an electronic device is provided, applied to an electronic device with a biological information acquisition module, and the device comprises:

[0010] an information acquisition unit configured to acquire historical detection information obtained by the electronic device detecting a target object appearing in a network detection environment within a preset time period;

[0011] a result prediction unit configured to determine a prediction result of a future behavior of the target object based on the historical detection information acquisition;

[0012] an action execution unit configured to switch the operation mode in response to a current time being a time before a first preset time length of a time when the future behavior determined based on the prediction result occurs.

[0013] According to a third aspect of the embodiments of the present specification, an electronic device is provided, comprising a wireless communication module, a biological information recognition module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to the first aspect when executing the program.

[0014] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores a computer program, wherein the program is executed by a processor to implement the steps of the method according to the first aspect.

[0015] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:

[0016] The present solution determines the identity of the target object by detecting the device fingerprint in the network environment and analyzing the behavior of the target object, and realizes the upgrading of the privacy protection mechanism, i.e., from passive response to active defense. The core is to build a prediction model based on historical detection information to prospectively analyze the future behavior trajectory of the target object. Based on this prediction, the system can trigger the corresponding mode switching in advance in the key window period before the actual occurrence of the target behavior. This active intervention mechanism significantly reduces the response delay in the traditional passive mode, greatly improves the timeliness and reliability of privacy protection, and thus reduces the risk of privacy leakage from the root.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present specification. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present specification and, together with the specification, serve to explain the principles of the present specification.

[0019] Figure 1 is a flowchart of an operation mode switching method of an electronic device according to an exemplary embodiment of the present specification.

[0020] Figure 2is a schematic diagram of determining an identity of a target object based on an index feature value according to an example embodiment of the present specification.

[0021] Figure 3 is a structural schematic diagram of an electronic device according to an example embodiment of the present specification.

[0022] Figure 4 is a block diagram of a running mode switching device of an electronic device according to an example embodiment of the present specification. DETAILED DESCRIPTION

[0023] The example embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, the same numbers refer to the same or similar elements throughout the drawings. The implementations described in the following example embodiments are not meant to represent all implementations consistent with the present specification. Rather, they are merely examples that are consistent with some aspects of the present specification as detailed in the appended claims.

[0024] The terminology used in the present specification is for the purpose of describing particular embodiments only and is not intended to limit the present specification. As used in the present specification and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0025] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used merely as labels to identify particular information. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present specification. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining" or "in response to ascertaining".

[0026] Currently, electronic security devices such as intelligent door locks and community monitoring systems are widely used in community and home scenarios. The core function of these devices is to identify the whereabouts of strangers and prevent illegal intrusion to protect the safety of the occupants and their property. It is worth noting that such devices generally integrate a biological information collection module: in addition to basic facial recognition, some high-end devices can even capture the physiological characteristics of surrounding organisms (such as heart rate, body posture, etc.).

[0027] However, for community or family members, the electronic security device can monitor the whereabouts of strangers, protect the privacy of biological information of specific persons such as the residents themselves or part of the neighbors, and the residents themselves also hope to avoid data leakage of their own devices.

[0028] The electronic security device can be an electronic device with a biological information collection module. For example, a monitoring device, a smart door lock, etc. The biological information collection module can be a facial information collection module, a biological behavior information collection module, a biological physiological information collection module, etc.

[0029] It can be understood that the electronic device with the biological information collection module has the risk of biological information privacy leakage due to the biological information collection function.

[0030] For example, in an exemplary resident entrance monitoring scenario, a smart door lock or independent monitoring device at the entrance of a family is generally equipped with a biological information collection module. For the sake of property and personal safety, the biological information collection module is always in an open state. However, the application purpose of the biological information collection module is to monitor the behavior of strangers, and the biological information of family members or neighbors does not need to be collected by the biological information collection module, so as to avoid the leakage of their biological information privacy.

[0031] Therefore, it is urgent to identify the object needing privacy protection from the monitored group.

[0032] Next, the embodiments of the present specification will be described in detail.

[0033] Figure 1 It is a flow chart of a running mode switching method of an electronic device according to an exemplary embodiment of the present specification. As shown in the figure, the method can be applied to an electronic device with a biological information collection module, including steps 101-103: Figure 1

[0034] Step 101: obtaining historical detection information obtained by the electronic device for detecting the target object appearing in the network detection environment within a preset time period.

[0035] Step 102: determining the prediction result of the future behavior of the target object based on the historical detection information.

[0036] Step 103: switching the running mode in response to the current time being the time before the first preset time length of the future behavior determined based on the prediction result.

[0037] ​Exemplarily, the preset time period can be the entire time from when the electronic device is initialized to the current time. Then, the prediction can be made based on all historical detection information obtained by detecting the target object appearing in the network detection environment.

[0038] The preset time period can also be a sliding time window of a certain length from the current time. For example, a time period of the last week, month, etc. from the current time. Then, the prediction can be made based on historical detection information obtained by the electronic device detecting the target object appearing in the network detection environment within the sliding time window. By using the latest historical detection information for prediction, the interference of invalid and outdated information is avoided.

[0039] Exemplarily, the future behavior occurrence time can be the time when the predicted target object will appear in the network detection environment. For example, assuming that the future appearance time of the target object is predicted to be 10:00 and the first preset length of time is 5 min, the running mode can be switched at 9:55. By reserving a safety time margin for the prediction result, the influence of prediction error can be reduced.

[0040] Of course, if the first preset length of time is 0, the running mode is switched in response to the current time being the future behavior occurrence time determined based on the prediction result.

[0041] Generally, the privacy protection mechanism is a passive reaction type, which usually identifies the identity of the target object before the target object appears in the monitoring range and executes the privacy protection mode corresponding to the identity of the target object. However, passive protection inevitably has a "time difference", that is, it cannot be guaranteed that the identity of the target object is identified exactly before the target object appears in the monitoring range, and it is possible that the identity of the target object is determined based on sufficient features of the target object at the moment when the target object has already appeared in the monitoring range. Of course, it is also possible that the identity of the target object is determined based on insufficient features of the target object before the target object appears in the monitoring range, but misidentification may still occur due to the insufficiency of the features.

[0042] In the present embodiment, the present scheme accurately determines the identity of the target object and the behavior analysis of the target object by detecting the device fingerprint in the network environment, realizes the paradigm upgrade of the privacy protection mechanism, that is, evolves from passive response to active defense. The core is to construct a prediction model based on historical detection information to prospectively judge the future behavior trajectory of the target object. Based on this prediction, the system can trigger the corresponding privacy protection action in advance in the key window period before the actual occurrence of the target behavior. This active intervention mechanism significantly reduces the response delay in the traditional passive mode, greatly improves the timeliness and reliability of privacy protection, and thus reduces the risk of privacy leakage from the root.

[0043] In an embodiment, the network detection environment can be understood as an effective signal range in which the electronic device can actually detect the object appearing in the surrounding. The network device fingerprint can be understood as a signal detected by the electronic device appearing in the network detection environment where the electronic device is located, which can be used to identify the identity of the network device. The signal can be a signal actively sent by the network device and intercepted by the electronic device, or a signal actively sent by the electronic device and returned after detecting the target object.

[0044] In an embodiment, the historical detection information can be only the network device fingerprint of the network device in the network detection environment, can be only the biological behavior information, or can be information integrating the network device fingerprint and the biological behavior information. The biological behavior information can include moving speed, moving direction, step, step frequency, moving range, and staying time in different spatial regions.

[0045] In an embodiment, the detection information obtained by detecting the network detection environment can be acquired, and the identity of the target object can be determined based on the detection information.

[0046] Exemplarily, the network device fingerprint of the network device in the network detection environment can be acquired, and the device identity can be identified based on the network device fingerprint, and the identity of the target object can be determined based on the network device fingerprint or in combination with the biological behavior information.

[0047] In an embodiment, the target object can be a biological object, which can exemplarily be a person holding the network device, or can be the network device itself.

[0048] If the target object is a person holding the network device, the network device identity and the person identity correspond. After the device identity is identified based on the network device fingerprint, the person identity can be determined according to the device identity. For example, in an exemplary monitoring scene at a resident's door, it is assumed that the network device is connected to the electronic device of the resident based on the network device fingerprint, and it is determined that the device is the device of the resident, and it is determined that the person holding the device is the resident himself.

[0049] If the target object is the device itself, the network device identity can be determined directly. For example, taking a door lock as a neighbor of the network device, and also in the example of monitoring the entrance of the house, if the electronic device detects that the signal strength of the Bluetooth signal of a network device in the detection environment is stable in a fixed interval and the duration is greater than a preset threshold, the network device can be determined as the neighbor's door lock according to these information, and the neighbor's door lock will automatically connect to the neighbor's portable device before the neighbor appears in the monitoring range, or send a sound signal in advance. According to the change of the connection state of the neighbor's door lock or the sound signal, for example, the connection state of the neighbor's door lock is switched from a continuous disconnected state to a connected state, or the door lock sends a sound such as "welcome home" which is captured by the electronic device, the electronic device can further determine that the neighbor will appear in the monitoring range according to these information, and switch the running mode of the electronic device to the mode corresponding to the neighbor's identity.

[0050] In an embodiment, if the identified target object is a biological object, the target object identity can be determined based on the network device fingerprint only, or based on the biological behavior information only, or based on the network device fingerprint and the biological behavior information. For example, based on the network device fingerprint, the device identity of the network device carried by the target person is determined, and the biological behavior information of the biological object is combined to determine the identity of the target person. If the identified target object is a network device, the identity of the network device can be determined based on the network device fingerprint.

[0051] For example, for the scenario where the network device is separated from the person, for example, the neighbor's door lock is not attached to the neighbor, the network device identity can be determined based on the network device fingerprint only. Of course, the identity of the biological object can also be determined by directly detecting the biological behavior information of the biological object in the network detection environment. Or when the biological object and the network device appear together, for example, the person holding the network device, the target object identity can be determined based on the network device fingerprint and the biological behavior information.

[0052] In an embodiment, when the electronic device is provided with a biological information collection module, the running mode includes a privacy protection mode, and when the privacy protection mode is switched, one or more of the following actions is performed:

[0053] The biological information collection module is turned off, the angle of the biological information collection module is adjusted to be misaligned with the target object, the resolution of the biological information collection module is reduced, and the image collected by the biological information collection module is blurred.

[0054] In an example monitoring scenario, such as a monitoring scenario of a household entrance, the operating mode of the electronic device can be switched based on the identity of the target object. For example, if it is not desirable for the monitoring device at the household entrance to collect the biological information of a neighbor, the electronic device can be switched from a regular monitoring mode to a privacy protection mode when it is identified that the target object about to appear in the monitoring range is the neighbor. The privacy protection mode can specifically be that the biological information collection module is turned off, the angle of the biological information collection module is adjusted so as not to be aimed at the specific target object, the resolution of the biological information collection module is reduced, or the image collected by the biological information collection module is blurred.

[0055] In the prior art, in order to meet the demand for privacy protection, the ID of the network device that needs privacy protection is stored in a whitelist. Generally, the network device and the monitoring device are in the same local area network, and the monitoring device can obtain the ID of the network device through the connection mode of the local area network, so as to switch to the privacy protection mode when it is determined that the ID of the network device appears in the whitelist.

[0056] However, the above-mentioned mode can only be used for internal network devices, and cannot identify non-internal network devices that do not establish a connection with the monitoring device. Of course, due to network security reasons, the monitoring device or the local area network where the monitoring device is located does not want to establish any connection with untrusted devices.

[0057] It can be understood that the existing scheme will fail when identifying the identity of non-internal network devices. For example, the prior art scheme cannot be applied to identify the identity information of a neighbor.

[0058] In an embodiment, the network device fingerprint in the network detection environment can be obtained in a non-connected state, and the non-connected state can be that the electronic device does not establish any effective communication link with the network device.

[0059] In this embodiment, the network device fingerprint is obtained in a non-connected manner, which can avoid the risk of data leakage of the electronic device itself, and can determine the identity of the target object based on the obtained network device fingerprint without authorization from others.

[0060] In an embodiment, the non-connected manner can be to detect micro-motion features (such as breathing rate, limb movement) in the network detection environment through a radar module.

[0061] The non-connected mode can also be achieved by the BLE (Bluetooth Low Energy) sniffing function of the Bluetooth device, which is used to capture the data packets transmitted by the network equipment appearing in the network detection environment through the BLE technology. For example, the device identification (Hashed MAC) in the captured broadcast packet, signal strength (RSSI, Received Signal Strength Indicator), service UUID (Universally Unique Identifier) (such as 0xFDEE private service of Apple), and transmission power. Random MAC tracking technology is adopted: the broadcast packet length, carrier frequency offset (CFO, Carrier Frequency Offset) of the same device are used to establish the radio frequency fingerprint (such as the CFO of the Apple device: 5.2 kHz ± 0.1 kHz).

[0062] The non-connected mode can be achieved by the deep analysis function of the Wi-Fi system to analyze the Wi-Fi signals appearing nearby. For example, the SSID (Service Set Identifier) (such as “Starbucks”) is extracted from the Probe Request frame, the Vendor OUI (Organizationally Unique Identifier) (such as Apple 00:17:F2), the supported rate (such as 802.11ax for mobile phones), and the like.

[0063] The non-connected mode can also be achieved by the UWB (Ultra-Wideband) positioning function to measure the distance of the network equipment and thus infer the location of the network equipment.

[0064] Through the above-mentioned manner, the network equipment fingerprint can be obtained based on the non-connected mode without establishing any effective communication link between the electronic device and the network equipment. It should be noted that one or more network equipment fingerprints can be obtained based on one or more of the above-mentioned manners.

[0065] In an embodiment, when acquiring the network device fingerprints in the network detection environment, the network device fingerprints in the network detection environment can be acquired based on multiple non-connection states. In the case of acquiring the network device fingerprints in the network detection environment based on multiple non-connection states, the first network device fingerprints in the network detection environment can be acquired in a non-connection state manner based on a low-power wireless communication module, and it is determined whether to activate a high-precision wireless communication module based on the first network device fingerprints. In the case of needing, the high-precision wireless communication module can be activated, and the second network device fingerprints detected by the high-precision wireless communication module are acquired, and the identity of the target object is determined based on the first network device fingerprints and the second network device fingerprints. The low-power wireless communication module can include a radar and a Bluetooth device; and the high-precision wireless communication module can include a UWB positioning device and a WI-FI system.

[0066] For example, if the behavior information of the object is determined based on the first network device fingerprints, and the behavior information indicates that the object will appear, it is determined that the high-precision wireless communication module needs to be activated. For example, in an exemplary home doorway monitoring scene, a person holding a network device who takes an elevator or other people holding a network device who appear near the corridor but do not walk towards the monitoring range are detected by the low-power wireless communication module. When it is determined that the person will not appear in the monitoring range based on the acquired network device fingerprints, the high-precision wireless communication module can not be activated. For example, assuming that the low-power wireless communication module is Bluetooth, the judgment basis can be that the Bluetooth signal gradually strengthens, it can be determined that someone is approaching nearby, and the high-precision wireless communication module can be activated to further identify the identity of the person. If the signal slightly strengthens and then rapidly weakens, it can be determined that the person is passing by near the monitoring range, and the high-precision wireless communication module can not be activated to further identify the identity of the person.

[0067] In the embodiment, the object appearing in the network detection environment is detected by the low-power wireless communication module, and the high-precision wireless communication module is started only when it is determined that the object needs to be identified to acquire the network device fingerprints in the network detection environment together with the low-power wireless communication module. On the one hand, the energy consumption of the high-precision wireless communication module can be saved, and the high-precision wireless communication module is not started unnecessarily. On the other hand, the identity of the target object can be identified based on multiple detection results of multiple wireless communication modules, and the accuracy of identification can be improved.

[0068] In an embodiment, the identity of the network device is identified based on the network device fingerprints, and the identity of the target object is determined; or the identity of the target object is determined in combination with biological behavior information.

[0069] In the process of identifying the network device identity based on the network device fingerprint, at least one feature index value of at least one dimension of device features can be extracted from the network device fingerprint, and the device identity of the corresponding network device can be determined according to the at least one feature index value.

[0070] The network device fingerprint can include physical layer information, protocol layer information and behavior layer information of the network device. Exemplarily, the dimensions of the device features can be divided into physical dimensions, protocol dimensions and behavior dimensions, wherein: the feature indexes of the physical features of the physical dimensions include carrier frequency offset, modulation error and signal strength fluctuation standard deviation; the feature indexes of the protocol features of the protocol dimensions include BLE service and Wi-Fi probe; and the feature indexes of the behavior features of the behavior dimensions include time period activity, stay duration and movement trajectory.

[0071] Wherein, the extraction manners of the carrier frequency offset, the modulation error and the signal strength fluctuation standard deviation are not subject to any limitation, the BLE service can exemplarily extract UUID, and the Wi-Fi probe can exemplarily extract Wi-Fi channel width or OUI.

[0072] And the time period activity, the stay duration and the movement trajectory can be indexes obtained by comprehensive calculation according to the detection results, and are used for analyzing the behavior information of the target object.

[0073] Exemplarily, the time period activity can calculate the active interval of the target object within 24 hours through -∑(p*log2(p)), so as to capture the regularity of the appearance time of the target object. Wherein, P can be the probability of appearing in a specific time interval. Wherein, the smaller the time period activity is, the more fixed the time period of the appearance of the target object is.

[0074] Exemplarily, the stay duration coefficient of variation can be represented by the ratio of the standard deviation and the average value of the stay duration of the same network device within a week. This index can be used to analyze the regularity of the stay duration of the target object.

[0075] Exemplarily, the movement trajectory smoothness can be measured by the ratio of the straight line distance and the actual path length of the target object. The closer the ratio is to 1, the closer the movement path of the target object is to a straight line, and the closer the ratio is to 0, the greater the degree of detour of the movement path of the target object is.

[0076] In an embodiment, in the process of determining the device identity of the corresponding network device according to the feature index value, at least one feature index value can be fused to obtain a first fusion index value of the corresponding dimension, and the device identity of the corresponding network device can be determined according to the first fusion index value of the corresponding dimension.

[0077] Exemplarily, if the device features are divided into one dimension, and there is only one feature index value in the dimension, the device identity of the corresponding network device can be determined directly based on the feature index value, and the fusion processing is to assign a weight of 1 to the feature index value.

[0078] Exemplarily, if the device features are divided into one dimension, and there are multiple feature index values in the dimension, the multiple feature index values can be fused to obtain a first fusion index value, and the device identity of the corresponding network device is determined according to the first fusion index value.

[0079] Exemplarily, if the device features are divided into multiple dimensions, and there is at least one feature index value in each dimension, at least part of the feature index values of at least part of the dimensions can be selected for fusion processing to obtain a first fusion index value of the corresponding dimension, and the device identity of the corresponding network device is determined according to the first fusion index value of the corresponding dimension.

[0080] For example, for A dimension, a1, a2 and a3 are included in the dimension; for B dimension, b1, b2 are included in the dimension; for C dimension, c1, c2 and c3 are included in the dimension. The first fusion index value of A dimension can be determined according to a1 and a2; the first fusion index value of B dimension can be determined according to b1 and b2; the first fusion index value of C dimension can be determined according to c1, c2 and c3, and the device identity of the corresponding network device is determined according to the first fusion index values of the respective dimensions.

[0081] In an embodiment, when the device identity of the corresponding network device is determined according to the first fusion index value of the corresponding dimension, the current environment scenario category can be determined, the first weight of each first fusion index value corresponding to the current environment scenario category is obtained, or the first weight of each first fusion index value is adaptively determined. The first fusion index values are fused according to the first weight to obtain a second fusion index value, and the device identity of the corresponding network device is determined according to the second fusion index value.

[0082] For example, the environment scenario can be divided into night mode, strong interference environment mode and long-term monitoring mode. For night mode, the physical layer weight is greater than the behavior layer weight, which is greater than the protocol layer weight; for strong interference environment mode, the physical layer weight is greater than or equal to the protocol layer weight, which is greater than the behavior layer weight; for long-term monitoring mode, the physical layer weight is less than the protocol layer weight, which is less than the behavior layer weight. The environment scenario category can be set by the user on the terminal, and the electronic device can fuse the first fusion index values of the respective dimensions using the weight corresponding to the environment scenario category set by the user to obtain a second fusion index value.

[0083] In an embodiment, when weighting, the confidence can also be weighted, as shown in the following formula:

[0084]

[0085] wherein, is a second fusion index value, is a weight of the kth layer dimension in the fusion process, is a confidence of the first fusion index value of the kth layer dimension, is the first fusion index value of the kth layer dimension.

[0086] In an embodiment, when determining the device identity of the corresponding network device according to the feature index value, the recognition result can be generated according to the feature index value, and the device identity of the corresponding network device is determined according to the recognition result.

[0087] The recognition result can be represented by any feature index value. For example, the size of the feature index value of the device feature of any dimension can be used as the recognition result.

[0088] The recognition result can also be represented by the first fusion index value or the second fusion index value. For example, the first fusion index value obtained by fusing a plurality of feature index values of any dimension can be used as the recognition result. For example, the second fusion index value obtained by fusing the first fusion index values of different dimensions can be used as the recognition result.

[0089] For example, in the exemplary monitoring scene of the home door, the recognition result can be normalized in [0, 1], assuming that [0, 0.3] corresponds to the identity of the household, (0.3, 0.6] corresponds to the identity of the neighbor, and (0.6, 1] corresponds to the identity of the other person. Then the identity of the corresponding target object can be determined according to the interval in which the recognition result is located. For example, if the recognition result is 0.45, the identity of the target object can be determined as a neighbor. The interval size of the value is closely related to the application scenario, and a person skilled in the art can reset the identity type corresponding to the numerical interval in which the different recognition results are located according to the actual needs.

[0090] Specifically, as shown in Figure 2 assuming that at least one feature index value of the physical dimension, the protocol dimension and the behavior dimension is extracted from the network device fingerprint. Any feature index value can be used alone as a recognition result for identity recognition. For example, the feature index value 1 of the physical dimension is 0.45, and the identity of the target object can be determined as a neighbor.

[0091] The plurality of characteristic indicator values in any dimension can be fused to obtain a first fused indicator value. Any fused indicator value can be used as a recognition result for identity recognition. For example, if the first fused indicator value corresponding to the physical dimension is 0.45, the identity of the target object can be determined as a neighbor. Or, if the first fused indicator value corresponding to the protocol dimension is 0.45, the identity of the target object can be determined as a neighbor. When performing fusion processing, the plurality of characteristic indicator values can be weighted, and higher weights are given to characteristic indicator values with higher importance, and lower weights are given to characteristic indicator values with lower importance.

[0092] The plurality of first fused indicator values can be fused to obtain a second fused indicator value. The second fused indicator value can be used as a recognition result for identity recognition. For example, if the second fused indicator value is 0.45, the identity of the target object can be determined as a neighbor. When performing fusion processing, the plurality of first fused indicator values can be weighted, and higher weights are given to first fused indicator values with higher importance, and lower weights are given to first fused indicator values with lower importance.

[0093] In an embodiment, for the recognition results determined at adjacent time instants in a continuous time period, if the difference between the recognition result at the current time instant and the recognition result at the previous time instant is greater than a preset threshold, the recognition result at the current time instant is adjusted according to a time decay strategy based on the recognition result at the previous time instant. The identity of the target object at the current time instant is determined according to the adjusted recognition result.

[0094] The recognition result at the current time instant can be adjusted based on the recognition result at the previous time instant by the following formula:

[0095]

[0096] wherein, is the recognition result at time instant t, is the recognition result at time instant t-1, is the difference between t and t-1.

[0097] For example, if network device fingerprints are obtained based on multiple wireless communication modules, and the recognition results of different wireless communication modules conflict, time decay arbitration can be introduced to prevent errors caused by conflicts.

[0098] For example, as shown in Table 1:

[0099] Table 1

[0100]

[0101] ​​​​In Table 1, at the moment t=0, the decision result based on BLE is that the decision state is to be confirmed, and the target object identity is not directly determined according to the current device fingerprint; at the moment t=2, the monitoring can be continued, and the decision based on UWB is that, as time elapses, the weight of the decision result of BLE at the previous moment in the UWB decision is reduced, and through continuous adjustment, the conflict between each other can be prevented. At the moment t=5, the response is terminated, and the monitoring is stopped.

[0102] In an embodiment, when the device identity is identified based on the network device fingerprint, the connection relationship between the target object and the electronic device can be determined according to the historical detection information of the target object, and the identity of the target object is determined based on the connection relationship and the historical detection information of the target object.

[0103] Specifically, in the case of the connection relationship being connected, whether the target object is a resident is determined according to the historical detection information of the target object; in the case of the connection relationship being unconnected, whether the target object is a neighbor is determined according to the historical detection information of the target object.

[0104] In an embodiment, after the network device fingerprint in the network detection environment is obtained, the current connection state between the network device and the electronic device can also be determined based on the network device fingerprint. When the device identity of the corresponding network device is determined based on the network device fingerprint, in response to the current connection state being connected, the device identity of the network device is determined as a first type of device; and in response to the current connection state being unconnected, the device identity of the network device is determined as a second type of device.

[0105] In an exemplary home doorway monitoring scene, the first type of device is a resident's device, and the second type of device is a device of other people.

[0106] In this embodiment, it is assumed that only residents can establish a connection with the electronic device, and based on the current connection state between the network device and the electronic device, the device that establishes a connection with the electronic device can be accurately identified as a resident's device.

[0107] In an embodiment, after the current connection state is determined to be unconnected, the historical activity of the network device can also be determined based on the network device fingerprint, and in the case that the historical activity is greater than a preset threshold, the network device is determined as a third type of device, wherein the third type of device is a neighbor's device.

[0108] In combination with the above two embodiments, the resident's device and the device of other people are distinguished by the connection state, and the neighbor's device can be further identified from the device of other people by the historical activity. In this way, without establishing a connection with any network device, the resident's device and the neighbor's device can also be identified.

[0109] The historical activity level can be measured by the period activity level, the coefficient of variation of the stay duration, and other characteristic indexes in the foregoing embodiments. If the period activity level is used for measurement, the historical activity level is the reciprocal of the period activity level. For example, if the period activity level is less than 1.5, it can be determined that the device of the neighbor. If the coefficient of variation of the stay duration is used for measurement, the historical activity level can be the reciprocal of the coefficient of variation of the stay duration. For example, if the coefficient of variation of the stay duration is less than 0.1, it can be determined that the device of the neighbor.

[0110] Of course, the present solution is not limited to the identity of other objects determined based on the network device fingerprint. For example, the identity of a service personnel can also be determined based on the UUID in the network device fingerprint.

[0111] In an embodiment, the network device fingerprint appearing in the network detection environment within a preset time period can be acquired, the device identity of the corresponding network device is determined based on the network device fingerprint, the management list having a mapping relationship with the network device identity is created based on the device identity, and the network device is added to the corresponding management list.

[0112] Exemplarily, the network device fingerprint appearing in the network detection environment within a preset time period can be the network device fingerprint in the network detection environment acquired in real time at present, or the network device fingerprint of the same network device in a historical time period.

[0113] In an embodiment, the network device fingerprint in the network detection environment is acquired, if the network device corresponding to the network device fingerprint is in any management list, the identity of the target object is determined as the identity corresponding to the any management list. If the network device corresponding to the network device fingerprint is not in any management list, the identity of the target object is determined based on the network device fingerprint.

[0114] In the present embodiment, different management lists correspond to the identities of different objects. After the device identity of the network device is determined, the network device can be added to the corresponding management list. When the network device in the management list appears again next time, the identity of the target object can be determined based on the management list, without the need to determine the identity of the target object based on the network device fingerprint again. On the one hand, the efficiency of determining the identity of the repeatedly appearing target object can be improved, and on the other hand, the error of determining the identity of the target object based on the network device fingerprint can be reduced.

[0115] In an embodiment, when the network device is added to the corresponding management list, the network device fingerprint representing the network device identity satisfying a first preset condition can be added to a first list; and the network device fingerprint representing the network device identity satisfying a second preset condition can be added to a second list.

[0116] Exemplarily, the network device fingerprint can uniquely identify the network device, and whether the network device appears in the management list can be determined by matching the network device fingerprint obtained in the network detection environment with the network device fingerprint stored in the management list.

[0117] Exemplarily, the first list can be a permanent white list for storing the network device fingerprint of the network device of the resident, and the second list can be a neighbor list for storing the network device fingerprint of the network device of the neighbor.

[0118] The first preset condition can be that the network device establishes a connection with the electronic device, or that the network device establishes a connection with the electronic device and the trajectory smoothness is greater than a preset value, or that the network device establishes a connection with the electronic device and an opening door action is detected within a preset time length. For example, the network device establishes a connection with the electronic device, which indicates that the network device is the device of the resident. For another example, the network device establishes a connection with the electronic device and the trajectory smoothness is greater than a preset threshold value, which indicates that the network device is in a straight line and approaches the electronic device. For another example, the network device establishes a connection with the electronic device and an opening door action is detected within a preset time, which can double-verify that the network device establishing the connection is the device of the resident.

[0119] According to the network device fingerprint, a heat map corresponding to the network device is generated, and the heat map is used to represent the time sequence aggregation feature and / or the area aggregation feature of the appearance of the corresponding network device. The second preset condition can be that the network device does not establish a connection with the electronic device and appears n times in a fixed time period. The fixed time period can be measured in the form of a heat map, and whether the network device appears n times in the fixed time period and the fixed area can be determined by measuring the similarity of the heat maps of the n appearances. The timestamp of the appearance of the network device can also be recorded, and only whether the network device appears n times in the fixed time period can be determined.

[0120] In an embodiment, the user can also manually add the network device to the management list. In response to a network device adding instruction containing a network device fingerprint, the network device is added to the corresponding management list.

[0121] The user can know the identity of the device, for example, the user can add the device of a family member to the corresponding management list.

[0122] The user does not know the identity of the device, for example, the user can search for a non-family member Bluetooth signal appearing nearby, and input the network device fingerprint of the network device sending the Bluetooth signal to the electronic device, so as to determine the corresponding management list and add the network device to the corresponding management list.

[0123] In an embodiment, the management list can further include an observation list, and when a network device is added to the corresponding management list, a network device fingerprint representing the identity of the network device meeting a third preset condition can be added to the observation list.

[0124] For example, the observation list can be a service personnel list, such as a delivery personnel list. The third preset condition can be that the network device fingerprint contains a UUID, and the UUID is a UUID exclusive to the service personnel. Of course, the observation list can also be a list of other personnel, and the third preset condition can be adjusted to correspond to the identity of the other personnel.

[0125] In an embodiment, for each network device in the observation list, the network device is removed from the management list when the network device meets a fourth preset condition.

[0126] For example, the fourth preset condition can be that there is no second occurrence within a preset time period in the future.

[0127] In an embodiment, the management list further includes a temporary list, and when a network device is added to the corresponding management list, the network device can be added to the temporary list, and the frequency of occurrence of the network device is updated. When the frequency of occurrence is greater than a frequency threshold, the network device is added to the management list corresponding to the identity of the personnel.

[0128] In this embodiment, by judging the frequency of occurrence in the temporary list, accidental occurrence or misidentification of the network device is prevented from being added to the management list, thereby improving the accuracy of the identity of the network device in the management list.

[0129] In an embodiment, different operation modes are executed based on different management lists.

[0130] For example, in an exemplary home doorway monitoring scenario, the management list can include a neighbor list and a resident list. In actual application, the operation mode can include a normal mode and a privacy protection mode. If a network device in the resident list is detected at the doorway of the home, the privacy protection mode is executed for the electronic devices indoors and outdoors. If a network device in the neighbor list is detected at the doorway of the home, the privacy protection mode is executed for the electronic devices outdoors. In other cases, the operation mode can be the normal mode. The privacy protection mode can be to turn off the biological information acquisition module, adjust the angle of the biological information acquisition module so as not to be aligned with the target object, reduce the resolution of the biological information acquisition module, and blur the image collected by the biological information acquisition module.

[0131] In actual application scenarios, the electronic device can perform real-time detection on the target object appearing in the network detection environment to obtain detection information, and determine the identity of the target object based on the current obtained detection information, and trigger the execution of a privacy protection action corresponding to the identity.

[0132] For a target object that appears for the first time, the identity of the target object can be determined in a timely manner based on the real-time acquired detection information since there is no historical detection information of the target object. The electronic device can work without waiting for accumulated historical data.

[0133] After the electronic device identifies the identity of the target object each time, the acquired detection information, the identity information of the target object, and the time stamp can be stored for subsequent use, such as Figure 1 The method based on historical detection information predicts the future behavior of the target object. For example, the timing of the future appearance of the target object is predicted.

[0134] When determining the prediction result of the future behavior of the target object based on historical detection information, there can be multiple prediction methods, and multiple prediction results are obtained:

[0135] In an embodiment, the identity of the target object can be pre-divided into different types. For example, in an exemplary home doorway monitoring scenario, the household members and neighbors can be regarded as target objects, when a household member is identified, the indoor and outdoor electronic devices can be switched to run in the mode, and when a neighbor is identified, the outdoor electronic device can be switched to run in the mode.

[0136] After detecting the target object appearing in the network detection environment each time to obtain detection information, the identity of the target object is determined according to the real-time acquired detection information, and a privacy protection action corresponding to the identity of the target object is executed. Then, the identity type of the target object corresponding to the detection information and the time stamp of the appearance of the target object can be recorded.

[0137] The recorded information content can be as shown in Table 2:

[0138] Table 2

[0139]

[0140] The above is the identification result obtained by real-time detection on different target objects appearing in succession in a continuous time period.

[0141] Based on the above data, the prediction model can predict the identity type and the time of appearance of the next appearing object. For example, it is predicted that a neighbor will appear on * year * month * day * hour * minute, or it is predicted that a household member will appear on * year * month * day * hour * minute.

[0142] In the embodiment, the continuous historical data can reveal the periodic pattern of the target object appearing. And the context correlation of the objects with different identities can also be captured by the above-mentioned prediction method. For example, the inhabitant B appears ten minutes after the neighbor A appears each time. At this time, the prediction of the appearing time of the inhabitant B can also be facilitated by the regularity of the neighbor A appearing.

[0143] In another embodiment, the device identity can be identified based on the network device fingerprint, and the target object identity is determined, or the target object identity is determined in combination with the biological behavior information. The future behavior of the target object meeting the preset condition is predicted. The preset condition can be the identity type requiring privacy protection. For example, when the identity of the target object needs to be protected, the future behavior of the object can be predicted, and the next appearing time of the object can be predicted based on all historical detection information of the same target object.

[0144] Exemplarily, when the prediction result of the future behavior of the target object is determined based on the historical detection information, the prediction result can be the future appearing time and the action type of the target object obtained by the behavior analysis model. The behavior analysis model is used to obtain the historical detection information of the target object in a preset time period, extract the spatio-temporal features and behavior features of the historical detection information, and construct a multi-dimensional feature vector. The spatio-temporal features can include time and space features, and the behavior features can be the connection relationship between the target object and the electronic device determined according to the historical detection information of the target object. A time series prediction model is trained based on the multi-dimensional feature vector, and the model associates the behavior features of the target object with the spatio-temporal context relationship. According to the real-time collected behavior features of the target object, the behavior analysis model outputs the probability distribution of the future appearing time and the action type of the target object.

[0145] For example, different action types can correspond to different types of target objects. For example, if the action type represents the moving rule that the target object first approaches the electronic device and then moves away from the electronic device, it can reflect that the target object is a neighbor. For another example, if the action type represents the moving rule that the target object gradually approaches the electronic device, it can reflect that the target object is a resident. The action type with the highest probability can be determined according to the probability distribution of the action type, and the identity of the target object corresponding to the action type can be further determined according to the action type, so that after the identity of the target object is determined, the corresponding privacy protection mode can be adaptively started.

[0146] Corresponding to the embodiments of the above-mentioned method, the present specification also provides embodiments of devices and terminals to which the devices are applied.

[0147] Figure 3 is a structural schematic diagram of an electronic device according to an exemplary embodiment. As shown in Figure 3As shown, at the hardware level, the electronic device 300 includes a biological information collection module, a processor 302, an internal bus 304, a network interface 306, a memory 308, and a non-volatile memory 310, and of course can also include other hardware required by the business. One or more embodiments of the present specification can be implemented in a software manner, such as reading a corresponding computer program from the non-volatile memory 310 into the memory 308 by the processor 302 and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logical module, but can also be hardware or logic devices.

[0148] Figure 4 is a block diagram of an operating mode switching apparatus of an electronic device according to an exemplary embodiment of the present specification. As shown, the apparatus can be applied to the electronic device 300 as shown in Figure 4 to implement the technical solutions of the present specification. The apparatus includes: Figure 3

[0149] An information acquisition unit 402 is configured to acquire historical detection information obtained by the electronic device detecting a target object appearing in a network detection environment within a preset time period.

[0150] A result prediction unit 404 is configured to determine a prediction result of a future behavior of the target object based on the historical detection information acquisition.

[0151] An action execution unit 406 is configured to switch the operating mode in response to a time point being a first preset time length before a time point at which the future behavior determined based on the prediction result occurs.

[0152] Optionally, the target object includes a network device and / or a living being. The historical detection information includes one or more of network device fingerprints and living being behavior information. The network device fingerprint includes physical layer information, protocol layer information, and behavior layer information of the network device.

[0153] Optionally, the network device fingerprint in the network detection environment is acquired in a non-connected state manner; the non-connected state is that the electronic device and the network device do not establish any effective communication link.

[0154] Optionally, the apparatus further includes a target object identity determination unit configured to identify the device identity and determine the target object identity based on the network device fingerprint before determining the prediction result of the future behavior of the target object based on the historical detection information acquisition; or determine the target object identity in combination with the living being behavior information.

[0155] ​The result prediction unit 404 is further configured to predict a future behavior of the target object that meets a preset condition.

[0156] Optionally, the target object identity determination unit is specifically configured to determine a connection relationship between the target object and the electronic device according to historical detection information of the target object, and determine the identity of the target object based on the connection relationship and the historical detection information of the target object.

[0157] Optionally, the prediction result is a future occurrence time and an action type of the target object output by a behavior analysis model. The behavior analysis model is configured to: acquire historical detection information of the target object within a preset time period; extract a spatio-temporal feature and a behavior feature of the historical detection information, and construct a multi-dimensional feature vector; train a time series prediction model based on the multi-dimensional feature vector, the model being associated with a spatio-temporal context relationship of the behavior feature of the target object; and output a probability distribution of a future occurrence time and an action type of the target object according to a real-time collected behavior feature of the target object by the behavior analysis model.

[0158] Optionally, when the electronic device is provided with a biological information collection module, the operation mode includes a privacy protection mode, and when the privacy protection mode is switched, one or more of the following actions is performed: the biological information collection module is turned off, the angle of the biological information collection module is adjusted to be misaligned with the target object, the resolution of the biological information collection module is reduced, and the image collected by the biological information collection module is blurred.

[0159] The implementation process of the functions and roles of the modules in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0160] For the device embodiment, since it basically corresponds to the method embodiment, the related parts are described in the part of the method embodiment. The above-described device embodiment is only illustrative, and the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., they can be located in one place or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purpose of the scheme of the present specification. Those skilled in the art can understand and implement it without creative labor.

[0161] The present specification also provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the operation mode switching method of the above-described electronic device.

[0162] In particular, computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0163] The specification also provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the operating mode switching method of any of the aforementioned electronic devices.

Claims

1. A method for switching operating modes of an electronic device, characterized in that, include: Acquire historical detection information obtained by electronic devices detecting target objects appearing in the network detection environment within a preset time period; The historical detection information includes one or more of network device fingerprints and biometric behavioral information; the network device fingerprint includes physical layer information, protocol layer information, and behavioral layer information of the network device; the network device fingerprint in the network detection environment is obtained in a disconnected state; the disconnected state means that no effective communication link has been established between the electronic device and the network device; The prediction result of the future behavior of the target object based on the historical detection information includes: identifying the device identity and determining the identity of the target object based on the network device fingerprint; predicting the future behavior of the target object that meets preset conditions; the identification of the device identity and determination of the target object based on the network device fingerprint also includes: determining the connection relationship between the target object and the electronic device based on the historical detection information of the target object; and judging the identity of the target object based on the fusion of the connection relationship and the historical detection information of the target object, wherein the connection relationship includes connected and not connected. The operating mode is switched at a time when the current moment is a first preset time before the time of the future behavior determined based on the prediction results.

2. The method according to claim 1, characterized in that, The prediction result is the future occurrence time and action type of the target object obtained through a behavior analysis model. The behavior analysis model is used for: Extract the spatiotemporal and behavioral features of the historical exploration information to construct a multidimensional feature vector; A time-series prediction model is trained based on the multi-dimensional feature vectors, and the model associates the behavioral features of the target object with the spatiotemporal context. Based on the real-time collected behavioral characteristics of the target object, the probability distribution of the future appearance time and action type of the target object is output through the behavior analysis model.

3. The method according to claim 1, characterized in that, When the electronic device is equipped with a bio-information acquisition module; The operating mode includes a privacy protection mode. When the operating mode is switched to privacy protection mode, one or more of the following actions are performed: The biological information acquisition module can be turned off, its angle adjusted to avoid aligning with the target object, its resolution reduced, and the images acquired by the biological information acquisition module blurred.

4. A switching device for the operating mode of an electronic device, characterized in that, include: The information acquisition unit is used to acquire historical detection information obtained by electronic devices detecting target objects appearing in the network detection environment within a preset time period. The historical detection information includes one or more of network device fingerprints and biometric behavioral information; the network device fingerprint includes physical layer information, protocol layer information, and behavioral layer information of the network device; the network device fingerprint in the network detection environment is obtained in a disconnected state; the disconnected state means that no effective communication link has been established between the electronic device and the network device; The result prediction unit is used to determine the prediction result of the future behavior of the target object based on the historical detection information, including: identifying the device identity and determining the identity of the target object based on the network device fingerprint; predicting the future behavior of the target object that meets preset conditions; the identification of the device identity and determination of the target object based on the network device fingerprint also includes: determining the connection relationship between the target object and the electronic device according to the historical detection information of the target object; and judging the identity of the target object based on the fusion of the connection relationship and the historical detection information of the target object, wherein the connection relationship includes connected and not connected. An action execution unit is used to switch operating modes in response to a time when the current time is a first preset time before the time of occurrence of the future behavior determined based on the prediction result.

5. An electronic device, characterized in that, include: Bioinformatics acquisition device, processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-3 by executing the executable instructions.

6. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-3.

Images