Data processing method, device, electronic device and storage medium
By detecting the user's status and entering the corresponding security mode through smart wearable devices, and performing protection operations, the problem of existing devices being unable to provide real-time protection is solved, and the user's security is improved under different activity states.
Patent Information
- Application Number
- CN202011379910.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-11-30
AI Technical Summary
Existing smart wearable devices lack a security mode and cannot implement protective measures in real time based on the user's status.
By acquiring data such as movement speed, EEG signals, heart rate, blood pressure, body surface temperature, and angular velocity of key parts detected by smart wearable devices, the current state is determined, and the corresponding safety mode is entered, such as sports safety mode, driving safety mode, or sleep safety mode, and corresponding protective operations are performed.
It enables real-time protection based on user status, improving user safety in different activity states. For example, it avoids accidents and unsafe situations through vibration, prompts, and wake-up.
Smart Images

Figure CN114639113B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of smart wearable devices, and in particular to a data processing method, device, electronic device and storage medium. Background Art
[0002] With the development of the Internet, various smart wearable devices have appeared in people's lives, such as smart watches, smart bracelets, smart Bluetooth headsets, etc. With the gradual popularization of artificial intelligence in people's lives, the functions of smart wearable devices are gradually increasing. However, the functions of most smart wearable devices in the prior art are limited to the user's life and work. For example, in terms of security, smart wearable devices in the prior art do not have a security mode and cannot perform corresponding protection measures on users in real time according to the user's status. Summary of the invention
[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a data processing method, device, electronic device and storage medium.
[0004] According to one aspect of an embodiment of the present application, a data processing method is provided, which is applied to a smart wearable device, including:
[0005] Acquire a detection data set obtained by the smart wearable device from detecting the current wearable object, wherein the detection data set includes multiple detection data, and the detection data includes: movement speed, electroencephalogram signal, heart rate, blood pressure, body surface temperature and angular velocity of key parts;
[0006] Acquire at least one candidate state corresponding to the detection data, and take a candidate state with a probability greater than a preset probability as a current state, wherein the current state includes: a motion state, a driving state, and a sleeping state;
[0007] Determine a safety mode corresponding to the current state, the safety mode including: a sports safety mode, a driving safety mode, and a sleeping safety mode;
[0008] Control the smart wearable device to perform a protection operation on the current wearable object according to the safety mode.
[0009] Further, when the safety mode is a sports safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes:
[0010] Acquire a current frame image captured by the smart wearable device, where the current frame image includes at least one object;
[0011] Performing differential processing on the current frame image using a previous frame image of the current frame image to obtain a target foreground image;
[0012] Determine a region of interest in the target foreground image, where the region of interest is obtained according to a gradient direction histogram feature of the target foreground image, and the region of interest carries at least one key part;
[0013] Tracking the object according to the region of interest to obtain the object quantity of the object;
[0014] A warning instruction is generated according to the number of objects, and the warning instruction is used to control the smart wearable device to perform a vibration operation according to a target vibration frequency corresponding to the number of objects.
[0015] Furthermore, tracking the object according to the region of interest to obtain the number of the object includes:
[0016] Extracting color features of the region of interest;
[0017] Determine a first peak sidelobe ratio and a first correlation graph of the gradient direction histogram feature, and a second peak sidelobe ratio and a second correlation graph of the color feature;
[0018] According to the first peak sidelobe ratio, the second peak sidelobe ratio, the first correlation graph, and the second correlation graph, a fusion weight ratio of the color feature in a target frame image is determined, where the target frame image is a next frame image of the current frame image, and a calculation formula for the fusion weight ratio is as follows:
[0019]
[0020] In the formula, k CN is the fusion weight ratio of the color features in the current frame image, PSLR HOG,s is the first peak sidelobe ratio in the current frame image, PSLR CN,s is the second peak sidelobe ratio in the current frame image, f HOG,s is the first correlation graph corresponding to the current frame image, f CN,s is the second correlation graph corresponding to the current frame image;
[0021] Determining the fusion weight of the color feature in the target frame image according to the fusion weight ratio;
[0022] Determine a target correlation graph of the gradient direction histogram feature and the color feature in the target frame image according to the fusion weight, the first correlation graph, and the second correlation graph;
[0023] The number of objects in the target frame image is determined according to the target correlation graph.
[0024] Further, determining the number of objects in the target frame image according to the target correlation graph includes:
[0025] determining a target peak value of the target correlation graph;
[0026] Determine a target region of interest in the target frame image according to the target peak value;
[0027] The number of objects is determined according to the number of key parts in the target region of interest.
[0028] Further, when the safety mode is a driving safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes:
[0029] Real-time detection of EEG data of the current wearer;
[0030] Extracting frequency domain feature information from the EEG data;
[0031] Inputting the frequency domain feature information into an analysis model, and obtaining the attention state of the current wearer according to the feature type corresponding to the frequency domain feature information by the analysis model;
[0032] A corresponding protection operation is performed on the current wearing object according to the attention state, where the attention state includes: focused attention, slightly distracted attention, and severely distracted attention.
[0033] Further, performing corresponding protection operations on the current worn object according to the attention state includes:
[0034] When the attention state is slightly distracted, controlling the smart wearable device to perform indirect vibration according to a first vibration frequency, and sending a first prompt message to a vehicle screen that is communicatively connected to the smart wearable device;
[0035] The performing corresponding protection operations on the current wearing object according to the attention state also includes:
[0036] When the attention state is severely distracted, the smart wearable device is controlled to perform continuous vibration according to the second vibration frequency, and a second prompt message is sent to the vehicle screen that is communicatively connected to the smart wearable device.
[0037] Further, when the safety mode is a sleep safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes:
[0038] Obtaining current displacement data of the smart wearable device;
[0039] When it is determined according to the current displacement data that the sleep condition of the current wearer is an abnormal sleep condition, a wake-up instruction is generated, and the wake-up instruction is used to wake up the current wearer according to a preset vibration frequency.
[0040] Further, when it is determined according to the current displacement data that the sleep condition of the current wearer is an abnormal sleep condition, generating a wake-up instruction includes:
[0041] Get historical displacement data;
[0042] Determine displacement change information according to the current displacement data and the historical displacement data;
[0043] When the displacement change information is greater than the preset change information, it is determined that the sleep condition of the current wearer is an abnormal sleep condition, and the wake-up instruction is generated.
[0044] According to another aspect of the embodiment of the present application, there is also provided a data processing device, which is applied to a smart wearable device, including:
[0045] An acquisition module is used to acquire a detection data set obtained by the smart wearable device from detecting the current wearable object, wherein the detection data set includes multiple detection data, and the detection data includes: movement speed, EEG signal, heart rate, blood pressure, body surface temperature and angular velocity of key parts;
[0046] an analysis module, configured to obtain at least one candidate state corresponding to the detection data, and take a candidate state with a probability greater than a preset probability as a current state, wherein the current state includes at least one of the following: a motion state, a driving state, and a sleeping state;
[0047] A determination module, configured to determine a safety mode corresponding to the current state, wherein the safety mode includes at least one of the following: a sports safety mode, a driving safety mode, and a sleeping safety mode;
[0048] A control module is used to control the smart wearable device to perform a protection operation on the current wearable object according to the safety mode.
[0049] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.
[0050] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.
[0051] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.
[0052] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages compared with the prior art: the embodiment of the present application determines the current state of the current worn object by detecting the detection data set obtained by detecting the current worn object, and the smart wearable device enters the corresponding safety mode according to the current state, thereby being able to provide different protection operations for the current worn object according to different states, thereby realizing real-time execution of corresponding protection measures for the user according to the user's state. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0055] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;
[0056] Figure 2 A flowchart of a data processing method provided in another embodiment of the present application;
[0057] Figure 3 A flowchart of a data processing method provided in another embodiment of the present application;
[0058] Figure 4 A flowchart of a data processing method provided in another embodiment of the present application;
[0059] Figure 5 A block diagram of a data processing device provided in an embodiment of the present application;
[0060] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0062] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another similar entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0063] The embodiments of the present application provide a control method, device and system for smart home devices. The method provided by the embodiments of the present invention can be applied to any required electronic device, for example, it can be a server, terminal and other electronic devices, which are not specifically limited here. For the convenience of description, they are referred to as electronic devices in the following.
[0064] According to one aspect of the embodiments of the present application, a method embodiment of a data processing method is provided, and the method is applied to a smart wearable device. Figure 1 A flowchart of a data processing method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:
[0065] Step S11, obtaining a detection data set obtained by the smart wearable device from detecting the current wearable object;
[0066] In the embodiment of the present application, the smart wearable device may be a smart bracelet, a smart watch, a smart foot ring, a smart Bluetooth headset, etc. The current wearable object is the user currently wearing the smart wearable device. The detection data set includes a plurality of detection data, and the detection data includes: movement speed, EEG signal, heart rate, blood pressure, and angular velocity of key parts. The angular velocity of key parts includes: angular velocity of wrist, angular velocity of ankle joint, etc.
[0067] Step S12, obtaining at least one candidate state corresponding to the detection data, and taking a candidate state with a probability greater than a preset probability as a current state, the current state including: a motion state, a driving state, and a sleeping state;
[0068] In the embodiment of the present application, step S12 includes the following steps A1-A3:
[0069] Step A1, obtaining at least one candidate state corresponding to the detection data;
[0070] In an embodiment of the present application, at least one candidate state corresponding to each detection data in the detection data set is obtained, and the candidate states include: motion state, driving state and sleeping state.
[0071] For example, at this time, the candidate states corresponding to the motion speed include: motion state, driving state, etc.; the candidate state corresponding to the heart rate is the motion state; the candidate state corresponding to the body surface temperature is the motion state; and the candidate state corresponding to the angular velocity of the wrist is the motion state.
[0072] Step A2, determining the probability of occurrence of the candidate state;
[0073] Step A3: taking the candidate state whose occurrence probability is greater than the preset probability as the current state.
[0074] Based on the above embodiment, the motion state has the highest probability of occurrence, and the motion state is taken as the current state.
[0075] Step S13, determining a safety mode corresponding to the current state, the safety modes including: sports safety mode, driving safety mode and sleeping safety mode;
[0076] In the embodiment of the present application, the current state and the safety mode are in a one-to-one correspondence, and the safety modes include: sports safety mode, driving safety mode and sleeping safety mode. For example: when the current state is the sports state, the safety mode is the sports safety mode; when the current state is the driving state, the safety mode is the driving safety mode; when the current state is the sleeping state, the safety mode is the sleeping safety mode.
[0077] Step S14, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode.
[0078] The embodiment of the present application determines the current state of the current worn object by detecting a set of detection data obtained from the current worn object, and the smart wearable device enters a corresponding safety mode according to the current state, thereby being able to provide different protection operations for the current worn object according to different states, thereby realizing real-time execution of corresponding protection measures for the user according to the user's state.
[0079] In the embodiment of the present application, when the safety mode is the motion safety mode, such as Figure 2 As shown, step S14, controlling the smart wearable device to perform a protection operation on the current wearable object in a safe mode, includes the following steps:
[0080] Step S21, obtaining a current frame image captured by the smart wearable device, where the current frame image includes at least one object;
[0081] In an embodiment of the present application, when the safety mode is the motion safety mode, the smart wearable device will shoot the current environment to obtain a current frame image. The objects included in the current frame image may be pedestrians, animals, etc.
[0082] Step S22, performing differential processing on the current frame image using the previous frame image of the current frame image to obtain a target foreground image;
[0083] In the embodiment of the present application, commonly used difference algorithms include: background difference method, frame difference method and three-frame difference method. The embodiment of the present application mainly uses the frame difference method to differentiate the current frame image, takes the current frame image as the background frame, uses the previous frame image of the current frame image to perform difference processing on the background frame, and when it is determined that there is a moving object (object), removes the image shadow to obtain the target foreground image. The use of the frame difference method can well suppress noise, adapt to changes in external scenes, and ensure the stability of the processing process.
[0084] Step S23, determining a region of interest in the target foreground image, the region of interest being obtained according to a gradient direction histogram feature of the target foreground image, and the region of interest carrying at least one key part;
[0085] In the embodiment of the present application, the object is a pedestrian, and the features of the pedestrian's head and shoulders are extracted and detected using the gradient directional histogram features and the classifier, the area of the pedestrian's head and shoulders in the target foreground image is determined, and the area where the head and shoulders are located is taken as the region of interest.
[0086] The specific process is as follows: First, convert the target foreground image into a grayscale image and perform correction. Then calculate the image pixel gradient using the following formula:
[0087] G a(a,b)=H(a+1,b)-H(a-1,b).
[0088] G b (a,b)=H(a,b+1)-H(a,b-1).
[0089] In the above formula, G a (a, b) is the horizontal gradient direction of pixel (a, b), G b (a,b) is the vertical gradient direction of pixel (a,b), and H(a,b) is the pixel value of pixel (a,b). The gradient magnitude and gradient direction of pixel (a,b) are as follows:
[0090]
[0091]
[0092] In the above formula, G(a,b) is the gradient amplitude and α(a,b) is the gradient direction.
[0093] It should be noted that, generally, a gradient operator is used to perform convolution operation on an image to obtain an image gradient. In this embodiment, a [-1, 0, 1] gradient operator is used to perform convolution operation on an image to obtain a horizontal gradient component. T The gradient operator performs a convolution operation to obtain the vertical gradient component, and finally obtains the gradient amplitude and gradient direction.
[0094] After obtaining the gradient magnitude and gradient direction, the gradient direction histogram is calculated based on the gradient magnitude and gradient direction. For example, for an image with 64×64 pixels, it is first divided into several connected regions of 8×8 pixels. Adjacent regions do not overlap. Each normalized block consists of 2×2 regions, so there are a total of (8-1)×(8-1)=49 blocks. The gradient direction is quantized into 9 histogram channels. The histogram channel of a certain direction is calculated in a weighted manner for each pixel in the region. The gradient magnitude of the pixel determines the size of the weight, and then the gradient direction histogram features are statistically calculated in each region.
[0095] Finally, the obtained gradient direction histogram features are input into the classifier, and the classifier analyzes the gradient direction histogram features to determine the region of interest, which includes the head and shoulders of the pedestrian (object).
[0096] Step S24, tracking the object according to the region of interest to obtain the number of objects;
[0097] In the embodiment of the present application, tracking the object according to the region of interest to obtain the number of objects includes the following steps B1-B6:
[0098] Step B1, extracting color features of the region of interest;
[0099] In the embodiment of the present application, the color feature is a CN (color name) feature.
[0100] Step B2, determining a first peak sidelobe ratio and a first correlation graph of a gradient direction histogram feature, and a second peak sidelobe ratio and a second correlation graph of a color feature;
[0101] In the embodiment of the present application, the first peak sidelobe ratio and the second peak sidelobe ratio are defined as follows: the ratio of the main lobe peak intensity to the peak intensity of the strongest sidelobe, and the correlation diagram is the correlation filter response diagram.
[0102] Step B3, determining the fusion weight ratio of the color feature in the target frame image according to the first peak sidelobe ratio, the second peak sidelobe ratio, the first correlation graph, and the second correlation graph, where the target frame image is the next frame image of the current frame image;
[0103] The calculation formula is as follows:
[0104]
[0105] In the formula, k CN is the fusion weight ratio of the color features in the current frame image, PSLR HOG,s is the first peak sidelobe ratio in the current frame image, PSLR CN,s is the second peak sidelobe ratio in the current frame image, f HOG,s is the first correlation graph corresponding to the current frame image, f CN,s is the second correlation graph corresponding to the current frame image. It can be understood that the s in the formula represents the sth frame image, that is, the current frame image.
[0106] In the embodiment of the present application, a gradient direction histogram feature and color feature with complementary characteristics are fused in an adaptive weighted manner. Since the peak sidelobe ratio can quantify the sharpness of the correlation peak, the larger the PSLR value, the higher the tracking confidence, and the more the current frame matches the target of the previous frame. In addition, the larger the peak value of the correlation filter response map, the more accurate the target position. Therefore, the feature fusion weight is calculated based on the peak sidelobe ratio of the two features and the peak value of the correlation filter response map.
[0107] Step B4, determining the fusion weight of the color feature in the target frame image according to the fusion weight ratio;
[0108] In the embodiment of the present application, the calculation formula of the fusion weight is as follows:
[0109] k CN,s =(1-β)×k CN,s-1 +β×k CN .
[0110] In the formula, β is the learning coefficient of the fusion weight, k CN,s is the fusion weight.
[0111] Step B5, determining a target correlation map of the gradient direction histogram features and the color features in the target frame image according to the fusion weight, the first correlation map and the second correlation map;
[0112] In the embodiment of the present application, the calculation formula of the target correlation graph is as follows:
[0113] f s (z) = (1-k CN,s )×f HOG,s (z)+k CN,s ×f CN,s (z).
[0114] In the formula, f s (z) is the target correlation graph.
[0115] Step B6, determining the number of objects in the target frame image according to the target correlation graph.
[0116] In the embodiment of the present application, determining the number of objects in the target frame image according to the target correlation graph includes steps B61 to B63:
[0117] Step B61, determining a target peak value of a target correlation graph;
[0118] Step B62, determining a target region of interest in the target frame image according to the target peak value;
[0119] Step B63, determining the number of objects according to the number of key parts in the target region of interest.
[0120] In the embodiment of the present application, according to the target correlation graph f t The peak position of (z) determines the region of interest in the target frame image, and the number of objects is determined according to the number of heads and shoulders of pedestrians in the region of interest in the target frame image.
[0121] The embodiment of the present application adds color features to improve the detection accuracy. When the object is completely blocked, the position of the region of interest in the next frame of the image can be predicted in advance, effectively solving the problem of tracking loss when the target is completely blocked. On this basis, the gradient direction histogram feature and the color feature are fused as a tracking detection mechanism to more accurately lock the position of the region of interest.
[0122] Step S25: generating a warning instruction according to the number of objects, wherein the warning instruction is used to control the smart wearable device to vibrate at a target vibration frequency corresponding to the number of objects.
[0123] In an embodiment of the present application, a correspondence between a preset number of objects and a vibration frequency is obtained, a target vibration frequency corresponding to the number of objects is determined based on the correspondence, and an early warning instruction is generated based on the target vibration frequency. The early warning instruction is used to control the smart wearable device to perform a vibration operation according to the target vibration frequency corresponding to the number of objects.
[0124] The embodiment of the present application tracks objects in real time to determine the number of objects. When the number of objects is large, the smart wearable device can perform a vibration operation to remind the current wearable object to reduce the movement speed to avoid accidents such as collisions.
[0125] Further, when the safety mode is the driving safety mode, such as Figure 3 As shown, step S14, controlling the smart wearable device to perform a protection operation on the current wearable object in a safe mode, includes:
[0126] Step S31, real-time detection of the EEG data of the current wearer;
[0127] In the embodiment of the present application, since there is serious noise interference in the original EEG signal, the original EEG signal is first preprocessed in this embodiment. The preprocessing process is as follows: a bandpass filter can remove part of the power frequency interference on the one hand, and can remove part of the myoelectric and electrooculographic noise on the other hand.
[0128] In addition, the high-frequency components in the original EEG signal can be removed well by using a bandpass filter, but the traditional digital filter noise reduction method cannot effectively filter out the low-frequency noise of the EEG signal and the noise of the overlapping frequency of the EEG signal. Therefore, wavelet threshold denoising is also required to obtain EEG data.
[0129] Step S32, extracting frequency domain feature information from the EEG data;
[0130] In the embodiment of the present application, the frequency domain feature information is a power spectrum, and the EEG data is divided into Q data segments using a sliding window, and the length of each data segment is p. At this time, the length of the EEG data is l=p×Q, and then the power spectrum of each data segment is calculated, and finally the power spectrum of each segment is combined to obtain a feature matrix;
[0131] The formula for calculating the power spectrum of each data segment is as follows:
[0132]
[0133] Moderate, W Q (ω) is the power spectrum of the Qth segment, ω(z) is the window function, U is the normalization coefficient, x Q (z) is the data observation value, e -d is a constant, d=0, 1, 2…
[0134] As an example, the signal length of EEG data is 2s, and the smart wearable device uses a 500HZ sampling frequency, so there are 1000 sampling points in the signal length. Then, the frequency domain feature information is extracted based on the 1000 sampling points. The specific process is as follows:
[0135] A rectangular window is selected to slide and intercept the EEG data. The window size is 128 sampling points, and the step length is 32 sampling points. Thus, 28 segments of EEG data O with a length of 128 sampling points are obtained, which are combined into a feature matrix R:
[0136] O = {y1, y2, ...y j}, i=28, j=128.
[0137] Step S33, inputting the frequency domain feature information into the analysis model, and the analysis model obtains the attention state of the current wearer according to the feature type corresponding to the frequency domain feature information;
[0138] In the embodiment of the present application, the above-mentioned feature matrix R is input into the analysis model, and the analysis model determines the attention state of the current wearer according to the feature matrix R. The analysis model in this embodiment includes:
[0139] (a) Input layer: The input is a 28×28 feature matrix, where the row vector represents the time sampling point and the column vector represents the frequency distribution.
[0140] (ii) The first convolutional layer is used to perform convolution transformation based on the local receptive field and extract features. The convolutional layer and the input layer are locally connected, and the input feature matrix is convolved with 6 convolution kernels of size 5×5, and after the activation function, the feature map is finally obtained, that is, 6 24×24 feature maps.
[0141] (III) The first pooling layer, the purpose of the pooling layer is to reduce the data dimension and reduce the computational complexity. In this embodiment, the first convolutional layer is reduced by half through a maximum pooling layer of size 2×2 to obtain 6 12×12 feature maps.
[0142] (iv) In the second convolutional layer, 6 12×12 feature maps are convolved with 12 3×3 convolution kernels and then passed through the activation function to finally obtain 72 feature maps of size 10×10.
[0143] (v) In the second pooling layer, the 72 feature maps of size 10×10 are reduced by half through a maximum pooling layer of size 2×2 to obtain 72 feature maps of size 5×5.
[0144] (VI) Fully connected layer: In this embodiment, the fully connected layer is used to classify feature maps, and its function is equivalent to that of a classifier.
[0145] (VII) Output layer: In this embodiment, three neurons are designed in the output layer to represent the analysis results (the analysis results include: focused attention, slightly distracted attention, and severely distracted attention).
[0146] Step S34, performing corresponding protection operations on the current wearable object according to the attention state.
[0147] In an embodiment of the present application, a corresponding protection operation is performed on the current wearable object according to the attention state, including: when the attention state is slightly distracted, controlling the smart wearable device to perform indirect vibration according to the first vibration frequency, and sending a first prompt message to the vehicle screen that is communicatively connected to the smart wearable device;
[0148] Performing corresponding protection operations on the current wearable object according to the attention state also includes: when the attention state is severely distracted, controlling the smart wearable device to perform continuous vibration according to the second vibration frequency, and sending a second prompt message to the vehicle screen that is communicatively connected to the smart wearable device.
[0149] The solution provided in the embodiment of the present application determines the attention state of the current wearer when the wearer is in a driving state, and automatically performs different prompt operations to the current wearer according to the attention state. At the same time, in conjunction with the vehicle screen connected to the smart device, the current wearer is given correct warning information, thereby effectively monitoring the user's driving conditions and improving safety during driving.
[0150] In the embodiment of the present application, when the safety mode is the sleep safety mode, such as Figure 4 As shown, step S14, controlling the smart wearable device to perform a protection operation on the current wearable object in a safe mode, includes:
[0151] Step S41, obtaining current displacement data of the smart wearable device;
[0152] In the embodiment of the present application, the current displacement data includes: displacement change data, and gravity point change data.
[0153] Step S42, when it is determined according to the current displacement data that the sleep condition of the current wearer is an abnormal sleep condition, a wake-up instruction is generated, and the wake-up instruction is used to wake up the current wearer according to a preset vibration frequency.
[0154] In the embodiment of the present application, when it is determined according to the current displacement data that the sleep condition of the current wearer is an abnormal sleep condition, a wake-up instruction is generated, including the following steps C1-C3:
[0155] Step C1, obtaining historical displacement data;
[0156] Step C2, determining displacement change information according to current displacement data and historical displacement data;
[0157] Step C3: when the displacement change information is greater than the preset change information, it is determined that the current sleep condition of the wearer is an abnormal sleep condition, and a wake-up instruction is generated.
[0158] In an embodiment of the present application, when the current wearer of the smart wearable device enters a sleep state, the smart wearable device will continue to analyze the sensed displacement data to determine the sleep condition of the current wearer.
[0159] If the displacement and gravity point changes of the smart wearable device within a predetermined length of time before the current time do not exceed the corresponding set limit values, it is determined that the current sleep status of the wearer is just falling asleep.
[0160] If the displacement and gravity point data tend to be stable and have almost no changes within a period of time from the current time, then it can be determined that the current wearer's sleep state is deep sleep.
[0161] For example, according to actual usage needs, the detection period can be set to 10 minutes. If the current wearer of the smart wearable device is female, and the historical horizontal displacement of the smart wearable device during the set detection period does not exceed 3 cm, and the gravity point change does not exceed 1.5 cm, then the current wearer is considered to be in deep sleep.
[0162] If the gender of the current wearer of the smart wearable device is female, and the historical horizontal displacement of the smart wearable device during the set detection period does not exceed 5 cm, and the gravity point change does not exceed 2.5 cm, then the sleep state of the current wearer is considered to be deep sleep.
[0163] Figure 5 This is a block diagram of a data processing device provided in an embodiment of the present application. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 5 As shown, the device comprises:
[0164] An acquisition module 51 is used to acquire a detection data set obtained by the smart wearable device from detecting the current wearable object, wherein the detection data set includes multiple detection data, and the detection data includes: movement speed, EEG signal, heart rate, blood pressure, body surface temperature and angular velocity of key parts;
[0165] An analysis module 52 is used to obtain at least one candidate state corresponding to the detection data, and take a candidate state with a probability greater than a preset probability as a current state, wherein the current state includes at least one of the following: a motion state, a driving state, and a sleeping state;
[0166] A determination module 53, configured to determine a safety mode corresponding to the current state, wherein the safety mode includes at least one of the following: a sports safety mode, a driving safety mode, and a sleeping safety mode;
[0167] The control module 54 is used to control the smart wearable device to perform a protection operation on the current wearable object according to the safety mode.
[0168] Further, when the safety mode is the sports safety mode, the control module 54 includes:
[0169] An acquisition submodule is used to acquire a current frame image captured by the smart wearable device, where the current frame image includes at least one object;
[0170] A processing submodule, used to perform differential processing on the current frame image using the previous frame image of the current frame image to obtain a target foreground image;
[0171] A determination submodule is used to determine a region of interest in a target foreground image, where the region of interest is obtained according to a gradient direction histogram feature of the target foreground image, and the region of interest carries at least one key part;
[0172] A tracking submodule is used to track the object according to the region of interest and obtain the object quantity of the object;
[0173] The generation submodule is used to generate a warning instruction according to the number of objects, and the warning instruction is used to control the smart wearable device to perform a vibration operation according to a target vibration frequency corresponding to the number of objects.
[0174] Furthermore, the tracking submodule is used to extract the color features of the region of interest; determine the first peak sidelobe ratio and the first correlation graph of the gradient direction histogram feature, and the second peak sidelobe ratio and the second correlation graph of the color feature; determine the fusion weight ratio of the color feature in the target frame image according to the first peak sidelobe ratio, the second peak sidelobe ratio, the first correlation graph and the second correlation graph, the target frame image is the next frame image of the current frame image, and the calculation formula of the fusion weight ratio is as follows:
[0175]
[0176] In the formula, k CN is the fusion weight ratio of the color features in the current frame image, PSLR HOG,s is the first peak sidelobe ratio in the current frame image, PSLR CN,sis the second peak sidelobe ratio in the current frame image, f HOG,s is the first correlation graph corresponding to the current frame image, f CN,s is the second correlation graph corresponding to the current frame image;
[0177] The fusion weight of the color feature in the target frame image is determined according to the fusion weight ratio; the target correlation graph of the gradient direction histogram feature and the color feature in the target frame image is determined according to the fusion weight, the first correlation graph and the second correlation graph; the number of objects in the target frame image is determined according to the target correlation graph.
[0178] Furthermore, the tracking submodule is used to determine the target peak value of the target correlation graph; determine the target region of interest in the target frame image according to the target peak value; and determine the number of objects according to the number of key parts in the target region of interest.
[0179] Further, when the safety mode is the driving safety mode, the control module 54 includes:
[0180] The detection submodule is used to detect the EEG data of the current wearer in real time;
[0181] The extraction submodule is used to extract the frequency domain feature information from the EEG data;
[0182] A processing submodule, used to input the frequency domain feature information into the analysis model, and the analysis model obtains the attention state of the current wearer according to the feature type corresponding to the frequency domain feature information;
[0183] The execution submodule is used to perform corresponding protection operations on the current wearable object according to the attention state, and the attention state includes: focused attention, slightly distracted attention, and severely distracted attention.
[0184] Further, the execution submodule is used to control the smart wearable device to perform indirect vibration according to the first vibration frequency when the attention state is slightly distracted, and send a first prompt information to the vehicle screen that is communicatively connected to the smart wearable device;
[0185] The execution submodule is used to control the smart wearable device to perform continuous vibration according to the second vibration frequency when the attention state is seriously distracted, and send a second prompt message to the vehicle screen that is communicatively connected to the smart wearable device.
[0186] Furthermore, when the safety mode is the sleep safety mode, the control module 54 is used to obtain the current displacement data of the smart wearable device; when it is determined according to the current displacement data that the sleep condition of the current wearable object is an abnormal sleep condition, a wake-up instruction is generated, and the wake-up instruction is used to wake up the current wearable object according to a preset vibration frequency.
[0187] The present application also provides an electronic device, such as Figure 6 As shown, the electronic device may include: a processor 1501 , a communication interface 1502 , a memory 1503 and a communication bus 1504 , wherein the processor 1501 , the communication interface 1502 , and the memory 1503 communicate with each other via the communication bus 1504 .
[0188] Memory 1503, used for storing computer programs;
[0189] The processor 1501 is used to implement the steps of the above embodiment when executing the computer program stored in the memory 1503.
[0190] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0191] The communication interface is used for communication between the above terminal and other devices.
[0192] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0193] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0194] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer executes the data processing method described in any one of the above embodiments.
[0195] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the data processing method described in any one of the above embodiments.
[0196] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk), etc.
[0197] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
[0198] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A data processing method, applied to a smart wearable device, characterized in that: include: Acquire a detection data set obtained by the smart wearable device from detecting the current wearable object, wherein the detection data set includes multiple detection data, and the detection data includes: movement speed, electroencephalogram signal, heart rate, blood pressure, body surface temperature and angular velocity of key parts; Acquire at least one candidate state corresponding to the detection data, and take a candidate state with a probability greater than a preset probability as a current state, wherein the current state includes: a motion state, a driving state, and a sleeping state; Determine a safety mode corresponding to the current state, the safety mode including: a sports safety mode, a driving safety mode, and a sleeping safety mode; Controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode; Wherein, when the safety mode is a sports safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes: Acquire a current frame image captured by the smart wearable device, where the current frame image includes at least one object; Performing differential processing on the current frame image using a previous frame image of the current frame image to obtain a target foreground image; Determine a region of interest in the target foreground image, where the region of interest is obtained according to a gradient direction histogram feature of the target foreground image, and the region of interest carries at least one key part; Tracking the object according to the region of interest to obtain the object quantity of the object; Generate a warning instruction according to the number of objects, wherein the warning instruction is used to control the smart wearable device to vibrate at a target vibration frequency corresponding to the number of objects; Tracking the object according to the region of interest to obtain the number of the object includes: Extracting color features of the region of interest; Determine a first peak sidelobe ratio and a first correlation graph of the gradient direction histogram feature, and a second peak sidelobe ratio and a second correlation graph of the color feature; According to the first peak sidelobe ratio, the second peak sidelobe ratio, the first correlation graph, and the second correlation graph, a fusion weight ratio of the color feature in a target frame image is determined, where the target frame image is a next frame image of the current frame image, and a calculation formula for the fusion weight ratio is as follows: ; In the formula, is the fusion weight ratio of the color features in the current frame image, is the first peak sidelobe ratio in the current frame image, is the second peak sidelobe ratio in the current frame image, is the first correlation graph corresponding to the current frame image, is the second correlation graph corresponding to the current frame image; Determining the fusion weight of the color feature in the target frame image according to the fusion weight ratio; Determine a target correlation graph of the gradient direction histogram feature and the color feature in the target frame image according to the fusion weight, the first correlation graph, and the second correlation graph; Determine the number of objects in the target frame image according to the target correlation graph; The step of determining the number of the objects in the target frame image according to the target correlation graph includes: determining a target peak value of the target correlation graph; Determine a target region of interest in the target frame image according to the target peak value; The number of objects is determined according to the number of key parts in the target region of interest.
2. The method according to claim 1, characterized in that When the safety mode is a driving safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes: Real-time detection of EEG data of the current wearer; Extracting frequency domain feature information from the EEG data; Inputting the frequency domain feature information into an analysis model, and obtaining the attention state of the current wearer according to the feature type corresponding to the frequency domain feature information by the analysis model; A corresponding protection operation is performed on the current wearing object according to the attention state, where the attention state includes: focused attention, slightly distracted attention, and severely distracted attention.
3. The method according to claim 2, characterized in that The performing a corresponding protection operation on the current worn object according to the attention state includes: When the attention state is slightly distracted, controlling the smart wearable device to perform indirect vibration according to a first vibration frequency, and sending a first prompt message to a vehicle screen that is communicatively connected to the smart wearable device; The performing corresponding protection operations on the current wearing object according to the attention state also includes: When the attention state is severely distracted, the smart wearable device is controlled to perform continuous vibration according to the second vibration frequency, and a second prompt message is sent to the vehicle screen that is communicatively connected to the smart wearable device.
4. The method according to claim 1, characterized in that: When the safety mode is a sleep safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes: Obtaining current displacement data of the smart wearable device; When it is determined according to the current displacement data that the sleep condition of the current wearer is an abnormal sleep condition, a wake-up instruction is generated, and the wake-up instruction is used to wake up the current wearer according to a preset vibration frequency.
5. A data processing device, applied to a smart wearable device, characterized in that: include: An acquisition module is used to acquire a detection data set obtained by the smart wearable device from detecting the current wearable object, wherein the detection data set includes multiple detection data, and the detection data includes: movement speed, EEG signal, heart rate, blood pressure, body surface temperature and angular velocity of key parts; an analysis module, configured to obtain at least one candidate state corresponding to the detection data, and take a candidate state with a probability greater than a preset probability as a current state, wherein the current state includes: a motion state, a driving state, and a sleeping state; A determination module, configured to determine a safety mode corresponding to the current state, wherein the safety mode includes: a sports safety mode, a driving safety mode, and a sleeping safety mode; A control module, used to control the smart wearable device to perform a protection operation on the current wearable object according to the safety mode; Wherein, when the safety mode is a sports safety mode, controlling the smart wearable device to perform a protection operation on the current wearable object according to the safety mode includes: Acquire a current frame image captured by the smart wearable device, where the current frame image includes at least one object; Performing differential processing on the current frame image using a previous frame image of the current frame image to obtain a target foreground image; Determine a region of interest in the target foreground image, where the region of interest is obtained according to a gradient direction histogram feature of the target foreground image, and the region of interest carries at least one key part; Tracking the object according to the region of interest to obtain the object quantity of the object; Generate a warning instruction according to the number of objects, wherein the warning instruction is used to control the smart wearable device to vibrate at a target vibration frequency corresponding to the number of objects; Tracking the object according to the region of interest to obtain the number of the object includes: Extracting color features of the region of interest; Determine a first peak sidelobe ratio and a first correlation graph of the gradient direction histogram feature, and a second peak sidelobe ratio and a second correlation graph of the color feature; According to the first peak sidelobe ratio, the second peak sidelobe ratio, the first correlation graph, and the second correlation graph, a fusion weight ratio of the color feature in a target frame image is determined, where the target frame image is a next frame image of the current frame image, and a calculation formula for the fusion weight ratio is as follows: ; In the formula, is the fusion weight ratio of the color features in the current frame image, is the first peak sidelobe ratio in the current frame image, is the second peak sidelobe ratio in the current frame image, is the first correlation graph corresponding to the current frame image, is the second correlation graph corresponding to the current frame image; Determining the fusion weight of the color feature in the target frame image according to the fusion weight ratio; Determine a target correlation graph of the gradient direction histogram feature and the color feature in the target frame image according to the fusion weight, the first correlation graph, and the second correlation graph; Determine the number of objects in the target frame image according to the target correlation graph; The step of determining the number of the objects in the target frame image according to the target correlation graph includes: determining a target peak value of the target correlation graph; Determine a target region of interest in the target frame image according to the target peak value; The number of objects is determined according to the number of key parts in the target region of interest.
6. A storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the method steps of any one of claims 1 to 4 when running.
7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the method steps of any one of claims 1 to 4 by running a program stored in a memory.
