Drinking state prediction method, system, electronic device and computer storage medium

Abstract

The present disclosure provides a drinking state prediction method, system, electronic device and computer storage medium. The drinking state prediction method includes: obtaining spectral information and physiological information corresponding to the object to be detected; based on the drinking state prediction model, feature fusion of the spectral information and physiological information to obtain fusion information, predicting the drinking state of the object to be detected according to the fusion information, and obtaining multiple prediction labels and their corresponding prediction probabilities, wherein the multiple prediction labels correspond to multiple different drinking states, and the drinking state prediction model carries the corresponding relationship between the drinking state of the object to be detected and the fusion information; determining the prediction label corresponding to the maximum value among the multiple prediction probabilities as the prediction result of the drinking state of the object to be detected.

Description

Technical Field

[0001] The present disclosure relates to the field of medical detection, and in particular, to a drinking status prediction method, system, electronic device and computer storage medium. Background Art

[0002] In the related art, the alcohol detection task can be completed by breath testing or blood testing, but blood testing is an invasive detection method and there is a risk of infection during the blood collection process; and breath testing can be interfered by external environmental factors, resulting in inaccurate test results.

[0003] In view of this, the related art proposes an alcohol detection method based on near-infrared spectroscopy technology, which realizes detection by combining spectral preprocessing methods with chemometric methods, such as near-infrared spectral analysis based on principal component analysis and partial least squares method.

[0004] However, the spectral preprocessing process involves scattering correction or denoising and smoothing of the spectral information, and the spectral information of human blood tissue is very weak. After preprocessing, some features or patterns may be weakened, and important information may be lost. At the same time, chemometric methods cannot capture and process the nonlinear characteristics of the spectrum, and there are a large number of such nonlinear characteristics in the human spectrum, so the existing alcohol detection model based on near-infrared spectroscopy technology has very limited ability to extract complex features.

[0005] Existing technologies usually require characteristic wavelength selection and dimensionality reduction processing of the high-dimensional features of spectral data to avoid the problem of dimensionality disaster, but this process is time-consuming and prone to loss of important feature information, resulting in the inability to guarantee the accuracy of the prediction results. Summary of the invention

[0006] The present disclosure aims to solve one of the technical problems in the related art to a certain extent. To this end, the present disclosure provides a drinking state prediction method, system, electronic device and computer storage medium.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for predicting drinking status, comprising:

[0008] Acquiring spectral information and physiological information corresponding to the object to be detected;

[0009] Based on the drinking status prediction model, the spectral information and the physiological information are feature fused to obtain fused information, and the drinking status of the subject to be detected is predicted according to the fused information to obtain a plurality of prediction labels and their corresponding prediction probabilities, wherein the plurality of prediction labels correspond to a plurality of different drinking statuses, respectively, and the drinking status prediction model carries the corresponding relationship between the drinking status of the subject to be detected and the fused information;

[0010] The predicted label corresponding to the maximum value among the multiple predicted probabilities is determined as the predicted result of the drinking status of the subject to be detected.

[0011] In some embodiments, the method further comprises:

[0012] Training the drinking status prediction model according to the training sample set until the drinking status prediction model carries the corresponding relationship;

[0013] The training sample set includes at least sample spectral information and sample physiological information.

[0014] In some embodiments,

[0015] The drinking state prediction model includes a feature extraction module, which includes a frequency domain feature extraction unit, a time domain feature extraction unit and a feature fusion unit.

[0016] The spectral information and the physiological information are subjected to feature fusion to obtain fusion information, including:

[0017] Based on the frequency domain feature extraction unit, feature extraction is performed on the spectral information to obtain spectral features;

[0018] Based on the time domain feature extraction unit, feature extraction is performed on the physiological information from two time dimensions, namely, forward and reverse, to obtain physiological features;

[0019] Based on the feature fusion unit, the spectral feature and the physiological feature are feature spliced ​​to obtain the fusion information.

[0020] In some embodiments, the physiological information includes an electrocardiogram signal and / or a photoplethysmogram signal,

[0021] The electrocardiogram signal is used to characterize at least one of heart rate and heart rate variability, and the photoplethysmogram signal is used to characterize at least one of blood pressure, blood oxygen saturation and respiratory rate.

[0022] In some embodiments, the plurality of prediction labels include not drinking, drinking, and being drunk.

[0023] In some embodiments, the drinking status prediction model further includes a prediction module.

[0024] The step of training the drinking status prediction model according to the training sample set until the drinking status prediction model carries the corresponding relationship includes:

[0025] Acquire the training sample set, wherein the training sample set includes a plurality of sample groups, each of the sample groups includes sample information corresponding to a plurality of target objects, the sample information includes sample spectral information, sample physiological information and sample labels, and the sample labels are used to represent the drinking status;

[0026] Inputting the training sample set into the feature extraction module, and obtaining sample fusion information according to the sample spectral information and the sample physiological information;

[0027] The sample fusion information is input into the prediction module to obtain multiple prediction labels and their corresponding prediction probabilities, and multiple error back propagation optimization training is performed in combination with the cross entropy loss function and the Adam algorithm to adjust the model parameters until the corresponding relationship is carried in the drinking status prediction model.

[0028] The sample fusion information is obtained based on the sample spectral information and the sample physiological information.

[0029] In some embodiments, the frequency domain feature extraction unit includes at least a stacked convolutional network and a pooling network, an activation function layer, and a fully connected layer, the time domain feature extraction unit includes at least a bidirectional long short-term memory network and a multi-layer perceptron, and the prediction module includes at least a fully connected layer and a Softmax layer.

[0030] The training sample set is input into the drinking state prediction model, and sample fusion information is obtained according to the sample spectral information and the sample physiological information; multiple prediction labels and their corresponding prediction probabilities are obtained according to the sample fusion information, and multiple error back propagation optimization training is performed in combination with the cross entropy loss function and the Adam algorithm, including:

[0031] The drinking state prediction model is subjected to multiple error back propagation optimization trainings, wherein each training includes the following steps:

[0032] Inputting the spectral information of multiple samples in the sample group into the frequency domain feature extraction unit to obtain the spectral feature information of the samples;

[0033] Inputting the physiological information of multiple samples in the sample group into the time domain feature extraction unit to obtain the physiological feature information of the samples;

[0034] Inputting the sample spectral feature information and the sample physiological feature information into the feature fusion unit for splicing and fusion to obtain sample fusion information;

[0035] Inputting the sample fusion features into the prediction module to obtain multiple prediction labels and their corresponding prediction probabilities;

[0036] According to the cross entropy loss function and the Adam algorithm, the loss value between the predicted label and the corresponding sample label is calculated until the loss value converges, thereby obtaining the drinking status prediction model carrying the corresponding relationship.

[0037] In a second aspect, an embodiment of the present disclosure provides a drinking status prediction system, the system comprising:

[0038] A drinking status prediction device, used to execute the drinking status prediction method according to the first aspect of the claims;

[0039] Near infrared spectrum detection equipment, used to detect the target object to obtain spectrum information;

[0040] Biosensor instruments are used to detect the physical indicators of the target object to obtain physiological information.

[0041] In some embodiments, the wavelength range detected by the near-infrared spectrum detection device is 800nm-2500nm, and the light source is a full-band halogen lamp light source.

[0042] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the drinking status prediction method described in the first aspect are performed.

[0043] In a fourth aspect, an embodiment of the present disclosure further provides a computer non-volatile readable storage medium, wherein a computer program is stored on the computer non-volatile readable storage medium, and when the computer program is executed by a processor, the steps of the drinking status prediction method as described in the first aspect are executed.

[0044] The drinking state prediction method provided in the embodiment of the present disclosure fuses the spectral information and the physiological information according to the drinking state prediction model, obtains the fused information, and performs prediction processing according to the fused information, thereby obtaining the corresponding drinking state prediction result. Among them, the drinking state prediction model is a deep learning algorithm model based on a convolutional neural network, which has the ability to automatically detect data patterns and automatically integrate feature extraction, so the collected spectral data does not need to be preprocessed, but the spectral information is directly input for analysis, and the optimal preprocessing filter can be found without manual intervention. The convolution kernel (filter) trained based on the optimization algorithm can perform a moving weighted average algorithm on the spectrum to different degrees. Compared with the traditional method, it can adapt to different spectral data characteristics more flexibly, remove noise more specifically, and solve the problem of information loss. In addition, the present scheme also combines the physiological information of the object to be detected, eliminates the interference caused by the difference in physiological indicators between different target objects, reduces the error caused by the detection of a single data type by fusing multiple data sources, and improves the accuracy of the drinking state prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic flow chart of a drinking status prediction method provided in an embodiment of the present disclosure.

[0046] Figure 2 A schematic flow chart of a method for training a drinking status prediction model provided in an embodiment of the present disclosure.

[0047] Figure 3 A schematic flowchart of a single training process for a drinking status prediction model provided in an embodiment of the present disclosure.

[0048] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0049] Figure 5 It is a module schematic diagram of a computer-readable medium provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments based on the embodiments are intended to be used to explain the present disclosure and cannot be understood as limiting the present disclosure.

[0051] References to "one embodiment" or "example" or "example" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment itself may be included in at least one embodiment disclosed in the present disclosure. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0052] Alcohol testing is an important safety measure in many fields, which can effectively protect people's lives and property safety. Alcohol affects people's work by affecting the activities of the central nervous system. When alcohol is consumed, it quickly enters the blood circulation and is transported to the brain through the blood. In the brain, alcohol interferes with neurotransmission and the balance of chemicals, resulting in a series of effects. After drinking, people may experience problems such as slow thinking, lack of concentration and impaired judgment. In daily life, excessive drinking may also cause arrhythmia, especially atrial fibrillation, and inhibit the myocardium, leading to heart failure. Among the effects on the cardiovascular system, frequent alcohol abuse may increase blood pressure and lead to arterial hypertension, which is also a risk factor for cardiovascular disease and sudden death.

[0053] In addition, in the field of transportation, drunk driving is one of the main causes of traffic accidents. By conducting alcohol tests, it is possible to detect whether the driver has been drinking in time to reduce the occurrence of drunk driving. This helps to protect the life and property safety of the driver himself and other road users. In addition to traffic safety issues, alcohol also poses potential risks to some other types of work. For example, for those engaged in high-altitude operations or other types of work that require delicate operations and are more dangerous, it is crucial to ensure that the operator is sober and focused. Therefore, in these types of work, alcohol tests are often required before taking up the job to ensure workplace safety.

[0054] In the related art, alcohol detection tasks can be completed through breath testing or blood testing, but blood testing is an invasive detection method. There is a risk of infection during the blood collection process. On the one hand, it is impossible to give results immediately, and on the other hand, it is impossible to complete a large number of detection tasks in a short period of time. It is not suitable for daily testing and scenarios where test results need to be obtained quickly; in addition, breath testing will be interfered by external environmental factors, resulting in inaccurate test results. At the same time, the breath test requires the target object to exhale through a straw in his mouth. In order to ensure hygiene during the detection process, a large amount of disposable consumables are required, which increases the detection cost.

[0055] In view of this, a method of alcohol detection based on near-infrared spectroscopy is proposed in the related art to improve the detection speed and accuracy while reducing the detection cost. Specifically, most of the existing alcohol content detection technologies based on near-infrared spectroscopy combine spectral preprocessing methods with chemometric methods, such as near-infrared spectral analysis based on principal component analysis and partial least squares method.

[0056] This detection method can achieve relatively ideal and stable results when applied to the analysis of pure substances. However, when used to analyze the content of components in human tissue, the accuracy and generalization that can be achieved are far from enough for the model to be applied to actual situations. Specifically, during the spectral preprocessing process, scatter correction or denoising smoothing is performed on the spectral information, and the spectral information of human blood tissue is very weak. After preprocessing, some features or patterns may be weakened and important information may be lost. In addition, since the final preprocessing method and parameters are not necessarily optimal, the preprocessing process usually requires repeated experiments, which is quite time-consuming.

[0057] In addition, the tissue composition of the human body is very complex. The internal components of human capillaries include water, hemoglobin, glucose, various salts, amino acids, phospholipids, etc. Each person's physiological state is different at different times and in different environments, and there are differences in individual physiological indicators between people that cannot be accurately quantified. These differences are reflected in the spectrum, and there will be complex linear and nonlinear changes, irregular absorption peak shifts, and a large amount of irrelevant information interference. The chemometric method cannot capture and process the nonlinear characteristics of the spectrum, and there are a large number of such nonlinear characteristics in the human spectrum, so the existing alcohol detection model based on near-infrared spectroscopy technology has very limited ability to extract complex features.

[0058] In order to solve at least one of the above technical problems, an embodiment of the present disclosure provides a drinking status prediction method, which combines spectral information and physiological information collected from the object to be detected to perform a multi-dimensional drinking status qualitative analysis on the drinking status of the object to be detected, thereby improving the accuracy of drinking status prediction.

[0059] Figure 1 A schematic flow chart of a drinking state prediction method provided by an embodiment of the present disclosure, as a first aspect of the present disclosure, such as Figure 1 As shown, the method includes steps S11 to S13, specifically:

[0060] Step S11, acquiring spectral information and physiological information corresponding to the object to be detected.

[0061] The above spectral information is obtained by using a near-infrared spectral detection device to detect the subcutaneous capillary densely distributed parts of the subject to be detected. For example, the wrist position, the inner and outer sides of the upper arm, the earlobe, the fingertips and other locations where the subcutaneous capillaries are densely distributed can be detected, and the embodiments of the present disclosure are not limited to this.

[0062] Step S12: Based on the drinking state prediction model, perform feature fusion on the spectral information and physiological information to obtain fusion information, and predict the drinking state of the object to be detected according to the fusion information, obtaining multiple prediction labels and their corresponding prediction probabilities. Among them, the multiple prediction labels respectively correspond to multiple different drinking states, and the drinking state prediction model carries the corresponding relationship between the drinking state of the object to be detected and the fusion information.

[0063] It should be noted that after a person drinks alcohol, the capillaries will dilate extensively, resulting in a decrease in blood pressure and an increase in heart rate. By analyzing physiological information such as heart rate and blood pressure, the drinking state of the object to be detected can be effectively determined. Therefore, using physiological information as auxiliary information for determining whether the object to be detected has drunk alcohol can improve the accuracy of the drinking state prediction model.

[0064] Step S13: Determine the prediction label corresponding to the maximum value among the multiple prediction probabilities as the prediction result of the drinking state of the object to be detected.

[0065] In the drinking state prediction method provided by the embodiments of the present disclosure, feature fusion is performed on the spectral information and physiological information according to the drinking state prediction model to obtain fusion information, and prediction processing is performed according to the fusion information, thereby obtaining the corresponding drinking state prediction result. Among them, the drinking state prediction model is a deep learning algorithm model based on a convolutional neural network, which has the ability to automatically detect data patterns and automatically integrate feature extraction. Therefore, the collected spectral data does not need to be preprocessed, but the spectral information is directly input for analysis. The automated preprocessing process based on the convolutional neural network can preprocess the spectral information more accurately and flexibly, avoiding information loss during manual intervention. In addition, the physiological information of the object to be detected is combined in this solution to eliminate the interference caused by differences in physiological indicators between different target objects, reduce the error caused by detecting with a single data type by fusing multiple data sources, and improve the accuracy of the drinking state prediction result.

[0066] In some embodiments, the multiple prediction labels include not drinking alcohol, drinking alcohol, and being drunk.

[0067] Specifically, in the embodiments of the present disclosure, the spectral data is classified according to the physical and chemical value of alcohol content, that is, blood alcohol concentration (BAC), specifically divided into 3 categories: BAC < 20mg / dL, not drinking state; 20mg / dL < BAC < 80mg / dL, drinking state; BAC > 80mg / dL, being drunk state.

[0068] In some embodiments, the drinking state prediction method further includes: training the drinking state prediction module according to the training sample set until the drinking state prediction model carries the above corresponding relationship.

[0069] The training sample set includes multiple sample groups, each sample group includes sample information corresponding to multiple target objects, the sample information includes sample spectral information, sample physiological information and sample labels, and the sample labels are used to represent drinking status.

[0070] It should be understood that since the drinking status prediction model is a deep learning algorithm model formed by convolutional neural network training, the convolutional neural network can be trained based on tens of thousands of data. The deep learning algorithm is data-driven. The more data, the better the model performance. However, the performance of existing technologies no longer improves or even decreases as the number of samples increases. Convolutional neural networks can achieve end-to-end spectral feature extraction and prediction, and have the ability to extract high-dimensional features, solving the problem of easy loss of information in feature selection in traditional methods.

[0071] Specifically, the training process of the drinking status prediction model is described in detail in the following embodiments and will not be repeated here.

[0072] In some embodiments, the drinking state prediction model includes a frequency domain feature extraction unit, a time domain feature extraction unit and a feature fusion unit. Then, in step S12, based on the drinking state prediction model, the spectral information and the physiological information are feature fused to obtain fused information, including:

[0073] Based on the frequency domain feature extraction unit, the spectral information is extracted to obtain the spectral features; based on the time domain feature extraction unit, the physiological information is extracted from the forward and reverse time dimensions to obtain the physiological features; based on the feature fusion unit, the spectral features and the physiological features are spliced ​​to obtain the fusion information.

[0074] In the disclosed embodiment, multimodal information, namely spectral information and physiological information, and multiple terminal instrument (device) modal information are used, and the drinking status prediction model is combined to predict and analyze the drinking status of the subject to be detected. Among them, spectral information belongs to frequency domain information, which has high-dimensional characteristics within a specific wavelength range; physiological information can be regarded as time domain information, such as the change of the electrical signal of the heart of the subject to be detected over time. Based on the above multimodal information, the information collected by different device ends is supplemented to make the information more comprehensive, thereby improving the accuracy of prediction. Furthermore, the drinking status prediction model is trained using the fusion information obtained according to the spectral characteristics and physiological characteristics, which can reduce the sensitivity of the model to noise and random changes.

[0075] In some embodiments, the physiological information includes an electrocardiogram signal and a photoplethysmogram signal, wherein the electrocardiogram signal is used to characterize at least one of heart rate and heart rate variability, and the photoplethysmogram signal is used to characterize at least one of blood pressure, blood oxygen saturation and respiratory rate.

[0076] In one example, physiological information may also include core body temperature changes.

[0077] The above-mentioned core body temperature refers to the temperature around life-related organs deep inside the body, such as the heart, brain, liver, etc.

[0078] It should be understood that after drinking, the capillaries of a person will be widely dilated, which will lead to lower blood pressure and faster heart rate. Therefore, by analyzing the electrocardiogram signal and the photoplethysmogram signal, the drinking state of the subject to be detected can be effectively determined. Specifically, using information such as heart rate and blood pressure as auxiliary information to determine whether the subject to be detected is drinking can improve the accuracy of the drinking state prediction model.

[0079] In the drinking status prediction method provided by the embodiment of the present disclosure, a variety of physiological information is used as a supplement to the spectral information to more accurately determine the drinking status of the subject to be detected, thereby avoiding misclassification of non-drinkers as drinkers and having higher test accuracy.

[0080] The training process of the drinking status prediction model in the embodiment of the present disclosure is described in detail below in conjunction with specific embodiments.

[0081] Figure 2 A schematic flow chart of a method for training a drinking status prediction model provided by an embodiment of the present disclosure, such as Figure 2 As shown, the steps of training the drinking status prediction module according to the training sample set include: Step S21 to Step S23, specifically:

[0082] Step S21, obtaining a training sample set.

[0083] The training sample set includes multiple sample groups, each sample group includes sample information corresponding to multiple target objects, the sample information includes sample spectral information, sample physiological information and sample labels, and the sample labels are used to represent drinking status.

[0084] Preferably, during the training process of the drinking status prediction model, a test sample set can be obtained while obtaining the training sample set, and the data volume ratio between the two can be 7:3, wherein the data of the training sample set is used to train the model, and the data of the test sample set is used to evaluate the model.

[0085] It should also be noted that the sample information in the training sample set and the test sample set is essentially the same, but the functions of the sample information in the two sets are different. In the entire sample set, a training sample set and a test sample set can be formed by random selection. And the quantity ratio of the two can be flexibly set by those skilled in the art. For example, the data volume ratio of the training sample set and the test sample set can also be 1:1, 3:2, 5:4, etc., which is not limited in the embodiments of the present disclosure.

[0086] Step S22, inputting the training sample set into the drinking status prediction model, and obtaining sample fusion information based on the sample spectral information and the sample physiological information.

[0087] Step S23, based on the sample fusion information, multiple prediction labels and their corresponding prediction probabilities are obtained, and multiple error back propagation optimization trainings are performed in combination with the cross entropy loss function and the Adam algorithm to adjust the model parameters until the drinking status prediction model carries the corresponding relationship.

[0088] The above correspondence refers to the correspondence between drinking status and fusion information.

[0089] In the training method of the drinking status prediction model provided in the embodiment of the present disclosure, the drinking status prediction model is a deep learning algorithm model formed based on the training of a convolutional neural network, which has the ability to extract high-dimensional features and can analyze complex nonlinear features. And precisely because the convolutional neural network can automate the spectral preprocessing process, the optimal preprocessing filter can be found without human intervention. The convolution kernel (filter) trained based on the optimization algorithm can perform different degrees of moving weighted average processing on the spectral information. Compared with traditional methods, it can adapt to different spectral data characteristics more flexibly, and can remove noise more specifically, solving the problem of weakening data features during training.

[0090] The above training steps are described in detail below in conjunction with specific embodiments.

[0091] Before obtaining the sample set in step S21, first, 20 male and female target subjects are screened, aged between 25 and 40 years old, in good health, without major diseases and hidden diseases. Since the sensor is easily disturbed when receiving signals, in order to prevent unnecessary noise, the target subjects are sitting and are required to remain as still as possible during the data acquisition phase. Then, the male target subject drinks 80g of alcohol and the female target subject drinks 50g of alcohol. The intake can be adjusted appropriately and the drinking is completed within 30 minutes. One intake makes the peak blood alcohol concentration reach more than 100mg / dL. Finally, after the intake is completed, a professional breath alcohol detector is used to perform a blow test every 10 minutes to detect the alcohol concentration. The target subject needs to rinse his mouth with water before blowing, and control the blowing speed when blowing. It should not be too fast and the time should not be too short.

[0092] In step S21, sample information is obtained, each sample information includes sample spectral information, sample physiological information and a sample label, and the sample label is used to indicate the drinking status.

[0093] It should be understood that the above sample information can be information in a training sample set or information in a test sample set, and the two are not distinguished. That is, after obtaining multiple sample information, a training sample set and a test sample set can be determined by random allocation.

[0094] Specifically, a near-infrared spectral reflectance detection platform is built, and spectral data of human wrist information before and after drinking is collected through a near-infrared spectral detector. While collecting the spectrum, the external environment is controlled to be consistent. The wavelength range of the above-mentioned near-infrared spectral detection equipment is 800nm-2500nm, and the light source is a full-band halogen lamp light source to obtain sample spectral information.

[0095] In addition, the spectral data may also be collected by detecting other locations where subcutaneous capillaries are densely distributed, such as the inner and outer sides of the upper arms, earlobes, fingertips, etc. of the subject to be detected, and the embodiments of the present disclosure are not limited to this.

[0096] The biosensor was used to collect the electrocardiogram (ECG) and photoplethysmogram (PPG) signals of 30 consecutive cardiac cycle windows. The sensor for collecting the PPG signal was placed on the middle finger of the right hand, and the electrodes for collecting the ECG signal were placed at specific positions of the left and right sternums and ribs to record the electrical activity of the heart in different directions. Then, a finite impulse response low-pass filter and a median filter were used to eliminate the baseline drift noise in the signal to obtain the sample physiological information.

[0097] In some embodiments, the drinking status category of each target object may be binary encoded using one-hot encoding to obtain a sample label corresponding to the target object.

[0098] Specifically, one-hot encoding uses an N-bit state register to encode N states. Each state has its own independent register bit, and only one of them is valid at any time. In the model training process, an important role of the classification label is to calculate the difference between the predicted label and the true label, so as to calculate the loss value. The smaller the loss value, the closer the predicted label is to the true label. The difference is actually the distance between the two labels. At this time, if the three drinking states are represented by 0 for no drinking, 1 for drinking, and 2 for drunkenness, then the distance between no drinking and drinking is 1, the distance between drinking and drunkenness is 1, and the distance between no drinking and drunkenness is 2. This is completely unacceptable when participating in the loss calculation. There should be no inequality between independent labels.

[0099] In this case, the one-hot encoding method can be used to represent the three drinking states, for example, [1,0,0], [0,1,0] and [0,0,1] respectively. In the three-dimensional coordinate system, the three vectors [1,0,0], [0,1,0] and [0,0,1] are perpendicular to each other, that is, they are orthogonal and independent. This solves the problem of unequal representation methods between independent labels mentioned above.

[0100] Furthermore, the sample spectral information, sample physiological information and sample labels after one-hot encoding are input into the drinking state prediction model for training. The drinking state prediction model includes a feature extraction module, a feature fusion unit and a prediction module. The feature extraction module includes a frequency domain feature extraction unit and a time domain feature extraction unit. The frequency domain feature extraction unit includes at least a stacked convolutional network and a pooling network, an activation function layer and a fully connected layer. The time domain feature extraction unit includes a bidirectional long short-term memory network and a multi-layer perceptron. The prediction module includes a fully connected layer and a Softmax layer.

[0101] Figure 3 A schematic flow chart of a single training process of a drinking state prediction model provided in an embodiment of the present disclosure, such as Figure 3 As shown, the above step S22 includes: each training of the drinking status prediction model includes steps S01 to S05, specifically:

[0102] Step S01: inputting a plurality of sample spectrum information in a sample group into a frequency domain feature extraction unit to obtain sample spectrum feature information.

[0103] In one example, the frequency domain feature extraction unit consists of 2 convolution blocks, 1 pooling layer, 1 Flatten layer, 1 fully connected layer and an output layer. The convolution block contains 1 one-dimensional convolution layer (1DCNN), 1 batch normalization layer (BN) and 1 ReLU activation function. Convolution and pooling are important operation processes in the model, which can perform deep extraction and dimensionality reduction of spectral features. In the convolution layer, a certain number of trainable convolution kernels are slid and translated to perform weighted summation operations on the input spectral feature vector. The result of the convolution operation is then transformed by the activation function to generate an output feature vector. Specifically, the convolution kernel corresponding to the first ( l -1) The j-th feature of the layer performs an element-wise product operation with the j-th position weight corresponding to the convolution kernel, and finally sums all the products. The above convolution operation can be expressed by formula 1:

[0104] Formula 1

[0105] in, It is Layer output features, It is the Layer output features, is the size of the convolution kernel, It is Layer Features and Layer The weight of the convolution kernel between features, is the bias value, It is The number of features of the layer, is the activation function.

[0106] In the pooling layer, the local maximum of the feature vector on each channel output by the convolutional layer is calculated by the 1D MaxPooling layer, which is to reduce the dimension of the feature vector and the weight vector. In addition to Max Pooling, Average Pooling and Global Average Pooling are also used for pooling operations.

[0107] Specifically, the following operations can be performed in the frequency domain feature extraction unit:

[0108] 1. Spectral data of size (Batch_size, Num_of_Features) Input to the first convolutional block, using The trainable size is *1 Conv1D convolves the input separately to obtain a multi-channel feature matrix . The output After Batch Normalization (BN), the ReLU activation function is used to get the output of the first convolutional block. . Batch_size is the number of samples in each Batch, and Num_of_Features is the number of spectral feature wavelengths.

[0109] 2. The output of the first convolutional block As the input of the second convolutional block, use The trainable size is *1 Conv1D convolves the input separately to obtain a multi-channel feature matrix . The output After passing through the BN layer, the ReLU activation function is used to obtain the output of the second convolutional block. .

[0110] 3. Use pool size The Max Pooling layer is Perform pooling operation, slide the pooling window to perform Max Pooling on each feature vector in the channel to obtain the local maximum value, so as to reduce the dimension and obtain the output of the pooling layer .

[0111] 4. Input to the Flatten layer for feature concatenation and flattening to generate a feature vector , this step is to be able to connect to the next fully connected layer.

[0112] 5. Connect to the fully connected layer, perform feature extraction and function mapping again, and get the final spectral feature output .

[0113] in The adjustment of parameters such as will have a significant impact on the prediction results and needs to be continuously adjusted according to the actual data training situation.

[0114] In the above training process, the receptive field (RF) of neurons is gradually expanded by using stacked convolution and pooling operations, so as to realize the learning and extraction of features at different abstract levels of spectral data. It can not only capture local wavelength features, but also capture a wider range of wavelength combination features as the network depth increases, and it is not easy to lose information.

[0115] Step S02: inputting physiological information of multiple samples in the sample group into a time domain feature extraction unit to obtain physiological feature information of the samples.

[0116] The time domain feature extraction unit is used to extract the features of the electrocardiogram signal and the photoplethysmogram signal. These features can characterize the heart rate, heart rate variability, respiratory rate, blood pressure and other information of the subject, which plays an auxiliary correction role in judging whether the subject is drinking alcohol, and is important multimodal biosensor information. Since it is a time series signal, it is extracted and characterized by a bidirectional long short-term memory network (BiLSTM) and a multi-layer perceptron (MLP). BiLSTM is a variant of a recurrent neural network (RNN) that combines information flows in both forward and reverse directions to capture features and patterns in sequence data. The outputs of the two LSTMs are connected or merged together at each time step.

[0117] Specifically, the following operations can be performed in the time domain feature extraction unit:

[0118] 1. The denoised data is of size (Batch_size, Time Steps, Num_of_ Signal Channels) Connect to the BiLSTM layer through the input layer. The parameter return_sequences is set to True and the number of hidden units is set to . Get the output features Time Steps is the number of time steps in the time series, which is the number of time points of the signal. Num_of Signal Channels is the number of channels for signal acquisition, which is the number of leads recorded for the ECG signal and the number of sensors for the photoplethysmography signal, which can be understood as the feature dimension of each time step.

[0119] 2. The features After being flattened, it is connected to the first hidden layer of the MLP, which consists of The input data is combined with the connection weights , Bias Calculate and then pass an activation function to get the feature .

[0120] 3. The features Through the Dropout layer, we get the features , where the Dropout rate is preferably 0.3.

[0121] 4. The features Through the second The hidden layer of neurons is connected with weights , Bias The calculation is then transformed through the activation function to obtain the feature .

[0122] 5. The features Through the Dropout layer, the final feature representation of this module is obtained .

[0123] in are the parameters that need to be adjusted before training. These are the parameters that need to be learned during model training.

[0124] In the time domain feature extraction unit, based on technologies such as Dropout and regularization, the noise level of the regression coefficient can be reduced, the sensitivity of the model to noise and random changes can be reduced, and overfitting can be avoided; based on optimization algorithms such as backpropagation and gradient descent, dynamic adjustment of learning rate and other technologies, the model can find the global optimal solution.

[0125] Step S03: input the sample spectral feature information and the sample physiological feature information into a feature fusion unit for splicing and fusion to obtain a sample fusion feature.

[0126] Specifically, the features extracted from the first two modules are and Perform splicing and fusion to generate fusion features , as the final representational feature of the target subject’s drinking state.

[0127] Step S04: input the sample fusion features into the prediction module to obtain multiple prediction labels and their corresponding prediction probabilities.

[0128] Specifically, the output after feature fusion is It will be input into a fully connected layer for final feature extraction and conversion. If overfitting occurs during training, the Dropout layer will be selectively added to the fully connected layer according to the situation. Finally, the output of the fully connected layer will enter the SoftMax function for classification calculation. The output of the SoftMax function is the probability distribution of each category, and the category corresponding to the largest output value is considered to be the prediction result of the model.

[0129] Step S05, calculate the loss function to measure the difference between the probability distribution predicted by the model and the true label of the corresponding sample, and iteratively optimize the loss function until convergence by adjusting the model parameters to improve the performance of the model in the classification task.

[0130] Among them, the loss function is the cross-entropy loss (Cross-Entropy Loss) in classification.

[0131] It should be noted that the number of network layers in the above-mentioned drinking status prediction model, such as the number of convolutional layers and pooling layers, can be flexibly adjusted by those skilled in the art according to the training conditions of the model, and is not limited in the embodiments of the present disclosure.

[0132] As a second aspect of the present disclosure, an embodiment of the present disclosure further provides a drinking status prediction system, the system comprising: a drinking status prediction device, a near infrared spectrum detection device and a biosensor instrument.

[0133] Wherein, the drinking status prediction device is used for the drinking status prediction method described in the first aspect.

[0134] The near-infrared spectrum detection device is used to detect the target object to obtain spectral information. Preferably, the wavelength range detected by the near-infrared spectrum detection device is 800nm-2500nm, and the light source is a full-band halogen lamp light source. Specifically, the detection can be performed on the wrist position of the object to be detected, the inner and outer sides of the upper arm, the earlobe, the fingertips and other locations where subcutaneous capillaries are densely distributed, which is not limited in the embodiments of the present disclosure.

[0135] Biosensor instruments are used to detect the physical indicators of the target object to obtain physiological information. For example, the sensor for collecting photoelectric volumetric signal is placed on the middle finger of the right hand, and the electrodes for collecting electrocardiogram signal are placed at specific positions of the left and right sternum and ribs to record the electrical activity of the heart in different directions.

[0136] The drinking status prediction system provided by the embodiment of the present disclosure includes a variety of sensor devices for acquiring information, including but not limited to the above-mentioned near-infrared spectroscopy detection equipment and biosensor instruments. Based on this, multimodal information is acquired, and the multimodal information includes at least spectral information and physiological information, and the drinking status of the subject to be detected is predicted and analyzed in combination with the drinking status prediction model. Based on the above-mentioned multimodal information, the information collected by different device ends is supplemented. By integrating multiple data sources, the error caused by the detection of a single data type is reduced, and the accuracy of the drinking status prediction results is improved.

[0137] The above-mentioned spectral information and physiological information are consistent with the description of the first aspect of the present disclosure and will not be repeated here.

[0138] As a third aspect of the present disclosure, an embodiment of the present disclosure further provides an electronic device, wherein, Figure 4 As shown, the electronic device may include:

[0139] One or more processors 101;

[0140] The memory 102 stores one or more computer programs. When the one or more computer programs are executed by the one or more processors 101, the one or more processors 101 implement the steps of the drinking status prediction method provided according to the first aspect of the present disclosure.

[0141] The electronic device may further include one or more I / O interfaces 103 connected between the processor and the memory and configured to implement information interaction between the processor and the memory.

[0142] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, and can realize information exchange between the processor and the memory, including but not limited to a data bus (Bus), etc.

[0143] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0144] As an optional implementation, the electronic device may also include a display, and the display is used to display a human-computer interaction interface, through which instructions can be received and status uploaded.

[0145] like Figure 5 As shown, the present invention also provides a computer-readable medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the drinking status prediction method provided in the first aspect of the present disclosure are implemented.

[0146] It can be understood by a person of ordinary skill in the art that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. Accordingly, the computer program can be stored in a non-volatile computer-readable storage medium, and when the computer program is executed, the method of any of the above-mentioned embodiments can be implemented. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0147] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Technical objects familiar with the art should understand that the present disclosure includes but is not limited to the contents described in the drawings and the above specific embodiments. Any modification that does not deviate from the functional and structural principles of the present disclosure will be included in the scope of the claims.

Claims

1. A method for predicting drinking status, characterized in that: include: Acquiring spectral information and physiological information corresponding to the object to be detected; Based on the drinking status prediction model, the spectral information and the physiological information are feature fused to obtain fused information, and the drinking status of the subject to be detected is predicted based on the fused information to obtain multiple prediction labels and their corresponding prediction probabilities, wherein the multiple prediction labels correspond to multiple different drinking statuses, and the drinking status prediction model carries the corresponding relationship between the drinking status of the subject to be detected and the fused information; wherein the spectral information is obtained by using a near-infrared spectral detection device to detect a part of the subject to be detected where subcutaneous capillaries are densely distributed, and the physiological information includes an electrocardiogram signal and / or a photoplethysmogram signal; Determine the prediction label corresponding to the maximum value among the plurality of prediction probabilities as the prediction result of the drinking state of the subject to be detected; The drinking state prediction model includes a feature extraction module, which includes a frequency domain feature extraction unit, a time domain feature extraction unit and a feature fusion unit. The frequency domain feature extraction unit includes at least a stacked convolutional network and a pooling network, an activation function layer and a fully connected layer. The spectral information and the physiological information are subjected to feature fusion to obtain fusion information, including: Based on the frequency domain feature extraction unit, feature extraction is performed on the spectral information to obtain spectral features; wherein the convolution network performs a weighted summation operation on the spectral information through a sliding translation of a trainable convolution kernel to obtain a convolution operation result, and converts the convolution operation result to output a spectral feature vector; Based on the time domain feature extraction unit, feature extraction is performed on the physiological information from two time dimensions, namely, forward and reverse, to obtain physiological features; Based on the feature fusion unit, the spectral feature and the physiological feature are feature spliced ​​to obtain the fusion information.

2. The drinking status prediction method according to claim 1, characterized in that: The method further comprises: Training the drinking status prediction model according to the training sample set until the drinking status prediction model carries the corresponding relationship; The training sample set includes at least sample spectral information and sample physiological information.

3. The drinking status prediction method according to claim 2, characterized in that: The drinking status prediction model also includes a prediction module. The step of training the drinking status prediction model according to the training sample set until the drinking status prediction model carries the corresponding relationship includes: Acquire the training sample set, wherein the training sample set includes a plurality of sample groups, each of the sample groups includes sample information corresponding to a plurality of target objects, the sample information includes sample spectral information, sample physiological information and sample labels, and the sample labels are used to represent the drinking status; Inputting the training sample set into the feature extraction module, and obtaining sample fusion information according to the sample spectral information and the sample physiological information; Inputting the sample fusion information into the prediction module to obtain multiple prediction labels and their corresponding prediction probabilities, and performing multiple error back propagation optimization trainings in combination with the cross entropy loss function and the Adam algorithm to adjust the model parameters until the corresponding relationship is carried in the drinking status prediction model; The sample fusion information is obtained based on the sample spectral information and the sample physiological information.

4. The drinking status prediction method according to claim 3, characterized in that: The time domain feature extraction unit at least includes a bidirectional long short-term memory network and a multi-layer perceptron, and the prediction module at least includes a fully connected layer and a normalized Softmax layer. Inputting the training sample set into the drinking status prediction model, and obtaining sample fusion information according to the sample spectral information and the sample physiological information; According to the sample fusion information, multiple prediction labels and their corresponding prediction probabilities are obtained, and multiple error back propagation optimization training is performed in combination with the cross entropy loss function and the optimized Adam algorithm, including: The drinking state prediction model is subjected to multiple error back propagation optimization trainings, wherein each training includes the following steps: Inputting the spectral information of multiple samples in the sample group into the frequency domain feature extraction unit to obtain the spectral feature information of the samples; Inputting the physiological information of multiple samples in the sample group into the time domain feature extraction unit to obtain the physiological feature information of the samples; Inputting the sample spectral feature information and the sample physiological feature information into the feature fusion unit for splicing and fusion to obtain sample fusion feature information; Inputting the sample fusion features into the prediction module to obtain multiple prediction labels and their corresponding prediction probabilities; According to the cross entropy loss function and the optimized Adam algorithm, the loss value between the predicted label and the corresponding sample label is calculated until the loss value converges, thereby obtaining the drinking status prediction model carrying the corresponding relationship.

5. The drinking status prediction method according to claim 1, characterized in that: The electrocardiogram signal is used to characterize at least one of heart rate and heart rate variability, and the photoplethysmogram signal is used to characterize at least one of blood pressure, blood oxygen saturation and respiratory rate.

6. The drinking status prediction method according to claim 1, characterized in that: The plurality of prediction labels include not drinking, drinking, and being drunk.

7. A drinking status prediction system, characterized in that: The system comprises: A drinking status prediction device, used to execute the drinking status prediction method according to any one of claims 1 to 6; Near infrared spectrum detection equipment, used to detect the target object to obtain spectrum information; Biosensor instruments are used to detect the physical indicators of the target object to obtain physiological information.

8. The drinking status prediction system according to claim 7, characterized in that: The wavelength range detected by the near-infrared spectrum detection equipment is 800nm-2500nm, and the light source is a full-band halogen lamp light source.

9. An electronic device, wherein: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the drinking status prediction method according to any one of claims 1 to 6 is executed.

10. A computer non-transitory readable storage medium, wherein: The computer non-transitory readable storage medium stores a computer program, and when the computer program is executed by a processor, the drinking status prediction method according to any one of claims 1 to 6 is executed.

Images