A method, device, equipment and readable storage medium for smoking identification

Abstract

The present application belongs to the field of intelligent detection technology, and discloses a method, device, equipment and readable storage medium for smoking identification, the method comprising: obtaining an image of a person to be tested to be identified; performing smoking identification on the image of the person to be tested to be identified based on a smoking identification model trained in advance, and obtaining a smoking identification result of the person to be tested. The method can achieve the effect of conveniently and accurately performing smoking detection on the person to be tested.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection technology, and more specifically, to a method, device, equipment and readable storage medium for smoking identification. Background Art

[0002] Currently, users' smoking health data is usually obtained through notification by the users or testing by testing agencies.

[0003] Under the existing technology, when detecting the smoking level of a user, personal information and facial features extracted using a face recognition model are usually combined to obtain the user's smoking data.

[0004] However, this method requires a large amount of personal information of the subject, and requires the subject to cooperate in providing a variety of personal information. When testing the user's smoking level, the testing process is relatively complicated and the test results are inaccurate.

[0005] Therefore, when detecting the smoking degree of a user, how to detect the smoking degree of the user conveniently and accurately is a technical problem that needs to be solved. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a method for smoking identification. Through the technical solution of the embodiments of the present application, the effect of conveniently and accurately detecting the smoking degree of a user can be achieved.

[0007] In a first aspect, an embodiment of the present application provides a method for smoking identification, comprising obtaining an image to be identified of a person to be tested; performing smoking identification on the image to be identified of the person to be tested based on a smoking identification model trained in advance, and obtaining a smoking identification result of the person to be tested, wherein the smoking identification model is constructed based on a neural network, and is used to determine the smoking degree of the person to be tested based on the predicted probability of each smoking category of the person to be tested.

[0008] In the above process, the smoking detection result of the subject to be tested can be directly obtained according to the image, so as to achieve the effect of convenient and accurate smoking detection of the subject to be tested.

[0009] In one embodiment, before obtaining the image to be recognized of the subject, the method further includes:

[0010] Acquire sample data for training a smoking recognition model, wherein the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image, and the smoking recognition data includes identity sample information of a subject and a smoking index of the sample of the subject;

[0011] Preprocessing each sample image to obtain a preprocessed image corresponding to each sample image;

[0012] According to each preprocessed image and identity sample information of each preprocessed image, an initial face recognition model is trained to obtain a trained face recognition model, wherein the initial face recognition model is constructed based on a neural network;

[0013] Based on the face recognition model, the smoking classification module and the smoking index prediction module, an initial smoking recognition model is constructed. The smoking classification module is used to determine the smoking category of the subject, and the smoking index prediction module is used to predict the smoking index of the subject.

[0014] Based on each preprocessed image and the smoking index corresponding to each preprocessed image, the initial smoking recognition model is trained to obtain a smoking recognition model.

[0015] In the above process, a smoking recognition model is constructed using a face recognition model, a smoking classification module and a smoking index prediction module. The smoking recognition model obtained by this method can make the prediction of the smoking recognition result of the final subject more accurate when used.

[0016] In one implementation, the initial smoking recognition model is trained based on each preprocessed image and the smoking index corresponding to each preprocessed image to obtain the smoking recognition model, including:

[0017] Based on each preprocessed image and the initial smoking recognition model, a predicted probability of smoking category of each subject and a predicted smoking index of the subject are obtained;

[0018] Obtain the smoking category loss based on the predicted probability of the smoking category corresponding to each preprocessed image;

[0019] Based on the predicted smoking index of the subject and the smoking index of the sample, the smoking index loss is obtained;

[0020] According to the smoking category loss and the smoking index loss, the parameters of the initial smoking recognition model are adjusted until a smoking recognition model is obtained.

[0021] In the above process, the model can be continuously adjusted by calculating the smoking category loss and the smoking index loss until the final trained smoking recognition model is obtained. The smoking recognition model trained by this method can reduce the error of the results when using the model to predict the smoking recognition of the subject, or even eliminate the error.

[0022] In one implementation, each sample image is preprocessed to obtain a preprocessed image corresponding to each sample image, including:

[0023] For each sample image, perform the following steps:

[0024] Performing face detection on a sample image to obtain a face detection area in the sample image;

[0025] Based on the face detection area, a sample image is intercepted to obtain a intercepted face image;

[0026] Extract key points from the face image to obtain at least two face key points corresponding to the face image;

[0027] According to at least two facial key points, the facial images are aligned to obtain a pre-processed image.

[0028] In the above process, preprocessing the acquired sample images by the above method can make the final facial image range smaller, and the efficiency of smoking recognition by the smoking recognition model is higher, which can save more resources.

[0029] In one embodiment, the smoking recognition model includes a feature extraction module, a smoking classification module, and a smoking index prediction module. Based on the pre-trained smoking recognition model, smoking recognition is performed on the image to be recognized to obtain the smoking recognition result of the subject to be tested, including:

[0030] Preprocessing the image to be identified of the subject to be tested to obtain a preprocessed image;

[0031] Input the preprocessed image into the feature extraction module to obtain the feature matrix;

[0032] Input the feature matrix into the smoking category module to obtain the predicted probabilities of multiple smoking categories of the subject;

[0033] The feature matrix is ​​input into the smoking index prediction module to obtain the smoking recognition result.

[0034] In the above process, the feature matrix of the subject is obtained through the feature extraction module, and the predicted probability of the smoking category of the subject is obtained through the smoking category module, so that the smoking recognition result of the subject can be obtained more conveniently. The accuracy of the smoking degree of the subject obtained by this method is higher.

[0035] In one implementation, the feature matrix is ​​input into a smoking index prediction module to obtain a smoking identification result, including:

[0036] Determine the maximum probability among the predicted probabilities for each smoking category;

[0037] If the smoking category corresponding to the maximum probability indicates that the subject does not smoke, then a smoking recognition result indicating that the subject does not smoke is obtained;

[0038] Otherwise, the smoking recognition result of the subject is obtained according to the category label value set for each smoking category and the probability of each smoking category.

[0039] In the above process, the predicted probability of each smoking category obtained by model prediction can be used to calculate the final smoking recognition result based on the predicted probability of each category and the corresponding category label value. The smoking recognition result calculated by this method is more accurate.

[0040] In one implementation, according to the category label values ​​set for each smoking category and the probability of each smoking category, the smoking recognition result of the subject is obtained, including:

[0041] Get the category label value set for each smoking category;

[0042] Based on the label values ​​of each category and the predicted probability of each smoking category, the predicted smoking index of the subject is obtained;

[0043] Obtaining a smoking category set for the predicted smoking index of the subject to be tested, where the smoking category is used to indicate the smoking degree of the subject to be tested;

[0044] Based on the smoking category set by the predicted smoking index, a smoking recognition result of the subject is obtained.

[0045] In the above process, the predicted smoking index of the subject is calculated by using the smoking category label value and the predicted probability of each category, and the smoking recognition result of the subject can also be determined according to the smoking index.

[0046] In a second aspect, an embodiment of the present application provides a smoking identification device, comprising:

[0047] An acquisition module, used for acquiring an image to be identified of a subject;

[0048] The recognition module is used to perform smoking recognition on the image to be recognized by the subject based on a pre-trained smoking recognition model to obtain the smoking recognition result of the subject, wherein the smoking recognition model is constructed based on a neural network and is used to determine the smoking degree of the subject according to the predicted probability of each smoking category of the subject.

[0049] In one embodiment, the device further comprises:

[0050] A second acquisition module is used for acquiring sample data for training a smoking recognition model before acquiring an image to be recognized of a subject to be tested, wherein the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image, and the smoking recognition data includes identity sample information of the subject to be tested and a smoking index of the sample of the subject to be tested;

[0051] Preprocessing each sample image to obtain a preprocessed image corresponding to each sample image;

[0052] According to each preprocessed image and identity sample information of each preprocessed image, an initial face recognition model is trained to obtain a trained face recognition model, wherein the initial face recognition model is constructed based on a neural network;

[0053] Based on the face recognition model, the smoking classification module and the smoking index prediction module, an initial smoking recognition model is constructed. The smoking classification module is used to determine the smoking category of the subject, and the smoking index prediction module is used to predict the smoking index of the subject.

[0054] Based on each preprocessed image and the smoking index corresponding to each preprocessed image, the initial smoking recognition model is trained to obtain a smoking recognition model.

[0055] In one implementation, the second acquisition module is specifically used to:

[0056] Based on each preprocessed image and the initial smoking recognition model, a predicted probability of smoking category of each subject and a predicted smoking index of the subject are obtained;

[0057] Obtain the smoking category loss based on the predicted probability of the smoking category corresponding to each preprocessed image;

[0058] Based on the predicted smoking index of the subject and the smoking index of the sample, the smoking index loss is obtained;

[0059] According to the smoking category loss and the smoking index loss, the parameters of the initial smoking recognition model are adjusted until a smoking recognition model is obtained.

[0060] In one implementation, the second acquisition module is specifically used to:

[0061] For each sample image, perform the following steps:

[0062] Performing face detection on a sample image to obtain a face detection area in the sample image;

[0063] Based on the face detection area, a sample image is intercepted to obtain a intercepted face image;

[0064] Extract key points from the face image to obtain at least two face key points corresponding to the face image;

[0065] According to at least two facial key points, the facial images are aligned to obtain a pre-processed image.

[0066] In one implementation, the identification module is specifically used to:

[0067] Preprocessing the image to be identified of the subject to be tested to obtain a preprocessed image;

[0068] Input the preprocessed image into the feature extraction module to obtain the feature matrix;

[0069] Input the preprocessed image into the smoking category module to obtain the predicted probabilities of multiple smoking categories of the subject;

[0070] The predicted probabilities of multiple smoking categories are input into the smoking index prediction module to obtain the smoking recognition results.

[0071] In one implementation, the identification module is specifically used for:

[0072] Determine the maximum probability among the predicted probabilities for each smoking category;

[0073] If the smoking category corresponding to the maximum probability indicates that the subject does not smoke, then a smoking recognition result indicating that the subject does not smoke is obtained;

[0074] Otherwise, the smoking recognition result of the subject is obtained according to the category label value set for each smoking category and the probability of each smoking category.

[0075] In one implementation, the identification module is specifically used for:

[0076] Get the category label value set for each smoking category;

[0077] Based on the label values ​​of each category and the predicted probability of each smoking category, the predicted smoking index of the subject is obtained;

[0078] Obtaining a smoking category set for the predicted smoking index of the subject to be tested, where the smoking category is used to indicate the smoking degree of the subject to be tested;

[0079] Based on the smoking category, the smoking recognition result of the subject is obtained.

[0080] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0081] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0082] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0084] Figure 1 A flowchart of a method for training a smoking recognition model provided in an embodiment of the present application;

[0085] Figure 2 A flowchart of a method for smoking identification provided in an embodiment of the present application;

[0086] Figure 3 A detailed flow chart of a method for smoking identification provided in an embodiment of the present application;

[0087] Figure 4 A detailed flow chart of a method for training a smoking recognition model provided in an embodiment of the present application;

[0088] Figure 5 A schematic block diagram of a smoking identification device provided in an embodiment of the present application;

[0089] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0090] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0091] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0092] First, some terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0093] Smoking index: It is the product of the average number of cigarettes smoked per day and the number of years of smoking. The smoking index shows the relationship between the cumulative amount of smoking and the risk of lung cancer. For example, if you smoke 20 cigarettes a day and start smoking at the age of 50, you will reach this danger line at the age of 70. If you start smoking at the age of 20, you will reach it at the age of 40. Therefore, the incidence of lung cancer is highest between the ages of 50 and 59, and the incidence of lung cancer continues to rise after the age of 40. Medical scientists regard smokers with a smoking index greater than 400 as a high-risk group for lung cancer. For the insurance industry, whether a customer smokes and the smoking index are one of the important bases for judging their health risk level. Usually, people can be divided into three categories: non-smokers, low-risk smokers (smoking index <= 400) and high-risk smokers (smoking index > 400).

[0094] Convolutional Neural Networks (CNN) are a type of feedforward neural network that includes convolutional calculations and has a deep structure. The convolutional neural network used in this application can be AlexNet (image recognition network), VGG (Visual Geometry Group Network, target image generator), ResNet (residual network) and MobileNet-v2 (mobile terminal network) and other convolutional neural networks.

[0095] Classifier: Classifier is a general term for methods of classifying samples in data mining, including algorithms such as decision trees, logistic regression, naive Bayes, and neural networks.

[0096] The above-mentioned smoking index is used to represent the smoking index of the sample in this application.

[0097] At present, when testing the smoking level of users, personal information and facial features extracted by face recognition models are usually combined to obtain the user's smoking data. This method requires a large amount of personal information of the tested person and requires the tested person to cooperate in providing a variety of personal information. When testing the smoking level of users, the testing process is relatively complicated and the test results are inaccurate.

[0098] To this end, the present application provides a smoking recognition model. By inputting the image of the subject to be tested into the model, the smoking degree of the subject to be tested can be directly obtained.

[0099] In the embodiment of the present application, the execution entity may be a smoking identification device in a smoking identification system. In actual applications, the smoking identification device may be an electronic device such as a terminal device, a server, or a firewall, and is not limited here.

[0100] In the embodiment of the present application, when detecting the smoking degree of the subject, a smoking recognition model is first trained, and then the image of the subject is input into the smoking recognition model to complete the detection of the smoking degree of the subject.

[0101] Please see Figure 1 , Figure 1 This is a flowchart of a method for training a smoking recognition model provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0102] Step 100: Obtain sample data for training a smoking recognition model.

[0103] Specifically, the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image.

[0104] A plurality of sample images and smoking recognition data corresponding to each sample image are obtained.

[0105] Specifically, the smoking identification data includes the identity sample information of the subject and the smoking index of the sample of the subject.

[0106] Among them, the sample data of the smoking recognition model can be some data stored in the historical system, or it can be data obtained from health tests of the subjects by some health testing institutions, or other data that can be used as data for training the smoking recognition model of this application. This application is not limited to this.

[0107] Step 110: pre-process each sample image to obtain a pre-processed image corresponding to each sample image.

[0108] When executing step 110, for each sample image, the following steps may be taken:

[0109] S1101: Perform face detection on a sample image to obtain a face detection area in the sample image.

[0110] Among them, the face area can be obtained by detecting the sample image through the target detection algorithm (SSD, Single Shot MultiBox Detector).

[0111] S1102: Based on the face detection area, a sample image is captured to obtain a captured face image.

[0112] In one implementation, the position of the face area of ​​the subject is extracted and the face image is captured.

[0113] Among them, the position of the face area of ​​the subject is extracted, and the face position is obtained by: [x start ,y start,x end ,y end ], where [x start ,y start ] and [x end ,y end ] are the upper left and lower right coordinates of the face bounding box obtained by the target detection algorithm, and the coordinates of the center point of the face bounding box [x center ,y center ]. The calculation method is as follows:

[0114] x center =(x start +x end ) / 2;

[0115] y center =(y start +y end ) / 2.

[0116] In one implementation, the following steps may be used to capture a face image based on the face detection area:

[0117] Calculate the distances between the four sides of the face bounding box and the center point respectively, take the maximum distance as d, set the scaling ratios s1, s2, s3, s4 in the four directions of up, down, left and right, and multiply them by d to get the new distances between the four sides of the face bounding box and the center point, and cut out the face image that is more suitable for face key point detection.

[0118] Among them, X represents the horizontal coordinate of the key point, Y represents the vertical coordinate of the key point, and X start is the horizontal coordinate of the upper left point, X end is the horizontal coordinate of the lower right point, Y start is the ordinate of the upper left point, Y start is the ordinate of the lower right point, X center is the horizontal coordinate of the center point, Y center is the vertical coordinate of the center point.

[0119] S1103: Extract key points from the face image to obtain at least two face key points corresponding to the face image.

[0120] In one embodiment, a key point detection model may be used to obtain some key points in the face.

[0121] For example, the key points of the corner of the eye, the middle of the eye, the tip of the nose and the two corners of the mouth can be: [x1, y1], [x2, y2]…[x M ,y M ].

[0122] Among them, X represents the horizontal coordinate of the key point, Y represents the vertical coordinate of the key point, and M represents the Mth key point.

[0123] S1104: Align the facial images according to at least two facial key points to obtain a pre-processed image.

[0124] In the above process, the target detection algorithm is first used to perform target detection on the sample image to obtain the face area, the obtained face area is cropped, and the key point detection model is used to perform key point detection on the cropped image. The face in the image can be aligned according to the key points detected in the face, and finally a preprocessed image is obtained. After preprocessing, the preprocessed image can be directly used for model training or for smoking detection of the subject, which can save more resources during model training and use, thereby improving the efficiency of smoking detection.

[0125] Step 120: Train the initial face recognition model according to each preprocessed image and the identity sample information of each preprocessed image to obtain a trained face recognition model.

[0126] The initial face recognition model is constructed based on a neural network. After training the initial face recognition model, its fully connected layer is removed and only its feature extraction part is retained. The identity sample information is used to indicate the identity of the subject.

[0127] Specifically, by training the initial face recognition model with the preprocessed images and the identity sample information corresponding to the images, a more accurate face recognition model can be obtained, and the results obtained when using the model to detect smoking in the subject are more accurate.

[0128] Step 130: constructing an initial smoking recognition model based on the face recognition model, the smoking classification module and the smoking index prediction module.

[0129] Specifically, the smoking classification module is used to determine the smoking category of the subject, and the smoking index prediction module is used to predict the smoking index of the subject, which can be further improved in subsequent steps to obtain the final smoking recognition model.

[0130] Step 140: training the initial smoking recognition model based on each pre-processed image and the smoking index corresponding to each pre-processed image to obtain a smoking recognition model.

[0131] Specifically, the initial smoking recognition model is trained by using the preprocessed image and the smoking index of the sample corresponding to the subject in the image to obtain the final smoking recognition model.

[0132] When executing step 140, the following steps may be taken:

[0133] S1401: Based on each pre-processed image and the initial smoking recognition model, the predicted probability of smoking category of each subject and the predicted smoking index of the subject are obtained.

[0134] Among them, the predicted probability of the smoking category is the predicted probability of each category obtained by inputting the face image into the initial smoking recognition model for prediction.

[0135] S1402: Obtain smoking category loss based on the predicted probability of the smoking category corresponding to each pre-processed image.

[0136] Among them, the smoking recognition results can be divided into three categories: non-smoking, low-risk smoking, and high-risk smoking, and the three label values ​​are 0, 1, and 2 respectively. The method for calculating the smoking category loss based on the predicted probability of the smoking category obtained by the face recognition model based on the key point data and the predicted probability of the sample image can be calculated using the following formula:

[0137]

[0138] Among them, L1 is the cross entropy loss function of the smoking classification module, N is the number of sample images, and y ic is a sign function (0 or 1), i represents the serial number of the sample image, and c is the smoking category (for example, c is 0, 1, or 2). ic is the predicted probability of smoking category c for sample i, and M represents the number of smoking categories.

[0139] S1403: Obtain the smoking index loss based on the predicted smoking index of the subject and the smoking index of the sample.

[0140] The smoking index of the sample is: x, where x is a continuous value.

[0141] Furthermore, x is discretized to obtain a discretized smoking index y. For example, x is discretized by dividing it by 50 and rounding it. Usually, the smoking index x ranges from [0, 1500]. x=0 means non-smoker, and y ranges from [0, 30] and y is an integer.

[0142] According to the predicted smoking index of the subject and the smoking index of the sample, the smoking index loss can be obtained using the following formula:

[0143]

[0144] si′=∑ i k*P′ ik .

[0145] Among them, L2 is the mean square error loss function of the smoking prediction module, N is the number of samples, si is the smoking index of sample i, si' is the smoking index of sample i predicted by the model, k is the label of the smoking category, and P'ik is the probability of the smoking category of category k.

[0146] S1404: According to the smoking category loss and the smoking index loss, the parameters of the initial smoking recognition model are adjusted until a smoking recognition model is obtained.

[0147] Among them, the formula for calculating the loss function of the entire model based on the smoking category loss and the smoking index loss is as follows:

[0148] L=αL1+βL2

[0149] Among them, α and β are weights, the sum of α and β is 1, L is the loss of the entire model, L1 is the smoking class loss, and L2 is the smoking index loss.

[0150] In the above process, the image is input into the initial smoking recognition model to obtain the predicted probability of the smoking category and the predicted smoking index, and then the predicted probability of the smoking category and the predicted smoking index are compared with the predicted probability of the smoking category and the smoking index in the sample to calculate the smoking category loss and the smoking index loss. The parameters of the model are adjusted by the smoking category loss and the smoking index loss until the final smoking category loss and the smoking index loss reach the expected value. The model at this time is the trained smoking recognition model. The smoking recognition model obtained by this method can make the final prediction of the smoking recognition result of the subject more accurate when the model is used.

[0151] Through the above method, a smoking recognition model can be obtained based on the training of the face recognition model, and then the smoking detection of the subject can be realized by detecting the image of the subject.

[0152] Combine the following Figure 2 The method for smoking identification in the embodiment of the present application is described in detail.

[0153] Please see Figure 2 , Figure 2 A flowchart of a method for smoking identification provided in an embodiment of the present application is shown in FIG. Figure 2 The smoking identification method shown includes:

[0154] Step 200: Obtain an image of a subject to be identified.

[0155] Specifically, the corresponding image to be identified may be directly acquired through the face of the subject to be tested, or the image to be identified of the subject to be tested may be provided by the system, but the present application is not limited thereto.

[0156] Step 210: Based on the pre-trained smoking recognition model, smoking recognition is performed on the image of the subject to be tested to obtain a smoking recognition result of the subject to be tested.

[0157] Specifically, the smoking recognition model is constructed based on a neural network and is used to determine the smoking degree of the subject according to the predicted probability of each smoking category of the subject. By inputting the image of the subject into the trained smoking recognition model, the corresponding smoking detection result of the subject can be obtained.

[0158] In one embodiment, the smoking recognition model includes a feature extraction module, a smoking classification module, and a smoking index prediction module. When executing step 210, the following steps may be used:

[0159] S2101: Preprocess the image to be recognized of the subject to be tested to obtain a preprocessed image.

[0160] S2102: Input the preprocessed image into the feature extraction module to obtain a feature matrix.

[0161] S2103: Input the feature matrix into the smoking category module to obtain the predicted probabilities of multiple smoking categories of the subject.

[0162] S2104: Input the feature matrix into the smoking index prediction module to obtain a smoking identification result.

[0163] In the above process, the feature matrix is ​​input into the feature extraction module, the smoking category module and the smoking index prediction module to process the image to be identified, so that the smoking identification result of the subject can be obtained more conveniently. The results obtained by this method are more efficient.

[0164] Among them, the smoking category module is used to confirm the predicted probability of the smoking category, and the smoking index prediction module is used to calculate the predicted smoking index of the subject to be tested based on the predicted probability of the smoking category. The feature matrix is ​​used to display the feature values ​​of the subject to be tested according to the matrix. The smoking recognition results include: non-smoking, low-risk smoking, high-risk smoking, smoking index corresponding to low-risk smoking and high-risk smoking, etc., and the present application is not limited to this.

[0165] In one implementation, when executing step 2104, the following steps may also be performed:

[0166] S21041 determines the maximum probability among the predicted probabilities for each smoking category.

[0167] S21042 If the smoking category corresponding to the maximum probability indicates that the subject does not smoke, then the smoking recognition result of the subject who does not smoke is obtained; otherwise, the smoking recognition result of the subject is obtained according to the category label values ​​set for each smoking category and the probability of each smoking category.

[0168] In the above process, determine which smoking category has the highest probability in the predicted probability of the smoking category. If it represents non-smoking, the recognition result of non-smoking is directly output. If it represents smoking, the category label value is set according to the degree of smoking, and then the smoking recognition result of the subject is calculated through the probability of the corresponding degree. The smoking recognition result calculated by this method is more accurate.

[0169] Among them, the non-smoking recognition result is used to indicate that the subject does not smoke, and the smoking recognition result includes: low-risk smoking and the smoking index corresponding to low-risk smoking or high-risk smoking and the smoking index corresponding to high-risk smoking.

[0170] In one implementation, when executing the above step S21043, the following steps may also be used:

[0171] Get the category label value set for each smoking category.

[0172] Based on the label values ​​of each category and the predicted probability of each smoking category, the predicted smoking index of the subject is obtained.

[0173] The smoking category set for the predicted smoking index of the subject is obtained, where the smoking category is used to indicate the smoking degree of the subject.

[0174] Based on the smoking category, the smoking recognition result of the subject is obtained.

[0175] In the above process, the predicted smoking index of the subject is calculated by using the smoking category label value and the predicted probability of each category, and the smoking recognition result of the subject can also be determined according to the smoking index.

[0176] Please see Figure 3 , Figure 3 A detailed flow chart of a method for smoking identification provided in an embodiment of the present application.

[0177] Step 300: For the image of the person to be tested, use a target detection algorithm to obtain a face frame, and capture the frame to obtain a face image.

[0178] Step 310: For the face image, use the key point detection model to obtain the key point coordinates, and align the face according to the key points.

[0179] Step 320: Send the processed face photo to the trained smoking recognition model, output the predicted probability of each smoking category, if the probability of non-smoking is the largest, output the smoking index, and output the smoking degree of the person being tested.

[0180] Specifically, when executing step 300 to step 320, the specific steps refer to the above-mentioned step 200 to step 210, which will not be repeated here.

[0181] Please see Figure 4 , Figure 4 A detailed flow chart of a method for training a smoking recognition model provided in an embodiment of the present application;

[0182] Step 400: Input the preprocessed face image into the convolutional neural network of the face recognition model and output the corresponding feature matrix.

[0183] Step 410: Input the feature matrix into the smoking classification module and the smoking index prediction module respectively, and output different prediction results.

[0184] Step 420: The feature matrix is ​​passed through the first fully connected layer and then through the first normalization function to obtain the probabilities of the three categories of non-smoker, low-risk smoker, and high-risk smoker.

[0185] Among them, the first normalization function (softmax1) is used to convert the feature values ​​into the predicted probabilities of the three smoking categories.

[0186] Step 430: Pass the feature matrix through the second fully connected layer and then through the second normalization function to obtain the probability of each type of smoking index after discretization. Use the probability of each type of smoking index as a weight and perform weighted summation on the label values ​​of the class to obtain the smoking index.

[0187] The second normalization function (softnax2) is used to calculate the probability of each category of the discretized smoking index.

[0188] Step 440: Calculate the total loss by using the predicted probabilities of the three categories of non-smoker, low-risk smoker and high-risk smoker and the smoking index of the sample.

[0189] Step 450: Adjust the parameters of the model according to the total loss to complete the training of the smoking recognition model.

[0190] In the above process, multi-task training combining classification and regression can effectively improve the accuracy of the model.

[0191] Specifically, when executing steps 400 to 450, please refer to the above steps 100 to 140 for relevant contents, and this application will not go into details.

[0192] In the embodiment of the present application, by obtaining an image of the person to be tested to be identified; based on a pre-trained smoking recognition model, smoking recognition is performed on the image of the person to be tested to be identified, and a smoking recognition result of the person to be tested is obtained, so as to achieve the effect of convenient and accurate smoking detection of the person to be tested.

[0193] To this end, the present application also provides a smoking identification device.

[0194] Please refer to Figure 5 , is a schematic block diagram of a smoking identification device 500 provided in an embodiment of the present application. The device 500 may be a module, a program segment or a code on an electronic device. The device 500 corresponds to the above method embodiment and can execute each step involved in the above method embodiment. The specific functions of the device 500 can be found in the description below. To avoid repetition, the detailed description is appropriately omitted here.

[0195] In one implementation, the device 500 includes:

[0196] An acquisition module 510 is used to acquire an image to be recognized of a subject;

[0197] The identification module 520 is used to perform smoking identification on the image to be identified of the subject to be tested based on a pre-trained smoking identification model to obtain a smoking identification result of the subject to be tested, wherein the smoking identification model is constructed based on a neural network and is used to determine the smoking degree of the subject to be tested based on the predicted probability of each smoking category of the subject to be tested.

[0198] In one implementation, the device 500 further includes:

[0199] A second acquisition module is used for acquiring sample data for training a smoking recognition model before acquiring an image to be recognized of a subject to be tested, wherein the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image, and the smoking recognition data includes identity sample information of the subject to be tested and a smoking index of the sample of the subject to be tested;

[0200] Preprocessing each sample image to obtain a preprocessed image corresponding to each sample image;

[0201] According to each preprocessed image and identity sample information of each preprocessed image, an initial face recognition model is trained to obtain a trained face recognition model, wherein the initial face recognition model is constructed based on a neural network;

[0202] Based on the face recognition model, the smoking classification module and the smoking index prediction module, an initial smoking recognition model is constructed. The smoking classification module is used to determine the smoking category of the subject, and the smoking index prediction module is used to predict the smoking index of the subject.

[0203] Based on each preprocessed image and the smoking index corresponding to each preprocessed image, the initial smoking recognition model is trained to obtain a smoking recognition model.

[0204] In one implementation, the second acquisition module is specifically used to:

[0205] Based on each preprocessed image and the initial smoking recognition model, a predicted probability of smoking category of each subject and a predicted smoking index of the subject are obtained;

[0206] Obtain the smoking category loss based on the predicted probability of the smoking category corresponding to each preprocessed image;

[0207] Based on the predicted smoking index of the subject and the smoking index of the sample, the smoking index loss is obtained;

[0208] According to the smoking category loss and the smoking index loss, the parameters of the initial smoking recognition model are adjusted until a smoking recognition model is obtained.

[0209] In one implementation, the second acquisition module is specifically used to:

[0210] For each sample image, perform the following steps:

[0211] Performing face detection on a sample image to obtain a face detection area in the sample image;

[0212] Based on the face detection area, a sample image is intercepted to obtain a intercepted face image;

[0213] Extract key points from the face image to obtain at least two face key points corresponding to the face image;

[0214] According to at least two facial key points, the facial images are aligned to obtain a pre-processed image.

[0215] In one implementation, the identification module 520 is specifically used to:

[0216] Preprocessing the image to be identified of the subject to be tested to obtain a preprocessed image;

[0217] Input the preprocessed image into the feature extraction module to obtain the feature matrix;

[0218] Input the feature matrix into the smoking category module to obtain the predicted probabilities of multiple smoking categories of the subject;

[0219] The feature matrix is ​​input into the smoking index prediction module to obtain the smoking recognition result.

[0220] In one implementation, the identification module 520 is specifically used to:

[0221] Determine the maximum probability among the predicted probabilities for each smoking category;

[0222] If the smoking category corresponding to the maximum probability indicates that the subject does not smoke, then a smoking recognition result indicating that the subject does not smoke is obtained;

[0223] Otherwise, the smoking recognition result of the subject is obtained according to the category label value set for each smoking category and the probability of each smoking category.

[0224] In one implementation, the identification module 520 is specifically used to:

[0225] Get the category label value set for each smoking category;

[0226] Based on the label values ​​of each category and the predicted probability of each smoking category, the predicted smoking index of the subject is obtained;

[0227] Obtaining a smoking category set for the predicted smoking index of the subject to be tested, where the smoking category is used to indicate the smoking degree of the subject to be tested;

[0228] Based on the smoking category, the smoking recognition result of the subject is obtained.

[0229] Please refer to Figure 6 This is a schematic block diagram of the structure of an electronic device 600 provided in an embodiment of the present application. The device may include a processor 620 and a memory 610. In one implementation, the device may also include: a communication interface 630 and a communication bus 640. The device corresponds to the above method embodiment and can execute each step involved in the above method embodiment. The specific functions of the device can be found in the description below.

[0230] Specifically, the memory 610 is used to store computer-readable instructions.

[0231] Processor 620 is used to process the readable instructions stored in the memory and can execute Figure 2 The various steps of method embodiments 200 to 210.

[0232] The communication interface 630 is used for signaling or data communication with other node devices, for example, for communication with a server or a terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0233] The communication bus 640 is used to realize direct connection and communication among the above components.

[0234] The communication interface 630 of the device in the embodiment of the present application is used to communicate signals or data with other node devices. The memory 610 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. In one embodiment, the memory 610 can also be at least one storage device located away from the aforementioned processor. The memory 610 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 620, the electronic device executes the aforementioned Figure 2The method process shown. The processor 620 can be used on the device 500 and used to perform the functions in the present application. Exemplarily, the above-mentioned processor 620 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the embodiments of the present application are not limited thereto.

[0235] The embodiment of the present application also provides a readable storage medium, when the computer program is executed by a processor, Figure 2 The method process in the method embodiment shown is executed by the electronic device.

[0236] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0237] In summary, the embodiments of the present application provide a method, device, electronic device and readable storage medium for smoking recognition, the method comprising: obtaining an image of a subject to be detected; performing smoking recognition on the image of the subject to be detected based on a smoking recognition model trained in advance, and obtaining a smoking recognition result of the subject to be detected. This method can achieve the effect of conveniently and accurately detecting the smoking of the subject to be detected.

[0238] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0239] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0240] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0241] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0242] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0243] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for identifying smoking, characterized in that: include: Acquire an image of a subject to be identified; Based on the pre-trained smoking recognition model, the smoking recognition is performed on the image of the subject to be recognized to obtain the smoking recognition result of the subject to be recognized, wherein the smoking recognition model is constructed based on a neural network and is used to determine the smoking degree of the subject to be recognized according to the predicted probability of each smoking category of the subject to be recognized; Before obtaining the image to be recognized of the subject to be tested, the method further includes: obtaining sample data for training the smoking recognition model, wherein the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image, and the smoking recognition data includes identity sample information of the subject to be tested and the smoking index of the sample of the subject to be tested; preprocessing each sample image to obtain a preprocessed image corresponding to each sample image; training an initial face recognition model according to each preprocessed image and the identity sample information of each preprocessed image to obtain a trained face recognition model, wherein the initial face recognition model is constructed based on a neural network; constructing an initial smoking recognition model based on the face recognition model, a smoking classification module and a smoking index prediction module, wherein the smoking classification module is used to determine the smoking category of the subject to be tested, and the smoking index prediction module is used to predict the smoking index of the subject to be tested; training the initial smoking recognition model based on each preprocessed image and the smoking index of the sample corresponding to each preprocessed image to obtain the smoking recognition model; The method of training the initial smoking recognition model based on each preprocessed image and the smoking index of the sample corresponding to each preprocessed image to obtain the smoking recognition model includes: obtaining the predicted probability of the smoking category of each subject and the predicted smoking index of the subject based on each preprocessed image and the initial smoking recognition model; obtaining the smoking category loss based on the predicted probability of the smoking category corresponding to each preprocessed image; obtaining the smoking index loss based on the predicted smoking index of the subject and the smoking index of the sample; and adjusting the parameters of the initial smoking recognition model according to the smoking category loss and the smoking index loss until the smoking recognition model is obtained.

2. The method according to claim 1, characterized in that: The preprocessing of each sample image to obtain a preprocessed image corresponding to each sample image includes: For each sample image, perform the following steps: Performing face detection on a sample image to obtain a face detection area in the sample image; Based on the face detection area, performing image interception on the sample image to obtain a intercepted face image; Extracting key points from the face image to obtain at least two face key points corresponding to the face image; The facial images are aligned according to the at least two facial key points to obtain the pre-processed image.

3. The method according to claim 1 or 2, characterized in that: The smoking recognition model includes a feature extraction module, a smoking classification module and a smoking index prediction module. The smoking recognition is performed on the image to be recognized based on the smoking recognition model trained in advance to obtain the smoking recognition result of the subject to be tested, including: Preprocessing the image to be identified of the subject to be tested to obtain a preprocessed image; Inputting the preprocessed image into the feature extraction module to obtain a feature matrix; Inputting the feature matrix into the smoking classification module to obtain the predicted probabilities of multiple smoking categories of the subject; The feature matrix is ​​input into the smoking index prediction module to obtain the smoking identification result.

4. The method according to claim 3, characterized in that The step of inputting the feature matrix into the smoking index prediction module to obtain the smoking identification result includes: Determine the maximum probability among the predicted probabilities for each smoking category; If the smoking category corresponding to the maximum probability indicates that the subject does not smoke, a smoking recognition result indicating that the subject does not smoke is obtained; Otherwise, the smoking recognition result of the subject is obtained according to the category label value set for each smoking category and the probability of each smoking category.

5. The method according to claim 4, characterized in that The step of obtaining the smoking recognition result of the subject according to the category label value set for each smoking category and the probability of each smoking category includes: Get the category label value set for each smoking category; Based on the label values ​​of each category and the predicted probability of each smoking category, obtaining the predicted smoking index of the subject; Obtaining a smoking category set for the predicted smoking index of the subject to be tested, wherein the smoking category is used to indicate the smoking degree of the subject to be tested; Based on the smoking category set by the predicted smoking index, a smoking recognition result of the subject is obtained.

6. A smoking identification device, characterized in that: include: An acquisition module, used for acquiring an image to be identified of a subject; a recognition module, used to perform smoking recognition on the image of the subject to be recognized based on a pre-trained smoking recognition model, and obtain a smoking recognition result of the subject to be recognized, wherein the smoking recognition model is constructed based on a neural network and is used to determine the smoking degree of the subject to be recognized according to the predicted probability of each smoking category of the subject to be recognized; Before the acquisition module acquires the image to be recognized of the subject to be tested, the device is further used to: acquire sample data for training the smoking recognition model, wherein the sample data includes a plurality of sample images and smoking recognition data corresponding to each sample image, and the smoking recognition data includes identity sample information of the subject to be tested and the smoking index of the sample of the subject to be tested; preprocess each sample image to obtain a preprocessed image corresponding to each sample image; train an initial face recognition model according to each preprocessed image and the identity sample information of each preprocessed image to obtain a trained face recognition model, wherein the initial face recognition model is constructed based on a neural network; construct an initial smoking recognition model based on the face recognition model, a smoking classification module and a smoking index prediction module, wherein the smoking classification module is used to determine the smoking category of the subject to be tested, and the smoking index prediction module is used to predict the smoking index of the subject to be tested; train the initial smoking recognition model based on each preprocessed image and the smoking index of the sample corresponding to each preprocessed image to obtain the smoking recognition model; The acquisition module is specifically used to: obtain the predicted probability of the smoking category of each subject and the predicted smoking index of the subject based on each preprocessed image and the initial smoking recognition model; obtain the smoking category loss based on the predicted probability of the smoking category corresponding to each preprocessed image; obtain the smoking index loss based on the predicted smoking index of the subject and the smoking index of the sample; and adjust the parameters of the initial smoking recognition model according to the smoking category loss and the smoking index loss until the smoking recognition model is obtained.

7. A smoking identification device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 are executed.

8. A computer-readable storage medium, characterized in that: include: A computer program, when the computer program is run on a computer, causes the computer to execute the method according to any one of claims 1 to 5.

Images