Smoking behavior determination method and device, electronic equipment and storage medium

Abstract

The invention discloses a smoking behavior determination method and device, electronic equipment and a storage medium, and relates to the computer technology, the method comprises the steps that a smoke area is determined from a plurality of continuous frame images, and the smoke area is used for indicating that cigarette analogues exist in the plurality of frame images; determining a cigarette end position in the cigarette analogue according to the smoke area; determining a cigarette stem area and a finger area according to the cigarette end position; when the cigarette rod area and the finger area meet preset conditions, it is determined that the cigarette analogue is a cigarette in a combustion state, and it is determined that the user has a smoking behavior according to the cigarette in the combustion state. According to the method, the detection efficiency of the smoking behavior can be improved by preferentially determining the smoke area from the plurality of frames of images; and each characteristic related to the smoking behavior is analyzed in sequence in a step-by-step progressive manner, so that the continuity and integrity of the scheme implementation process can be ensured, and the beneficial effect of improving the accuracy of determining the smoking behavior is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for determining smoking behavior. Background Art

[0002] With the rapid development of automobile technology, there are more and more cars in the world and traffic is becoming more and more developed. However, the safety awareness of drivers has not been enhanced accordingly, resulting in more frequent traffic accidents; therefore, safety assisted driving technology has also received more and more attention. When the driver smokes while driving, it will distract the driver's driving attention. Therefore, when it is detected that the driver smokes while driving the vehicle, using safe driving technology to issue safety warnings to the user is an effective way to avoid the driver's attention being distracted. Then, how to accurately and quickly determine the user's smoking behavior is a key issue that needs to be solved urgently. Summary of the invention

[0003] The present application provides a smoking behavior determination method, device, electronic device and storage medium, which can improve the efficiency and accuracy of determining smoking behavior.

[0004] In a first aspect, the present application provides a method for determining smoking behavior, comprising: determining a smoke area from a plurality of consecutive frame images, wherein the smoke area is used to indicate the presence of a cigarette analog in the plurality of frame images; determining a cigarette butt position in the cigarette analog based on the smoke area; determining a cigarette rod area and a finger area based on the cigarette butt position; and determining that the cigarette analog is a cigarette in a burning state when the cigarette rod area and the finger area meet preset conditions, so as to determine that the user has smoking behavior based on the cigarette in a burning state.

[0005] Optionally, determining the smoke area from a number of consecutive frame images includes: for a number of consecutive frame images, obtaining a grayscale image corresponding to each frame image; performing frame difference calculation on every two adjacent grayscale images to obtain frame difference grayscale images corresponding to the number of frame images; and determining the smoke area from the frame difference grayscale images corresponding to the number of frame images.

[0006] Optionally, determining the smoke area from the frame difference grayscale images corresponding to the several frame images includes: for at least two frame difference grayscale images, obtaining the corresponding frame difference binary images, and obtaining the corresponding smoke contour from the at least two frame difference binary images; when extracting smoke features from the smoke contour, determining the smoke contour to be the smoke area; the smoke features include an upward movement feature and a diffusion behavior feature, the upward movement feature is obtained by determining the longitudinal coordinate of the top pixel point in the smoke contour, and the diffusion behavior feature is obtained by calculating the area increase ratio of the contour area corresponding to the smoke contour in at least two of the frame difference binary images.

[0007] Optionally, determining the position of the cigarette butt in the cigarette analog based on the smoke region includes: for any frame image, determining the coordinate position of the bottom pixel point in the smoke region as the position of the cigarette butt in the cigarette analog.

[0008] Optionally, determining the cigarette rod area and the finger area according to the cigarette butt position includes: determining a circular area with the cigarette butt position as the center and a preset distance as the radius; dividing the circular area to obtain at least two sub-areas to be measured; calculating the average grayscale value of at least two sub-areas to be measured, and determining the sub-area to be measured with the largest average grayscale value as the target area; determining the image to be measured based on the target area, inputting the image to be measured into an image segmentation model, and obtaining a target segmentation map including the cigarette rod area and the finger area.

[0009] Optionally, when the cigarette rod area and the finger area meet preset conditions, determining that the cigarette analog is a cigarette in a burning state includes: determining a first proportion of the cigarette rod area in the target segmentation map, and determining a second proportion of the finger area in the target segmentation map; when the first proportion is not less than a first threshold value, and the second proportion is not less than a second threshold value, determining that the cigarette analog is a cigarette in a burning state.

[0010] Optionally, after determining that the user has smoking behavior based on the cigarette in a burning state, it also includes: determining at least one target hardware information to be linked; calling at least one target hardware device corresponding to the target hardware information to provide a corresponding linkage service based on at least one target hardware device.

[0011] In a second aspect, the present application provides a smoking behavior determination device, the device comprising:

[0012] A smoke determination module, used for determining a smoke region from a plurality of consecutive frames of images, wherein the smoke region is used for indicating the presence of cigarette analogs in the plurality of frames of images;

[0013] a position determination module, for determining the position of the cigarette butt in the cigarette analog according to the smoke area;

[0014] An area determination module, used to determine a cigarette rod area and a finger area according to the position of the cigarette butt;

[0015] The cigarette determination module is used to determine that the cigarette analog is a cigarette in a burning state when the cigarette rod area and the finger area meet preset conditions, so as to determine that the user has smoking behavior based on the cigarette in a burning state.

[0016] In a third aspect, the present application further provides an electronic device, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the smoking behavior determination method described in any embodiment of the present application.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the smoking behavior determination method described in any embodiment of the present application when executed.

[0021] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the smoking behavior determination method described in any embodiment of the present application.

[0022] The smoking behavior determination scheme provided in the embodiment of the present application takes into account that smoke is a unique feature of smoking behavior, and can improve the detection efficiency of smoking behavior by preferentially determining the smoke area from a number of frames of images; further, the embodiment determines the position of the cigarette butt in the cigarette analog from the smoke area, and gradually determines the cigarette rod area and the finger area according to the position of the cigarette butt, and when the cigarette rod area and the finger area meet the preset conditions, it can be determined that the cigarette analog is a cigarette in a burning state, and then determine that the user has smoking behavior. This embodiment analyzes each feature related to smoking behavior in turn in a step-by-step manner, which can ensure the continuity and integrity of the scheme implementation process, and achieves the beneficial effect of improving the accuracy of determining smoking behavior.

[0023] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium, wherein the computer-readable storage medium may be packaged together with the processor of the smoking behavior determination device, or may be packaged separately from the processor of the smoking behavior determination device, which is not limited in this application.

[0024] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description.

[0026] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 is a flow chart of a method for determining smoking behavior provided in an embodiment of the present application;

[0029] Figure 2a is another flow chart of the method for determining smoking behavior provided in an embodiment of the present application;

[0030] Figure 2b is a schematic diagram of a smoke area provided in an embodiment of the present application;

[0031] Figure 2c is a schematic diagram of a circular area provided in an embodiment of the present application;

[0032] Figure 2d This is a schematic diagram of determining a cigarette rod area and a finger area provided in an embodiment of the present application;

[0033] Figure 3 is a structural schematic diagram of a smoking behavior determination device provided in an embodiment of the present application;

[0034] Figure 4 It is a structural schematic diagram of an electronic device of a smoking behavior determination method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the present application will be clearly and completely described below in conjunction with the drawings in the present embodiment. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0038] Figure 1 A flowchart of a smoking behavior determination method provided in an embodiment of the present application. This embodiment can be applied to determine whether a user has smoking behavior. The method can be performed by a smoking behavior determination device, which can be implemented in the form of hardware and / or software and integrated in an electronic device that executes the method. Preferably, the electronic device in the embodiment of the present application can be a server, or a computer device, etc.

[0039] refer to Figure 1 The smoking behavior determination method of this embodiment includes but is not limited to the following steps:

[0040] S110 , determining a smoke region from a plurality of consecutive image frames, where the smoke region is used to indicate that a cigarette analogue exists in the plurality of image frames.

[0041] In this embodiment, a plurality of continuous frames of images can be obtained from the video stream received by the server during the driving process of the vehicle. Among them, the way for the server to receive the video stream can be to install a high-resolution camera at an appropriate position inside the vehicle (such as the A-pillar or the steering wheel) so as to clearly capture the driver's face and upper body area, so as to facilitate the analysis of the user's possible smoking behavior based on the captured video stream. The camera should be configured to continuously record the video stream, so that the video stream captured by the camera is sent to the server, so as to perform frame extraction on the received video stream at the server, and a plurality of continuous frames of images can be extracted from the video stream, and then each frame of image is subjected to subsequent analysis operations, so as to achieve the purpose of determining whether the driver has smoking behavior. Optionally, in order to accurately analyze each frame of image, the server can also perform preprocessing operations on each frame of image to improve the quality of each frame of image. Among them, the current preprocessing operation may include denoising operation, brightness adjustment operation and contrast optimization operation, etc., and the specific way of preprocessing each frame of image is not limited here.

[0042] In this embodiment, the purpose of analyzing several consecutive frames of images is that this scheme first needs to determine the smoke area from several consecutive frames of images. Since smoke is a unique feature of smoking, when the smoke area exists in several consecutive frames of images, it proves that the current user may have smoking behavior, so that subsequent steps are executed for accurate analysis; when there is no smoke area, no subsequent operation is required; this embodiment helps to improve the detection efficiency of determining smoking behavior by preferentially determining the smoke area from several frames of images.

[0043] Among them, the above-mentioned continuous multiple frame images can be frame images within a preset time. The current preset time can be 1 second to 3 seconds. Generally, a 1-second video stream can contain 25 frames of images. When a 1-second to 3-second video stream is selected, the continuous multiple frame images in this embodiment can contain 25 frames to 75 frames. Preferably, the preset time can be 2 seconds, then the continuous multiple frame images are 50 frames, and the selection of the specific preset time and the selection of the corresponding number of image frames are not limited here.

[0044] Since smoke generally has the characteristics of being upward and diffused when smoking, the method for determining the smoke area from several consecutive frames of images in this embodiment can be: obtaining the smoke area from several consecutive frames of images, and analyzing whether the adjacent N frames of images contain an area with upward and diffuse characteristics. If so, it indicates that there is a smoke area. The current adjacent N frames of images can be 3 frames, 5 frames or 10 frames, etc., and the specific selection of N is not limited here; optionally, a convolutional neural network model can be established to analyze several consecutive frames of images according to the neural network model, so as to determine whether several consecutive frames of images contain a smoke area, etc. The specific method of determining the smoke area from several consecutive frames of images is not limited here.

[0045] When the smoke area can be determined from several consecutive frames of images, it indicates that cigarette analogs exist in the several frames of images, so it is necessary to further determine whether the cigarette analogs are cigarettes in a burning state in combination with subsequent steps, thereby improving the accuracy of determining smoking behavior recognition.

[0046] It should be noted that, in this embodiment, cigarette analogs are not used to indicate objects that are similar in appearance to cigarettes, but are used to indicate the presence of objects in the cabin that are similar in characteristics to the smoke region. In this embodiment, taking the method of determining the smoke characteristics as analyzing whether there is an area with upward and diffusing characteristics in a number of consecutive frames of images as an example, when a vehicle is traveling on a bumpy road, pendants and other items in the vehicle will also move at varying distances as the vehicle travels, so for a number of consecutive frames of images, it is also possible to obtain an area with upward and diffusing characteristics; in this embodiment, objects that contain an area with characteristics similar to the determined smoke region are all confirmed as cigarette analogs.

[0047] S120: Determine the position of the cigarette butt in the cigarette analog according to the smoke area.

[0048] In this embodiment, the cigarette butt position indicates the location of the burned area in the cigarette. Generally, a burning cigarette can be divided into a burning area and an unburned area. The burning area is the area where the cigarette produces black and gray products during the burning process; the unburned area is the area of ​​the cigarette rod that is not burned in the current state, and the cigarette rod area is generally white.

[0049] Generally, the smoke area is generated during the burning process of the cigarette. Then, the method of determining the position of the cigarette butt in the cigarette analogue according to the smoke area is to determine the position corresponding to the burning area as the cigarette butt position.

[0050] The purpose of determining the position of the cigarette butt in this embodiment is to include the finger features and cigarette rod features when determining the user's smoking behavior based on the cigarette butt area in the subsequent steps. As the cigarette burns for a certain period of time, the black-gray products will gradually increase. If the position corresponding to the burning area where all the black-gray products are located is determined as the cigarette butt position, there is a situation where the determined finger features and cigarette rod features are incomplete, which affects the accurate determination of the smoking behavior.

[0051] Preferably, when the present embodiment determines the position of the cigarette butt in the cigarette analog based on the smoke area, the position of the bottommost pixel in the smoke area can be determined as the cigarette butt position; or after determining the burning area based on the smoke area, the position of the pixel closest to the cigarette rod in the burning area can be determined as the cigarette butt position; since a black cigarette butt ring will be generated between the burning area and the unburned area during the cigarette burning process, the position of any pixel in the black cigarette butt ring can also be determined as the cigarette butt position, etc. The specific method of determining the cigarette butt position in the present embodiment is not limited here.

[0052] If the cigarette butt position cannot be determined according to any of the methods provided in this embodiment, the subsequent operation steps may not be performed to determine that the user does not have any smoking behavior.

[0053] S130: Determine the cigarette stem area and the finger area according to the position of the cigarette butt.

[0054] If the user holds or clamps the cigarette rod with his fingers when smoking, the present embodiment further determines the cigarette rod area and the finger area based on the position of the cigarette butt to accurately detect the smoking behavior.

[0055] In some vehicle regulations, if a cigarette is placed in the mouth and not held by the hand, it may not be determined as smoking behavior. Then this embodiment can meet the current vehicle requirements by determining the existence of the finger area, thereby reducing the false detection rate.

[0056] In one implementation, the method for determining the cigarette rod area and the finger area according to the cigarette butt position in this embodiment can be: for a current frame image including the smoke area and the cigarette butt position, the current frame image is input into an image segmentation model to output a segmentation map including the finger area and the cigarette rod area according to the model; optionally, since there may be more interference factors in the current frame image, in order to improve the recognition accuracy of determining the cigarette rod area and the finger area, this embodiment can first determine a circular area with the cigarette butt position as the center and a preset distance as the radius; divide the circular area to obtain at least two sub-areas to be tested; calculate the average grayscale value of at least two sub-areas to be tested, and determine the sub-area to be tested with the largest average grayscale value as the target area, and the current target area includes the cigarette rod area; cut out the image based on the target area to obtain the image to be tested, input the image to be tested into the image segmentation model, and obtain a target segmentation map including the cigarette rod area and the finger area. In this embodiment, first, the target area includes the cigarette rod area, and then the image is extended and cut out based on the finger direction, so that the obtained image to be tested includes the cigarette rod area and the finger area, and the current image to be tested contains fewer interference factors, which facilitates the image segmentation model to quickly identify the finger area and cigarette rod area contained in the image to be tested.

[0057] If the cigarette rod area and the finger area cannot be determined according to any of the methods provided in this embodiment, the subsequent operation steps may not be performed to determine that the user does not have any smoking behavior.

[0058] This embodiment first determines the smoke area from a number of consecutive frames of images, and then determines the position of the cigarette butt from the smoke area. Compared with the conventional smoking behavior detection scheme that first identifies the shape of the object, this method of first detecting the smoke area and the position of the cigarette butt can avoid false detection caused by other slender objects. Further, by detecting the cigarette rod area and the finger area, the detection range is gradually expanded, the accuracy of real-time monitoring is improved, and the accuracy and reliability of the scheme implementation process are ensured.

[0059] S140. When the cigarette rod area and the finger area meet preset conditions, determine that the cigarette analog is a cigarette in a burning state, so as to determine that the user has smoking behavior based on the cigarette in a burning state.

[0060] To ensure the accuracy of the final determination of smoking behavior, the solution provided in this embodiment further detects whether the cigarette rod area and the finger area meet the preset conditions after determining the cigarette rod area and the finger area. If the preset conditions are met, the cigarette analog can be determined to be a cigarette in a burning state, so as to determine that the user has a smoking behavior based on the cigarette in a burning state. The solution provided in this embodiment can realize accurate detection of smoking behavior through lightweight image processing technology, and the implementation process is simple and the calculation amount is small.

[0061] The above preset conditions may include analyzing whether the cigarette rod area meets the cigarette rod feature; analyzing whether the finger area meets the finger feature, etc.; or determining whether the area occupied by the cigarette rod area in the current frame image is greater than a first threshold, and determining whether the area occupied by the finger area in the current frame image is greater than a second threshold, etc. The setting of the specific preset conditions is not limited here.

[0062] The smoking behavior determination method provided in this embodiment takes into account that smoke is a unique feature of smoking behavior, and can improve the detection efficiency of smoking behavior by preferentially determining the smoke area from a number of frames of images; further, this embodiment determines the position of the cigarette butt in the cigarette analog from the smoke area, and gradually determines the cigarette rod area and the finger area according to the cigarette butt position, and when the cigarette rod area and the finger area meet the preset conditions, it can be determined that the cigarette analog is a cigarette in a burning state, and then determine that the user has smoking behavior. This embodiment analyzes each feature related to smoking behavior in turn in a step-by-step manner, which can ensure the continuity and integrity of the solution implementation process, and achieves the beneficial effect of improving the accuracy of determining smoking behavior.

[0063] Figure 2aIt is another flow chart of the smoking behavior determination method provided in an embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiments, and the specific optimization is: this embodiment explains in detail the specific implementation methods of the related processes of "determining the smoke area from a number of consecutive frame images", "determining the cigarette butt position in the cigarette analogue according to the smoke area", "determining the cigarette rod area and the finger area according to the cigarette butt position", and "determining the cigarette analogue as a cigarette in a burning state when the cigarette rod area and the finger area meet the preset conditions" in the above embodiments.

[0064] See also Figure 2a The smoking behavior determination method of this embodiment includes but is not limited to the following steps:

[0065] S210 . For a plurality of consecutive image frames, obtain a grayscale image corresponding to each image frame.

[0066] In the current step, by performing grayscale conversion on each of the consecutive frames of images, a grayscale image corresponding to each frame of image can be obtained.

[0067] S211 , performing frame difference calculation on every two adjacent grayscale images to obtain frame difference grayscale images corresponding to a plurality of image frames.

[0068] First, two consecutive frames of images are selected from a number of consecutive frames of images, such as I t and I t+1 , further through I t and I t+1 The corresponding grayscale images are subjected to frame difference calculation to obtain a frame difference grayscale image corresponding to the current two consecutive frames of images.

[0069] The specific calculation method is as follows:

[0070] D(x,y)=|I t+1 (x,y)-I t (x,y)| (1)

[0071] Among them, the above formula D(x,y) represents the frame difference grayscale image, I t+1 (x,y) and I t (x, y) represents the grayscale value of the pixel (x, y) in two consecutive frames of images.

[0072] In this embodiment, taking 20 consecutive frames of images as an example, after the frame difference calculation is performed in the current step, 19 frame difference grayscale images can be obtained.

[0073] S212: Determine the smoke area from the frame difference grayscale images corresponding to the plurality of frame images.

[0074] When determining the smoke area based on the frame difference grayscale image, in order to improve the recognition accuracy of the smoke area, the solution provided in this embodiment can first pre-process the frame difference grayscale images corresponding to several frame images before determining the smoke area. The current pre-processing method can include binarization processing and noise filtering processing.

[0075] In this embodiment, the purpose of binarization is to represent the frame difference grayscale image in the form of a binary image, which helps to accurately analyze the relevant features of the smoke area in the subsequent steps; the purpose of noise filtering is that the image after the frame difference grayscale image is binarized may contain some spots or small areas in addition to large smoke areas. By calculating the area of ​​each connected domain in the binary image, if the area exceeds a certain threshold (for example, 50), it is considered to be a possible smoke area and retained, otherwise it is considered to be noise and deleted.

[0076] In a preferred implementation, in this embodiment, the above step S212 can be implemented as follows:

[0077] For at least two frame difference grayscale images, corresponding frame difference binary images are obtained, and corresponding smoke contours are obtained from the at least two frame difference binary images; when smoke features are extracted from the smoke contours, the smoke contours are determined to be smoke areas.

[0078] Since smoke has a typical upward movement characteristic due to thermal rise, and smoke usually gradually spreads out during the rising process, in this embodiment, the smoke characteristics include upward movement characteristics and diffusion behavior characteristics. Then, smoke contour analysis is performed on at least two frame difference grayscale images through the upward movement characteristics and diffusion behavior characteristics, so that when the smoke characteristics can be extracted from the smoke contour, the smoke contour is determined to be a smoke area.

[0079] For details, please refer to Figure 2b , Figure 2b Schematic diagram of the smoke area provided in the embodiment of the present application. Figure 2b , three frame difference binary images are shown in the figure. The current three frame difference binary images can be obtained based on three continuous frame difference grayscale images, or based on three non-continuous frame difference grayscale images. It is only necessary to keep the frame number of the frame difference grayscale image increasing from left to right. Exemplarily, the current three frame difference binary images can be frame difference binary images corresponding to the frame difference grayscale images of the 5th, 6th, and 7th frames; or they can be frame difference binary images corresponding to the frame difference grayscale images of the 5th, 10th, and 15th frames. When the frame number difference corresponding to two adjacent frame difference grayscale images is larger, the difference in smoke characteristics of the smoke contours in each two frame difference binary images is larger, which is convenient for accurate analysis.

[0080] In this embodiment, the upward motion feature is obtained by determining the ordinate of the top pixel point in the smoke contour. Figure 2b In the example, for each area position in the first binary image, its vertical upward movement in subsequent frames is detected. Specifically, by determining the ordinate of the top pixel in the smoke outline, the current top pixel can be the pixel with the maximum ordinate in each frame (taking the origin of the coordinate at the center of the smoke outline as an example). Based on this, by analyzing the ordinate of the top pixel in each frame, if it gradually increases, it is considered to meet the "upward movement" condition, that is, it meets the upward movement feature; if it does not meet the upward movement feature, it is deleted and considered not to be smoke.

[0081] In this embodiment, the diffusion behavior feature is obtained by calculating the area increase ratio of the contour area corresponding to the smoke contour in at least two frame difference binary images.

[0082] The specific method of determining the diffusion behavior characteristics can be achieved through the following formula:

[0083]

[0084] The above α represents the area increase ratio; Area at(t) and Area at(t+1) respectively represent the contour areas corresponding to the smoke contours in two adjacent frame difference binary images.

[0085] If the α values ​​of at least two frame difference binary images are calculated to be greater than 1, the contour is considered to be smoke; otherwise, it is deleted and considered not to be smoke.

[0086] In this embodiment, when the upward movement feature and the diffusion behavior feature can be extracted from the smoke outline at the same time, it is considered to be generated smoke and enters the subsequent judgment process.

[0087] For the traditional solution of smoking behavior detection, the existing solution is often to add post-processing logic after the process, such as cutting out the area containing cigarettes and sending it to the classification network to determine whether it belongs to cigarettes. However, this method will cause a relatively large waste of resources. For example, if it is finally judged that it is not a cigarette, then the previous cigarette detection, cigarette area cutting and cigarette classification are all silent costs, and the overall process is too long, which is easy to cause resource waste. This embodiment can improve the detection efficiency of determining smoking behavior by giving priority to detecting the smoke generated by smoking and identifying possible smoke areas by using the frame difference method.

[0088] S220: For any frame image, determine the coordinate position of the bottom pixel point in the smoke area as the position of the cigarette butt in the cigarette analog.

[0089] Simply determining the smoke area through smoke features in the aforementioned steps cannot directly lead to the presence of smoking behavior by the driver during driving. In order to improve the accuracy of determining smoking behavior, this embodiment further performs hierarchical expansion detection in subsequent steps to gradually determine the position of the cigarette butt based on the smoke area, and then analyzes the position of the cigarette butt to segment the target segmentation map including the cigarette rod area and the finger area based on the image segmentation model, so as to determine that the smoke area is generated by the cigarette, thereby improving the accuracy of the overall judgment. This embodiment improves the accuracy of determining smoking behavior by verifying the smoking features contained in the smoking behavior at each stage layer by layer, while also ensuring the continuity and integrity of the entire detection process. Among them, the above-mentioned bottom pixel point can be understood as the Figure 2b In the example, taking the coordinate origin at the center of the smoke outline, the lowest pixel point in the vertical direction is searched, and the coordinate corresponding to the lowest pixel point is determined as the coordinate position of the bottom pixel point, and then the position is determined as the cigarette butt position in the cigarette analog, which can also be understood as the potential position of the cigarette butt in the cigarette analog.

[0090] S230: Determine a circular area with the cigarette butt position as the center and the preset distance as the radius.

[0091] This embodiment performs extended analysis based on the position of the cigarette butt to determine the cigarette rod features and finger features that are consistent with the smoking behavior.

[0092] For details, please refer to Figure 2c , Figure 2c It is a schematic diagram of a circular area provided in an embodiment of the present application. In this embodiment, a detection range is first determined, that is, a prototype area is determined with the cigarette butt position as the center and the preset distance as the radius. The current circular area at least includes the unburned cigarette rod. The reason is that since the cigarette butt position is the coordinate position of the bottom pixel point in the smoke area, it can also be understood as the combination of the burning area and the unburned area in the actual smoking process. Then, when the circular area is determined with the cigarette butt position as the center and the preset distance as the radius, the current circular area must include part of the unburned cigarette butt area.

[0093] The preset distance may be 6 mm to 10 mm, preferably 7.7 mm, which is the diameter of most cigarettes in the national cigarette standard. The selection of the specific preset distance is not limited here.

[0094] S231 . Divide the circular area to obtain at least two sub-areas to be detected.

[0095] Please continue to refer to Figure 2c, the way to divide the circular area in this embodiment can be to divide the circular area horizontally with the cigarette butt position as the center of the circle to obtain two upper and lower sub-areas to be tested; divide the circular area vertically to obtain two left and right sub-areas to be tested; preferably, in this embodiment, the cigarette butt position is taken as the center of the circle, and two straight lines passing through the center of the circle are used to divide the circular area horizontally and vertically to obtain four equal sub-areas to be tested. The purpose of doing so is that among the four divided sub-areas to be tested, one part must contain more cigarette rod areas (such as the upper left part), and one part must contain more cigarette butt areas (such as the lower right part), which is helpful for accurately analyzing the cigarette rod area in subsequent steps.

[0096] S232, calculating the average grayscale values ​​of at least two sub-regions to be tested, and determining the sub-region to be tested with the largest average grayscale value as the target region.

[0097] In the current step, the grayscale transition of at least two sub-areas to be tested is analyzed by calculating the average grayscale value of at least two sub-areas to be tested. Based on the burning characteristics of cigarettes, there is an obvious color difference between the burning cigarette butt and the unburned cigarette rod, because the ash area after burning is usually darker, showing black-gray, while the unburned cigarette rod is brighter, showing white, creating an obvious grayscale transition boundary. Therefore, this embodiment calculates the average grayscale value of at least two sub-areas to be tested to distinguish the possible ash area and cigarette rod area, and determines the sub-area to be tested that contains the cigarette rod area as the target area.

[0098] Since the cigarette rod area is white, when calculating the average gray value, the average gray value of the sub-area to be tested containing the cigarette rod area is generally close to 255. Compared with other sub-areas to be tested containing ash or other interfering debris, the average gray value of the sub-area to be tested containing the cigarette rod area must be the largest. Therefore, the sub-area to be tested with the largest average gray value is determined as the target area. Correspondingly, the sub-area to be tested with the smallest average gray value can be determined as the ash area.

[0099] Preferably, in order to improve the accuracy of determining the cigarette rod area, this embodiment can also calculate the grayscale difference between the sub-area to be measured with the largest average grayscale value and the sub-area to be measured with the smallest average grayscale value after calculating the average grayscale value of at least two sub-areas to be measured. When the grayscale difference exceeds a preset threshold, the sub-area to be measured with the largest average grayscale value is determined to be the target area, so as to execute subsequent steps; otherwise, it indicates that the target area cannot be determined, indicating that there is no smoking behavior currently.

[0100] The preset threshold value can be obtained by calculating the average gray value corresponding to the ash area and the average gray value corresponding to the cigarette rod area when a conventional cigarette is burning.

[0101] S233, determining the image to be tested based on the target area, inputting the image to be tested into the image segmentation model, and obtaining a target segmentation map including the cigarette rod area and the finger area.

[0102] The target area obtained by the above steps may include the cigarette rod area, and then the image to be tested needs to be determined based on the target area. In this embodiment, the method of determining the image to be tested can be to take the cigarette butt position as the center of the circle and a preset fixed length as the radius (the radius is required to be greater than the length of the cigarette rod and less than 3 times the length of the cigarette rod), extend in the direction of the area, take a rectangle of the area, and cut it out to obtain the image to be tested. For details, please refer to Figure 2d , Figure 2d : is a schematic diagram of determining the cigarette rod area and the finger area provided in the embodiment of the present application. Figure 2d The first picture in is the original picture of the captured current frame image. After determining the circular area from the current frame image and calculating the average gray value of each sub-area to be tested in the circular area, the target area is obtained, and then the image to be tested is obtained by cutting out the target area, that is, Figure 2d The second image in .

[0103] The above image segmentation model is obtained by training the image semantic segmentation model. The segmentation categories are set to 3 categories: background, finger and cigarette holder. The cut-out image to be tested is sent to the segmentation model to obtain the segmented result. Figure 2d As shown in the figure, the area at the upper left corner of the circle center is deducted from the left image to obtain the segmentation result of the finger and the cigarette rod ( Figure 2d The theoretical basis for using this method is that users always put their hands on the cigarette rod area when smoking. In addition, according to relevant vehicle regulations, when the cigarette is placed in the mouth and not held by the hand, it is not judged as smoking, and the method proposed by the present invention can fully meet such requirements. In this embodiment, since the input image of the semantic segmentation is a small area that is cropped and cut out, the amount of calculation is relatively small, and this function can be implemented using a very small segmentation model, which will not bring a large computational burden to the system.

[0104] S240: Determine a first proportion of the cigarette rod area in the target segmentation map, and determine a second proportion of the finger area in the target segmentation map.

[0105] In the current step, the first proportion and the second proportion are calculated to determine whether the target segmentation map contains the cigarette rod feature and the finger feature. The first proportion and the second proportion can be calculated by calculating the pixels corresponding to the cigarette rod area and the pixels corresponding to the finger area.

[0106] S241. When the first proportion is not less than the first threshold value and the second proportion is not less than the second threshold value, determine that the cigarette analog is a cigarette in a burning state, so as to determine that the user has a smoking behavior based on the cigarette in a burning state.

[0107] In this embodiment, when it is determined that the first proportion is not less than the first threshold value and the second proportion is not less than the second threshold value, it is determined that the cigarette analog is a cigarette in a burning state, and it is considered that the driver is in a smoking state. Since the cigarette rod will gradually decrease during smoking, the threshold for the cigarette rod can be appropriately set to be smaller, for example, the first threshold value can be 0.05, the second threshold value can be 0.2, etc., and the specific determination of the first threshold value and the second threshold value is not limited here.

[0108] The improved solution of the embodiment of the present application avoids false detection caused by other slender objects by starting detection from smoke and cigarette butts. The method of gradually expanding to complete cigarette rods and hands ensures the accuracy and reliability of the entire process. By using traditional image processing methods to obtain a target segmentation map containing the cigarette rod area and the finger area, rather than a complex deep learning model, the computational burden of this solution is lighter and can be run in an environment with low computing resources. In addition, this embodiment uses traditional image processing technology in the application of smoking detection, especially in a resource-constrained environment, and can effectively perform the detection of user smoking behavior in the vehicle internal monitoring system.

[0109] In a preferred embodiment, after determining that the user has a smoking behavior based on the cigarette in a burning state, the method further includes: determining at least one target hardware information to be linked; calling a target hardware device corresponding to the at least one target hardware information to provide a corresponding linkage service based on the at least one target hardware device. The purpose of this is to remind the user not to smoke through the linkage service to avoid dangerous accidents.

[0110] In an application scenario, taking the detection of the driver's smoking behavior while driving a vehicle as an example, at least one target hardware information to be linked may be the voice module information and window information in the cockpit. Thus, after determining that the user has smoking behavior, the window can be opened by the user based on the window information to ensure air flow. At the same time, a voice reminder can be provided to the user based on the voice module information to remind the user of abnormal behavior and to pay attention to safe driving, thereby reducing the incidence of driving accidents.

[0111] Another application example, the linkage service of this embodiment can further track the location of cigarettes when a burning cigarette is detected, and detect the cigarette butt close to the car window. If a cigarette butt is detected close to the car window and suddenly disappears, it may mean that the burning cigarette has been thrown out of the window. When the act of throwing a burning cigarette out of the window is detected, the fire protection system can be immediately triggered to prevent safety problems such as fires, so as to maintain road traffic order and environmental sanitation.

[0112] The smoking behavior determination method provided in the embodiment of the present application takes into account that smoke is a unique feature of smoking behavior, and can improve the detection efficiency of determining smoking behavior by preferentially determining the smoke area from a number of frames of images; further, the embodiment determines the position of the cigarette butt in the cigarette analog from the smoke area, and gradually determines the cigarette rod area and the finger area according to the position of the cigarette butt, and when the cigarette rod area and the finger area meet the preset conditions, it can be determined that the cigarette analog is a cigarette in a burning state, and then determine that the user has smoking behavior. This embodiment analyzes each feature related to smoking behavior in turn in a step-by-step manner, which can ensure the continuity and integrity of the solution implementation process, and achieves the beneficial effect of improving the accuracy of determining smoking behavior.

[0113] Figure 3 is a structural diagram of a smoking behavior determination device provided in an embodiment of the present application, and the device is suitable for executing the smoking behavior determination method provided in an embodiment of the present application. Figure 3 As shown, the device may specifically include: a smoke determination module 310, a position determination module 320, an area determination module 330 and a cigarette determination module 340, wherein:

[0114] A smoke determination module 310, configured to determine a smoke region from a plurality of consecutive image frames, wherein the smoke region is used to indicate the presence of cigarette analogs in the plurality of image frames;

[0115] A position determination module 320, for determining the position of the cigarette butt in the cigarette analog according to the smoke area;

[0116] An area determination module 330, used to determine a cigarette rod area and a finger area according to the cigarette butt position;

[0117] The cigarette determination module 340 is used to determine that the cigarette analog is a cigarette in a burning state when the cigarette rod area and the finger area meet preset conditions, so as to determine that the user has smoking behavior based on the cigarette in a burning state.

[0118] The smoking behavior determination device provided in the embodiment of the present application takes into account that smoke is a unique feature of smoking behavior, and can improve the detection efficiency of determining smoking behavior by preferentially determining the smoke area from a number of frames of images; further, the embodiment determines the position of the cigarette butt in the cigarette analog from the smoke area, and gradually determines the cigarette rod area and the finger area according to the position of the cigarette butt, and when the cigarette rod area and the finger area meet the preset conditions, it can be determined that the cigarette analog is a cigarette in a burning state, and then determine that the user has smoking behavior. This embodiment analyzes each feature related to smoking behavior in turn in a step-by-step manner, which can ensure the continuity and integrity of the solution implementation process, and achieves the beneficial effect of improving the accuracy of determining smoking behavior.

[0119] In one embodiment, the smoke determination module 310 includes a grayscale image acquisition unit, a frame difference calculation unit and a smoke determination unit, wherein:

[0120] A grayscale image acquisition unit, used for acquiring a grayscale image corresponding to each frame of a plurality of consecutive frames of images;

[0121] A frame difference calculation unit, used for performing frame difference calculation on every two adjacent grayscale images to obtain frame difference grayscale images corresponding to the plurality of image frames;

[0122] The smoke determination unit is used to determine the smoke area from the frame difference grayscale images corresponding to the plurality of frame images.

[0123] In one embodiment, the smoke determination unit is specifically used to obtain corresponding frame difference binary images for at least two frame difference grayscale images, and obtain corresponding smoke contours from at least two of the frame difference binary images; when extracting smoke features from the smoke contours, the smoke contours are determined to be the smoke area; the smoke features include upward movement features and diffusion behavior features, the upward movement features are obtained by determining the vertical coordinates of the top pixel points in the smoke contours, and the diffusion behavior features are obtained by calculating the area increase ratios of the contour areas corresponding to the smoke contours in at least two of the frame difference binary images.

[0124] In one embodiment, the position determination module 320 is specifically configured to determine, for any frame image, the coordinate position of the bottom pixel point in the smoke region as the cigarette butt position in the cigarette analog.

[0125] In one embodiment, the region determination module 330 includes a region determination unit, a region segmentation unit, a grayscale calculation unit, and a model input unit, wherein:

[0126] An area determination unit, used to determine a circular area with the cigarette butt position as the center and a preset distance as the radius;

[0127] A region segmentation unit, used for segmenting the circular region to obtain at least two sub-regions to be measured;

[0128] A grayscale calculation unit, used for calculating the average grayscale value of at least two of the sub-regions to be detected, and determining the sub-region to be detected with the largest average grayscale value as the target region;

[0129] The model input unit is used to determine the image to be tested based on the target area, input the image to be tested into the image segmentation model, and obtain a target segmentation map including the cigarette rod area and the finger area.

[0130] In one embodiment, the cigarette determination module 340 includes a proportion determination unit and a cigarette determination unit, wherein:

[0131] A proportion determining unit, used to determine a first proportion of the cigarette rod area in the target segmentation map, and determine a second proportion of the finger area in the target segmentation map;

[0132] The cigarette determining unit is configured to determine that the cigarette analog is a cigarette in a burning state when the first proportion is not less than a first threshold value and the second proportion is not less than a second threshold value.

[0133] In one embodiment, the apparatus further includes: an information determination module and a device calling module, wherein:

[0134] An information determination module, used to determine at least one target hardware information to be linked;

[0135] The device calling module is used to call at least one target hardware device corresponding to the target hardware information to provide a corresponding linkage service based on at least one target hardware device.

[0136] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0137] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the smoking behavior determination method described in any embodiment of the present application.

[0138] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the smoking behavior determination method described in any embodiment of the present application when executed.

[0139] Reference below Figure 4 , Figure 4 1 is a schematic diagram of the structure of an electronic device for a method for determining smoking behavior provided in an embodiment of the present application, and shows a schematic diagram of the structure of a computer system 500 suitable for implementing the electronic device of the embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0140] like Figure 4 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0141] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0142] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0143] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the above.

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The modules and / or units involved in the embodiments described in the present application may be implemented by software or hardware. The modules and / or units described may also be set in a processor, for example, it may be described as: a processor includes a smoke determination module, a position determination module, an area determination module and a cigarette determination module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.

[0146] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: determining a smoke area from a plurality of consecutive frame images, the smoke area is used to indicate the presence of a cigarette analog in the plurality of frame images; determining the position of a cigarette butt in the cigarette analog based on the smoke area; determining a cigarette rod area and a finger area based on the position of the cigarette butt; when the cigarette rod area and the finger area meet preset conditions, determining that the cigarette analog is a cigarette in a burning state, so as to determine that the user has a smoking behavior based on the cigarette in a burning state.

[0147] According to the technical solution of this embodiment, considering that smoke is a unique feature of smoking behavior, the detection efficiency of determining smoking behavior can be improved by preferentially determining the smoke area from several frames of images; further, this embodiment determines the position of the cigarette butt in the cigarette analog from the smoke area, and gradually determines the cigarette rod area and the finger area according to the position of the cigarette butt, and when the cigarette rod area and the finger area meet the preset conditions, it can be determined that the cigarette analog is a cigarette in a burning state, and then determine that the user has smoking behavior. This embodiment analyzes each feature related to smoking behavior in turn in a step-by-step manner, which can ensure the continuity and integrity of the solution implementation process, and achieves the beneficial effect of improving the accuracy of determining smoking behavior.

[0148] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for determining smoking behavior, characterized in that: include: Determining a smoke region from a plurality of consecutive image frames, wherein the smoke region is used to indicate the presence of cigarette analogs in the plurality of image frames; determining a cigarette butt position in the cigarette analog based on the smoke region; Determine the cigarette rod area and the finger area according to the cigarette butt position; When the cigarette rod area and the finger area meet preset conditions, the cigarette analog is determined to be a cigarette in a burning state, so as to determine that the user has a smoking behavior based on the cigarette in a burning state.

2. The method for determining smoking behavior according to claim 1, characterized in that: The step of determining the smoke area from a plurality of consecutive frames of images includes: For a number of consecutive frames of images, obtain the grayscale image corresponding to each frame of image; Calculating the frame difference between every two adjacent grayscale images to obtain the frame difference grayscale images corresponding to the plurality of image frames; The smoke area is determined from the frame difference grayscale images corresponding to the plurality of frames of images.

3. The method for determining smoking behavior according to claim 2, characterized in that: The determining the smoke area from the frame difference grayscale images corresponding to the plurality of frame images includes: For at least two frame difference grayscale images, obtain corresponding frame difference binary images, and obtain corresponding smoke contours from the at least two frame difference binary images; When extracting smoke features from the smoke contour, the smoke contour is determined to be the smoke area; the smoke features include upward movement features and diffusion behavior features, the upward movement features are obtained by determining the vertical coordinate of the top pixel point in the smoke contour, and the diffusion behavior features are obtained by calculating the area increase ratio of the contour area corresponding to the smoke contour in at least two of the frame difference binary images.

4. The method for determining smoking behavior according to claim 1, characterized in that: The step of determining the position of the cigarette butt in the cigarette analog according to the smoke area comprises: For any frame image, the coordinate position of the bottom pixel point in the smoke area is determined as the cigarette butt position in the cigarette analog.

5. The method for determining smoking behavior according to claim 1, characterized in that: The determining of the cigarette rod area and the finger area according to the cigarette butt position includes: Determine a circular area with the cigarette butt position as the center and the preset distance as the radius; Segmenting the circular area to obtain at least two sub-areas to be measured; Calculating average grayscale values ​​of at least two of the sub-regions to be tested, and determining the sub-region to be tested with the largest average grayscale value as the target region; An image to be tested is determined based on the target area, and the image to be tested is input into an image segmentation model to obtain a target segmentation map including the cigarette rod area and the finger area.

6. The method for determining smoking behavior according to claim 5, characterized in that: When the cigarette rod area and the finger area meet preset conditions, determining that the cigarette analog is a cigarette in a burning state includes: Determine a first proportion of the cigarette rod area in the target segmentation map, and determine a second proportion of the finger area in the target segmentation map; When the first proportion is not less than a first threshold value, and the second proportion is not less than a second threshold value, it is determined that the cigarette analog is a cigarette in a burning state.

7. The method for determining smoking behavior according to claim 1, characterized in that: After determining that the user has smoking behavior based on the burning cigarette, the method further includes: Determine at least one target hardware information to be linked; At least one target hardware device corresponding to the target hardware information is called to provide a corresponding linkage service based on the at least one target hardware device.

8. A smoking behavior determination device, characterized in that: include: smoke A determination module, used for determining a smoke region from a plurality of consecutive frames of images, wherein the smoke region is used for indicating the presence of cigarette analogs in the plurality of frames of images; a position determination module, for determining the position of the cigarette butt in the cigarette analog according to the smoke area; An area determination module, used to determine a cigarette rod area and a finger area according to the position of the cigarette butt; The cigarette determination module is used to determine that the cigarette analog is a cigarette in a burning state when the cigarette rod area and the finger area meet preset conditions, so as to determine that the user has smoking behavior based on the cigarette in a burning state.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the smoking behavior determination method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the smoking behavior determination method according to any one of claims 1 to 7 is implemented.

Images