A smoking behavior detection method, device and medium

Abstract

The application discloses a smoking behavior detection method and device and a medium. The method comprises the following steps: determining a sample data set; the sample data set comprises multiple images of smoking behaviors; determining a first multi-scale target detection network; determining the size of a to-be-increased feature map according to the input image size of the first multi-scale target detection network; increasing the number of feature maps of the head of the first multi-scale target detection network according to the size of the to-be-increased feature map, to obtain a second multi-scale target detection network; training the second multi-scale target detection network according to the sample data set, to obtain a smoking behavior detection network model; acquiring a to-be-detected image; and identifying the smoking behavior of a user by using the to-be-detected image and the smoking behavior detection network model. The utilization of the shallow position information of the smoking behavior detection network model is improved by increasing the feature map, so that the identification effect of the cigarette small target and the smoking behavior detection accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method, device and medium for detecting smoking behavior. Background Technology

[0002] Smoking is not only harmful to human health, but also a safety hazard that causes traffic accidents, fires, explosions, and other incidents. Especially in public places such as warehouses and forests, fires caused by smoking can lead to irreparable loss of life and property. Therefore, research on the identification of smoking behavior is particularly important.

[0003] Currently, smoking behavior detection technology uses traditional machine learning algorithms or smoke sensors based on image feature matching to study targets. However, due to the inconspicuous nature of cigarette features and the small size of the target itself, it is impossible to accurately detect cigarette features when facing smoking behavior in public places, especially in surveillance videos, resulting in low accuracy in smoking behavior detection. Summary of the Invention

[0004] This application provides a smoking behavior detection method, device, and medium to address the problem of low accuracy in smoking behavior detection.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] On one hand, embodiments of this application provide a smoking behavior detection method, which includes: determining a sample dataset; the sample dataset including multiple images of smoking behavior; determining a first multi-scale object detection network; determining the size of a feature map to be added based on the input image size of the first multi-scale object detection network; increasing the number of head feature maps in the first multi-scale object detection network based on the size of the feature map to be added, thereby obtaining a second multi-scale object detection network; training the second multi-scale object detection network based on the sample dataset to obtain a smoking behavior detection network model; acquiring an image to be detected; and identifying a user's smoking behavior by comparing the image to be detected with the smoking behavior detection network model.

[0007] In one example, the step of increasing the number of head feature maps in the first multi-scale object detection network according to the size of the feature map to be added, to obtain the second multi-scale object detection network, specifically includes: if the first multi-scale object detection network is a YOLOX-tiny network, then the number of head feature maps in the first multi-scale object detection network is determined to be 3; according to the size of the feature map to be added, the number of head feature maps in the first multi-scale object detection network is increased from 3 to 4, to obtain the second multi-scale object detection network.

[0008] In one example, the method further includes: the feature maps of the head in the first multi-scale object detection network have sizes of 13*13*384, 26*26*192 and 52*52*96, respectively, and the size of the feature map to be added is 104*104*48.

[0009] In one example, the step of identifying a user's smoking behavior by using the image to be detected and the smoking behavior detection network model specifically includes: inputting the image to be detected into a pre-built user detection model to obtain the user's coordinate position information in the image to be detected; cropping the image to be detected based on the coordinate position information to obtain a user image to be detected; and identifying the user's smoking behavior by using the smoking behavior detection network model and the user image.

[0010] In one example, cropping the image to be detected based on the coordinate position information to obtain the user image to be detected specifically includes: generating a rectangular bounding box of the user based on the coordinate position information; expanding the width of the bounding box according to a preset ratio to obtain an updated bounding box; and cropping the user in the image to be detected based on the updated bounding box to obtain the user image to be detected.

[0011] In one example, the step of identifying a user's smoking behavior using the smoking behavior detection network model and the user image specifically includes: determining whether the size of the user image is consistent with the size of the input image of the smoking behavior detection network model; if not, adjusting the size of the user image until it is consistent with the size of the input image of the smoking behavior detection network model; inputting the adjusted user image into the smoking behavior detection network model to determine whether the adjusted user image includes smoking behavior, thereby identifying the user's smoking behavior.

[0012] In one example, training the second multi-scale object detection network based on the sample dataset to obtain a smoking behavior detection network model specifically includes: dividing the sample dataset into a training set and a validation set; performing data augmentation on the training set, inputting the training set into the second multi-scale object detection network, and training the second multi-scale object detection network using a bounding box regression loss function; validating the trained second multi-scale object detection network using a validation set until the validation is successful, thereby obtaining the smoking behavior detection network model.

[0013] In one example, the data augmentation of the training set specifically includes: stitching the training set together using the Mosaic data augmentation algorithm; and / or performing HSV color space transformation on the training set; and / or horizontally flipping, randomly rotating, and randomly scaling the training set.

[0014] On the other hand, embodiments of this application provide a smoking behavior detection device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: determine a sample dataset; the sample dataset including multiple images of smoking behavior; determine a first multi-scale object detection network; determine the size of a feature map to be added based on the input image size of the first multi-scale object detection network; increase the number of head feature maps in the first multi-scale object detection network based on the size of the feature map to be added, to obtain a second multi-scale object detection network; train the second multi-scale object detection network based on the sample dataset to obtain a smoking behavior detection network model; acquire an image to be detected, and identify a user's smoking behavior by comparing the image to be detected with the smoking behavior detection network model.

[0015] On the other hand, embodiments of this application provide a non-volatile computer storage medium for smoking behavior detection, storing computer-executable instructions, wherein the computer-executable instructions are configured to: determine a sample dataset; the sample dataset includes multiple images of smoking behavior; determine a first multi-scale object detection network; determine the size of the feature map to be added based on the input image size of the first multi-scale object detection network; increase the number of head feature maps in the first multi-scale object detection network based on the size of the feature map to be added, thereby obtaining a second multi-scale object detection network; train the second multi-scale object detection network based on the sample dataset to obtain a smoking behavior detection network model; acquire an image to be detected; and identify the user's smoking behavior by comparing the image to be detected with the smoking behavior detection network model.

[0016] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0017] For the first multi-scale target detection network, the number of feature maps of the head in the first multi-scale target detection network is increased, which improves the ability of the smoking behavior detection network model to process regional details of smoking behavior images. By increasing the number of feature maps, the smoking behavior detection network model can make greater use of shallow position information, thereby improving the recognition effect of small cigarette targets and thus improving the accuracy of smoking behavior detection. Attached Figure Description

[0018] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which:

[0019] Figure 1A schematic flowchart illustrating a smoking behavior detection method provided in an embodiment of this application;

[0020] Figure 2 A schematic diagram of an improved YOLOX-tiny network provided for an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the structure of a smoking behavior detection device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 This is a flowchart illustrating a smoking behavior detection method provided in an embodiment of this application. This method can be applied to different business areas, and certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0025] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.

[0026] It should be noted that the server can be a single device or a system composed of multiple devices, i.e., a distributed server. This application does not make any specific limitations on this.

[0027] Figure 1 The process may include the following steps:

[0028] S101: Determine the sample dataset; the sample dataset includes multiple images of smoking behavior.

[0029] This involves collecting multiple open-source images containing smoking behavior from the internet, creating a dataset in VOC format, dividing the sample dataset into training, validation, and test sets, and performing data augmentation on the training set.

[0030] When creating the dataset in VOC format, create a folder containing three subfolders: JPEGImages, Annotations, and ImageSets. Place all sample datasets in the JPEGImages folder and all XML format annotation files in the Annotations folder. Generate three text files, train.txt, val.txt, and test.txt, in the ImageSets / Main folder at an 8:1:1 ratio. These text files contain the image and annotation file indexes for the training, validation, and test sets, respectively.

[0031] Among them, when performing data augmentation on the training set, the Mosaic data augmentation algorithm is used to stitch the training set together, thereby supplementing the small target samples in the training set, and / or the HSV color space transformation is performed on the training set, thereby highlighting more target details, and / or the training set is horizontally flipped, randomly rotated and randomly scaled, thereby increasing the positional diversity of the targets.

[0032] S102: Determine the first multi-scale target detection network.

[0033] The first multi-scale target detection network is the YOLOX-tiny network.

[0034] S103: Determine the size of the feature map to be added based on the size of the input image of the first multi-scale target detection network.

[0035] S104: Based on the size of the feature map to be added, increase the number of feature maps of the head in the first multi-scale object detection network to obtain the second multi-scale object detection network.

[0036] In some embodiments of this application, when the first multi-scale object detection network is a YOLOX-tiny network, the number of feature maps of the head in the first multi-scale object detection network is 3. According to the size of the feature maps to be added, the number of feature maps of the head in the first multi-scale object detection network is increased from 3 to 4 to obtain the second multi-scale object detection network.

[0037] In the first multi-scale object detection network, the feature maps of the head correspond to sizes of 13*13*384, 26*26*192 and 52*52*96, respectively, and the size of the feature map to be added is 104*104*48.

[0038] More intuitively, Figure 2 This is a schematic diagram of an improved YOLOX-tiny network.

[0039] exist Figure 2In the YOLOX-tiny network, the detection head contains feature maps of three sizes: 13*13*384, 26*26*192, and 52*52*96. However, considering the problem of small targets in the sample dataset, a feature map of size 104*104*48 was added on the basis of the original size to improve the detection accuracy of small targets.

[0040] S105: Train the second multi-scale target detection network based on the sample dataset to obtain a smoking behavior detection network model.

[0041] In some embodiments of this application, the training set is input into a second multi-scale object detection network, and the second multi-scale object detection network is trained using a bounding box regression loss function.

[0042] The trained second-scale object detection network is validated using a validation set until it passes validation, thus obtaining the smoking behavior detection network model.

[0043] Finally, the smoking behavior detection network model is tested on the test set. If the test is passed, the final smoking behavior detection network model is obtained.

[0044] Specifically, yolox_voc_tiny.py is rewritten based on the exps / default / yolox_tiny.py file, and this file is moved to the exps / example / yolox_voc folder, so that the YOLOX network can train the tiny model.

[0045] Furthermore, the loss function formula is as follows:

[0046]

[0047] Where IoU is the area intersection-union ratio between the predicted bounding box and the label bounding box, Δ is the distance cost between the predicted bounding box and the label bounding box, and Ω is the shape cost between the predicted bounding box and the label bounding box.

[0048] in,

[0049]

[0050]

[0051] Where, ρ t It is the square of the ratio of the distance between the center point of the prediction box and the label box and the corner point of the diagonal of the outer rectangle in the x and y directions; Λ is the angle cost; α is the angle value between the center point of the prediction box and the label box in the x or y direction, taking the smaller value between the x and y direction angles.

[0052] in,

[0053]

[0054] Wherein, θ is the overall loss in terms of shape cost, with a value range of [2,6], and a value of 4 is recommended; It is the absolute value of the ratio of the difference between the width or length of the prediction box and the label box to the larger of the two.

[0055] S106: Obtain the image to be detected, and identify the user's smoking behavior by comparing the image to be detected with the smoking behavior detection network model.

[0056] In some embodiments of this application, two detection models are cascaded. First, the user detection model performs an initial screening of the user in the image to be detected, filtering out invalid areas in the original image and extracting valid user areas, thereby increasing the proportion of smoking behavior in the detection image and improving the accuracy of the smoking recognition model. Finally, the image to be detected is cropped.

[0057] Specifically, the image to be detected is input into a pre-built user detection model to obtain the user's coordinate position information in the image to be detected.

[0058] Then, the image to be detected is cropped according to the coordinate position information to obtain the user image to be detected, that is, multiple user images.

[0059] In this process, when cropping the image to be detected based on coordinate location information, a user's bounding box is first generated based on the coordinate location information. Then, the width of the bounding box is expanded according to a preset ratio to obtain an updated bounding box. Finally, the user in the image to be detected is cropped based on the updated bounding box to obtain the user image to be detected. This proportional expansion of the bounding box width ensures the integrity of the user image.

[0060] Finally, smoking behavior is identified by using a smoking behavior detection network model and user images.

[0061] This involves determining whether the size of the user's image matches the size of the input image for the smoking behavior detection network model.

[0062] If not, the size of the user image is adjusted until it matches the size of the input image of the smoking behavior detection network model.

[0063] Finally, the adjusted user image is input into the smoking behavior detection network model to determine whether the adjusted user image includes smoking behavior, so as to identify the user's smoking behavior.

[0064] It should be noted that, although the embodiments in this application are based on... Figure 1Steps S101 to S106 will be described sequentially, but this does not mean that steps S101 to S106 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S106 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S106 can be appropriately adjusted according to actual needs.

[0065] pass Figure 1 The proposed method increases the number of feature maps for the head in the first multi-scale target detection network, thereby improving the ability of the smoking behavior detection network model to process regional details of smoking behavior images. By increasing the number of feature maps, the smoking behavior detection network model can make greater use of shallow location information, thus improving the recognition effect of small cigarette targets and improving the accuracy of smoking behavior detection.

[0066] Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.

[0067] Figure 3 This is a schematic diagram of a smoking behavior detection device provided in an embodiment of this application. The device includes:

[0068] At least one processor; and,

[0069] A memory communicatively connected to the at least one processor; wherein,

[0070] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0071] Determine the sample dataset; the sample dataset includes multiple images of smoking behavior;

[0072] Determine the first multi-scale target detection network;

[0073] The size of the feature map to be added is determined based on the input image size of the first multi-scale target detection network.

[0074] Based on the size of the feature map to be added, the number of feature maps in the head of the first multi-scale object detection network is increased to obtain the second multi-scale object detection network.

[0075] The second multi-scale target detection network is trained based on the sample dataset to obtain a smoking behavior detection network model;

[0076] The system acquires an image to be detected and uses the image to be detected and the smoking behavior detection network model to identify the user's smoking behavior.

[0077] Some embodiments of this application provide a non-volatile computer storage medium for detecting smoking behavior, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0078] Determine the sample dataset; the sample dataset includes multiple images of smoking behavior;

[0079] Determine the first multi-scale target detection network;

[0080] The size of the feature map to be added is determined based on the input image size of the first multi-scale target detection network.

[0081] Based on the size of the feature map to be added, the number of feature maps in the head of the first multi-scale object detection network is increased to obtain the second multi-scale object detection network.

[0082] The second multi-scale target detection network is trained based on the sample dataset to obtain a smoking behavior detection network model;

[0083] The system acquires an image to be detected and uses the image to be detected and the smoking behavior detection network model to identify the user's smoking behavior.

[0084] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0085] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0086] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0091] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. A method for detecting smoking behavior, characterized in that, The method includes: Determine the sample dataset; the sample dataset includes multiple images of smoking behavior; Determine the first multi-scale target detection network; The size of the feature map to be added is determined based on the input image size of the first multi-scale target detection network. Based on the size of the feature map to be added, the number of feature maps in the head of the first multi-scale object detection network is increased to obtain the second multi-scale object detection network. The second multi-scale target detection network is trained based on the sample dataset to obtain a smoking behavior detection network model; The system acquires an image to be detected and uses the image to be detected and the smoking behavior detection network model to identify the user's smoking behavior. The step of increasing the number of feature maps in the head of the first multi-scale object detection network according to the size of the feature map to be added, to obtain the second multi-scale object detection network, specifically includes: If the first multi-scale object detection network is a YOLOX-tiny network, then the number of feature maps of the head in the first multi-scale object detection network is determined to be 3. Based on the size of the feature map to be added, the number of feature maps of the head in the first multi-scale object detection network is increased from 3 to 4 to obtain the second multi-scale object detection network. The method further includes: The feature maps of the head in the first multi-scale target detection network have sizes of 13*13*384, 26*26*192 and 52*52*96, respectively, and the feature map to be added has a size of 104*104*48. The step of identifying a user's smoking behavior by comparing the image to be detected with the smoking behavior detection network model specifically includes: The image to be detected is input into a pre-built user detection model to obtain the coordinate position information of the user in the image to be detected. The image to be detected is cropped based on the coordinate position information to obtain the user image to be detected; The smoking behavior of the user is identified by using the smoking behavior detection network model and the user image. The step of cropping the image to be detected based on the coordinate position information to obtain the user image to be detected specifically includes: Generate the user's bounding box based on the coordinate position information; The width of the rectangle is expanded according to a preset ratio to obtain an updated rectangle. Based on the updated bounding box, the user in the image to be detected is cropped to obtain the user image to be detected; The step of identifying a user's smoking behavior using the smoking behavior detection network model and the user image specifically includes: Determine whether the size of the user image is consistent with the size of the input image of the smoking behavior detection network model; If not, the size of the user image is adjusted until it matches the size of the input image of the smoking behavior detection network model; The adjusted user image is input into the smoking behavior detection network model to determine whether the adjusted user image includes smoking behavior, so as to identify the user's smoking behavior. The step of training the second multi-scale target detection network based on the sample dataset to obtain a smoking behavior detection network model specifically includes: The sample dataset is divided into a training set and a validation set; The training set is augmented with data, and then input into the second multi-scale object detection network. The second multi-scale object detection network is trained using the bounding box regression loss function. The trained second-scale target detection network is validated using a validation set until it passes validation, thus obtaining the smoking behavior detection network model. The data augmentation of the training set specifically includes: The training set is concatenated using the Mosaic data augmentation algorithm; and / or Perform HSV color space transformation on the training set; and / or The training set is horizontally flipped, randomly rotated, and randomly scaled. The loss function formula is as follows: in, It is the area intersection-union ratio between the predicted bounding box and the label bounding box. It is the distance cost between the prediction box and the label box. It is the shape cost between the prediction box and the label box; in, in, It is the square of the ratio of the distance between the center point of the prediction box and the label box and the corner point of the diagonal of the outer rectangle in the x and y directions; It is the cost of perspective; It is the angle value of the center point of the prediction box and the label box in the x or y direction, taking the smaller value between the x and y direction angles; in, in, It is the overall loss in terms of shape cost, with a value range of [2,6]. It is the absolute value of the ratio of the difference between the width or length of the prediction box and the label box to the larger of the two.

2. A smoking behavior detection device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to: perform a smoking behavior detection method according to any one of claims 1.

3. A non-volatile computer storage medium for detecting smoking behavior, storing computer-executable instructions, characterized in that, The computer-executable instructions are configured to execute a smoking behavior detection method as described in any one of claims 1.

Images