Behavior detection methods, devices, equipment, media, and computer program products

Abstract

This application discloses a behavior detection method, apparatus, device, medium, and computer program product. The behavior detection method includes: acquiring N frames of images within a first preset time period, each frame including at least one face detection region; acquiring facial image information corresponding to each face detection region in each frame based on a face detection model; inputting the facial image information corresponding to each face detection region in each frame into a smoking recognition model for each frame to obtain a first smoking recognition result for each face image information in each frame; determining that smoking behavior is included within the first preset time period when the number of first smoking recognition results for each face image information in each frame and a first threshold range are within the second threshold range. According to the embodiments of this application, smoking behavior can be quickly identified, improving the accuracy of smoking behavior detection results.

Description

Technical Field

[0001] This application belongs to the field of computer vision, and in particular relates to a behavior detection method, apparatus, device, medium and computer program product. Background Technology

[0002] The harm of smoking to society is not only reflected in its health hazards, but also in the fact that smokers' uncivilized behavior greatly increases the risk of fire. Therefore, more and more places are beginning to establish no-smoking areas.

[0003] Currently, common methods for monitoring whether smoking occurs in no-smoking areas include setting up smoke sensors or cigarette detection. However, due to the low sensitivity of the sensors and the difficulty in recognizing cigarette images, there are instances of missed or false detections of smoking behavior. Summary of the Invention

[0004] This application provides a behavior detection method, apparatus, device, medium, and computer program product that can quickly identify smoking behavior and improve the accuracy of smoking behavior detection results.

[0005] In a first aspect, embodiments of this application provide a behavior detection method, including:

[0006] Obtain N frames of images within a first preset time period, wherein each frame of image includes at least one face detection region, and N is a positive integer;

[0007] Based on the face detection model, obtain the face image information corresponding to each face detection region in each frame of the image;

[0008] For each frame of the image, the facial image information corresponding to each facial detection region in the image is input into the smoking recognition model to obtain the first smoking recognition result for each facial image information in each frame of the image.

[0009] The second smoking recognition result for each frame is determined based on the number of first smoking recognition results for each facial image information in each frame and the first threshold range.

[0010] When the number of target smoking recognition results among the N second smoking recognition results of N frames of images is within the second threshold range, it is determined that smoking behavior is included within the first preset time period.

[0011] In some possible implementations of the first aspect, the facial image information includes a sub-image corresponding to each facial detection region and a facial identifier corresponding to each facial detection region; for each frame of image, the facial image information corresponding to each facial detection region in the image is input into the smoking recognition model to obtain a first smoking recognition result for each facial image information in each frame of image, including:

[0012] The sub-images corresponding to the facial detection regions in each frame are input into the smoking recognition model to obtain smoking behavior detection information for each sub-image;

[0013] For each facial identifier, smoking behavior detection information of the sub-image corresponding to the facial detection region is obtained from N frames of images, resulting in M ​​smoking behavior detection information for each facial identifier, where M is a positive integer less than or equal to N.

[0014] Determine the number of smoking behavior detection data points, including smoking behavior information, corresponding to each facial identifier (M data points).

[0015] Based on the number of smoking behavior information corresponding to each facial identifier and the third threshold range, the first smoking recognition result for each facial image information in each frame is determined.

[0016] In some possible implementations of the first aspect, when the amount of smoking behavior information is within the range of a third threshold, the smoking behavior information corresponding to the facial identifier is determined, and the facial identifier corresponding to the smoking behavior information is taken as the target identifier.

[0017] For each frame of image, the first smoking recognition result that determines the facial image information corresponding to the target identifier includes the target smoking recognition result.

[0018] In some possible implementations of the first aspect, based on a face detection model, facial image information corresponding to each face detection region in each frame of the image is obtained, including:

[0019] The face detection model identifies the face detection regions included in each frame of the image, and determines the initial sub-image corresponding to each face detection region and the face identifier corresponding to each face detection region.

[0020] The resolution of the initial sub-image corresponding to the face detection region in each frame is adjusted to the target resolution to obtain the sub-image corresponding to the face detection region.

[0021] In some possible implementations of the first aspect, determining a second smoking recognition result for each frame of an image based on the number of first smoking recognition results for each facial image information in each frame and a first threshold range includes:

[0022] For each frame of image, when the number of first smoking recognition results that are target smoking recognition results is within a first threshold range, the second smoking recognition result of the image is determined to include the target smoking recognition result.

[0023] In some possible implementations of the first aspect, after determining that smoking behavior is included within a first preset time period when the number of target smoking recognition results included in the N second smoking recognition results of N frames of images is within a second threshold range, the method further includes:

[0024] The time interval between acquiring N frames of images for a first preset duration and the next acquisition of N frames of images for the first preset duration is the second preset duration.

[0025] Secondly, embodiments of this application provide a behavior detection device, comprising:

[0026] The acquisition module is used to acquire N frames of images within a first preset time period, wherein each frame of image includes at least one face detection region, and N is a positive integer;

[0027] The acquisition module is also used to acquire facial image information corresponding to each facial detection region in each frame of the image based on the facial detection model;

[0028] The recognition module is used to input the facial image information corresponding to each facial detection area in the image into the smoking recognition model for each frame of the image, so as to obtain the first smoking recognition result for each facial image information in each frame of the image.

[0029] The processing module is used to determine the second smoking recognition result of each frame image based on the number of first smoking recognition results of each facial image information in each frame image and a first threshold range;

[0030] The processing module is also used to determine that smoking behavior is included within a first preset time period when the number of target smoking recognition results included in the N second smoking recognition results of N frames of images is within a second threshold range.

[0031] Thirdly, this application provides a behavior detection device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the behavior detection method described in the first aspect or any implementable method of the first aspect.

[0032] Fourthly, this application provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the behavior detection method described in the first aspect or any implementable embodiment of the first aspect.

[0033] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the behavior detection method as described in the first aspect or any implementable embodiment of the first aspect.

[0034] This application discloses a behavior detection method, apparatus, device, medium, and computer program product. The acquired images include face detection regions. Next, facial image information corresponding to each face detection region in each frame is acquired, and this facial image information is input into a smoking recognition model to determine the smoking recognition result for each frame. Since the image size corresponding to the face detection region is significantly smaller than the size of the complete image, the smoking recognition model's recognition speed for smoking behavior can be improved. Then, based on the number of first smoking recognition results for each face image information in each frame and a first threshold range, a second smoking recognition result for each frame is determined. Finally, the number of N smoking recognition results in N frames that are target recognition results is compared with a second threshold range to determine whether smoking behavior is included within the first preset time period, effectively improving the accuracy of smoking behavior recognition. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a behavior detection method provided in an embodiment of this application;

[0037] Figure 2 This is a schematic diagram of a process for determining a first smoking identification result provided in an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of the structure of a behavior detection device provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of the structure of a behavior detection device provided in an embodiment of this application. Detailed Implementation

[0040] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0041] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0042] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0043] The harm of smoking to society is not only reflected in its health hazards, but also in the fact that smokers' uncivilized behavior greatly increases the risk of fire. Therefore, more and more places are beginning to establish no-smoking areas.

[0044] Currently, monitoring for smoking in no-smoking areas commonly employs methods such as setting up smoke sensors or cigarette detection. However, due to low sensor sensitivity and the difficulty in recognizing cigarette images, the detection results often result in missed or false detections. For example, if the detection model uses images too close to the camera, while it easily captures clear cigarette features, it fails to effectively detect smoking in large groups of people. Therefore, the applicability of models for smoking detection is currently limited. Another example is when the detection model directly detects low-resolution cigarettes within high-resolution images, leading to low accuracy in detecting smoking.

[0045] To address this, embodiments of this application provide a behavior detection method, apparatus, device, medium, and computer program product. This method includes face detection in acquired images, obtains facial image information corresponding to each face detection region in each frame, inputs the facial image information corresponding to each face detection region into a smoking recognition model, determines the smoking recognition result for each frame, then determines the second smoking recognition result for each frame based on the number of first smoking recognition results for each facial image information in each frame and a first threshold range, and finally compares the number of N smoking recognition results in N frames that are the target recognition results with a second threshold range to determine whether smoking behavior is included within a first preset time period. This effectively improves the accuracy of identifying smoking behavior.

[0046] The behavior detection method provided in the embodiments of this application will now be described in conjunction with the accompanying drawings. Figure 1 A flowchart illustrating a behavior detection method according to an embodiment of this application is shown. Figure 1 As shown, the method may include the following steps:

[0047] Step 110: Obtain N frames of images within the first preset time period.

[0048] Each frame of the image includes at least one face detection region, where N is a positive integer.

[0049] Step 120: Based on the face detection model, obtain the facial image information corresponding to each face detection region in each frame image.

[0050] Step 130: For each frame of image, input the facial image information corresponding to each facial detection region in the image into the smoking recognition model to obtain the first smoking recognition result for each facial image information in each frame of image.

[0051] Step 140: Determine the second smoking recognition result for each frame of image based on the number of first smoking recognition results for each facial image information in each frame and the first threshold range.

[0052] Step 150: When the number of target smoking recognition results included in the N second smoking recognition results of N frames of images is within the second threshold range, it is determined that smoking behavior is included within the first preset time period.

[0053] The specific implementation methods of each of the above steps will be described in detail below.

[0054] The behavior detection method of this application embodiment includes face detection in the acquired images. Next, it acquires facial image information corresponding to each face detection region in each frame of the image, and inputs the facial image information corresponding to each face detection region into a smoking recognition model to determine the smoking recognition result of each frame of the image. Since the image size corresponding to the face detection region is significantly smaller than the size of the complete image, the smoking recognition model can improve the recognition speed of smoking behavior. Then, based on the number of first smoking recognition results of each face image information in each frame of the image and a first threshold range, a second smoking recognition result of each frame of the image is determined. Finally, the number of N smoking recognition results of N frames of the image that are the target recognition results is compared with a second threshold range to determine whether smoking behavior is included within the first preset time period, which can effectively improve the accuracy of smoking behavior recognition.

[0055] The specific implementation methods for each of the above steps are described below.

[0056] Specifically, step 110 involves acquiring N frames of images within a first preset time period.

[0057] In scenarios requiring the detection of smoking behavior, image acquisition devices can be used to capture images. These devices can be, for example, cameras, vision sensors, or other image-capturing equipment. The image acquisition devices can acquire images of the behavior detection scenario in real time.

[0058] For example, taking a camera as an image acquisition device, in order to avoid the camera's placement affecting the normal activities of the user being monitored, and to ensure that the acquired images include as much facial feature information as possible, the camera can be positioned according to the actual acquisition scenario. As a specific example, the elevation angle between the camera and the boundary of the effective area should be greater than 45 degrees and less than 85 degrees, and the camera's field of view distance should be between 15 meters and 20 meters.

[0059] In some embodiments, video data can be acquired in real time using an image acquisition device. Optionally, the acquired video data can be saved to a preset memory for retrieval when smoking behavior detection is required. Optionally, real-time smoking behavior detection can also be performed on the acquired video data to improve the timeliness of the detection results. Optionally, the resolution of the acquired video data can be adjusted according to the actual shooting capability of the image acquisition device. For example, the resolution of the acquired video can be set to 1920×1080, without specific limitations.

[0060] In some embodiments, the first preset duration can be set according to the behavior detection requirements, and no specific limitation is made here. N images can be extracted from the video. To improve detection speed and reduce the processing of redundant data, video data of the first preset duration can be acquired first, and one frame can be extracted from each preset number of frames in the video data, thus obtaining N frames within the first preset duration. For example, the preset number of frames can be 5, and one image can be extracted from 5 consecutive frames. This effectively reduces the amount of data processing and improves the speed of obtaining behavior detection results.

[0061] Step 120 involves obtaining facial image information corresponding to each facial detection region in each frame of the image based on the facial detection model.

[0062] Optionally, to improve face detection speed, each frame of image can be preprocessed before being input into the preset face detection model to improve the recognition efficiency and accuracy of the face detection region. For example, preprocessing of each frame may include scaling the pixels of each frame to a target size, such as 640*480. The target size can also be set according to the specific input requirements of the face detection model and is not specifically limited here.

[0063] Preprocessing of each frame of image can also include converting a three-channel color image into a single-channel grayscale image. This reduces the amount of data processing required by the preset face detection model while ensuring recognition accuracy and improving the recognition efficiency of the face detection region.

[0064] In some embodiments, the face detection model can be, for example, a trained ultra-lightweight general-purpose face detection model (Ultra-Light-Fast-Generic-Face-Detector). Due to its small size, this model is suitable for electronic devices such as edge computing devices, mobile devices, or personal computers. By using a pre-defined recognition model, face detection regions in each frame of image can be quickly detected, thereby improving the scene adaptability of the behavior detection method.

[0065] To facilitate subsequent detection of smoking behavior, the facial detection regions in each detected image frame can be extracted to obtain facial image information corresponding to each facial detection region. Optionally, to improve detection accuracy, the extracted facial image information corresponding to each facial detection region can be filtered according to preset screening conditions. For example, screening conditions may include one or more of the following: the height of the facial image is not less than 80 pixels, the width is not less than 50 pixels, and the distance between the face in the image and the camera's field of view is less than 5 meters. It is understood that the facial detection model can be trained according to detection requirements. For example, faces captured from different angles, such as frontal and side views, can all be within the recognition range of the preset facial detection model. Based on the preset facial recognition model, information such as the image, location, and number of faces in the image can be identified.

[0066] In some embodiments, step 120 may further include the following steps: identifying the face detection regions included in each frame image according to the face detection model, determining the initial sub-image corresponding to each face detection region and the face identifier corresponding to each face detection region; adjusting the resolution of the initial sub-image corresponding to the face detection region in each frame image to the target resolution to obtain the sub-image corresponding to the face detection region.

[0067] For each frame, the resolution of the initial sub-image corresponding to each face detection region is adjusted to the target resolution to obtain the sub-image corresponding to the face detection region. This facilitates more accurate identification of cigarettes in the face detection region, improving the accuracy of detecting smoking behavior. For example, the target resolution can be 640×640 to achieve finer pixel-level edge segmentation, so as to identify cigarettes included in the face image information as much as possible.

[0068] In step 130, for each frame of the image, the facial image information corresponding to each facial detection region in the image is input into the smoking recognition model to obtain the first smoking recognition result for each facial image information in each frame of the image.

[0069] According to the steps described in the embodiments of this application, at least one facial image information can be obtained for each frame of image. The facial image information corresponding to each facial detection region in the image is input into the smoking recognition model to obtain a first smoking recognition result for each facial image information in each frame of image. For example, the first smoking recognition result may include smoking behavior in the facial image information or may not include smoking behavior in the facial image information.

[0070] Optionally, the smoking recognition model can be trained based on the YOLOv5 detection model. The trained smoking recognition model can identify information such as the image, location, and quantity of cigarettes in each image.

[0071] In some embodiments, facial image information includes a sub-image corresponding to each facial detection region and a facial identifier corresponding to each facial detection region.

[0072] Figure 2 This is a flowchart illustrating a method for determining a first smoking identification result according to an embodiment of this application. Step 130 in this embodiment may further include steps 131 to 134, in conjunction with... Figure 2 As shown.

[0073] Step 131: Input the sub-image corresponding to the face detection region in each frame into the smoking recognition model to obtain smoking behavior detection information for each sub-image.

[0074] The smoking recognition model can identify whether each sub-image contains a cigarette.

[0075] Step 132: For each facial identifier, obtain the smoking behavior detection information of the sub-image corresponding to the facial detection region from the N frames of images, and obtain M smoking behavior detection information for each facial identifier, where M is a positive integer less than or equal to N.

[0076] Since facial image information includes a sub-image corresponding to each facial detection region and a facial identifier corresponding to each facial detection region, and since the facial identifier is used to distinguish different facial information, that is, in N frames of images, facial information from the same object can be associated with a facial identifier, thereby obtaining a sub-image of the object in multiple frames of images. For example, the object to be identified can be a face in the captured images.

[0077] After identifying whether each sub-image contains a cigarette, smoking behavior detection information can be obtained from the sub-images corresponding to the facial detection region in N frames for each facial identifier, resulting in M ​​smoking behavior detection information for each facial identifier.

[0078] Step 133: Determine the number of smoking behavior detection information items (including smoking behavior information) corresponding to each facial identifier M.

[0079] Step 134: Based on the number of smoking behavior information corresponding to each facial identifier and the third threshold range, determine the first smoking recognition result for each facial image information in each frame.

[0080] For example, the third threshold range could be greater than a fixed threshold or a dynamic threshold. A dynamic threshold could be, for example, the ratio of the amount of smoking behavior information to M.

[0081] In some embodiments, when the amount of smoking behavior information is within a third threshold range, smoking behavior information corresponding to a facial identifier is determined, and the facial identifier corresponding to the smoking behavior information is designated as the target identifier; for each frame of image, the first smoking recognition result of the facial image information corresponding to the target identifier includes the target smoking recognition result. For example, the target smoking recognition result is that the facial image information includes smoking behavior.

[0082] Thus, by detecting whether smoking behavior exists in the facial detection area corresponding to the same facial identifier, the first smoking recognition result of each facial image information in each frame can be determined, which can effectively improve the accuracy of detecting smoking behavior.

[0083] Step 140: Determine the second smoking recognition result for each frame of image based on the number of first smoking recognition results for each facial image information in each frame and the first threshold range.

[0084] In some embodiments, since at least one facial image information can be obtained for each frame, a second smoking recognition result for each frame can be determined based on the number of first smoking recognition results for each facial image information in each frame and a first threshold range. Optionally, the first threshold range can be set according to detection requirements.

[0085] Specifically, for each frame of image, when the number of first smoking recognition results that are target smoking recognition results is within a first threshold range, the second smoking recognition result of the image is determined to include the target smoking recognition result. The target smoking recognition result is facial image information that includes smoking behavior.

[0086] In one example, the first threshold range can be a specific numerical range. For example, the first preset range can be such that when the first smoking recognition result of at least one facial image information is the target smoking recognition result, the second smoking recognition result of each frame of the image can be determined, that is, the image includes smoking behavior.

[0087] In another example, the first threshold range can be determined based on the number of facial image information corresponding to each frame. Specifically, for example, if a frame corresponds to X facial image information and has X first recognition results, when the number of target recognition results included in the X first recognition results exceeds 30% of the first recognition results, a second smoking recognition result is determined for each frame, that is, the image includes smoking behavior.

[0088] According to the embodiments of this application, it is possible to quickly determine whether each frame of an image includes smoking behavior.

[0089] In step 150, when the number of target smoking recognition results included in the N second smoking recognition results of N frames of images is within a second threshold range, it is determined that smoking behavior is included within a first preset time period.

[0090] Within a first preset time period, N frames of images can yield N second smoking recognition results. Specifically, for N frames of images within the first preset time period, smoking behavior is determined to have occurred within the first preset time period if the number of second smoking recognition results within the N frames that are target smoking recognition results falls within a second threshold range. For example, the second threshold range may be greater than 3. When more than 3 target smoking recognition results are included among the N smoking recognition results, it can be determined that smoking behavior is included within the first preset time period.

[0091] In some embodiments, the preset quantity threshold can be set according to actual detection needs. For example, the quantity threshold can be adjusted according to the size of N.

[0092] As a concrete example, when determining whether smoking behavior is included within the first preset time period, a sliding window can also be used for judgment. Specifically, the order of N second smoking recognition results is determined according to the chronological order of the image acquisition times. Optionally, the width of the preset sliding window can be greater than 1 and less than N. The preset sliding window slides sequentially from farthest to nearest according to the chronological order, where the sliding step size can be 1. No specific limitation is made on the sliding step size here. When all the second smoking recognition results in a sliding window are the target smoking recognition results, it is determined that smoking behavior is included within the first preset time period.

[0093] According to the embodiments of this application, smoking behavior can be quickly and accurately determined within a first preset time period. This allows for better identification of smoking behavior in different scenarios, thus enabling better adaptation to various detection situations.

[0094] In some embodiments, after detecting smoking behavior within a first preset time period, a smoking behavior prompt message may be generated.

[0095] After determining that smoking behavior is included within the first preset duration, the behavior detection method may further include the following steps: the time interval between acquiring N frames of images for the first preset duration and the next acquisition of N frames of images for the first preset duration is a second preset duration.

[0096] Specifically, the second preset duration can be determined based on the pedestrian flow speed in the detection scenario. For example, the slower the pedestrian flow speed, the longer the second preset duration. By performing behavior detection every second preset duration on the image data acquired by the image acquisition device, repeated detection of the same object can be avoided, thus preventing a waste of computing resources.

[0097] Based on the same inventive concept, this application also provides a behavior detection device 300 corresponding to the above-described behavior detection method. (Specifically combined with...) Figure 3 Please provide a detailed explanation.

[0098] Figure 3 This is a schematic diagram of the structure of a behavior detection device provided in an embodiment of this application, as shown below. Figure 3 As shown, the behavior detection device 300 may include:

[0099] The acquisition module 310 is used to acquire N frames of images within a first preset time period, wherein each frame of image includes at least one face detection region, and N is a positive integer;

[0100] The acquisition module 310 is also used to acquire facial image information corresponding to each facial detection region in each frame of the image based on the facial detection model;

[0101] The recognition module 320 is used to input the facial image information corresponding to each facial detection area in the image into the smoking recognition model for each frame of the image, so as to obtain the first smoking recognition result of each facial image information in each frame of the image.

[0102] Processing module 330 is used to determine the second smoking recognition result of each frame image based on the number of first smoking recognition results of each facial image information in each frame image and a first threshold range;

[0103] The processing module 330 is further configured to determine that smoking behavior is included within a first preset time period when the number of target smoking recognition results included in the N second smoking recognition results of N frames of images is within a second threshold range.

[0104] In some embodiments, facial image information includes a sub-image corresponding to each facial detection region and a facial identifier corresponding to each facial detection region;

[0105] The recognition module 320 is also used to input the sub-image corresponding to the face detection area in each frame image into the smoking recognition model to obtain smoking behavior detection information for each sub-image;

[0106] The processing module 330 is also used to obtain smoking behavior detection information of the sub-image corresponding to the face detection area from N frames of images for each face identifier, so as to obtain M smoking behavior detection information for each face identifier, where M is a positive integer less than or equal to N.

[0107] The processing module 330 is also used to determine the number of smoking behavior detection information items, including smoking behavior information, corresponding to each facial identifier M;

[0108] The processing module 330 is also used to determine the first smoking recognition result of each facial image information in each frame image based on the number of smoking behavior information corresponding to each facial identifier and the third threshold range.

[0109] In some embodiments, the processing module 330 is further configured to determine the smoking behavior information corresponding to the facial identifier when the number of smoking behavior information is within a third threshold range, and to set the facial identifier corresponding to the smoking behavior information as the target identifier;

[0110] The processing module 330 is also used to determine, for each frame of image, the first smoking recognition result, including the target smoking recognition result, of the facial image information corresponding to the target identifier.

[0111] In some embodiments, the processing module 330 is further configured to identify the face detection regions included in each frame image according to the face detection model, and determine the initial sub-image corresponding to each face detection region and the face identifier corresponding to each face detection region.

[0112] The processing module 330 is also used to adjust the resolution of the initial sub-image corresponding to the face detection region in each frame image to the target resolution, so as to obtain the sub-image corresponding to the face detection region.

[0113] In some embodiments, the processing module 330 is further configured to determine, for each frame of image, that the second smoking recognition result of the image includes the target smoking recognition result when the number of the first smoking recognition result is within a first threshold range.

[0114] In some embodiments, the processing module 330 is further configured to set the time interval between acquiring N frames of images for a first preset duration and the next acquisition of N frames of images for the first preset duration as a second preset duration.

[0115] It is understood that the behavior detection device 200 in this application embodiment can correspond to the execution subject of the behavior detection method provided in this application embodiment. The specific details of the operation and / or function of each module / unit of the behavior detection device 200 can be found in the above-described embodiments of this application. Figure 1 The descriptions of the relevant parts of the behavior detection method are omitted here for the sake of brevity.

[0116] The behavior detection device of this application embodiment can collect images including face detection. Next, it acquires facial image information corresponding to each face detection region in each frame of the image, and inputs the facial image information corresponding to each face detection region into a smoking recognition model to determine the smoking recognition result of each frame of the image. Since the image size corresponding to the face detection region is significantly smaller than the size of the complete image, the smoking recognition model can improve the recognition speed of smoking behavior. Then, based on the number of first smoking recognition results of each face image information in each frame of the image and a first threshold range, a second smoking recognition result of each frame of the image is determined. Finally, the number of N smoking recognition results of N frames of the image that are the target recognition results is compared with a second threshold range to determine whether smoking behavior is included within the first preset time period, which can effectively improve the accuracy of smoking behavior recognition.

[0117] Based on the same inventive concept, this application also provides a behavior detection device 400 corresponding to the above-described behavior detection method. (Specifically combined with...) Figure 3 Please provide a detailed explanation.

[0118] Figure 4 A schematic diagram of the structure of a behavior detection device according to an embodiment of this application is shown. Figure 4 As shown, the device may include a processor 401 and a memory 402 storing computer program instructions.

[0119] Specifically, the processor 401 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0120] Memory 402 may include mass storage for information or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 402 may include removable or non-removable (or fixed) media, or memory 402 may be non-volatile solid-state memory. Memory 402 may be internal or external to the behavior detection device.

[0121] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0122] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement the method described in the embodiments of this application and achieve the corresponding technical effects achieved by executing the method in the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0123] In one example, the behavior detection device may also include a communication interface 403 and a bus 410. Wherein, as Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0124] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0125] Bus 410 includes hardware, software, or both, that couples components of an online information flow metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0126] The behavior detection device can execute the behavior detection method in the embodiments of this application, thereby achieving the corresponding technical effects of the behavior detection method described in the embodiments of this application.

[0127] Furthermore, in conjunction with the behavior detection methods in the above embodiments, this application embodiment can provide a readable storage medium for implementation. This readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the behavior detection methods in the above embodiments. Examples of readable storage media can be non-transitory machine-readable media, such as electronic circuits, semiconductor memory devices, read-only memory (ROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, etc.

[0128] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0129] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0130] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0131] Furthermore, in conjunction with the behavior detection method, apparatus, and readable storage medium described in the above embodiments, this application can provide a computer program product for implementation. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs any of the behavior detection methods described in the above embodiments.

[0132] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0133] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A behavior detection method, characterized in that, include: Obtain N frames of images within a first preset time period, wherein each frame of image includes at least one face detection region, and N is a positive integer; Based on the face detection model, obtain the face image information corresponding to each face detection region in each frame of the image; For each frame of the image, the facial image information corresponding to each facial detection area in the image is input into the smoking recognition model to obtain the first smoking recognition result for each facial image information in each frame of the image; For each frame of image, when the number of the first smoking recognition results that are target smoking recognition results is within a first threshold range, it is determined that the second smoking recognition result of the image includes the target smoking recognition result, wherein the target smoking recognition result is that the facial image information includes smoking behavior; When the number of target smoking recognition results among the N second smoking recognition results of the N frames of images is within the second threshold range, it is determined that smoking behavior is included within the first preset time period.

2. The method according to claim 1, characterized in that, The facial image information includes a sub-image corresponding to each facial detection region and a facial identifier corresponding to each facial detection region. For each frame of the image, the facial image information corresponding to each facial detection region in the image is input into the smoking recognition model to obtain a first smoking recognition result for each facial image information in each frame, including: The sub-images corresponding to the facial detection regions in each frame are input into the smoking recognition model to obtain smoking behavior detection information for each sub-image; For each facial identifier, smoking behavior detection information of the sub-image corresponding to the facial detection region is obtained from the N frames of images, resulting in M ​​smoking behavior detection information for each facial identifier, where M is a positive integer less than or equal to N; Determine the number of smoking behavior detection information items, including smoking behavior information, corresponding to each of the facial identifiers; Based on the number of smoking behavior information corresponding to each facial identifier and the third threshold range, the first smoking recognition result for each facial image information in each frame is determined.

3. The method according to claim 2, characterized in that, When the number of smoking behavior information is within the third threshold range, the smoking behavior information corresponding to the facial identifier is determined, and the facial identifier corresponding to the smoking behavior information is set as the target identifier. For each frame of image, the first smoking recognition result for determining the facial image information corresponding to the target identifier includes the target smoking recognition result.

4. The method according to claim 2, characterized in that, The facial image information corresponding to each facial detection region in each frame of the image is obtained based on the facial detection model, including: The face detection model identifies the face detection regions included in each frame of the image, and determines the initial sub-image corresponding to each face detection region and the face identifier corresponding to each face detection region. The resolution of the initial sub-image corresponding to the face detection region in each frame is adjusted to the target resolution to obtain the sub-image corresponding to the face detection region.

5. The method according to claim 1, characterized in that, After determining that smoking behavior is included within the first preset time period when the number of target smoking recognition results included in the N second smoking recognition results of the N frames of images is within a second threshold range, the method further includes: The time interval between acquiring N frames of images for a first preset duration and the next acquisition of N frames of images for the first preset duration is the second preset duration.

6. A behavior detection device, characterized in that, The device includes: The acquisition module is used to acquire N frames of images within a first preset time period, wherein each frame of image includes at least one face detection region, and N is a positive integer; The acquisition module is also used to acquire facial image information corresponding to each facial detection region in each frame of the image based on the facial detection model; The recognition module is used to input the facial image information corresponding to each facial detection area in the image into the smoking recognition model for each frame of the image, so as to obtain the first smoking recognition result for each facial image information in each frame of the image; The processing module is used to determine, for each frame of image, that when the number of the first smoking recognition results that are target smoking recognition results is within a first threshold range, the second smoking recognition result of the image includes the target smoking recognition result, wherein the target smoking recognition result is that the facial image information includes smoking behavior; The processing module is further configured to determine that smoking behavior is included within the first preset time period when the number of target smoking recognition results included in the N second smoking recognition results of the N frames of images is within a second threshold range.

7. A behavior detection device, characterized in that, The device includes: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the behavior detection method as described in any one of claims 1-5.

8. A readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which, when executed by a processor, implement the behavior detection method as described in any one of claims 1-5.

9. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the behavior detection method as described in any one of claims 1-5.

Images