A smoking detection method and electronic device

Abstract

The application relates to the technical field of image processing, and discloses a smoking detection method and an electronic device. The method comprises the following steps: after receiving a to-be-processed image, inputting the to-be-processed image into a first model, determining initial cigarette areas and a human head area in the to-be-processed image through the first model; determining the slope between any two initial cigarette areas, and filtering the initial cigarette areas based on the slope to obtain first target cigarette areas; and determining whether a smoking event exists in a target position corresponding to the to-be-processed image according to the first target cigarette areas and the human head area. In the embodiment, the initial cigarette areas are filtered based on the slope between the initial cigarette areas, and first target cigarette areas which are randomly arranged and recognized as cigarettes are obtained; according to the first target cigarette areas and the human head area, whether a smoking event exists in the target position, i.e. whether an object represented by the human head area smokes, can be accurately determined.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a smoking detection method and electronic device. Background Technology

[0002] Smoking poses significant risks in certain settings. For example, in the context of rail transit, escape is difficult in the event of a fire, potentially leading to substantial casualties. Therefore, timely smoking detection is necessary.

[0003] In related technologies, images are captured at a location to determine whether cigarettes are present in the images, thereby determining whether smoking has occurred.

[0004] However, there are many lights in rail transit scenarios. When the lights are reflected on the ground, they form a long white strip. If a pedestrian passes by, the pedestrian's head overlaps with the light reflection strip, which can easily be identified as smoking, thus generating a false alarm. Summary of the Invention

[0005] This application provides a smoking detection method and an electronic device for accurately detecting smoking.

[0006] In a first aspect, embodiments of this application provide a smoking detection method, the method comprising:

[0007] After receiving the image to be processed, the image to be processed is input into the first model, and the initial cigarette region and head region in the image to be processed are determined by the first model.

[0008] Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain a first target cigarette region;

[0009] Based on the first target cigarette region and the head region, determine whether a smoking event exists in the target location corresponding to the image to be processed.

[0010] The above scheme, since lights are usually arranged in an orderly manner while the position of cigarettes is random, filters the initial cigarette regions based on the slope between them after identifying the initial cigarette regions and the head regions in the image to be processed, and obtains the first target cigarette regions that are randomly arranged and identified as cigarettes. Based on the first target cigarette regions and the head regions, it is possible to accurately determine whether there is a smoking event at the target location, that is, whether the object represented by the head region is smoking.

[0011] Secondly, embodiments of this application provide an electronic device, including a communication unit and a processor;

[0012] The communication unit is used to transmit data with the image acquisition device at the target location;

[0013] The processor is configured to, upon receiving an image to be processed, input the image to be processed into a first model, determine an initial cigarette region and a head region in the image to be processed using the first model; determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain a first target cigarette region; and determine whether a smoking event exists in the target location corresponding to the image to be processed based on the first target cigarette region and the head region.

[0014] Thirdly, embodiments of this application provide a smoking detection device, comprising:

[0015] The first recognition module is used to input the image to be processed into the first model after receiving the image to be processed, and to determine the initial cigarette region and the head region in the image to be processed through the first model.

[0016] The first filtering module is used to determine the slope between any two initial cigarette regions and filter the initial cigarette regions based on the slope to obtain a first target cigarette region.

[0017] The second recognition module is used to determine whether a smoking event exists in the target location corresponding to the image to be processed, based on the first target cigarette area and the head area.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the smoking detection method as described in any of the first aspects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating the first application scenario provided in the embodiments of this application;

[0021] Figure 2 A schematic flowchart illustrating the first smoking detection method provided in this application embodiment;

[0022] Figure 3 This is a schematic diagram of a light mapping strip provided in an embodiment of this application;

[0023] Figure 4 A schematic diagram of the slope provided for an embodiment of this application;

[0024] Figure 5 A schematic flowchart illustrating the filtering of an initial cigarette area provided in an embodiment of this application;

[0025] Figure 6 A schematic flowchart illustrating the second smoking detection method provided in this application embodiment;

[0026] Figure 7 A schematic flowchart illustrating the smoking event determination method provided in this application embodiment;

[0027] Figure 8 This is a schematic diagram of the first type of downsampling provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of a second type of downsampling provided in an embodiment of this application;

[0029] Figure 10 A schematic flowchart illustrating the third smoking detection method provided in this application embodiment;

[0030] Figure 11 A schematic flowchart illustrating the fourth smoking detection method provided in this application embodiment;

[0031] Figure 12 A schematic flowchart illustrating the fifth smoking detection method provided in this application embodiment;

[0032] Figure 13 A schematic flowchart illustrating the color value determination method provided in the embodiments of this application;

[0033] Figure 14 A schematic flowchart illustrating the head and shoulder sub-image processing method provided in this application embodiment;

[0034] Figure 15 This is a schematic diagram of the stitched image provided in an embodiment of this application;

[0035] Figure 16 A schematic flowchart illustrating the sixth smoking detection method provided in this application embodiment;

[0036] Figure 17 This is a schematic diagram of the structure of the first smoking detection device provided in the embodiments of this application;

[0037] Figure 18 This is a schematic diagram of the structure of the second smoking detection device provided in the embodiments of this application;

[0038] Figure 19 A schematic block diagram of an electronic device provided in an embodiment of this application;

[0039] Figure 20 A schematic diagram of the program product provided in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0042] In the description of this application, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, it can refer to a direct connection, an indirect connection through an intermediate medium, or a connection within two devices. Those skilled in the art can understand the specific meaning of the above term in this application based on the specific circumstances.

[0043] Smoking poses significant risks in certain locations. For example, in rail transit systems, escape is difficult in the event of a fire, potentially leading to substantial casualties. Although subways are equipped with smoke detectors that can detect smoke in the air, if smoking occurs, the detectors may brake suddenly. However, a sudden stop of a high-speed train is extremely dangerous and would also cause equipment damage and energy consumption. Therefore, timely smoke detection is necessary.

[0044] In related technologies, images are captured at a location to determine whether cigarettes are present in the images, thereby determining whether smoking has occurred.

[0045] However, there are many lights in rail transit scenarios. When the lights are reflected on the ground, they form a long white strip. If a pedestrian passes by, the pedestrian's head overlaps with the light reflection strip, which can easily be identified as smoking, thus generating a false alarm.

[0046] Based on this, this application provides a smoking detection method, the method comprising: after receiving an image to be processed, inputting the image to be processed into a first model, determining an initial cigarette region and a head region in the image to be processed through the first model; determining the slope between any two initial cigarette regions, and filtering the initial cigarette regions based on the slope to obtain a first target cigarette region; and determining whether a smoking event exists in the target location corresponding to the image to be processed based on the first target cigarette region and the head region.

[0047] See Figure 1 The illustration shows an application scenario provided by an embodiment of this application. This application scenario includes an electronic device and an image acquisition device positioned at a target location. Figure 1 Taking one image acquisition device as an example, the number of image acquisition devices can be set according to the size of the area at the target location during implementation;

[0048] In some embodiments, the electronic device communicates with the image acquisition device via data. The electronic device may communicate with the image acquisition device via a local area network (LAN), wireless local area network (WLAN), or other networks.

[0049] In practice, the image acquisition device can acquire images at a certain frequency and send the acquired images (images to be processed) to electronic devices;

[0050] After receiving the image to be processed, the electronic device inputs the image to be processed into a first model, and determines the initial cigarette region and the head region in the image to be processed through the first model; determines the slope between any two initial cigarette regions, and filters the initial cigarette regions based on the slope to obtain a first target cigarette region; and determines whether there is a smoking event in the target location corresponding to the image to be processed based on the first target cigarette region and the head region.

[0051] The electronic devices in the embodiments of this application may include one or more types of servers.

[0052] The above scheme, since lights are usually arranged in an orderly manner while the position of cigarettes is random, filters the initial cigarette regions based on the slope between them after identifying the initial cigarette regions and the head regions in the image to be processed, and obtains the first target cigarette regions that are randomly arranged and identified as cigarettes. Based on the first target cigarette regions and the head regions, it is possible to accurately determine whether there is a smoking event at the target location, that is, whether the object represented by the head region is smoking.

[0053] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0054] This application provides a first method for detecting smoking, which can be applied to the aforementioned electronic devices, such as... Figure 2 As shown, the method may include:

[0055] Step S201: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model.

[0056] In practice, the image acquisition device set up at the target location acquires images of the target location and sends the acquired images to be processed (containing objects at the target location, and may also include items such as cigarettes) to electronic devices.

[0057] After receiving the image to be processed, the electronic device performs feature extraction using the first model to determine whether the acquired image contains cigarettes and human heads, and to determine the regions corresponding to cigarettes and human heads in the image.

[0058] Step S202: Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain the first target cigarette region.

[0059] During implementation, when the light is reflected onto the ground, it forms a long white strip (light reflection strip), which is very similar to the shape of a cigarette. Therefore, some of the cigarettes identified by the first model may be the light reflection strip.

[0060] See Figure 3 As shown, since lights are usually arranged in an orderly manner, the light projection strips formed on the ground are also arranged in an orderly manner; however, the position of the cigarette is random and not fixed.

[0061] Based on this, after identifying the initial cigarette region and the head region in the image to be processed, this embodiment filters the initial cigarette region based on the slope between the initial cigarette regions (filtering out the ordered initial cigarette regions) to obtain the first target cigarette region that is randomly arranged and identified as a cigarette.

[0062] This embodiment does not specifically limit the method for determining the slope mentioned above; please refer to [reference needed]. Figure 4 As shown, by determining the center points of two initial cigarette regions, the slope between these two initial cigarette regions is determined based on the coordinates of these two center points in the processed image. For example, the slope between initial cigarette region i and initial cigarette region j... Among them, y i Let y be the ordinate of the center point of the initial cigarette region i. j Let x be the ordinate of the center point of the initial cigarette region j. i Let x be the x-coordinate of the center point of the initial cigarette region i. j Let x be the x-coordinate of the center point of the initial cigarette region j.

[0063] Step S203: Based on the first target cigarette region and the head region, determine whether there is a smoking event in the target location corresponding to the image to be processed.

[0064] In practice, based on the aforementioned slope, the initially ordered cigarette regions are filtered out, and the first target cigarette regions that are randomly arranged and identified as cigarettes are filtered out, thus eliminating some light interference.

[0065] For example, if a first target cigarette area exists, a smoking event is directly determined, and the smoking event corresponds to a head area that matches the first target cigarette area; or, the smoking event can be determined by referring to the following embodiments.

[0066] The above scheme, since lights are usually arranged in an orderly manner while the position of cigarettes is random, filters the initial cigarette regions based on the slope between them after identifying the initial cigarette regions and the head regions in the image to be processed, and obtains the first target cigarette regions that are randomly arranged and identified as cigarettes. Based on the first target cigarette regions and the head regions, it is possible to accurately determine whether there is a smoking event at the target location, that is, whether the object represented by the head region is smoking.

[0067] See Figure 5 As shown, in some optional embodiments, the filtering of the initial cigarette region based on the slope described above can be achieved, but is not limited to, in the following ways:

[0068] Step S501: For any slope, determine whether there are other slopes that are the same as the slope.

[0069] If yes, proceed to step S502; otherwise, proceed to step S503.

[0070] See above Figure 3 As shown, since the lights are usually arranged in an orderly manner, the white stripes (light reflection strips) formed by the light reflecting on the ground are also arranged in an orderly manner, and these white stripes will form the same slope;

[0071] Based on this, this embodiment determines whether every two slopes are the same. If they are the same, it means that the corresponding initial cigarette areas are arranged in an orderly manner, and the light mapping strip may be mistakenly detected as cigarettes. These initial cigarette areas are then filtered out, i.e., step S502 is executed. Conversely, if there are no other slopes that are the same as the slope, it means that the corresponding initial cigarette areas are arranged randomly, and these initial cigarette areas cannot be filtered out based on the sorting, i.e., step S503 is executed.

[0072] Step S502: Filter out the initial cigarette region corresponding to the slope, and filter out the initial cigarette regions corresponding to other slopes of the same value.

[0073] For example, if there are other slopes that are the same as this slope, it means that the corresponding initial cigarette areas are arranged in an orderly manner. It is possible that the light mapping strip is misdetected as cigarettes, and these initial cigarette areas are filtered out.

[0074] Step S503: Retain the two initial cigarette regions corresponding to the slope.

[0075] For example, if there are no other slopes with the same slope, it means that the corresponding initial cigarette regions are randomly arranged and cannot be filtered out based on the sorting. In other words, the first target cigarette region contains the initial cigarette region corresponding to the slope.

[0076] The above scheme, for each slope, determines whether the corresponding initial cigarette area is arranged in an orderly manner by judging whether there are other slopes that are the same as the slope, and then filters out the orderly arranged initial cigarette areas to reduce the interference of orderly arranged lights.

[0077] Correspondingly, this application provides a second smoking detection method, which can be applied to the aforementioned electronic devices, such as... Figure 6 As shown, the method may include:

[0078] Step S601: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model;

[0079] Step S602: Determine the slope between any two initial cigarette regions.

[0080] Step S603: For any slope, determine whether there are other slopes that are the same as the slope.

[0081] Step S604: If yes, then filter out the initial cigarette region corresponding to the slope, and filter out other initial cigarette regions corresponding to the same slope; otherwise, retain the two initial cigarette regions corresponding to the slope.

[0082] Step S605: Based on the first target cigarette region obtained by filtering and the head region, determine whether there is a smoking event in the target location corresponding to the image to be processed.

[0083] The specific implementation of steps S601 to S605 can be referred to the above embodiments, and will not be repeated here.

[0084] See Figure 7 As shown, in some optional embodiments, step S203 can be implemented in, but is not limited to, the following ways:

[0085] Step S701: Input the head and shoulder sub-image corresponding to the first target cigarette region into the second model, and determine whether the second target cigarette region exists in the head and shoulder sub-image through the second model.

[0086] The first model has a greater number of downsampling operations than the second model; the head and shoulder sub-image includes the first target cigarette region and the matched head region.

[0087] In this embodiment, after selecting the first target cigarette area, the first target cigarette area is matched with the head area, such as determining the head area that is closest to the first target cigarette area; while the cigarette may not only appear in the head area, but may also be present near the head.

[0088] Based on this, this embodiment magnifies the head region to the head and shoulder region to obtain a head and shoulder sub-image. By magnifying the head region to the head and shoulder sub-image, smoking detection is performed on the head and shoulder sub-image, avoiding missed detections caused by cigarettes next to the face, and further improving the accuracy of smoking detection.

[0089] In implementation, since the first model needs to recognize both cigarettes and human faces, and cigarettes have smaller pixels while human faces have larger pixels, the first model requires smaller-scale sampling and can use a conventional model. However, the second model only needs to recognize cigarettes. If a conventional model is used, the cigarette features will be gradually lost during the continuous downsampling process. Based on this, in this embodiment, the number of downsampling operations for the first model is greater than that for the second model.

[0090] See Figure 8 As shown, when the input image is 640*640, downsampling yields a first feature map of 80*80 pixels, scaled by 8 times, which allows detection of an 8*8 pixel target. Further downsampling yields a second feature map of 40*40 pixels, scaled by 16 times, resulting in a 16*16 target. A third feature map of 20*20 pixels yields a 32*32 feature map. However, when the input image is 640*640, the length and width of the cigarette image are mostly around 20 pixels; the third downsampling process gradually loses the cigarette's features.

[0091] See Figure 9 As shown, when the input image is 640*640, downsampling yields a first feature map of 80*80 pixels, scaled by 8 times, which allows the detection of an 8*8 pixel target. Further downsampling yields a second feature map of 40*40 pixels, scaled by 16 times, resulting in a 16*16 target. Since the second model eliminates the third-scale feature extraction process, retaining only two scales of feature maps, high-resolution feature maps can be used to detect cigarettes.

[0092] The above Figure 8 and Figure 9 This is merely an illustrative example and is not intended to limit the scope of this application.

[0093] Step S702: If a second target cigarette region exists in the head and shoulder sub-image, then it is determined that a smoking event exists at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0094] For example, by using a high-resolution feature map to detect cigarettes using a second model, it is possible to further eliminate the interference of light mapping bands in the first target cigarette region and obtain the second target cigarette region.

[0095] The above scheme sets up two models. The first model detects both the head and the cigarette simultaneously, while the second model only detects the cigarette. By reducing the downsampling frequency of the second model, it enables the second model to use high-resolution feature maps to detect the cigarette. This further eliminates the interference of light mapping bands in the first target cigarette region, thus obtaining the second target cigarette region. If the second target cigarette region exists in the head and shoulder sub-image, it can more accurately determine that there is a smoking event at the target location, and that the smoking event corresponds to the head region in the head and shoulder sub-image.

[0096] In some alternative implementations, the first model is trained using a first sample, and the second model is trained using a second sample; the first sample and the negative samples in the second sample include data from the light mapping strip.

[0097] For example, the first model is trained using the first sample, where the positive samples include head data and cigarette data; the negative samples of the first sample include data from the light mapping strip, and the ratio of positive to negative samples is 1:1; the second model is trained using the second sample, where the positive samples of the second sample include cigarette data; the negative samples of the second sample include data from the light mapping strip, and the ratio of positive to negative samples is 1:1.

[0098] The above scheme uses the data from the light mapping strip as negative samples to train the first and second models, enabling the two models to better learn the difference between cigarettes and the light mapping strip.

[0099] Correspondingly, this application provides a third method for detecting smoking, which can be applied to the aforementioned electronic devices, such as... Figure 10 As shown, the method may include:

[0100] Step S1001: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model.

[0101] Step S1002: Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain the first target cigarette region.

[0102] Step S1003: Input the head and shoulder sub-image corresponding to the first target cigarette region into the second model, and determine whether the second target cigarette region exists in the head and shoulder sub-image through the second model.

[0103] Step S1004: If a second target cigarette region exists in the head and shoulder sub-image, then it is determined that a smoking event exists at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0104] The specific implementation of steps S1001 to S1004 can be found in the above embodiments, and will not be repeated here.

[0105] This application provides a fourth method for detecting smoking, which can be applied to the aforementioned electronic devices, such as... Figure 11 As shown, the method may include:

[0106] Step S1101: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model.

[0107] Step S1102: Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain the first target cigarette region.

[0108] Step S1103: Input the head and shoulder sub-image corresponding to the first target cigarette region into the second model, and determine whether the second target cigarette region exists in the head and shoulder sub-image through the second model.

[0109] The specific implementation of steps S1101 to S1103 can be referred to the above embodiments, and will not be repeated here.

[0110] Step S1104: Filter the second target cigarette region based on its color value.

[0111] The color value is determined based on the hue, saturation, and brightness of the second target cigarette region.

[0112] As mentioned above, when light is projected onto the ground, it forms a long white strip (light projection strip), which closely resembles the shape of a cigarette. In practice, if the lights are not arranged in an orderly manner and the shape of the light projection strip is very similar to that of a cigarette, it is still difficult to eliminate this part of the light projection strip using the slope and the second model.

[0113] Regardless of the shape of the light, it is usually quite bright, so the color values ​​of the cigarette and the light reflection zone will be quite different; based on this, this embodiment further filters the second target cigarette area based on the color value of the second target cigarette area, thereby accurately identifying the cigarette.

[0114] Step S1105: If a second target cigarette region exists in the head and shoulder sub-image, then it is determined that a smoking event exists at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0115] The specific implementation of step S1105 can be referred to the above embodiments, and will not be repeated here.

[0116] The above scheme, because the light is usually quite bright, will result in a large difference in color values ​​between the cigarette and the light mapping zone. By further filtering the second target cigarette area based on the color value of the second target cigarette area, cigarette identification can be performed accurately.

[0117] This application provides a fifth method for detecting smoking, which can be applied to the aforementioned electronic devices, such as... Figure 12 As shown, the method may include:

[0118] Step S1201: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model.

[0119] Step S1202: Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain the first target cigarette region.

[0120] Step S1203: Input the head and shoulder sub-image corresponding to the first target cigarette region into the second model, and determine whether the second target cigarette region exists in the head and shoulder sub-image through the second model.

[0121] The specific implementation of steps S1201 to S1203 can be referred to the above embodiments, and will not be repeated here.

[0122] Step S1204: Filter the second target cigarette area whose color value is greater than the preset color threshold.

[0123] For example, since the light-reflecting band is brighter than a cigarette, this embodiment sets a preset color threshold. If the color value is greater than the preset color threshold, it is likely a light-reflecting band, and these second target cigarette areas are filtered out; conversely, if the color value is not greater than the preset color threshold, it is likely a cigarette, and these second target cigarette areas are retained.

[0124] Step S1205: If a second target cigarette region exists in the head and shoulder sub-image, it is determined that a smoking event exists at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0125] The specific implementation of step S1205 can be found in the above embodiments, and will not be repeated here.

[0126] The above scheme uses a light-mapping area that is brighter than a cigarette. If the color value is greater than a preset color threshold, it indicates that the second target cigarette area is bright and is likely a light-mapping area. By filtering these second target cigarette areas, smoking can be accurately identified.

[0127] See Figure 13 As shown, the color value of the second target cigarette region can be determined in the following way:

[0128] Step S1301: For any pixel in the second target cigarette region, determine the hue, saturation, and brightness of the pixel.

[0129] In this embodiment, the image to be processed is an RGB (red, green and blue) image, which can obtain the r (red) value, g (green) value and b (blue) value of each pixel in the second target cigarette area, and calculate the hue (h), saturation (l) and brightness (s) of each pixel.

[0130] For example, hue

[0131]

[0132]

[0133] Where max is the maximum value among r, g, and b values ​​of the corresponding pixel, and min is the minimum value among r, g, and b values ​​of the corresponding pixel.

[0134] Step S1302: Determine the average value of the hue, saturation and brightness of the pixel as the color value of the pixel.

[0135] For example, after calculating the hue (h), saturation (l), and brightness (s) of a pixel, the average of the three is used as the color value of that pixel.

[0136] Step S1303: Determine the color value of the second target cigarette region based on the color values ​​of all pixels in the second target cigarette region.

[0137] For example, after calculating the color values ​​of all pixels in the second target cigarette region, the color value of the second target cigarette region is determined based on the color values ​​of these pixels (such as the average of the color values ​​of all pixels).

[0138] See Figure 14 As shown, in some optional implementations, the head and shoulder sub-image can be processed as follows:

[0139] Step S1401: If there are multiple head and shoulder sub-images, scale each head and shoulder sub-image based on a preset size;

[0140] Step S1402: Stitch together multiple scaled head and shoulder sub-images to obtain a stitched image, and input the stitched image into the second model.

[0141] When there are multiple head and shoulder sub-images, performing a smoking detection for each head and shoulder sub-image may take a lot of time.

[0142] Based on this, if there are multiple head and shoulder sub-images, the head and shoulder sub-images are scaled according to a preset size, that is, each head and shoulder image is scaled to a preset size. The image at the preset size can ensure that the presence of cigarettes can be detected, and false detections and false misses can be avoided. This embodiment does not limit the specific size of the preset size.

[0143] Then, the head and shoulder sub-images of the preset size are stitched together to obtain a stitched image. For example, after the head and shoulder sub-images are arranged from left to right to fill a row, they are arranged in the next row in the same order from left to right until the stitched image is filled. The pixel value of the stitched image is a set value, and the pixel value of the stitched image is adapted to the pixel value of the input image of the second model.

[0144] See Figure 15 As shown, after scaling the head and shoulder sub-images, the 16 scaled head and shoulder sub-images are stitched together in a 4x4 pattern to obtain the stitched image.

[0145] The above Figure 15This is merely an illustrative example, and the specific splicing method is not limited in this embodiment.

[0146] The above scheme reduces the detection frequency and detection time of the second model by scaling and unifying the size of the head and shoulder sub-images, then stitching the head and shoulder sub-images together, and inputting the stitched image into the second model. This improves the efficiency of smoking detection.

[0147] Correspondingly, this application provides a sixth method for detecting smoking, which can be applied to the aforementioned electronic devices, such as... Figure 16 As shown, the method may include:

[0148] Step S1601: After receiving the image to be processed, input the image to be processed into the first model, and determine the initial cigarette region and head region in the image to be processed through the first model.

[0149] Step S1602: Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain the first target cigarette region.

[0150] Step S1603: If there are multiple head and shoulder sub-images, scale each head and shoulder sub-image based on a preset size.

[0151] Step S1604: Stitch together multiple scaled head and shoulder sub-images to obtain a stitched image, and input the stitched image into the second model to determine whether a second target cigarette region exists in the head and shoulder sub-image.

[0152] Step S1605: If a second target cigarette region exists in the head and shoulder sub-image, then it is determined that a smoking event exists at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0153] The specific implementation of steps S1601 to S1605 can be referred to the above embodiments, and will not be repeated here.

[0154] like Figure 17 As shown, based on the same inventive concept, this application provides a first smoking detection device 1700, comprising:

[0155] The first recognition module 1701 is used to input the image to be processed into a first model after receiving the image to be processed, and to determine the initial cigarette region and the head region in the image to be processed through the first model.

[0156] The first filtering module 1702 is used to determine the slope between any two initial cigarette regions and filter the initial cigarette regions based on the slope to obtain a first target cigarette region.

[0157] The second identification module 1703 is used to determine whether a smoking event exists in the target location corresponding to the image to be processed based on the first target cigarette area and the head area.

[0158] In some optional implementations, the second identification module 1703 is specifically used for:

[0159] The head and shoulder sub-image corresponding to the first target cigarette region is input into the second model, and the second model determines whether the second target cigarette region exists in the head and shoulder sub-image; wherein, the downsampling number of the first model is greater than the downsampling number of the second model; the head and shoulder sub-image includes the first target cigarette region and the matched head region;

[0160] If a second target cigarette region exists in a head and shoulder sub-image, then a smoking event is determined to exist at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0161] See Figure 18 As shown, in some optional embodiments, this application provides a second smoking detection device 1800, which, based on the above-described smoking detection device 1700, further includes a second filtering module 1704, used for:

[0162] Before the second identification module 1703 determines that a smoking event exists in the target location, the second target cigarette region is filtered based on the color value of the second target cigarette region; wherein the color value is determined based on the hue, saturation and brightness of the second target cigarette region.

[0163] In some alternative implementations, the second filtering module 1704 is specifically used for:

[0164] Filter out the second target cigarette area whose color value is greater than the preset color threshold.

[0165] In some alternative implementations, the color value of the second target cigarette region is determined by the following method:

[0166] For any pixel in the second target cigarette region, determine the hue, saturation, and brightness of the pixel;

[0167] The average value among the hue, saturation, and brightness of the pixel is determined as the color value of the pixel;

[0168] The color value of the second target cigarette region is determined based on the color values ​​of all pixels in the second target cigarette region.

[0169] In some optional implementations, the second identification module 1703 is specifically used for:

[0170] If there are multiple head and shoulder sub-images, each head and shoulder sub-image is scaled based on a preset size;

[0171] Multiple scaled head and shoulder sub-images are stitched together to obtain a stitched image, which is then input into the second model.

[0172] In some alternative implementations, the first model is trained using a first sample, and the second model is trained using a second sample; the first sample and the negative samples in the second sample include data from the light mapping strip.

[0173] In some alternative implementations, the first filtering module 1702 is specifically used for:

[0174] For any given slope, determine whether there are any other slopes that are the same as the given slope;

[0175] If so, filter out the initial cigarette region corresponding to the slope, and filter out other initial cigarette regions corresponding to the same slope; otherwise, retain the two initial cigarette regions corresponding to the slope.

[0176] Since this device is the same as the device in the method of this application embodiment, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0177] like Figure 19 As shown, based on the same inventive concept, this application provides an electronic device 1900, including: a processor 1901 and a memory 1902;

[0178] Memory 1902 may be volatile memory, such as random-access memory (RAM); memory 1902 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1902 may be any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1902 may be a combination of the above-described memories.

[0179] The processor 1901 may include one or more central processing units (CPUs), graphics processing units (GPUs), or digital processing units, etc.

[0180] This application embodiment does not limit the specific connection medium between the memory 1902 and the processor 1901. This application embodiment... Figure 19 The memory 1902 and the processor 1901 are connected via a bus 1903, and the bus 1903 is in... Figure 19 The bus 1903, represented by thick lines, can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 19 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0181] The memory 1902 stores program code, which, when executed by the processor 1901, causes the processor 1901 to perform the following processes:

[0182] After receiving the image to be processed, the image to be processed is input into the first model, and the initial cigarette region and head region in the image to be processed are determined by the first model.

[0183] Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain a first target cigarette region;

[0184] Based on the first target cigarette region and the head region, determine whether a smoking event exists in the target location corresponding to the image to be processed.

[0185] In some alternative implementations, the processor 1901 specifically performs:

[0186] The head and shoulder sub-image corresponding to the first target cigarette region is input into the second model, and the second model determines whether the second target cigarette region exists in the head and shoulder sub-image; wherein, the downsampling number of the first model is greater than the downsampling number of the second model; the head and shoulder sub-image includes the first target cigarette region and the matched head region;

[0187] If a second target cigarette region exists in a head and shoulder sub-image, then a smoking event is determined to exist at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

[0188] In some alternative implementations, before determining that a smoking event exists at the target location, the processor 1901 further performs:

[0189] The second target cigarette region is filtered based on its color values; wherein the color values ​​are determined based on the hue, saturation, and brightness of the second target cigarette region.

[0190] In some alternative implementations, the processor 1901 specifically performs:

[0191] Filter out the second target cigarette area whose color value is greater than the preset color threshold.

[0192] In some alternative implementations, the color value of the second target cigarette region is determined by the following method:

[0193] For any pixel in the second target cigarette region, determine the hue, saturation, and brightness of the pixel;

[0194] The average value among the hue, saturation, and brightness of the pixel is determined as the color value of the pixel;

[0195] The color value of the second target cigarette region is determined based on the color values ​​of all pixels in the second target cigarette region.

[0196] In some alternative implementations, the processor 1901 specifically performs:

[0197] If there are multiple head and shoulder sub-images, each head and shoulder sub-image is scaled based on a preset size;

[0198] Multiple scaled head and shoulder sub-images are stitched together to obtain a stitched image, which is then input into the second model.

[0199] In some alternative implementations, the first model is trained using a first sample, and the second model is trained using a second sample; the first sample and the negative samples in the second sample include data from the light mapping strip.

[0200] In some alternative implementations, the processor 1901 specifically performs:

[0201] For any given slope, determine whether there are any other slopes that are the same as the given slope;

[0202] If so, filter out the initial cigarette region corresponding to the slope, and filter out other initial cigarette regions corresponding to the same slope; otherwise, retain the two initial cigarette regions corresponding to the slope.

[0203] Since the electronic device is the same electronic device that executes the method in the embodiments of this application, and the principle of the electronic device in solving the problem is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.

[0204] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the steps of the smoking detection method described above. The readable storage medium can be a non-volatile readable storage medium.

[0205] In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when the program product is run on an electronic device, causes the processor of the electronic device to perform the steps of the smoking detection method according to the various exemplary embodiments of this application described above.

[0206] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0207] like Figure 20 As shown, a program product 2000 according to an embodiment of this application is described, which may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0208] The present application has been described above with reference to block diagrams and / or flowcharts illustrating methods, apparatus (systems), and / or computer program products according to embodiments of the present application. It should be understood that a block of a block diagram and / or flowchart, as well as combinations of blocks of block diagrams and / or flowcharts, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, and / or other programmable means to produce a machine, such that the instructions, executable via the computer processor and / or other programmable means, create methods for implementing the functions / actions specified in the blocks of the block diagrams and / or flowcharts.

[0209] Accordingly, this application can also be implemented using hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, this application can take the form of a computer program product on a computer-usable or computer-readable storage medium, having computer-usable or computer-readable program code implemented in the medium for use by or in conjunction with an instruction execution system. In the context of this application, a computer-usable or computer-readable medium can be any medium that can contain, store, communicate, transmit, or deliver a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0210] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0211] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting smoking, characterized in that, The method includes: After receiving the image to be processed, the image to be processed is input into the first model, and the initial cigarette region and head region in the image to be processed are determined by the first model. Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain a first target cigarette region; Based on the first target cigarette area and the head area, determine whether there is a smoking event at the target location corresponding to the image to be processed; Filtering the initial cigarette region based on the slope includes: For any given slope, determine whether there are any other slopes that are the same as the given slope; If so, the initial cigarette region corresponding to the slope is filtered out, and other initial cigarette regions corresponding to the same slope are also filtered out; otherwise, the two initial cigarette regions corresponding to the slope are retained, wherein the first target cigarette region includes the two initial cigarette regions corresponding to the slope. Based on the first target cigarette region and the head region, determining whether a smoking event exists at the target location corresponding to the image to be processed includes: The head and shoulder sub-image corresponding to the first target cigarette region is input into the second model, and the second model determines whether the second target cigarette region exists in the head and shoulder sub-image; wherein, the downsampling number of the first model is greater than the downsampling number of the second model; the head and shoulder sub-image includes the first target cigarette region and the matched head region; If a second target cigarette region exists in a head and shoulder sub-image, then a smoking event is determined to exist at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

2. The method as described in claim 1, characterized in that, Before determining that a smoking event exists at the target location, the method further includes: The second target cigarette region is filtered based on its color values; wherein the color values ​​are determined based on the hue, saturation, and brightness of the second target cigarette region.

3. The method as described in claim 2, characterized in that, Based on the color values ​​of the second target cigarette region, the second target cigarette region is filtered, including: Filter out the second target cigarette area whose color value is greater than the preset color threshold.

4. The method as described in claim 2, characterized in that, The color value of the second target cigarette region is determined using the following method: For any pixel in the second target cigarette region, determine the hue, saturation, and brightness of the pixel; The average value among the hue, saturation, and brightness of the pixel is determined as the color value of the pixel; The color value of the second target cigarette region is determined based on the color values ​​of all pixels in the second target cigarette region.

5. The method as described in claim 1, characterized in that, The head and shoulder sub-image corresponding to the first target cigarette region is input into the second model, including: If there are multiple head and shoulder sub-images, each head and shoulder sub-image is scaled based on a preset size; Multiple scaled head and shoulder sub-images are stitched together to obtain a stitched image, which is then input into the second model.

6. The method as described in claim 1, characterized in that, The first model is trained using the first sample, and the second model is trained using the second sample; the negative samples in the first sample and the second sample include data from the light mapping strip.

7. An electronic device, characterized in that, Includes communication units and processors; The communication unit is used to transmit data with the image acquisition device at the target location; The processor is configured to input the image to be processed into a first model after receiving the image to be processed, and determine the initial cigarette region and the head region in the image to be processed through the first model; Determine the slope between any two initial cigarette regions, and filter the initial cigarette regions based on the slope to obtain a first target cigarette region; Based on the first target cigarette area and the head area, determine whether there is a smoking event at the target location corresponding to the image to be processed; The processor is specifically used for: For any given slope, determine whether there are any other slopes that are the same as the given slope; If so, the initial cigarette region corresponding to the slope is filtered out, and other initial cigarette regions corresponding to the same slope are also filtered out; otherwise, the two initial cigarette regions corresponding to the slope are retained, wherein the first target cigarette region includes the two initial cigarette regions corresponding to the slope. The processor is specifically used for: The head and shoulder sub-image corresponding to the first target cigarette region is input into the second model, and the second model determines whether the second target cigarette region exists in the head and shoulder sub-image; wherein, the downsampling number of the first model is greater than the downsampling number of the second model; the head and shoulder sub-image includes the first target cigarette region and the matched head region; If a second target cigarette region exists in a head and shoulder sub-image, then a smoking event is determined to exist at the target location, and the smoking event corresponds to the head region in the head and shoulder sub-image.

Images