Laughing gas inhalation detection and early warning method, system, device and storage medium

Abstract

The present invention relates to the field of image recognition technology, and specifically to a nitrous oxide inhalation detection and early warning method, system, device and storage medium. To solve the problem of low accuracy and efficiency of nitrous oxide inhalation detection in the prior art, the present invention improves target detection accuracy by adding a global frequency feature information extraction module to the C2f module of a YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module. At the same time, by comparing the intersection-and-union features of detection frames corresponding to a balloon and a person on the same motor vehicle at a first road camera and a second road camera, and combining the image acquisition time, the nitrous oxide inhalation probability of the target motor vehicle is calculated, and a target motor vehicle with a nitrous oxide inhalation probability greater than an inhalation probability threshold is alarmed as a nitrous oxide inhalation illegal motor vehicle. The method is applied to the detection of nitrous oxide inhalation behavior on the road, and can improve the detection accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a nitrous oxide inhalation detection and early warning method, system, device and storage medium. Background Art

[0002] Inhaling nitrous oxide can affect the function of the nervous system, cause mood swings, and increase fatigue and drowsiness. Inhaling nitrous oxide while driving or riding in a motor vehicle on the road is an extremely dangerous behavior. It not only poses a serious threat to the life safety of the driver or passengers themselves, but also brings huge risks to other road users, seriously undermining road traffic safety and traffic order.

[0003] Currently, law enforcement agencies are using target detection technology in intelligent traffic monitoring systems to identify nitrous oxide balloons. They then manually identify images of balloons in the cab to determine if there is dangerous nitrous oxide inhalation. However, testing has shown that this method has poor detection accuracy and efficiency, primarily for the following reasons: First, nitrous oxide balloons are small targets, smaller than other objects in the scene, making it difficult for conventional target detection methods to accurately and reliably detect them. Furthermore, manually determining whether nitrous oxide inhalation has occurred based on image analysis, which is subjective, unstable, and costly. This reduces the accuracy and efficiency of nitrous oxide inhalation detection and identification in road traffic. Summary of the Invention

[0004] The purpose of the present invention is to provide a nitrous oxide inhalation detection and early warning method, system, equipment and storage medium.

[0005] The technical solutions of the present invention are as follows:

[0006] A method for detecting and warning of nitrous oxide inhalation, comprising the following operations:

[0007] S1. When a motor vehicle travels to a first distance from a first road camera, acquiring an image of the motor vehicle, and if a balloon is present in the image of the motor vehicle, using the image of the motor vehicle as an image to be detected;

[0008] S2. The image to be detected is processed by the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input is processed by Wavelet transform processing is performed to obtain initial wavelet transform features; low-frequency component features in the initial wavelet transform features are subjected to wavelet transform processing to obtain low-frequency component wavelet transform features; low-frequency component wavelet transform features are subjected to convolution and inverse wavelet transform to obtain low-frequency filter component features; low-frequency filter component features are superimposed with each feature in the initial wavelet transform features and then convolved to obtain frequency enhancement features; frequency enhancement features are subjected to inverse wavelet transform to obtain low-frequency enhancement features; the input is convolved several times and then superimposed with the low-frequency enhancement features to obtain global frequency features, which are used to perform the superposition operation in the Bottleneck submodule in the C2f module;

[0009] S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0010] The processing process in the global and local information fusion module in S2 is specifically as follows: the input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation respectively to obtain global enhanced features and local attention features; the global enhanced features and the convolution features are processed by Hadamard product, and then dilated convolution is performed to obtain global initial aggregation features; the global initial aggregation features and the convolution features are added element by element to obtain global aggregation features; the local attention features and the global enhancement features are processed by local information integration to obtain local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain local aggregation features; the global aggregation features and the local aggregation features are stacked to obtain global and local fusion features, which are used to execute the operations in the Concat module.

[0011] The specific operation of global information enhancement processing is as follows: the convolution features are respectively subjected to convolution processing with several convolution kernels of different sizes to obtain several secondary convolution features; the secondary convolution features corresponding to the convolution kernel size of the last one are transposed to obtain transposed features; the transposed features are element-wise multiplied with the secondary convolution features corresponding to the convolution kernel size of the second last one, and the global enhanced features are obtained by nonlinear processing with other secondary convolution features.

[0012] The specific operations of local attention aggregation processing are as follows: the convolution features are subjected to convolution, batch normalization and nonlinear processing to obtain the initial local features; based on the pixel values ​​of each position point in the initial local features and the corresponding learning pixel values, the pixel values ​​of each position point are subjected to channel attention processing, and then multiplied with the corresponding scaling factor to obtain the local initial attention features; the local initial attention features are subjected to nonlinear processing to obtain the local attention features.

[0013] The operation of obtaining the local initial attention feature is obtained by the following formula:

[0014] ,

[0015] r i is the first local initial attention feature i The pixel value of the position point, f i is the first local feature in the initial i The scaling factor for each position point, x i,c is the first local feature in the initial i Position No. c The pixel value of each channel, s i,c is the first local feature in the initial i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

[0016] The specific operation of local information integration processing is as follows: after the local attention features and global enhancement features are batch normalized and nonlinearly processed, the pixel values ​​of all channels of each position point are weighted averaged to obtain the local integrated features; after the local initial overall features are fully connected, they are element-wise multiplied with the convolution features to obtain the local initial aggregation features.

[0017] The probability of nitrous oxide inhalation in S3 is obtained by the following formula:

[0018] ,

[0019] P The probability of inhaling nitrous oxide,α 、 β 、 γ is the intersection-to-union ratio variation parameter, time probability parameter, and probability compensation value, ΔIoU is the difference between the intersection of the detection frames of the balloon and the person in the image to be detected and the intersection of the detection frames of the balloon and the person in the risk vehicle image, f(23:59,T) Acquisition time for risky motor vehicle images T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding night time function.

[0020] A nitrous oxide inhalation detection and early warning system, used to implement the above-mentioned nitrous oxide inhalation detection and early warning method, comprising:

[0021] a module for generating an image to be detected, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and use the image of the motor vehicle as the image to be detected if a balloon is present in the image of the motor vehicle;

[0022] The target motor vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frame corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: :The input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; the low-frequency component wavelet transform features are convolved and inverse wavelet transformed to obtain low-frequency filter component features; the low-frequency filter component features are superimposed with each feature in the initial wavelet transform features and then convolved to obtain frequency enhancement features; the frequency enhancement features are inverse wavelet transformed to obtain low-frequency enhancement features; the input is convolved several times and then superimposed with the low-frequency enhancement features to obtain global frequency features, which are used to perform the superposition operation in the Bottleneck submodule in the C2f module;

[0023] The nitrous oxide inhalation probability generation and alarm module is used to obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; based on the time when the risky motor vehicle image is obtained and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the nitrous oxide inhalation probability is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0024] A nitrous oxide inhalation detection and warning device comprises a processor and a memory, wherein the processor implements the above-mentioned nitrous oxide inhalation detection and warning method when executing a computer program stored in the memory.

[0025] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the above-mentioned nitrous oxide inhalation detection and early warning method is implemented.

[0026] The beneficial effects of the present invention are:

[0027] The present invention provides a method for detecting and warning nitrous oxide inhalation. The method first obtains an image of a motor vehicle traveling near a first road camera to check whether there are balloons related to nitrous oxide inhalation in and near the motor vehicle cab. If there are balloons in the motor vehicle image, the motor vehicle image is used as the image to be detected, and an improved YOLOv8 model is used for target detection to obtain an initial detection result of nitrous oxide inhalation. The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module, so that the model can better understand the relative position of the target ball with respect to the background information. The relationship below can quickly locate the target and obtain high semantic features, enhance the feature expression ability of the image in darker scenes, and improve the target detection accuracy; then, if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the result of the first intersection-over-union ratio threshold, then the intersection-over-union features of the detection frames corresponding to the balloon and the person on the same motor vehicle at the first road camera and the second road camera are compared, and combined with the image acquisition time, the nitrous oxide inhalation probability of the target motor vehicle is calculated, and the target motor vehicle with a nitrous oxide inhalation probability greater than the inhalation probability threshold is alarmed as a nitrous oxide inhalation illegal motor vehicle; this method is used in the detection and identification of nitrous oxide inhalation behavior on the road, which can improve the accuracy and efficiency of detection. DETAILED DESCRIPTION

[0028] This embodiment provides a method for detecting and warning of nitrous oxide inhalation, including the following operations:

[0029] S1. When a motor vehicle travels to a first distance from a first road camera, acquiring an image of the motor vehicle, and if a balloon is present in the image of the motor vehicle, using the image of the motor vehicle as an image to be detected;

[0030] S2. The image to be detected is processed by the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input is processed by Wavelet transform processing is performed to obtain initial wavelet transform features; low-frequency component features in the initial wavelet transform features are subjected to wavelet transform processing to obtain low-frequency component wavelet transform features; low-frequency component wavelet transform features are subjected to convolution and inverse wavelet transform to obtain low-frequency filter component features; low-frequency filter component features are superimposed with each feature in the initial wavelet transform features and then convolved to obtain frequency enhancement features; frequency enhancement features are subjected to inverse wavelet transform to obtain low-frequency enhancement features; the input is convolved several times and then superimposed with the low-frequency enhancement features to obtain global frequency features, which are used to perform the superposition operation in the Bottleneck submodule in the C2f module;

[0031] S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0032] S1. When a motor vehicle travels to a first distance from a first road camera, an image of the motor vehicle is acquired. If a balloon exists in the image of the motor vehicle, the image of the motor vehicle is used as an image to be detected.

[0033] When the motor vehicle travels to a first distance from the first road camera, the motor vehicle image is obtained to check whether there are balloons related to nitrous oxide inhalation in and near the motor vehicle cab. If there are balloons in the motor vehicle image, the motor vehicle image is used as the image to be detected, and further detection in S2 is performed.

[0034] In order to prevent the motor vehicle from traveling too fast and causing the motor vehicle image to become blurred, thereby preventing the accuracy of balloon detection from being reduced, this embodiment is configured to further determine whether the motor vehicle image is blurred before determining whether there are balloons in the motor vehicle image; if blurred, the motor vehicle image is deblurred to obtain a deblurred motor vehicle image, which is used to perform the operation of determining whether there are balloons in the motor vehicle image; if not blurred, the motor vehicle image directly performs the operation of determining whether there are balloons in the motor vehicle image.

[0035] The above-mentioned operation of determining whether the motor vehicle image is blurred is specifically as follows: performing Fourier transform processing on the motor vehicle image to obtain a spectrum diagram; based on the spectrum diagram, obtaining the high-frequency energy ratio (spectral energy with energy greater than an energy threshold); if the high-frequency energy ratio is greater than the ratio threshold, the motor vehicle image is not blurred; if the high-frequency energy ratio is not greater than the ratio threshold, the motor vehicle image is blurred.

[0036] In addition, the above-mentioned deblurring operation can be achieved by replacing the pixel value of each position point in the motor vehicle image with the median value of the pixel values ​​in its neighborhood. This can not only quickly deblur the image, but also retain the edge information of the image to a certain extent, which is conducive to subsequent balloon recognition.

[0037] S2. The image to be detected is processed by the improved YOLOv8 model to obtain an initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than a first intersection-over-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle.

[0038] First, the image to be detected is processed by the improved YOLOv8 model. The detection head module in the improved YOLOv8 model will perform target detection processing based on the detection frame on the image to be detected after feature extraction (the target detection processing in the detection head module is an existing technology and will not be described in detail here to save space). The cab is detected first, and then the detection frame area corresponding to the cab with a confidence level higher than the confidence threshold (preferably 0.4) is further subjected to target detection processing for balloons and people to obtain the initial detection result of nitrous oxide inhalation.

[0039] Then, the initial detection results of nitrous oxide inhalation are analyzed. If the intersection-over-union (IoU) of the detection frames corresponding to the balloon and the person is greater than a first IoU threshold, and the IoU of the detection frames corresponding to the balloon and the person's head is greater than a second IoU threshold, then there is a risk of nitrous oxide inhalation in the cab of the motor vehicle corresponding to the image to be detected. Therefore, target detection processing is performed on the image to be detected (which can be implemented based on the improved YOLOv8 model or other neural network models), and the license plate number of the motor vehicle is obtained based on the image to be detected. The motor vehicle corresponding to the license plate number, that is, the image to be detected, is used as the target motor vehicle, and S3 is executed to further determine whether there is nitrous oxide inhalation behavior.

[0040] The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module.

[0041] Since nitrous oxide inhalation is likely to occur at night, and the balloons storing nitrous oxide are small targets, the color, shape and deep features of the balloons are often easily obscured by noise in the night environment. Directly applying the existing YOLOv8 model for target detection is difficult to effectively capture the characteristics of the balloons in motor vehicles, which will lead to detection failure. To solve the above technical problems, this embodiment adds a global frequency feature information extraction module to the C2f module of the YOLOv8 model, so that the model can better understand the contextual relationship between the target and the background information at a lower computational cost, thereby effectively improving the model's detection accuracy for the target. Secondly, a global and local information fusion module is added between the SPPF module and the Concat module in the YOLOv8 model to strengthen feature fusion, so that the positioning and high semantic features of the improved YOLOv8 model are richer, which can enhance the feature expression ability of image data in darker scenes, thereby further improving the model's detection accuracy for targets, as well as the overall generalization ability and robustness of the model.

[0042] Among them, the processing process in the global frequency feature information extraction module in the improved YOLOv8 model is specifically as follows: the input is processed by wavelet transform to obtain the initial wavelet transform feature; the low-frequency component features in the initial wavelet transform feature are processed by wavelet transform to obtain the low-frequency component wavelet transform feature; the low-frequency component wavelet transform feature is convolved and inverse wavelet transformed to obtain the low-frequency filter component feature; the low-frequency filter component feature is superimposed with each feature in the initial wavelet transform feature and then convolved to obtain the frequency enhancement feature; the frequency enhancement feature is inverse wavelet transformed to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition (element-by-element addition operation) operation in the Bottleneck submodule in the C2f module.

[0043] The processing process in the global and local information fusion module is as follows.

[0044] Step 1: The input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation to obtain global enhancement features and local attention features.

[0045] Among them, the operation of global information enhancement processing is specifically as follows: the convolution features are respectively subjected to convolution processing with several convolution kernels of different sizes to obtain several secondary convolution features; the secondary convolution features corresponding to the convolution kernel size of the last one are transposed to obtain transposed features; the transposed features are element-wise multiplied with the secondary convolution features corresponding to the convolution kernel size of the second last one, and then the transposed features are subjected to nonlinear processing (which can be achieved through the softmax function) with other secondary convolution features to obtain global enhanced features.

[0046] At the same time, the specific operations of local attention aggregation processing are as follows: the convolution features are subjected to convolution, batch normalization and nonlinear processing (which can be achieved through the ReLu activation function) to obtain the initial local features; based on the pixel value of each position point in the initial local features and the corresponding learning pixel value, the pixel value of each position point is subjected to channel attention processing, and then multiplied with the corresponding scaling factor to obtain the local initial attention feature; the local initial attention feature is subjected to nonlinear processing (which can be achieved through the softmax function) to obtain the local attention feature.

[0047] The operation of obtaining the local initial attention feature is achieved through the following formula:

[0048] ,

[0049] r i is the first local initial attention feature i The pixel value of the position point, f i is the first local feature in the initial i The scaling factor for each position point, x i,c is the first local feature in the initial i Position No. c The pixel value of each channel, s i,c is the first local feature in the initial i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

[0050] Step 2: After the global enhancement features and convolution features are processed by Hadamard product, dilated convolution is performed to obtain the global initial aggregation features; the global initial aggregation features and convolution features are added element by element to obtain the global aggregation features.

[0051] At the same time, the local attention features and the global enhancement features are processed by local information integration to obtain the local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain the local aggregation features.

[0052] The local information integration process is as follows: After batch normalization and nonlinear processing (which can be achieved through the ReLu activation function) of the local attention features and global enhancement features, the pixel values ​​of all channels at each location are weighted averaged to obtain the local integrated features; after the local initial global features are fully connected, they are element-wise multiplied with the convolutional features to obtain the local initial aggregated features. The local initial aggregated features are obtained by fully connecting the local initial global features and then processing them with the Sigmoid activation function to obtain channel weights. The channel weights are then multiplied by the values ​​of the convolutional features at the corresponding locations, so that the convolutional features obtain the relative importance of the channel dimension, thus obtaining the local initial aggregated features.

[0053] Step 3: Global aggregation features and local aggregation features are stacked to obtain global and local fusion features, which are used to perform operations in the Concat module.

[0054] S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0055] By comparing the intersection and comparison features of the detection frames corresponding to the balloon and the person on the same motor vehicle in the first road camera and the second road camera, and combining the image acquisition time, the probability of nitrous oxide inhalation of the target motor vehicle is calculated. If the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0056] First, when there is a possibility that a motor vehicle (target motor vehicle) captured by the first road camera may be inhaling nitrous oxide, in order to improve the accuracy of the monitoring results, when the target motor vehicle travels to a second distance from the second road camera, an image of the target motor vehicle is obtained as a risk motor vehicle image.

[0057] Then, based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained.

[0058] The probability of nitrous oxide inhalation is obtained by the following formula:

[0059] ,

[0060] P The probability of inhaling nitrous oxide, α 、 β 、 γis the intersection-to-union ratio variation parameter, time probability parameter, and probability compensation value, ΔIoU is the difference between the intersection of the detection frames of the balloon and the person in the image to be detected and the intersection of the detection frames of the balloon and the person in the risk vehicle image, f(23:59,T) Acquisition time for risky motor vehicle images T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding night time function, T and 23:59 The closer it is, the more likely it is that illegal activities will occur at night. The smaller the night time function value is, the greater the probability of nitrous oxide inhalation is.

[0061] When the second distance is equal to the first distance, the image ratio of the risky motor vehicle image and the image to be detected is the same. By directly comparing the risky motor vehicle image with the image to be detected, the intersection and union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image can be obtained. ΔIoU When the second distance is not equal to the first distance, the image ratio of the risky motor vehicle image is converted to the same as that of the image to be detected, and the intersection and union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image and the image to be detected is compared to obtain ΔIoU .

[0062] The probability of nitrous oxide inhalation can also be obtained by the following formula:

[0063] ,

[0064] P The probability of inhaling nitrous oxide, a, b are the first parameter and probability compensation term respectively, IoU is the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, T is the time when the risky motor vehicle image is acquired, or the average acquisition time of the risky motor vehicle image and the image to be detected, P(T) Acquisition time for risky motor vehicle images T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding nighttime illegal behavior generation value, T and 23:59 The closer it is, the greater the value of illegal activities at night and the greater the probability of nitrous oxide inhalation.

[0065] The probability of nitrous oxide inhalation can also be obtained by the following formula:

[0066] ,

[0067] P The probability of inhaling nitrous oxide, a, b are the first parameter and probability compensation term respectively, IoU is the intersection-union ratio of the detection boxes corresponding to the balloon and the person in the risky motor vehicle image.

[0068] Finally, if the probability of nitrous oxide inhalation is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0069] This embodiment further provides a nitrous oxide inhalation detection and warning system for implementing the above-mentioned nitrous oxide inhalation detection and warning method, comprising:

[0070] a module for generating an image to be detected, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and use the image of the motor vehicle as the image to be detected if a balloon is present in the image of the motor vehicle;

[0071] The target motor vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frame corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: :The input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; the low-frequency component wavelet transform features are convolved and inverse wavelet transformed to obtain low-frequency filter component features; the low-frequency filter component features are superimposed with each feature in the initial wavelet transform features and then convolved to obtain frequency enhancement features; the frequency enhancement features are inverse wavelet transformed to obtain low-frequency enhancement features; the input is convolved several times and then superimposed with the low-frequency enhancement features to obtain global frequency features, which are used to perform the superposition operation in the Bottleneck submodule in the C2f module;

[0072] The nitrous oxide inhalation probability generation and alarm module is used to obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; based on the time when the risky motor vehicle image is obtained and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the nitrous oxide inhalation probability is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0073] This embodiment further provides a nitrous oxide inhalation detection and warning device, comprising a processor and a memory, wherein the processor implements the above-mentioned nitrous oxide inhalation detection and warning method when executing a computer program stored in the memory.

[0074] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned nitrous oxide inhalation detection and warning method.

[0075] The present embodiment provides a method for detecting and warning nitrous oxide inhalation. First, an image of a motor vehicle traveling near a first road camera is obtained to check whether there are balloons related to nitrous oxide inhalation in and near the motor vehicle cab. If there are balloons in the motor vehicle image, the motor vehicle image is used as the image to be detected, and the improved YOLOv8 model is used for target detection to obtain an initial detection result of nitrous oxide inhalation. The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module, so that the model can better understand the relative position of the target ball with respect to the background information. The relationship below can quickly locate the target and obtain high semantic features, enhance the feature expression ability of the image in darker scenes, and improve the target detection accuracy; then, if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the result of the first intersection-over-union ratio threshold, then the intersection-over-union features of the detection frames corresponding to the balloon and the person on the same motor vehicle at the first road camera and the second road camera are compared, and combined with the image acquisition time, the nitrous oxide inhalation probability of the target motor vehicle is calculated, and the target motor vehicle with a nitrous oxide inhalation probability greater than the inhalation probability threshold is alarmed as a nitrous oxide inhalation illegal motor vehicle; this method is used in the detection and identification of nitrous oxide inhalation behavior on the road, which can improve the accuracy and efficiency of detection.

Claims

1. A method for detecting and warning of nitrous oxide inhalation, characterized in that: The following operations are included: S1. When a motor vehicle travels to a first distance from a first road camera, acquiring an image of the motor vehicle, and if a balloon is present in the image of the motor vehicle, using the image of the motor vehicle as an image to be detected; S2. The image to be detected is processed using the improved YOLOv8 model to obtain an initial nitrous oxide inhalation detection result. If, in the initial nitrous oxide inhalation detection result, the intersection-over-union ratio of the detection frames corresponding to the balloon and the person is greater than a first intersection-over-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle. The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; The processing process in the global frequency feature information extraction module is as follows: the input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; The low-frequency component wavelet transform features are convolved and inverse wavelet transformed to obtain the low-frequency filter component features; The low-frequency filter component features and each feature in the initial wavelet transform feature are superimposed and then convolved to obtain the frequency enhancement feature; The frequency enhancement feature is inversely transformed into a low-frequency enhancement feature. The input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain a global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule of the C2f module. The processing process in the global and local information fusion module is as follows: the input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation respectively to obtain global enhanced features and local attention features; the global enhanced features and the convolution features are processed by Hadamard product, and then dilated convolution is performed to obtain global initial aggregation features; the global initial aggregation features and the convolution features are added element by element to obtain global aggregation features; the local attention features and the global enhancement features are processed by local information integration to obtain local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain local aggregation features; the global aggregation features and the local aggregation features are stacked to obtain global and local fusion features, which are used to perform the operations in the Concat module; S3. Acquire an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risky motor vehicle image; Based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

2. The laughing gas inhalation detection and early warning method according to claim 1, characterized in that: The operations of the global information enhancement process are specifically as follows: The convolution features are processed by several convolution kernels with different sizes to obtain several secondary convolution features. The secondary convolution features corresponding to the penultimate convolution kernel size are transposed to obtain transposed features. The transposed features are element-wise multiplied with the secondary convolution features corresponding to the penultimate convolution kernel size, and then processed nonlinearly with other secondary convolution features to obtain global enhanced features.

3. The method for detecting and warning of nitrous oxide inhalation according to claim 1, wherein: The operations of the local attention aggregation processing are specifically as follows: The convolution feature is subjected to convolution, batch normalization and nonlinear processing to obtain an initial local feature; based on the pixel value of each position point in the initial local feature and the corresponding learning pixel value, the pixel value of each position point is subjected to channel attention processing and multiplied with the corresponding scaling factor to obtain a local initial attention feature; the local initial attention feature is subjected to nonlinear processing to obtain the local attention feature.

4. The method for detecting and warning of nitrous oxide inhalation according to claim 1, wherein: The operation of obtaining the local initial attention feature is obtained by the following formula: r i is the first local initial attention feature i The pixel value of the position point, f i is the first local feature in the initial i The scaling factor for each position point, x i,c is the first local feature in the initial i Position No. c The pixel value of each channel, s i,c is the first local feature in the initial i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

5. The method for detecting and warning of nitrous oxide inhalation according to claim 1, wherein: The operations of the local information integration processing are specifically as follows: After batch normalization and nonlinear processing of local attention features and global enhancement features, the pixel values ​​of all channels at each location are weighted averaged to obtain local integrated features; After the local initial overall features are fully connected, they are multiplied element-by-element with the convolutional features to obtain the local initial aggregation features.

6. The method for detecting and warning of nitrous oxide inhalation according to claim 1, wherein: In S3, the probability of nitrous oxide inhalation is obtained by the following formula: P The probability of inhaling nitrous oxide, α 、 β 、 γ is the intersection-to-union ratio variation parameter, time probability parameter, and probability compensation value, ΔIoU is the difference between the intersection of the detection frames of the balloon and the person in the image to be detected and the intersection of the detection frames of the balloon and the person in the risk vehicle image, f(23:59,T) Acquisition time for risky motor vehicle images T The corresponding night time function.

7. A nitrous oxide inhalation detection and early warning system, used to implement the nitrous oxide inhalation detection and early warning method according to claim 1, characterized in that: include: a module for generating an image to be detected, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and use the image of the motor vehicle as the image to be detected if a balloon is present in the image of the motor vehicle; The target vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; If, in the initial detection result of nitrous oxide inhalation, the intersection-and-union ratio of the detection frames corresponding to the balloon and the person is greater than a first intersection-and-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; The low-frequency component wavelet transform features are convolved and inverse wavelet transformed to obtain the low-frequency filter component features; The low-frequency filter component features and each feature in the initial wavelet transform feature are superimposed and then convolved to obtain the frequency enhancement feature; The frequency enhancement feature is inversely transformed into a low-frequency enhancement feature. The input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain a global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule of the C2f module. a nitrous oxide inhalation probability generation and alarm module, configured to obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; Based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

8. A nitrous oxide inhalation detection and warning device, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the nitrous oxide inhalation detection and early warning method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, the nitrous oxide inhalation detection and early warning method according to any one of claims 1 to 6 is implemented.

Images