Nitrous oxide suction detection and early warning method, system and equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a laughing gas suction detection early warning method, system and device and a storage medium. In order to solve the problem of low accuracy and efficiency of laughing gas suction detection in the prior art, a global frequency feature information extraction module is added into a C2f module of a YOLOv8 model, and a global and local information fusion module is added between an SPPF module and a Concat module, so that the target detection accuracy is improved; meanwhile, the laughing gas suction probability of the target motor vehicle is calculated by comparing the intersection-to-union ratio characteristics of the balloon on the same motor vehicle and the detection frame corresponding to the person at the first road camera and the second road camera and combining the image acquisition time, and the target motor vehicle with the laughing gas suction probability larger than the suction probability threshold value is selected as the target motor vehicle with the laughing gas suction probability larger than the suction probability threshold value. An alarm is given when laughing gas sucks illegal motor vehicles; the method is applied to road laughing gas suction behavior detection, and the detection accuracy and efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method, system, device and storage medium for detecting and warning of nitrous oxide inhalation. Background Art

[0002] Inhaling nitrous oxide can affect the function of the nervous system, cause mood swings, and increase fatigue and drowsiness. Inhaling nitrous oxide while driving or riding in a motor vehicle on the road is an extremely dangerous behavior. It not only poses a serious threat to the life safety of the driver or passengers themselves, but also brings huge risks to other road users, seriously undermining road traffic safety and traffic order.

[0003] At present, relevant law enforcement departments have begun to identify balloons storing nitrous oxide through target detection technology in intelligent traffic monitoring systems, and then manually identify pictures of balloons in the cab to determine whether there is dangerous behavior of inhaling nitrous oxide. However, after testing, the detection accuracy and efficiency of this method are poor, mainly due to the following reasons: First, balloons storing nitrous oxide are small targets, and their size is smaller than other targets in the scene. Ordinary target detection methods are difficult to accurately and stably detect balloons; and judging whether there is nitrous oxide inhalation behavior from pictures based on manual experience will have problems in subjectivity, stability and cost, which will reduce the accuracy and efficiency of nitrous oxide inhalation detection and identification in road traffic. Summary of the invention

[0004] The purpose of the present invention is to provide a laughing gas inhalation detection and early warning method, system, equipment and storage medium.

[0005] The technical solution of the present invention is as follows: A method for detecting and warning of nitrous oxide inhalation, comprising the following operations: S1. When the motor vehicle travels to a first distance from the first road camera, an image of the motor vehicle is acquired, and if a balloon exists in the image of the motor vehicle, the image of the motor vehicle is used as an image to be detected; S2. The image to be detected is processed by the improved YOLOv8 model to obtain an initial detection result of nitrous oxide inhalation; if in the initial detection result of nitrous oxide inhalation, the intersection and union ratio of the detection frames corresponding to the balloon and the person is greater than the first intersection and union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input Wavelet transform processing is performed to obtain initial wavelet transform features; low-frequency component features in the initial wavelet transform features are subjected to wavelet transform processing to obtain low-frequency component wavelet transform features; low-frequency component wavelet transform features are subjected to convolution and inverse wavelet transform to obtain low-frequency filter component features; low-frequency filter component features are subjected to convolution processing after being superimposed with each feature in the initial wavelet transform features to obtain frequency enhancement features; frequency enhancement features are subjected to inverse wavelet transform to obtain low-frequency enhancement features; input is subjected to convolution several times and then superimposed with low-frequency enhancement features to obtain global frequency features, which are used to perform superposition operations in the Bottleneck submodule in the C2f module; S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0006] The processing process in the global and local information fusion module in S2 is specifically as follows: the input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation respectively to obtain global enhanced features and local attention features; the global enhanced features and the convolution features are processed by Hadamard product, and then dilated convolution is performed to obtain global initial aggregation features; the global initial aggregation features and the convolution features are added element by element to obtain global aggregation features; the local attention features and the global enhancement features are processed by local information integration to obtain local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain local aggregation features; the global aggregation features and the local aggregation features are stacked to obtain global and local fusion features, which are used to execute the operations in the Concat module.

[0007] The specific operation of global information enhancement processing is as follows: the convolution features are respectively subjected to convolution processing with several convolution kernels of different sizes to obtain several secondary convolution features; the secondary convolution features corresponding to the penultimate convolution kernel size are transposed to obtain transposed features; the transposed features are element-wise multiplied with the secondary convolution features corresponding to the second-to-last convolution kernel size, and then nonlinearly processed with other secondary convolution features to obtain global enhanced features.

[0008] The specific operations of local attention aggregation processing are as follows: the convolution features are subjected to convolution, batch normalization and nonlinear processing to obtain the initial local features; based on the pixel value of each position point in the initial local features and the corresponding learning pixel value, the pixel value of each position point is subjected to channel attention processing, and then multiplied with the corresponding scaling factor to obtain the local initial attention feature; the local initial attention feature is subjected to nonlinear processing to obtain the local attention feature.

[0009] The operation of obtaining the local initial attention feature is obtained by the following formula: , r i is the local initial attention feature i The pixel value of the location point, f i is the first local feature in the initial i The scaling factor for each position point is x i,c is the first local feature in the initial i Position No. c The pixel value of each channel, s i,c is the first local feature in the i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

[0010] The specific operation of local information integration processing is as follows: after the local attention features and global enhancement features are batch normalized and nonlinearly processed, the pixel values ​​of all channels of each position point are weighted averaged to obtain the local integrated features; after the local initial overall features are fully connected, they are multiplied element-by-element with the convolution features to obtain the local initial aggregation features.

[0011] The probability of nitrous oxide inhalation in S3 is obtained by the following formula: , P The probability of inhaling nitrous oxide, α , β , γis the intersection-and-combination ratio variation parameter, time probability parameter, and probability compensation value, ΔIoU is the difference between the intersection of the detection frames corresponding to the balloon and the person in the image to be detected and the intersection of the detection frames corresponding to the balloon and the person in the risk motor vehicle image. f(23:59,T) Get the time for the risky motor vehicle image T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding night time function.

[0012] A laughing gas inhalation detection and early warning system, used to implement the above-mentioned laughing gas inhalation detection and early warning method, comprising: a to-be-detected image generation module, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and if a balloon exists in the image of the motor vehicle, use the image of the motor vehicle as the to-be-detected image; The target motor vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frame corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: :The input is processed by wavelet transform to obtain the initial wavelet transform feature; the low-frequency component feature in the initial wavelet transform feature is processed by wavelet transform to obtain the low-frequency component wavelet transform feature; the low-frequency component wavelet transform feature is convolved and inverse wavelet transformed to obtain the low-frequency filter component feature; the low-frequency filter component feature is convolved with each feature in the initial wavelet transform feature to obtain the frequency enhancement feature; the frequency enhancement feature is inverse wavelet transformed to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule in the C2f module; The nitrous oxide inhalation probability generation and alarm module is used to obtain an image of a target motor vehicle when the target motor vehicle travels to a second distance from a second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the nitrous oxide inhalation probability is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0013] A laughing gas inhalation detection and early warning device comprises a processor and a memory, wherein the above-mentioned laughing gas inhalation detection and early warning method is implemented when the processor executes a computer program stored in the memory.

[0014] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the above-mentioned nitrous oxide inhalation detection and early warning method is implemented.

[0015] The beneficial effects of the present invention are: The present invention provides a method for detecting and warning nitrous oxide inhalation. First, an image of a motor vehicle traveling near a first road camera is obtained to check whether there are balloons related to nitrous oxide inhalation in and near the motor vehicle cab. If there are balloons in the motor vehicle image, the motor vehicle image is used as an image to be detected, and an improved YOLOv8 model is used for target detection to obtain an initial detection result of nitrous oxide inhalation. The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module, so that the model can better understand the relative position of the target ball to the background information. The following relationship can quickly locate the target and obtain high semantic features, enhance the feature expression ability of the image in darker scenes, and improve the target detection accuracy; then, if in the initial detection result of nitrous oxide inhalation, the intersection-and-union ratio of the detection frames corresponding to the balloon and the person is greater than the result of the first intersection-and-union ratio threshold, the intersection-and-union features of the detection frames corresponding to the balloon and the person on the same motor vehicle at the first road camera and the second road camera are compared, and combined with the image acquisition time, the nitrous oxide inhalation probability of the target motor vehicle is calculated, and the target motor vehicle with a nitrous oxide inhalation probability greater than the inhalation probability threshold is alarmed as a nitrous oxide inhalation illegal motor vehicle; this method is applied to the detection and identification of road nitrous oxide inhalation behavior, which can improve the accuracy and efficiency of detection. DETAILED DESCRIPTION

[0016] This embodiment provides a method for detecting and warning of nitrous oxide inhalation, including the following operations: S1. When the motor vehicle travels to a first distance from the first road camera, an image of the motor vehicle is acquired, and if a balloon exists in the image of the motor vehicle, the image of the motor vehicle is used as an image to be detected; S2. The image to be detected is processed by the improved YOLOv8 model to obtain an initial detection result of nitrous oxide inhalation; if in the initial detection result of nitrous oxide inhalation, the intersection and union ratio of the detection frames corresponding to the balloon and the person is greater than the first intersection and union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input Wavelet transform processing is performed to obtain initial wavelet transform features; low-frequency component features in the initial wavelet transform features are subjected to wavelet transform processing to obtain low-frequency component wavelet transform features; low-frequency component wavelet transform features are subjected to convolution and inverse wavelet transform to obtain low-frequency filter component features; low-frequency filter component features are subjected to convolution processing after being superimposed with each feature in the initial wavelet transform features to obtain frequency enhancement features; frequency enhancement features are subjected to inverse wavelet transform to obtain low-frequency enhancement features; input is subjected to convolution several times and then superimposed with low-frequency enhancement features to obtain global frequency features, which are used to perform superposition operations in the Bottleneck submodule in the C2f module; S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0017] S1. When a motor vehicle travels to a first distance from a first road camera, an image of the motor vehicle is acquired. If a balloon exists in the image of the motor vehicle, the image of the motor vehicle is used as an image to be detected.

[0018] When the motor vehicle travels to a first distance from the first road camera, the motor vehicle image is obtained to check whether there are balloons related to nitrous oxide inhalation in the motor vehicle cab and its vicinity. If there are balloons in the motor vehicle image, the motor vehicle image is used as the image to be detected, and further detection in S2 is performed.

[0019] In order to prevent the motor vehicle from traveling too fast and causing the motor vehicle image to be blurred, thereby preventing the accuracy of balloon detection from being reduced, the present embodiment is arranged to determine whether the motor vehicle image is blurred before determining whether there is a balloon in the motor vehicle image; if blurred, the motor vehicle image is deblurred to obtain a deblurred motor vehicle image, which is used to execute the operation of determining whether there is a balloon in the motor vehicle image; if not blurred, the motor vehicle image directly executes the operation of determining whether there is a balloon in the motor vehicle image.

[0020] The above-mentioned operation of judging whether the motor vehicle image is blurred is specifically as follows: performing Fourier transform processing on the motor vehicle image to obtain a spectrum diagram; based on the spectrum diagram, obtaining the high-frequency energy proportion (spectral energy whose energy is greater than the energy threshold); if the high-frequency energy proportion is greater than the proportion threshold, the motor vehicle image is not blurred; if the high-frequency energy proportion is not greater than the proportion threshold, the motor vehicle image is blurred.

[0021] In addition, the above deblurring operation can be achieved by replacing the pixel value of each position point in the motor vehicle image with the median value of the pixel value in its neighborhood. This can not only quickly deblur the image, but also retain the edge information of the image to a certain extent, which is beneficial to subsequent balloon recognition.

[0022] S2. The image to be detected is processed by the improved YOLOv8 model to obtain an initial detection result of nitrous oxide inhalation; if in the initial detection result of nitrous oxide inhalation, the intersection-and-union ratio of the detection frames corresponding to the balloon and the person is greater than the first intersection-and-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle.

[0023] First, the image to be detected is processed by the improved YOLOv8 model. The detection head module in the improved YOLOv8 model will perform target detection processing based on the detection frame on the image to be detected after feature extraction (the target detection processing in the detection head module is the existing technology and will not be described in detail here to save space). The cab is detected first, and then the detection frame area corresponding to the cab with a confidence level higher than the confidence threshold (preferably 0.4) is further processed for balloon and human target detection to obtain the initial detection result of nitrous oxide inhalation.

[0024] Then, the initial detection results of nitrous oxide inhalation are analyzed. If the intersection-and-union ratio of the detection frames corresponding to the balloon and the person is greater than the first intersection-and-union ratio threshold, and the intersection-and-union ratio of the detection frames corresponding to the balloon and the human head is greater than the second intersection-and-union ratio threshold, there is a risk of nitrous oxide inhalation in the cab of the motor vehicle corresponding to the image to be detected. Therefore, the image to be detected is processed for target detection (which can be implemented based on the improved YOLOv8 model or other neural network models), and the license plate number of the motor vehicle is obtained based on the image to be detected. The motor vehicle corresponding to the license plate number of the motor vehicle, that is, the image to be detected, is used as the target motor vehicle, and S3 is executed to further determine whether there is nitrous oxide inhalation behavior.

[0025] The above-mentioned improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module.

[0026] Since the act of inhaling nitrous oxide is likely to occur at night, and the balloon storing nitrous oxide is a small target, the color, shape and deep features of the balloon are often easily obscured by noise in the night environment. Directly applying the existing YOLOv8 model for target detection is difficult to effectively capture the features of the balloon in the motor vehicle, which will lead to detection failure. To solve the above technical problems, this embodiment adds a global frequency feature information extraction module to the C2f module of the YOLOv8 model, so that the model can better understand the contextual relationship between the target and the background information at a lower computational cost, and effectively improves the model's detection accuracy for the target. Secondly, a global and local information fusion module is added between the SPPF module and the Concat module in the YOLOv8 model to strengthen feature fusion, so that the positioning and high semantic features of the improved YOLOv8 model are richer, and the feature expression ability of image data in darker scenes can be enhanced, thereby further improving the model's detection accuracy for the target, as well as the overall generalization ability and robustness of the model.

[0027] Among them, the processing process in the global frequency feature information extraction module in the improved YOLOv8 model is specifically as follows: the input is processed by wavelet transform to obtain the initial wavelet transform feature; the low-frequency component features in the initial wavelet transform feature are processed by wavelet transform to obtain the low-frequency component wavelet transform feature; the low-frequency component wavelet transform feature is convolved and inverse wavelet transformed to obtain the low-frequency filter component feature; the low-frequency filter component feature is convolved with each feature in the initial wavelet transform feature after superposition to obtain the frequency enhancement feature; the frequency enhancement feature is inverse wavelet transformed to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition (element-by-element addition operation) operation in the Bottleneck submodule in the C2f module.

[0028] The processing process in the global and local information fusion module is as follows.

[0029] Step 1: The input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation to obtain global enhanced features and local attention features.

[0030] The operation of global information enhancement processing is as follows: the convolution features are respectively subjected to convolution processing with several convolution kernels of different sizes to obtain several secondary convolution features; the secondary convolution features corresponding to the penultimate convolution kernel size are transposed to obtain transposed features; the transposed features are element-wise multiplied with the secondary convolution features corresponding to the penultimate convolution kernel size, and then nonlinearly processed with other secondary convolution features (which can be achieved through the softmax function) to obtain global enhanced features.

[0031] At the same time, the operations of local attention aggregation processing are as follows: the convolution features are subjected to convolution, batch normalization and nonlinear processing (which can be achieved through the ReLu activation function) to obtain the initial local features; based on the pixel value of each position point in the initial local features and the corresponding learning pixel value, the pixel value of each position point is subjected to channel attention processing, and then multiplied with the corresponding scaling factor to obtain the local initial attention feature; the local initial attention feature is subjected to nonlinear processing (which can be achieved through the softmax function) to obtain the local attention feature.

[0032] The operation of obtaining the local initial attention feature is implemented by the following formula: , r i is the local initial attention feature i The pixel value of the location point, f i is the first local feature in the i The scaling factor for each position point is x i,c is the first local feature in the i Position No. c The pixel value of each channel, s i,c is the first local feature in the i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

[0033] Step 2: After the global enhanced features and convolution features are processed by Hadamard product, dilated convolution is performed to obtain the global initial aggregation features; the global initial aggregation features and convolution features are added element by element to obtain the global aggregation features.

[0034] At the same time, the local attention features and the global enhancement features are processed by local information integration to obtain the local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain the local aggregation features.

[0035] The specific operation of local information integration processing is as follows: after the local attention features and global enhancement features are batch normalized and nonlinearly processed (which can be achieved through the ReLu activation function), the pixel values ​​of all channels of each position point are weighted averaged to obtain the local integrated features; after the local initial overall features are fully connected, they are element-wise multiplied with the convolution features to obtain the local initial aggregation features. The specific operation of obtaining the local initial aggregation features is as follows: after the local initial overall features are fully connected, they are processed by the Sigmoid activation function to obtain the channel weights, and the channel weights are multiplied with the values ​​of the corresponding position points of the convolution features so that the convolution features obtain the relative importance of the channel dimension and obtain the local initial aggregation features.

[0036] Step 3: Global aggregation features and local aggregation features are stacked to obtain global and local fusion features, which are used to perform operations in the Concat module.

[0037] S3. Obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risk motor vehicle image; obtain the probability of nitrous oxide inhalation based on the time when the risk motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risk motor vehicle image; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, record the license plate number of the target motor vehicle and issue an alarm.

[0038] By comparing the intersection and comparison features of the detection frames corresponding to the balloon and the person on the same motor vehicle in the first road camera and the second road camera, and combining the image acquisition time, the probability of nitrous oxide inhalation of the target motor vehicle is calculated. If the probability of nitrous oxide inhalation is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0039] First, when there is a possibility that nitrous oxide is inhaled in a motor vehicle (target motor vehicle) captured by the first road camera, in order to improve the accuracy of the monitoring results, when the target motor vehicle travels to a second distance from the second road camera, an image of the target motor vehicle is acquired as a risky motor vehicle image.

[0040] Then, based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained.

[0041] The probability of nitrous oxide inhalation is obtained by the following formula: , P The probability of inhaling nitrous oxide, α , β , γ is the intersection-and-combination ratio variation parameter, time probability parameter, and probability compensation value, ΔIoUis the difference between the intersection of the detection frames corresponding to the balloon and the person in the image to be detected and the intersection of the detection frames corresponding to the balloon and the person in the risk motor vehicle image. f(23:59,T) Get the time for the risky motor vehicle image T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding night time function, T and 23:59 The closer it is, the more likely it is that illegal behavior will occur at night. The smaller the night time function value is, the greater the probability of nitrous oxide inhalation is.

[0042] When the second distance is equal to the first distance, the image ratio of the risky motor vehicle image and the image to be detected is the same. By directly comparing the detection box intersection and union ratio of the balloon and the person in the risky motor vehicle image with the image to be detected, we can get ΔIoU When the second distance is not equal to the first distance, the image ratio of the risky motor vehicle image is converted to the same as that of the image to be detected, and the intersection and union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image and the image to be detected is compared to obtain ΔIoU .

[0043] The probability of nitrous oxide inhalation can also be obtained through the following formula: , P The probability of inhaling nitrous oxide, a, b are the first parameter and probability compensation term respectively, IoU is the intersection-union ratio of the detection boxes corresponding to the balloon and the person in the risky motor vehicle image, T is the time when the risky motor vehicle image is acquired, or the average time when the risky motor vehicle image and the image to be detected are acquired. P(T) Get the time for the risky motor vehicle image T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding nighttime illegal behavior generation value, T and 23:59 The closer it is, the greater the value of illegal activities at night and the greater the probability of nitrous oxide inhalation.

[0044] The probability of nitrous oxide inhalation can also be obtained through the following formula: , P The probability of inhaling nitrous oxide, a, b are the first parameter and probability compensation term respectively, IoU It is the intersection-union ratio of the detection boxes corresponding to the balloon and the person in the risky motor vehicle image.

[0045] Finally, if the probability of nitrous oxide inhalation is greater than the inhalation probability threshold, the target motor vehicle license plate number is recorded and an alarm is issued.

[0046] This embodiment also provides a nitrous oxide inhalation detection and early warning system, which is used to implement the above-mentioned nitrous oxide inhalation detection and early warning method, including: a to-be-detected image generation module, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and if a balloon exists in the image of the motor vehicle, use the image of the motor vehicle as the to-be-detected image; The target motor vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frame corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: :The input is processed by wavelet transform to obtain the initial wavelet transform feature; the low-frequency component feature in the initial wavelet transform feature is processed by wavelet transform to obtain the low-frequency component wavelet transform feature; the low-frequency component wavelet transform feature is convolved and inverse wavelet transformed to obtain the low-frequency filter component feature; the low-frequency filter component feature is convolved with each feature in the initial wavelet transform feature to obtain the frequency enhancement feature; the frequency enhancement feature is inverse wavelet transformed to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule in the C2f module; The nitrous oxide inhalation probability generation and alarm module is used to obtain an image of a target motor vehicle when the target motor vehicle travels to a second distance from a second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the nitrous oxide inhalation probability is obtained; if the nitrous oxide inhalation probability is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

[0047] The present embodiment further provides a nitrous oxide inhalation detection and warning device, comprising a processor and a memory, wherein the processor implements the above-mentioned nitrous oxide inhalation detection and warning method when executing a computer program stored in the memory.

[0048] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned nitrous oxide inhalation detection and early warning method when executed by a processor.

[0049] The present embodiment provides a method for detecting and warning nitrous oxide inhalation. First, an image of a motor vehicle traveling near a first road camera is obtained to check whether there are balloons related to nitrous oxide inhalation in and near the motor vehicle cab. If there are balloons in the motor vehicle image, the motor vehicle image is used as the image to be detected, and the improved YOLOv8 model is used for target detection to obtain an initial detection result of nitrous oxide inhalation. The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module, so that the model can better understand the relative position of the target ball to the background information. The following relationship can quickly locate the target and obtain high semantic features, enhance the feature expression ability of the image in darker scenes, and improve the target detection accuracy; then, if in the initial detection result of nitrous oxide inhalation, the intersection-and-union ratio of the detection frames corresponding to the balloon and the person is greater than the result of the first intersection-and-union ratio threshold, the intersection-and-union features of the detection frames corresponding to the balloon and the person on the same motor vehicle at the first road camera and the second road camera are compared, and combined with the image acquisition time, the nitrous oxide inhalation probability of the target motor vehicle is calculated, and the target motor vehicle with a nitrous oxide inhalation probability greater than the inhalation probability threshold is alarmed as a nitrous oxide inhalation illegal motor vehicle; this method is applied to the detection and identification of road nitrous oxide inhalation behavior, which can improve the accuracy and efficiency of detection.

Claims

1. A method for detecting and warning of nitrous oxide inhalation, characterized in that: The following operations are included: S1. When the motor vehicle travels to a first distance from the first road camera, an image of the motor vehicle is acquired, and if a balloon exists in the image of the motor vehicle, the image of the motor vehicle is used as an image to be detected; S2. The image to be detected is processed by the improved YOLOv8 model to obtain an initial detection result of nitrous oxide inhalation; if the intersection-over-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-over-union ratio threshold, the license plate number of the motor vehicle is obtained based on the image to be detected, and the motor vehicle corresponding to the license plate number is used as the target motor vehicle; The improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model and adding a global and local information fusion module between the SPPF module and the Concat module; The processing process in the global frequency feature information extraction module is as follows: the input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; The wavelet transform features of the low-frequency components are subjected to convolution and inverse wavelet transform to obtain the low-frequency filtering component features; The low-frequency filter component feature and each feature in the initial wavelet transform feature are superimposed and then convolved to obtain the frequency enhancement feature; The frequency enhancement feature is transformed by inverse wavelet transform to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule in the C2f module; S3, obtaining an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as the risky motor vehicle image; Based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained; if the probability of nitrous oxide inhalation is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

2. The laughing gas inhalation detection and early warning method according to claim 1, characterized in that: In S2, the processing process in the global and local information fusion module is specifically as follows: The input is processed by convolution to obtain convolution features; the convolution features are processed by global information enhancement and local attention aggregation to obtain global enhancement features and local attention features; After the global enhancement feature and the convolution feature are processed by Hadamard product, the dilated convolution is performed to obtain the global initial aggregation feature; the global initial aggregation feature and the convolution feature are added element by element to obtain the global aggregation feature; The local attention features and the global enhancement features are processed by local information integration to obtain the local initial aggregation features; the local initial aggregation features and the convolution features are added element by element to obtain the local aggregation features; The global aggregation features and the local aggregation features are stacked to obtain global and local fusion features, which are used to perform operations in the Concat module.

3. The laughing gas inhalation detection and early warning method according to claim 2, characterized in that: The operations of the global information enhancement processing are specifically as follows: The convolution features are processed by several convolutions with different convolution kernel sizes to obtain several secondary convolution features; the secondary convolution features corresponding to the penultimate convolution kernel size are transposed to obtain transposed features; the transposed features are element-wise multiplied with the secondary convolution features corresponding to the second-to-last convolution kernel size, and then processed nonlinearly with other secondary convolution features to obtain global enhanced features.

4. The method for detecting and warning of nitrous oxide inhalation according to claim 2, characterized in that: The operations of the local attention aggregation processing are specifically as follows: The convolution feature is subjected to convolution, batch normalization and nonlinear processing to obtain an initial local feature; based on the pixel value of each position point in the initial local feature and the corresponding learning pixel value, the pixel value of each position point is subjected to channel attention processing and then multiplied with the corresponding scaling factor to obtain a local initial attention feature; the local initial attention feature is subjected to nonlinear processing to obtain the local attention feature.

5. The laughing gas inhalation detection and early warning method according to claim 2, characterized in that: The operation of obtaining the local initial attention feature is obtained by the following formula: ， r i is the local initial attention feature i The pixel value of the location point, f i is the first local feature in the initial i The scaling factor for each position point is x i,c is the first local feature in the initial i Position No. c The pixel value of each channel, s i,c is the first local feature in the i Position No. c The learning pixel value corresponding to each channel, C is the total number of channels.

6. The method for detecting and warning of nitrous oxide inhalation according to claim 2, characterized in that: The operations of the local information integration processing are specifically as follows: After batch normalization and nonlinear processing, the local attention features and global enhancement features are weighted averaged on the pixel values ​​of all channels at each location point to obtain the local integrated features. After the local initial overall features are fully connected, they are multiplied element-by-element with the convolutional features to obtain the local initial aggregation features.

7. The method for detecting and warning of nitrous oxide inhalation according to claim 1, characterized in that: In S3, the probability of nitrous oxide inhalation is obtained by the following formula: ， P The probability of inhaling nitrous oxide, α , β , γ is the intersection-and-combination ratio variation parameter, time probability parameter, and probability compensation value, ΔIoU is the difference between the intersection of the detection frames corresponding to the balloon and the person in the image to be detected and the intersection of the detection frames corresponding to the balloon and the person in the risk motor vehicle image. f(23:59,T) Get the time for the risky motor vehicle image T , or the average acquisition time of the risk vehicle image and the image to be detected T The corresponding night time function.

8. A nitrous oxide inhalation detection and early warning system, used to implement the nitrous oxide inhalation detection and early warning method according to claim 1, characterized in that: include: a module for generating an image to be detected, configured to obtain an image of the motor vehicle when the motor vehicle travels to a first distance from the first road camera, and if a balloon exists in the image of the motor vehicle, use the image of the motor vehicle as the image to be detected; The target motor vehicle acquisition module is used to process the image to be detected through the improved YOLOv8 model to obtain the initial detection result of nitrous oxide inhalation; If the intersection-and-union ratio of the detection frames corresponding to the balloon and the person in the initial detection result of nitrous oxide inhalation is greater than the first intersection-and-union ratio threshold, the motor vehicle license plate number is obtained based on the image to be detected, and the motor vehicle corresponding to the motor vehicle license plate number is used as the target motor vehicle; the improved YOLOv8 model is obtained by adding a global frequency feature information extraction module to the C2f module of the YOLOv8 model, and adding a global and local information fusion module between the SPPF module and the Concat module; the processing process in the global frequency feature information extraction module is specifically as follows: the input is processed by wavelet transform to obtain initial wavelet transform features; the low-frequency component features in the initial wavelet transform features are processed by wavelet transform to obtain low-frequency component wavelet transform features; The wavelet transform features of the low-frequency components are subjected to convolution and inverse wavelet transform to obtain the low-frequency filtering component features; The low-frequency filter component feature and each feature in the initial wavelet transform feature are superimposed and then convolved to obtain the frequency enhancement feature; The frequency enhancement feature is transformed by inverse wavelet transform to obtain the low-frequency enhancement feature; the input is convolved several times and then superimposed with the low-frequency enhancement feature to obtain the global frequency feature, which is used to perform the superposition operation in the Bottleneck submodule in the C2f module; The nitrous oxide inhalation probability generation and alarm module is used to obtain an image of the target motor vehicle when the target motor vehicle travels to a second distance from the second road camera within a neighborhood time or neighborhood area, as a risky motor vehicle image; Based on the time when the risky motor vehicle image is acquired and the intersection-union ratio of the detection frames corresponding to the balloon and the person in the risky motor vehicle image, the probability of nitrous oxide inhalation is obtained; if the probability of nitrous oxide inhalation is greater than the inhalation probability threshold, the license plate number of the target motor vehicle is recorded and an alarm is issued.

9. A laughing gas inhalation detection and warning device, characterized in that: The method comprises a processor and a memory, wherein when the processor executes a computer program stored in the memory, the method for detecting and warning the inhalation of nitrous oxide as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the nitrous oxide inhalation detection and early warning method as described in any one of claims 1 to 7 is implemented.

Images