A method, apparatus, device, and storage medium for detecting dangerous driving behavior.

Abstract

This application discloses a method, apparatus, device, and storage medium for detecting dangerous driving behavior. The method includes: acquiring an initial driving behavior image corresponding to the driver during vehicle operation, wherein the initial driving behavior image is an initial multi-channel infrared image; extracting any one of multiple initial single-channel infrared images with the same pixel value corresponding to the initial driving behavior image as a target image; performing image transformation processing based on the target image to obtain multiple target sub-images with different pixel values ​​corresponding to the target image; merging the image channels of the multiple target sub-images to obtain a target driving behavior image; and inputting the target driving behavior image into a dangerous driving behavior detection model to detect dangerous driving behavior and obtain a behavior detection result. The technical solution of this application allows the input image to contain more information without changing the image size of the input model, thereby improving the accuracy of the behavior detection result.

Description

Technical Field

[0001] This application relates to the field of image signal processing technology, specifically to a method, apparatus, device, and storage medium for detecting dangerous driving behavior. Background Technology

[0002] Currently, vehicle accidents caused by dangerous driving behaviors are increasing year by year, and people are paying urgent attention to safety issues during vehicle operation. In existing dangerous driving behavior detection tasks, the infrared image of the driving behavior is usually reduced to the size of the input image corresponding to the deep learning neural network, and then the dangerous driving behavior is detected based on the reduced infrared image using the deep learning neural network.

[0003] However, since the pixel values ​​of the three RGB channels of an infrared image are the same, when a deep learning neural network extracts features from the three channels of an infrared image, it is equivalent to extracting features from only one channel. This results in redundant information extracted from the other two channels, which wastes the model's computational resources and affects the accuracy of dangerous driving behavior detection. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for detecting dangerous driving behavior. It allows the input image to contain more information while maintaining the same image size, thereby improving the accuracy of behavior detection results while avoiding wasting computational resources. The technical solution of this application is as follows:

[0005] On the one hand, a method for detecting dangerous driving behavior is provided, the method comprising:

[0006] During vehicle operation, an initial driving behavior image corresponding to the driver is acquired, wherein the initial driving behavior image is an initial multi-channel infrared image;

[0007] Extract any one initial single-channel infrared image from multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the pixel values ​​of the multiple initial single-channel infrared images are the same.

[0008] Based on the target image, image transformation processing is performed to obtain multiple target sub-images corresponding to the target image, and the pixel values ​​of each of the multiple target sub-images are different.

[0009] Image channels of the multiple target sub-images are merged to obtain the target driving behavior image;

[0010] The target driving behavior image is input into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection results.

[0011] On the other hand, a dangerous driving behavior detection device is provided, the device comprising:

[0012] The image acquisition module is used to acquire an initial driving behavior image corresponding to the driver during vehicle operation, wherein the initial driving behavior image is an initial multi-channel infrared image;

[0013] The target image extraction module is used to extract any one of the multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the multiple initial single-channel infrared images have the same pixel value.

[0014] An image conversion processing module is used to perform image conversion processing based on the target image to obtain multiple target sub-images corresponding to the target image, wherein the multiple target sub-images have different pixel values.

[0015] An image channel merging module is used to merge the multiple target sub-images to obtain a target driving behavior image;

[0016] The dangerous driving behavior detection module is used to input the target driving behavior image into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection result.

[0017] On the other hand, a dangerous driving behavior detection device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the dangerous driving behavior detection method as described above.

[0018] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the dangerous driving behavior detection method as described above.

[0019] The dangerous driving behavior detection method, device, equipment, and storage medium provided in this application have the following technical effects:

[0020] Using the technical solution provided in this application, during vehicle operation, an initial driving behavior image corresponding to the driver is acquired. This initial driving behavior image is an initial multi-channel infrared image containing multiple initial single-channel infrared images, each with the same pixel value. Then, image transformation processing is performed on any one of the multiple initial single-channel infrared images to obtain multiple target sub-images corresponding to the target image, each with different pixel values. Compared to the reduced infrared image obtained by directly downscaling the infrared driving behavior image in the prior art, the target multi-channel infrared image obtained by merging the image channels of multiple target sub-images has a larger entropy value in its pixel values ​​and contains more information. Therefore, inputting this target multi-channel infrared image into a dangerous driving behavior detection model can improve the accuracy of dangerous driving behavior detection. Furthermore, since the pixel values ​​of the multiple single-channel infrared images corresponding to the target multi-channel infrared image are different, redundant calculations in the dangerous driving behavior detection model can be avoided, thus preventing waste of computational resources. Attached Figure Description

[0021] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a dangerous driving behavior detection method provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a process for image transformation processing based on a target image to obtain multiple target sub-images corresponding to the target image, provided by an embodiment of this application;

[0024] Figure 3 This is a schematic diagram illustrating an image scaling method provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a process for sampling pixels in a target direction of a target image to obtain pixel values ​​corresponding to multiple sampled pixels in the target image, provided by an embodiment of this application.

[0026] Figure 5a -c is a schematic diagram of a set of pixel samples provided in the embodiments of this application;

[0027] Figure 6This is a schematic diagram of a process provided in this application embodiment to input a target driving behavior image into a dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection result;

[0028] Figure 7 This is a flowchart of a dangerous driving behavior early warning method based on the above-described dangerous driving behavior detection method provided in an embodiment of this application;

[0029] Figure 8 This is a schematic flowchart of a model training method provided in an embodiment of this application;

[0030] Figure 9 This is a flowchart illustrating another model training method provided in an embodiment of this application;

[0031] Figure 10 This is a flowchart illustrating another model training method provided in an embodiment of this application;

[0032] Figure 11 This is a schematic diagram of a dangerous driving behavior detection method and device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0035] The following describes a method for detecting dangerous driving behavior provided by an embodiment of this application. Figure 1This is a flowchart illustrating a method for detecting dangerous driving behavior provided in an embodiment of this application. It should be noted that this specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual systems or products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the above method may include:

[0036] S101, during vehicle operation, acquire the initial driving behavior image corresponding to the driver. The initial driving behavior image is an initial multi-channel infrared image.

[0037] In the embodiments of this specification, the initial driving behavior image can be an initial multi-channel infrared image obtained by capturing the driver's driving behavior.

[0038] In one specific embodiment, obtaining the initial driving behavior image corresponding to the driver may include: using a monocular infrared camera to acquire a sequence of infrared image frames of the driver driving the vehicle in the driver's seat; extracting one infrared image frame from the infrared image frame sequence as the original driving behavior image; and obtaining the initial driving behavior image based on the original driving behavior image.

[0039] In one specific embodiment, obtaining an initial driving behavior image based on the original driving behavior image may include scaling the original driving behavior image frame to obtain the initial driving behavior image.

[0040] S102, extract any one of the initial single-channel infrared images from the multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the pixel values ​​of the multiple initial single-channel infrared images are the same.

[0041] In the embodiments described in this specification, the initial driving behavior image can be an infrared image containing multiple image channels. In an optional embodiment, the multiple image channels may include three image channels.

[0042] In practical applications, the pixel values ​​of the three image channels in an RGB infrared image are all the same, meaning the brightness of the three image channels is identical. In a specific embodiment, the three image channels can be the R channel, G channel, and B channel, respectively. Correspondingly, the multiple initial single-channel infrared images corresponding to the initial driving behavior image can include: an initial R channel infrared image, an initial G channel infrared image, and an initial B channel infrared image. Extracting any one of the multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image can include: extracting any one of the initial R channel infrared image, the initial G channel infrared image, and the initial B channel infrared image as the target image.

[0043] S103, perform image transformation processing based on the target image to obtain multiple target sub-images corresponding to the target image, and the pixel values ​​of the multiple target sub-images are different.

[0044] In the embodiments described in this specification, the target sub-image can be an image obtained by filling in some pixel values ​​in the target image.

[0045] In the embodiments of this specification, the image conversion process here can be to fill the pixel values ​​obtained after pixel sampling of the target image into multiple single-channel images to be filled, thereby obtaining multiple target sub-images.

[0046] In a specific embodiment, such as Figure 2 As shown, the above image transformation processing based on the target image, which yields multiple target sub-images corresponding to the target image, can include:

[0047] S201, Obtain multiple single-channel images to be filled;

[0048] S202, perform pixel sampling in the target direction on the target image to obtain the pixel values ​​corresponding to multiple sampled pixel points in the target image;

[0049] S203, fill the pixel values ​​corresponding to multiple sampled pixel points into multiple single-channel images to be filled, and obtain multiple target sub-images;

[0050] Wherein, the first image size is the product of the second image size and the first quantity, the first image size is the image size of the target image, the second image size is the image size corresponding to any one of the multiple single-channel images to be filled, and the first quantity is the number of multiple single-channel images to be filled.

[0051] In one specific embodiment, the single-channel image to be filled can be a preset single-channel image used for pixel filling.

[0052] In one specific embodiment, the target direction can be the sampling direction of a pixel. Specifically, the target direction can include: horizontal sampling direction, vertical sampling direction, and diagonal sampling direction.

[0053] In one specific embodiment, the first image size can be the image size of the target image. Specifically, the first image size can include: a first image length and a first image width. Optionally, the first image length is equal to the first image width.

[0054] In one specific embodiment, the second image size can be the image size of the single-channel image to be filled. Since the target sub-image is the image obtained after pixel filling of the single-channel image to be filled, the second image size can also be the image size of the target sub-image. Specifically, the second image size can include: a second image length and a second image width. Optionally, the second image length is equal to the second image width.

[0055] In one specific embodiment, the first quantity can be set according to the image processing requirements in the actual application. For example, the first quantity can be 3, and correspondingly, the first image size can be 3 times the second image size; the first quantity can be 4, and correspondingly, the first image size can be 4 times the second image size.

[0056] In one specific embodiment, the input image size of the dangerous driving behavior detection model can be used as the second image size, then a first quantity can be determined, and the first image size can be obtained based on the first quantity and the second image size.

[0057] As can be seen from the above embodiments, by performing pixel sampling and pixel value filling on the target image (any one of the multiple initial single-channel infrared images with the same pixel value), multiple sub-images with different pixel values ​​are obtained, which can improve the amount of information contained in the subsequently synthesized target multi-channel infrared image.

[0058] In one specific embodiment, before obtaining the initial driving behavior image corresponding to the driver, the method may further include:

[0059] Based on the input image size of the dangerous driving behavior detection model, determine the second image size; determine the first quantity; based on the first quantity and the second image size, determine the first image size;

[0060] Accordingly, obtaining the driver's initial driving behavior image may include:

[0061] The original driving behavior image is acquired and scaled to obtain an initial driving behavior image of the first image size.

[0062] Specifically, the original driving behavior image here can be any frame from an infrared image frame sequence captured by an infrared camera.

[0063] See Figure 3 , Figure 3 This is a schematic diagram of image size scaling provided in an embodiment of this application. Taking a first quantity of 3 as an example, assuming the image size of the original driving behavior image is M×N and the input image size of the dangerous driving behavior detection model is n×n, then the second image size is determined to be n×n, and the first image size is (3×n)×(3×n). The original driving behavior image is scaled to obtain an initial driving behavior image of size (3×n)×(3×n). That is, multiple initial single-channel infrared images of size (3×n)×(3×n) can be obtained, and one initial single-channel infrared image is extracted from them as the target image.

[0064] In a specific embodiment, such as Figure 4 As shown, the above-mentioned pixel sampling of the target image in the target direction to obtain the pixel values ​​corresponding to multiple sampled pixel points in the target image may include:

[0065] S401, determine a second number of regions to be sampled in the target image, where the second number is the square of the second image size.

[0066] In one specific embodiment, the sampling area can be a square sampling area, and correspondingly, the side length of the sampling area can be an integer multiple of a first number. For example, if the first number is 3, the sampling area can be a 3x3 grid sampling area with a side length of 3.

[0067] In one specific embodiment, none of the second number of sampled regions have overlapping pixels.

[0068] S402, traverse the second number of regions to be sampled.

[0069] S403, perform pixel sampling in the target direction on the currently traversed sampling area to obtain the pixel value corresponding to the first number of sampled pixels in the currently traversed sampling area.

[0070] Accordingly, the process of filling multiple target sub-images by adding the pixel values ​​corresponding to multiple sampled pixels into multiple single-channel images to be filled can include:

[0071] S404, determine the correspondence between each sampled pixel in the first number of sampled pixels and each single-channel image to be filled in the first number of single-channel images to be filled.

[0072] In a specific embodiment, the correspondence can be a one-to-one correspondence between each sampled pixel and each single-channel image to be filled.

[0073] S405, based on the correspondence, fill the pixel values ​​corresponding to each sampled pixel point into each single-channel image to be filled, and obtain the first number of filled single-channel images.

[0074] S406, after traversing the second number of regions to be sampled, the current first number of filled single-channel images are used as multiple target sub-images.

[0075] In one example, the target image is 6×6 in size, and the single-channel image to be filled and the target sub-image are both 2×2 in size. Pixel sampling is performed on the target image in the target direction, and then the pixel values ​​corresponding to the sampled pixels are used to fill the single-channel image to be filled, resulting in the target sub-image. The target direction can include: horizontal sampling direction, vertical sampling direction, and diagonal sampling direction. Specifically, the horizontal, vertical, and diagonal sampling methods are as follows: Figure 5a , Figure 5b and Figure 5c As shown. Taking three target sub-images as an example, the three single-channel images to be filled represent the R channel image to be filled, the G channel image to be filled, and the B channel image to be filled, respectively.

[0076] For details, see Figure 5a The steps of the horizontal sampling method are as follows: First, determine four sampling regions in the target image, designated as Grid 1, Grid 2, Grid 3, and Grid 4. Then, iterate through these four Grids. Grid 1 is the first traversal; horizontal pixel sampling is performed on Grid 1 to obtain three pixel values ​​in the first row: 0, 1, and 2. These three pixel values ​​are then filled into the top-left corner pixels of the R-channel, G-channel, and B-channel images to be filled, respectively. Next, Grid 2 is the second traversal; horizontal pixel sampling is performed on Grid 2 to obtain three pixel values ​​in the first row: 10, 11, and 12. These three pixel values ​​are then filled into the top-right corner pixels of the R-channel, G-channel, and B-channel images to be filled, respectively. This process continues until all four Grids are traversed, resulting in three target sub-images: an R-channel sub-image, a G-channel sub-image, and a B-channel sub-image.

[0077] For details, see Figure 5bThe steps of the vertical sampling method are as follows: First, determine four sampling regions in the target image, designated as Grid 1, Grid 2, Grid 3, and Grid 4. Then, iterate through these four Grids. Grid 1 is the first traversal; horizontal pixel sampling is performed on Grid 1 to obtain three pixel values ​​in the first column: 0, 3, and 6. These three pixel values ​​are then filled into the top-left corner pixels of the R-channel, G-channel, and B-channel images to be filled, respectively. Next, Grid 2 is the second traversal; vertical pixel sampling is performed on Grid 2 to obtain three pixel values ​​in the first column: 10, 13, and 16. These three pixel values ​​are then filled into the top-right corner pixels of the R-channel, G-channel, and B-channel images to be filled, respectively. This process continues until all four Grids have been traversed, resulting in three target sub-images: an R-channel sub-image, a G-channel sub-image, and a B-channel sub-image.

[0078] For details, see Figure 5c The steps for diagonal sampling are as follows: First, identify four regions to be sampled in the target image, designated as Grid 1, Grid 2, Grid 3, and Grid 4. Then, iterate through these four Grids, starting with Grid 1. Perform diagonal pixel sampling on Grid 1 to obtain three pixel values ​​along its diagonal: 0, 4, and 8. These three pixel values ​​are then used to fill the corresponding R-channel, G-channel, and B-channel images to be filled, respectively. The first iteration is the top-left pixel. The second iteration is the 2-grid 3. Pixels are sampled diagonally in 2-grid 3, resulting in three pixel values: 10, 14, and 18. These three pixel values ​​are then used to fill the top-right pixel of the R-channel image, G-channel image, and B-channel image to be filled, respectively. This process continues until all four 2-grid 3 iterations are completed, resulting in three target sub-images: the R-channel sub-image, the G-channel sub-image, and the B-channel sub-image.

[0079] As can be seen from the above embodiments, image conversion processing is performed using multiple sampling methods, thereby improving the applicability of the image conversion method.

[0080] S104, merge the image channels of multiple target sub-images to obtain the target driving behavior image.

[0081] In the embodiments described in this specification, the target driving behavior image can be an infrared image containing multiple target image channels. These multiple target image channels can be image channels corresponding to multiple target sub-images.

[0082] In existing technologies, the initial driving behavior image is typically scaled directly to obtain the target driving behavior image. Here, continuing with the example of an initial driving behavior image of 6×6 pixels and a target driving behavior image of 2×2 pixels, the direct scaling process is similar to sampling 4 pixels from the 36 pixels of the initial driving behavior image to obtain the target driving behavior image; that is, the target driving behavior image contains the information of 4 pixels. Since the pixel values ​​of the R, G, and B channels are the same in the infrared image, this application selects one channel image from the three channels (R channel image, G channel image, and B channel image) of the initial driving behavior image as the target image, based on... Figure 5a The sampling method shown in -c samples the pixel values ​​of 12 pixels out of the 36 pixels in the target image, and fills them into three 2×2 single-channel images. Based on the three filled single-channel images, the image channels are merged to obtain the target driving behavior image. That is, the target driving behavior image contains the information of 12 pixels. Therefore, the technical solution of this application can contain more information in the image of the input model without changing the size of the input image of the dangerous driving behavior detection model, thereby improving the accuracy of the detection results.

[0083] S105, Input the target driving behavior image into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection result.

[0084] In the embodiments described in this specification, the above-mentioned dangerous driving behavior detection model may include: a face detection model, a face key point localization model, and a behavior detection model, such as... Figure 6 As shown, the above-mentioned input of the target driving behavior image into the dangerous driving behavior detection model for dangerous driving behavior detection can yield the following behavior detection results:

[0085] S601, input the target driving behavior image into the face detection model to perform face detection and obtain the driver's face information.

[0086] Specifically, the driver's facial information can be the face detection bounding box information in the target driving behavior image.

[0087] Specifically, the face detection model may include, but is not limited to, OpenCV models, deep convolutional neural networks, etc., and this application does not limit it.

[0088] In one specific embodiment, the face detection model can be obtained by training a preset face detection model based on sample driving behavior images and the labeled driver face information corresponding to the sample driving behavior images.

[0089] In a specific embodiment, the sample driving behavior image can be a multi-channel infrared image obtained based on the image conversion method of steps S102-S104, and the pixel values ​​of the multiple single-channel infrared images corresponding to the multi-channel infrared image are different.

[0090] S602, input the driver's facial information into the facial key point localization model to perform key point localization and obtain facial key point location information;

[0091] Specifically, facial landmark location information can characterize the location information of facial feature points of the driver in the target driving behavior image. Among them, facial feature points may include, but are not limited to: points of eyebrows, eyes, nose, mouth, and facial contour areas.

[0092] Specifically, the facial landmark localization model may include, but is not limited to, ERT algorithm model, Top-Bottom model, deep learning regression model, etc., and this application does not limit it.

[0093] In one specific embodiment, the facial landmark localization model can be obtained by training a preset facial landmark localization model on facial landmark localization based on the sample driver's facial information and the labeled facial landmark location information corresponding to the sample driver's facial information.

[0094] S603 extracts the target region image from the target driving behavior image based on the location information of facial key points.

[0095] In the embodiments of this specification, the target detection region image can be an ROI (Regions of Interest) image of dangerous driving behavior.

[0096] In one specific embodiment, a target region image can be extracted from a target driving behavior image using preset ROI size information and facial key point location information.

[0097] Specifically, the preset ROI size information can be pre-set in conjunction with the type of dangerous driving behavior. The preset ROI size information can include: ROI height and ROI width. Taking smoking as an example of dangerous driving behavior, the target area image can be the image around the driver's mouth. The height of the ROI can be controlled by the nose key point and the chin key point, and the width of the ROI can be controlled by the left and right key points of the corners of the mouth. The image around the mouth can be extracted from the target driving behavior image using the width and height.

[0098] Specifically, the image cropping process can be implemented using a preset script, which can be a small program written in Python. Based on the preset ROI size information and facial key point location information, the script can crop the target area image from the target driving behavior image.

[0099] S604: Input the target region image into the behavior detection model to detect dangerous driving behavior and obtain the behavior detection results.

[0100] In the embodiments of this specification, the behavior detection results are used to indicate whether the driver is engaging in dangerous driving behavior. Specifically, dangerous driving behavior may include, but is not limited to: the driver smoking, the driver not wearing a seatbelt, the driver taking both hands off the steering wheel, and the driver using a mobile phone. In one specific embodiment, the behavior detection results may include: a behavior detection tag.

[0101] Specifically, the behavior detection model may include, but is not limited to, deep convolutional neural networks, etc., and this application does not limit it.

[0102] In one specific embodiment, the behavior detection model can be obtained by training a preset behavior detection model to detect dangerous driving behavior based on the sample target region image and the labeled behavior detection results corresponding to the sample target region image.

[0103] As can be seen from the above embodiments, when the target multi-channel infrared image obtained by the image conversion method in steps S102-S104 is input into the dangerous driving behavior detection model for dangerous driving behavior detection, since the target multi-channel infrared image contains more information, the dangerous driving behavior detection model can extract more features from the target multi-channel infrared image, thereby improving the accuracy of dangerous driving behavior detection.

[0104] See Figure 7 , Figure 7 This is a flowchart of a dangerous driving behavior early warning method based on the above-described dangerous driving behavior detection method, provided in an embodiment of this application.

[0105] S701, use a monocular infrared camera to collect the original three-channel infrared image frame sequence of the vehicle when the driver is driving in the driver's seat. Use the image transformation method in steps S102-S104 to perform image transformation processing on the original three-channel infrared image training set to obtain the target three-channel infrared image. Use the target three-channel infrared image as input to train the face detection model, the face key point localization model and the behavior detection model.

[0106] S702: Use the target three-channel infrared image transformed by S701 to generate training set 1. Use the annotation tool to annotate the face bounding box information in training set 1. Then input training set 1 and the face bounding box information in training set 1 into the preset face detection model for face detection training to obtain the face detection model.

[0107] S703 uses training set 1 as input and an annotation tool to generate facial landmark information from dataset 1. Using dataset 1 and its facial landmark information as input, it trains a pre-defined facial landmark localization model to obtain the final facial landmark localization model.

[0108] S704 uses Dataset 1 and its facial landmark information as input to generate a ROI dataset of dangerous driving behaviors using a script. The dangerous driving behavior ROI dataset is then fed into a pre-defined image classification deep learning model for training the dangerous driving behavior detection, resulting in a behavior detection model.

[0109] S705, during vehicle operation, the monocular camera captures a real-time sequence of raw three-channel infrared image frames from the driver's seat. This raw three-channel infrared image frame sequence is then converted into a target three-channel infrared image frame sequence using the image transformation methods in steps S102-S104. The target three-channel infrared image frame sequence is input into the face detection model generated in step S702 to obtain the driver's facial information. This driver's facial information is then input into the face key point localization model generated in step S703 to obtain the driver's face key point location information. Using the face key point location information, a Region of Interest (ROI) image of the dangerous driving behavior detection area is extracted from the target three-channel infrared image frame sequence. This ROI image is input into the behavior detection model generated in step S704 to determine whether safe driving behavior exists within the ROI image area. If the model detects dangerous driving behavior, a voice prompt is issued to the driver; otherwise, the detection continues with the next image frame sequence.

[0110] In the embodiments described in this specification, such as Figure 8 As shown, the above-mentioned dangerous driving behavior detection model is trained in the following way:

[0111] S801, acquire the sample driving behavior image and the first annotation detection result corresponding to the sample driving behavior image, the pixel values ​​of the multiple sample single-channel infrared images corresponding to the sample driving behavior image are different.

[0112] S802, input the sample driving behavior image into the first preset detection model to detect dangerous driving behavior and obtain the first sample detection result.

[0113] In a specific embodiment, the first preset detection model may include: a preset face detection model, a preset face key point localization model, and a preset behavior detection model. Specifically, the detailed steps of "inputting the sample driving behavior image into the first preset detection model to detect dangerous driving behavior and obtain the first sample detection result" are similar to the detailed steps of S105, and will not be repeated here.

[0114] S803, based on the first labeled detection results and the first sample detection results, train the first preset detection model to obtain the dangerous driving behavior detection model.

[0115] In a specific embodiment, training a first preset detection model based on the first labeled detection result and the first sample detection result to obtain a dangerous driving behavior detection model may include:

[0116] Based on the first labeled detection results and the first sample detection results, the detection result loss information is determined; based on the detection result loss information, the first preset detection model is trained to obtain the dangerous driving behavior detection model.

[0117] In an optional embodiment, the first annotation detection result may include dangerous behavior annotation labels of the sample driving behavior image, the first sample detection result may include dangerous behavior prediction labels of the sample driving behavior image, and correspondingly, the detection result loss information may include: label loss information.

[0118] In a specific embodiment, determining the detection result loss information based on the first annotation detection result and the first sample detection result may include determining the label loss information between the dangerous behavior annotation label and the dangerous behavior prediction label based on a preset loss function.

[0119] In one specific embodiment, the label loss information can characterize the difference between the hazardous behavior annotation label and the hazardous behavior prediction label. In one specific embodiment, the preset loss function may include, but is not limited to, the cross-entropy loss function, the logistic loss function, the exponential loss function, etc.

[0120] In an optional embodiment, training a first preset detection model based on the detection result loss information to obtain a dangerous driving behavior detection model may include: updating the network parameters of the first preset detection model based on the detection result loss information; repeating step S802 to the training iteration operation of updating the network parameters of the first preset detection model based on the detection result loss information based on the updated first preset detection model until the training convergence condition is met; and using the first preset detection model obtained when the training convergence condition is met as the dangerous driving behavior detection model.

[0121] In an optional embodiment, the aforementioned training convergence condition can be that the number of training iterations reaches a preset number of training iterations. Optionally, the target point detection convergence condition can also be that the loss information at the current position is less than a specified threshold. In the embodiments of this specification, the preset number of training iterations and the specified threshold can be preset in conjunction with the training speed and accuracy of the network in practical applications.

[0122] As can be seen from the above examples, training the model using the loss information from the detection results can improve the model's generalization learning ability and the accuracy of dangerous driving detection.

[0123] In practical applications, when the training set for model training is large, such as reaching millions, the trained model cannot learn the features of a small number of samples in a specific scenario. Therefore, it is necessary to simplify the model training set, remove redundant data, and train the model based on the simplified training set.

[0124] In one specific embodiment, the aforementioned sample driving behavior images can be any sample images from the initial sample image set. Specifically, the initial sample image set can be a set of sample images with obvious dangerous driving behavior characteristics, and the initial sample image set can include positive sample images and negative sample images. Optionally, the initial sample image set can include: 3000 positive sample images and 3000 negative sample images.

[0125] In a specific embodiment, such as Figure 9 As shown, when the aforementioned sample driving behavior images can be any sample images from the initial sample image set, the above-mentioned training of the first preset detection model based on the first annotation detection result and the first sample detection result to obtain the dangerous driving behavior detection model may include:

[0126] S901, Based on the first labeled detection result and the first sample detection result, train the first preset detection model to obtain the initial detection model.

[0127] S902, obtain the set of sample images to be screened, and determine the sample screening probability threshold corresponding to the set of sample images to be screened.

[0128] Specifically, the sample image set to be screened can be a large set of sample images that need to be streamlined.

[0129] Specifically, the sample screening probability threshold can be a threshold set for the predicted probability of a sample used for sample screening.

[0130] In one specific embodiment, the sample screening probability threshold may include a positive sample screening probability threshold and a negative sample screening probability threshold. Optionally, the positive sample screening probability threshold may be 0.1 and the negative sample screening probability threshold may be 0.9.

[0131] S903, input the image set of samples to be screened into the initial detection model to detect dangerous driving behavior, and obtain the second sample detection result corresponding to the image set of samples to be screened. The second sample detection result includes the sample prediction probability.

[0132] Specifically, each time, 10% of the positive and negative sample images can be selected from the set of sample images to be screened and input into the initial detection model for dangerous driving behavior detection.

[0133] S904, select target sample images from the set of sample images to be screened that have a corresponding sample prediction probability that meets the sample screening probability threshold; add the target sample images to the initial sample image set to obtain the updated initial sample image set.

[0134] Specifically, with a positive sample selection probability threshold of 0.1 and a negative sample selection probability threshold of 0.9, when selecting positive samples, positive sample images with a sample prediction probability lower than 0.1 are added to the initial training set for training; when selecting negative samples, negative sample images with a sample prediction probability greater than 0.9 are added to the initial training set for training.

[0135] Specifically, before adding to the initial training set, the positive and negative sample images to be added are browsed, and negative samples are mixed into positive samples and positive samples are mixed into negative samples.

[0136] S905, based on the updated initial sample image set, trains the initial detection model to detect dangerous driving behavior, and obtains the trained initial detection model.

[0137] S906, based on the trained initial detection model, jump to input the set of sample images to be screened into the initial detection model to detect dangerous driving behavior, and obtain the second sample detection result corresponding to the set of sample images to be screened, until the third number of the current target sample images meets the preset screening end condition.

[0138] Specifically, the preset screening end condition can be pre-set according to the sample screening needs in actual application. For example, when the number of samples in the sample image set to be screened is 1 million, the preset screening end condition can be that the number of the third target sample images is less than 1,000.

[0139] S907 uses the current initial detection model as the dangerous driving behavior detection model.

[0140] As can be seen from the above embodiments, by selecting samples, the initial detection model can learn more features with less data, which is more conducive to the model learning the image features of a small number of samples in special scenarios, thereby reducing the false alarm rate of the model and improving the model's generalization learning ability and the accuracy of dangerous driving detection.

[0141] In a specific embodiment, such as Figure 10 As shown, after the third number of the current target sample images meets the preset screening termination condition, the above method may further include:

[0142] S908, the initial detection model when the third quantity meets the preset screening termination condition is used as the target detection model.

[0143] S909, Update the sample selection probability threshold to obtain the updated sample selection probability threshold.

[0144] S910, based on the updated sample screening probability threshold and target detection model, jump to input the image set to be screened into the initial detection model to detect dangerous driving behavior, and obtain the second sample detection result corresponding to the image set to be screened, until the current sample screening probability threshold meets the threshold update condition.

[0145] S911 uses the current target detection model as a dangerous driving behavior detection model.

[0146] Specifically, the positive sample screening probability threshold can be updated sequentially to 0.1, 0.2, 0.3 and 0.4 to screen positive sample images, and the negative sample screening probability threshold can be updated sequentially to 0.9, 0.8, 0.7 and 0.6 to screen negative sample images.

[0147] In an optional embodiment, after obtaining the dangerous driving behavior detection model, the model is used to detect false positive samples from video frames extracted from in-vehicle videos. Undetected positive and negative samples are added to a simplified dataset for training, while detected positive and negative samples are added to a larger, unsimplified dataset for later iterations. When adding undetected positive and negative samples to the simplified training set, the added data is reviewed, and negative samples are mixed with positive samples, and positive samples are mixed with negative samples, to improve model accuracy and provide a final, stable dangerous driving behavior detection model.

[0148] As can be seen from the above examples, by updating the sample selection probability threshold, the model can learn more sample features, thereby improving the model's generalization learning ability.

[0149] As can be seen from the above embodiments, using the technical solution provided in this specification, during vehicle operation, an initial driving behavior image corresponding to the driver is acquired. This initial driving behavior image is an initial multi-channel infrared image containing multiple initial single-channel infrared images, each with the same pixel value. Then, image transformation processing is performed on any one of the multiple initial single-channel infrared images to obtain multiple target sub-images corresponding to the target image, each with different pixel values. Compared to the prior art of directly reducing the size of the infrared driving behavior image, the target multi-channel image obtained by merging image channels of multiple target sub-images... Infrared images have higher pixel entropy values ​​and contain more information. Therefore, inputting the multi-channel infrared image of the target into a dangerous driving behavior detection model can improve the accuracy of dangerous driving behavior detection. In addition, since the pixel values ​​of the multiple single-channel infrared images corresponding to the multi-channel infrared image of the target are different, redundant calculations in the dangerous driving behavior detection model can be avoided, thus avoiding waste of computing resources. Furthermore, through sample selection, the initial detection model can learn more features with less data, which is more conducive to the model learning the image features of a small number of samples in special scenarios, thereby reducing the false alarm rate of the model and further improving the model's generalization learning ability and the accuracy of dangerous driving detection.

[0150] This application provides a dangerous driving behavior detection device, such as... Figure 11 As shown, the above-mentioned device may include:

[0151] The image acquisition module 1110 is used to acquire the initial driving behavior image corresponding to the driver during the vehicle's operation. The initial driving behavior image is an initial multi-channel infrared image.

[0152] The target image extraction module 1120 is used to extract any one of the multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the pixel values ​​of the multiple initial single-channel infrared images are the same.

[0153] The image conversion processing module 1130 is used to perform image conversion processing based on the target image to obtain multiple target sub-images corresponding to the target image, and the pixel values ​​of the multiple target sub-images are different.

[0154] The image channel merging module 1140 is used to merge multiple target sub-images to obtain a target driving behavior image;

[0155] The dangerous driving behavior detection module 1150 is used to input the target driving behavior image into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection results.

[0156] In one specific embodiment, the image conversion processing module 1130 described above may include:

[0157] A single-channel image acquisition unit is used to acquire multiple single-channel images to be filled.

[0158] The first pixel sampling unit is used to sample pixels in the target direction of the target image to obtain the pixel values ​​corresponding to multiple sampled pixels in the target image.

[0159] The first pixel filling unit is used to fill the pixel values ​​corresponding to multiple sampled pixel points into multiple single-channel images to be filled, thereby obtaining multiple target sub-images.

[0160] Wherein, the first image size is the product of the second image size and the first quantity, the first image size is the image size of the target image, the second image size is the image size corresponding to any one of the multiple single-channel images to be filled, and the first quantity is the number of multiple single-channel images to be filled.

[0161] In one specific embodiment, the first pixel sampling unit described above may include:

[0162] The sampling region determination unit is used to determine a second number of sampling regions in the target image, where the second number is the square of the second image size;

[0163] The sampling region traversal unit is used to traverse the second number of sampling regions;

[0164] The second pixel sampling unit is used to sample pixels in the target direction of the currently traversed sampling area to obtain the pixel values ​​corresponding to the first number of sampled pixels in the currently traversed sampling area.

[0165] Accordingly, the aforementioned first pixel padding unit may include:

[0166] The correspondence determination unit is used to determine the correspondence between each sampled pixel in the first number of sampled pixels and each single-channel image to be filled in the first number of single-channel images to be filled;

[0167] The second pixel filling unit is used to fill the pixel value corresponding to each sampled pixel point into each single-channel image to be filled based on the correspondence relationship, so as to obtain a first number of filled single-channel images.

[0168] The target sub-image unit is used to take the current first number of filled single-channel images as multiple target sub-images after traversing the second number of sampled regions.

[0169] In the embodiments described in this specification, the above-mentioned dangerous driving behavior detection model may include: a face detection model, a face key point localization model, and a behavior detection model; the above-mentioned dangerous driving behavior detection module 1150 may include:

[0170] The face detection unit is used to input the target driving behavior image into the face detection model to perform face detection and obtain the driver's face information;

[0171] The key point localization unit is used to input the driver's facial information into the facial key point localization model to perform key point localization and obtain facial key point location information;

[0172] The target region image cropping unit is used to crop the target region image from the target driving behavior image based on the facial key point location information;

[0173] The dangerous driving behavior detection unit is used to input the target area image into the behavior detection model to detect dangerous driving behavior and obtain the behavior detection results.

[0174] In the embodiments described in this specification, the above-mentioned dangerous driving behavior detection model is trained using the following device:

[0175] The sample acquisition module is used to acquire sample driving behavior images and the first annotation detection results corresponding to the sample driving behavior images. The pixel values ​​of the multiple single-channel infrared images corresponding to the sample driving behavior images are different.

[0176] The first sample detection result module is used to input the sample driving behavior image into the first preset detection model to detect dangerous driving behavior and obtain the first sample detection result.

[0177] The model training module is used to train a first preset detection model based on the first labeled detection results and the first sample detection results to obtain a dangerous driving behavior detection model.

[0178] In one specific embodiment, the aforementioned sample driving behavior images can be any sample images from the initial sample image set, and the aforementioned model training module can include:

[0179] The initial detection model unit is used to train the first preset detection model based on the first labeled detection result and the first sample detection result to obtain the initial detection model;

[0180] The sample screening probability threshold determination unit is used to acquire the set of sample images to be screened and determine the sample screening probability threshold corresponding to the set of sample images to be screened.

[0181] The second sample detection result unit is used to input the image set to be screened into the initial detection model to detect dangerous driving behavior and obtain the second sample detection result corresponding to the image set to be screened. The second sample detection result includes the sample prediction probability.

[0182] The sample filtering unit is used to filter out target sample images whose predicted probability meets the sample filtering probability threshold from the sample image set to be filtered; and to add the target sample images to the initial sample image set to obtain the updated initial sample image set.

[0183] The model training unit is used to train the initial detection model for dangerous driving behavior detection based on the updated initial sample image set, so as to obtain the trained initial detection model.

[0184] The first repeated execution unit is used to jump to input the set of sample images to be screened into the initial detection model to detect dangerous driving behavior based on the trained initial detection model, and obtain the second sample detection result corresponding to the set of sample images to be screened, until the third number of the current target sample images meets the preset screening end condition.

[0185] The first dangerous driving behavior detection model unit is used to take the current initial detection model as the dangerous driving behavior detection model.

[0186] In one specific embodiment, the above-described apparatus may further include:

[0187] The target detection model unit is used to take the initial detection model when the third quantity meets the preset screening termination condition as the target detection model;

[0188] The sample selection probability threshold update unit is used to update the sample selection probability threshold and obtain the updated sample selection probability threshold.

[0189] The second repeated execution unit is used to jump to input the image set to be screened into the initial detection model to detect dangerous driving behavior based on the updated sample screening probability threshold and the target detection model, and obtain the second sample detection result corresponding to the image set to be screened until the current sample screening probability threshold meets the threshold update condition.

[0190] The second dangerous driving behavior detection model unit is used to take the current target detection model as the dangerous driving behavior detection model.

[0191] It should be noted that the apparatus in the device embodiment and the method embodiment are based on the same inventive concept.

[0192] This application provides a dangerous driving behavior detection device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded and executed by the processor to implement the dangerous driving behavior detection method provided in the above method embodiments.

[0193] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the aforementioned devices. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0194] The method embodiments provided in this application can be executed in an in-vehicle terminal or a similar computing device, that is, the computer device mentioned above may include an in-vehicle terminal or a similar computing device.

[0195] This application embodiment also provides a storage medium, which can be set in a server to store at least one instruction or at least one program related to implementing a dangerous driving behavior detection method in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the dangerous driving behavior detection method provided in the above method embodiment.

[0196] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0197] As can be seen from the embodiments of the dangerous driving behavior detection method, apparatus, device, or storage medium provided in this application, using the technical solution provided in the embodiments of this specification, during vehicle operation, an initial driving behavior image corresponding to the driver is acquired. This initial driving behavior image is an initial multi-channel infrared image containing multiple initial single-channel infrared images, each with the same pixel value. Then, image transformation processing is performed based on any one of the multiple initial single-channel infrared images to obtain multiple target sub-images corresponding to the target image, each with different pixel values. Compared with the prior art of directly reducing the size of the infrared driving behavior image to obtain a reduced infrared image, image transformation processing is performed on multiple target sub-images. The multi-channel infrared image of the target obtained by combining channels has a larger entropy value of pixel values ​​and contains more information. Therefore, inputting this multi-channel infrared image of the target into a dangerous driving behavior detection model can improve the accuracy of dangerous driving behavior detection. In addition, since the pixel values ​​of the multiple single-channel infrared images corresponding to the multi-channel infrared image of the target are different, redundant calculations in the dangerous driving behavior detection model can be avoided, thus avoiding waste of computing resources. Furthermore, through sample selection, the initial detection model can learn more features with less dataset, which is more conducive to the model learning the image features of a small number of samples in special scenarios, thereby reducing the false alarm rate of the model and further improving the model's generalization learning ability and the accuracy of dangerous driving detection.

[0198] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0199] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0200] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0201] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting dangerous driving behavior, characterized in that, The method includes: During vehicle operation, an initial driving behavior image corresponding to the driver is acquired, wherein the initial driving behavior image is an initial multi-channel infrared image; Extract any one initial single-channel infrared image from multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the pixel values ​​of the multiple initial single-channel infrared images are the same. Acquire multiple single-channel images to be filled; Pixel sampling is performed on the target image in the target direction to obtain pixel values ​​corresponding to multiple sampled pixel points in the target image; The pixel values ​​corresponding to the plurality of sampled pixels are filled into the plurality of single-channel images to be filled to obtain a plurality of target sub-images corresponding to the target image, and the pixel values ​​of the plurality of target sub-images are different; the first image size is the product of the second image size and the first quantity, the first image size is the image size of the target image, the second image size is the image size corresponding to any single-channel image to be filled in the plurality of single-channel images to be filled, and the first quantity is the number of the plurality of single-channel images to be filled; Image channels of the multiple target sub-images are merged to obtain the target driving behavior image; The target driving behavior image is input into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection results.

2. The method according to claim 1, characterized in that, The step of sampling pixels in the target direction of the target image to obtain pixel values ​​corresponding to multiple sampled pixels in the target image includes: A second number of regions to be sampled are determined in the target image, wherein the second number is the square of the size of the second image; Traverse the second number of regions to be sampled; Pixel sampling is performed on the currently traversed sampling area in the target direction to obtain the pixel values ​​corresponding to the first number of sampled pixels in the currently traversed sampling area; The step of filling the pixel values ​​corresponding to the plurality of sampled pixels into the plurality of single-channel images to be filled, to obtain the plurality of target sub-images corresponding to the target image, includes: Determine the correspondence between each sampled pixel in the first number of sampled pixels and each single-channel image to be filled in the first number of single-channel images to be filled; Based on the correspondence, the pixel values ​​corresponding to each sampled pixel point are filled into each single-channel image to be filled, to obtain the first number of filled single-channel images; After traversing the second number of sampled regions, the current first number of filled single-channel images are used as the plurality of target sub-images.

3. The method according to claim 1, characterized in that, The dangerous driving behavior detection model includes: a face detection model, a face key point localization model, and a behavior detection model. The step of inputting the target driving behavior image into the dangerous driving behavior detection model to perform dangerous driving behavior detection yields the following behavior detection results: The target driving behavior image is input into the face detection model for face detection to obtain the driver's face information; The driver's facial information is input into the facial key point localization model to perform key point localization and obtain facial key point location information; Based on the facial key point location information, a target region image is extracted from the target driving behavior image; The target region image is input into the behavior detection model to detect dangerous driving behavior, and the behavior detection result is obtained.

4. The method according to any one of claims 1 to 3, characterized in that, The dangerous driving behavior detection model is trained in the following way: Acquire sample driving behavior images and corresponding first annotation detection results for the sample driving behavior images, wherein the pixel values ​​of the multiple single-channel infrared images corresponding to the sample driving behavior images are different; The sample driving behavior image is input into a first preset detection model to detect dangerous driving behavior, and the first sample detection result is obtained. Based on the first labeled detection results and the first sample detection results, the first preset detection model is trained to obtain the dangerous driving behavior detection model.

5. The method according to claim 4, characterized in that, The sample driving behavior image is any sample image from the initial sample image set. The step of training the first preset detection model based on the first labeled detection result and the first sample detection result to obtain the dangerous driving behavior detection model includes: Based on the first labeled detection results and the first sample detection results, the first preset detection model is trained to obtain the initial detection model; Obtain the set of sample images to be screened, and determine the sample screening probability threshold corresponding to the set of sample images to be screened; The set of sample images to be screened is input into the initial detection model to detect dangerous driving behavior, and a second sample detection result corresponding to the set of sample images to be screened is obtained. The second sample detection result includes the sample prediction probability. Select target sample images from the set of sample images to be screened, whose corresponding predicted probability satisfies the sample screening probability threshold; add the target sample images to the initial sample image set to obtain the updated initial sample image set; Based on the updated initial sample image set, the initial detection model is trained to detect dangerous driving behavior, resulting in a trained initial detection model. Based on the trained initial detection model, the process jumps to inputting the set of sample images to be screened into the initial detection model to detect dangerous driving behavior, and obtains the second sample detection result corresponding to the set of sample images to be screened, until the third number of the current target sample images meets the preset screening end condition. The current initial detection model will be used as the dangerous driving behavior detection model.

6. The method according to claim 5, characterized in that, After the third number of the current target sample images meets the preset screening termination condition, the method further includes: The initial detection model when the third quantity satisfies the preset screening termination condition is used as the target detection model; Update the sample selection probability threshold to obtain the updated sample selection probability threshold; Based on the updated sample screening probability threshold and the target detection model, the process jumps to inputting the image set to be screened into the initial detection model to detect dangerous driving behavior, and obtains the second sample detection result corresponding to the image set to be screened, until the current sample screening probability threshold meets the threshold update condition. The current target detection model will be used as the dangerous driving behavior detection model.

7. A dangerous driving behavior warning device, characterized in that, The device includes: The image acquisition module is used to acquire an initial driving behavior image corresponding to the driver during vehicle operation, wherein the initial driving behavior image is an initial multi-channel infrared image; The target image extraction module is used to extract any one of the multiple initial single-channel infrared images corresponding to the initial driving behavior image as the target image, wherein the multiple initial single-channel infrared images have the same pixel value. An image conversion and processing module is used to acquire multiple single-channel images to be filled; to sample pixels in the target image in the target direction to obtain pixel values ​​corresponding to multiple sampled pixels in the target image; to fill the multiple single-channel images to be filled with the pixel values ​​corresponding to the multiple sampled pixels to obtain multiple target sub-images corresponding to the target image, wherein the pixel values ​​of each of the multiple target sub-images are different; the first image size is the product of the second image size and the first quantity, wherein the first image size is the image size of the target image, the second image size is the image size corresponding to any single-channel image to be filled among the multiple single-channel images to be filled, and the first quantity is the number of the multiple single-channel images to be filled; An image channel merging module is used to merge the multiple target sub-images to obtain a target driving behavior image; The dangerous driving behavior detection module is used to input the target driving behavior image into the dangerous driving behavior detection model to detect dangerous driving behavior and obtain the behavior detection result.

8. A dangerous driving behavior warning device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the dangerous driving behavior warning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the dangerous driving behavior warning method as described in any one of claims 1 to 6.

Images