Driver drowsiness detection data set construction method and related equipment

Abstract

The invention discloses a construction method of a driver drowsiness detection data set and related equipment, and relates to the technical field of image processing, and the method comprises the steps: responding to a data set construction instruction, extracting continuous multi-frame driver driving images from a driver drowsiness video library, carrying out the automatic pre-labeling of the driver driving images through a preset multi-mode large model, and obtaining a pre-labeled driver driving image; the method comprises the steps of obtaining a pre-annotation result, sending the pre-annotation result to an annotation inspection device for verifying and correcting the pre-annotation result to obtain a target annotation result meeting annotation requirements, and constructing a data set meeting a YOLO model training format based on the target annotation result and a driver driving image, the data set being used for training a YOLO model. The YOLO model is used for real-time driver drowsiness detection. According to the method, based on the synergistic effect between large model pre-annotation and manual verification, the workload of workers is reduced, the accuracy of the annotation result is also improved, and then the driver drowsiness detection data set is efficiently and accurately constructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a driver drowsiness detection dataset construction method and related equipment. BACKGROUND

[0002] With the rapid development of intelligent driving technology, driver monitoring system (DMS) has become a key research direction to improve traffic safety. Fatigue driving is one of the main causes of traffic accidents, and real-time detection of the drowsiness state of the driver and triggering of an alarm or intervention are of great significance for reducing accidents and ensuring public traffic safety. The combination of computer vision (CV) and deep learning technology provides a new solution for driver drowsiness detection. Compared with traditional physiological sensors (such as electrooculogram and electroencephalogram monitoring), the detection scheme based on video images has the advantages of non-invasiveness, strong real-time performance, and low deployment cost, so it has attracted widespread attention from academia and industry.

[0003] However, the related art has significant defects in the construction of a driver drowsiness detection dataset. For example, the traditional eye ear ratio (EAR) method relies on the stable detection of face key points, but in complex scenes (such as changes in lighting, drivers wearing glasses, and side face poses), key point extraction is prone to failure, resulting in a significant decrease in drowsiness recognition accuracy. In addition, although the existing public dataset (such as the RLDD dataset of the University of Texas at Arlington) has diversity, it does not provide high-precision frame-by-frame face bounding box labeling, and researchers need to perform secondary labeling to apply it to object detection or deep learning training. This process not only consumes time and effort, but also is prone to labeling bias, leading to inconsistencies in labeling standards among different research teams and affecting the comparability of experiments.

[0004] In summary, how to efficiently and accurately construct a driver drowsiness detection dataset is a technical problem that needs to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent an admission that the above content is related art. SUMMARY

[0006] The main purpose of the present application is to provide a driver drowsiness detection dataset construction method and related equipment, aiming to solve the technical problem of how to efficiently and accurately construct a driver drowsiness detection dataset.

[0007] To achieve the above purpose, the present application provides a driver drowsiness detection dataset construction method, which comprises: extracting a plurality of continuous driver driving images from the driver drowsiness video library in response to a data set construction instruction; automatically pre-labeling the driver driving images using a preset multi-modal large model to obtain a pre-labeling result; sending the pre-labeling result to a labeling checking device for checking and correcting the pre-labeling result to obtain a target labeling result meeting labeling requirements; constructing a data set meeting a YOLO model training format based on the target labeling result and the driver driving images, the data set being used for training a YOLO model, and the YOLO model being used for real-time driver drowsiness detection.

[0008] In an embodiment, after the step of constructing a data set meeting a YOLO model training format based on the target labeling result and the driver driving images, the method further comprises: training a YOLO model based on the data set to obtain a target YOLO model meeting user requirements; labeling real-time collected driver driving images based on the target YOLO model to obtain a labeling result; performing driver drowsiness detection based on the labeling result corresponding to the continuous N frames of real-time collected driver driving images and a preset label setting strategy to obtain a driver drowsiness detection result, the value of N being preset.

[0009] In an embodiment, the step of performing driver drowsiness detection based on the labeling result corresponding to the continuous N frames of real-time collected driver driving images and a preset label setting strategy to obtain a driver drowsiness detection result further comprises: determining a closed-eye frequency and an open-eye frequency corresponding to the continuous N frames of real-time collected driver driving images based on the labeling result corresponding to the continuous N frames of real-time collected driver driving images; performing driver drowsiness detection based on the closed-eye frequency, the open-eye frequency and a preset label setting strategy to obtain a driver drowsiness detection result, the driver drowsiness detection result including wakefulness and drowsiness.

[0010] In an embodiment, after the step of performing driver drowsiness detection based on the closed-eye frequency, the open-eye frequency and a preset label setting strategy to obtain a driver drowsiness detection result, if the driver drowsiness detection result is wakefulness, the method further comprises: determining a change trend corresponding to the open-eye frequency and the closed-eye frequency; predicting an open-eye frequency and a closed-eye frequency corresponding to a future frame of driver driving images based on the change trend and a pre-constructed prediction model; determine a driver drowsiness detection result corresponding to the future frame based on the open-eye frequency and the closed-eye frequency of the driver in the future frame; if the driver drowsiness detection result corresponding to the future frame is drowsiness, issue a drowsiness warning.

[0011] In an embodiment, before the step of using the preset multi-modal large model to automatically pre-annotate the driver driving image to obtain a pre-annotation result, the method further comprises: obtain sample image data, wherein the annotation result corresponding to the sample image data is a first annotation result; use the current multi-modal large model to process the sample image data to obtain a second annotation result; determine whether the first annotation result and the second annotation result are consistent; if not consistent, adjust the parameters of the current multi-modal large model, and return to the step of using the current multi-modal large model to process the sample image data to obtain a second annotation result based on the current multi-modal large model after adjusting the parameters, until the first annotation result and the second annotation result are consistent, and obtain a preset multi-modal large model.

[0012] In an embodiment, the step of extracting continuous multiple frames of driver driving images from the driver drowsiness video library in response to the data set construction instruction further comprises: extract a driver drowsiness video from the driver drowsiness video library in response to the data set construction instruction; perform an image frame extraction operation on the driver drowsiness video based on a preset image frame extraction rate to obtain continuous multiple frames of driver driving images.

[0013] In an embodiment, the step of using the preset multi-modal large model to automatically pre-annotate the driver driving image to obtain a pre-annotation result further comprises: determine a region of interest in the driver driving image based on prior knowledge to obtain an initial region of interest; perform a correction operation on the initial region of interest based on a preset region of interest correction strategy to obtain a target region of interest; use the preset multi-modal large model to automatically pre-annotate the image corresponding to the target region of interest to obtain a pre-annotation result.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a driver drowsiness detection data set construction device, which comprises: an extraction module, configured to extract continuous multiple frames of driver driving images from a driver drowsiness video library in response to a data set construction instruction; A pre-labeling module is configured to perform automatic pre-labeling on the driver driving image using a preset multi-modal large model to obtain a pre-labeling result. A verification and correction module is configured to send the pre-labeling result to a labeling checking device for verification and correction of the pre-labeling result to obtain a target labeling result meeting labeling requirements. A construction module is configured to construct a data set meeting a YOLO model training format based on the target labeling result and the driver driving image, where the data set is used to train a YOLO model for real-time driver drowsiness detection.

[0015] In an embodiment, the driver drowsiness detection data set construction apparatus further includes a detection module, which includes: A model training unit is configured to train a YOLO model based on the data set to obtain a target YOLO model meeting user requirements. A labeling unit is configured to label real-time collected driver driving images based on the target YOLO model to obtain a labeling result. A first detection unit is configured to perform driver drowsiness detection based on the labeling result corresponding to the continuous N frames of real-time collected driver driving images and a preset label setting strategy to obtain a driver drowsiness detection result, where the value of N is preset.

[0016] In an embodiment, the detection module further includes: A first determination unit is configured to determine the closed-eye frequency and the open-eye frequency corresponding to the continuous N frames of real-time collected driver driving images based on the labeling result corresponding to the continuous N frames of real-time collected driver driving images. A second detection unit is configured to perform driver drowsiness detection based on the closed-eye frequency, the open-eye frequency, and a preset label setting strategy to obtain a driver drowsiness detection result, where the driver drowsiness detection result includes wakefulness and drowsiness.

[0017] In an embodiment, if the driver drowsiness detection result is wakefulness, the detection module further includes: A second determination unit is configured to determine the change trend of the open-eye frequency and the closed-eye frequency. A prediction unit is configured to predict the open-eye frequency and the closed-eye frequency corresponding to a future frame of driver based on the change trend and a pre-constructed prediction model. A first judgment unit is configured to judge the driver drowsiness detection result corresponding to the future frame of driver based on the open-eye frequency and the closed-eye frequency corresponding to the future frame of driver. An early warning unit is configured to issue a drowsiness early warning if the driver is determined to be drowsy in the future frame.

[0018] In an embodiment, the detection module further includes: An acquisition unit is configured to acquire sample image data, wherein the sample image data corresponds to a first annotation result. A data processing unit is configured to process the sample image data using a current multi-modal large model to obtain a second annotation result. A second judgment unit is configured to determine whether the first annotation result is consistent with the second annotation result. An iterative training unit is configured to adjust the parameters of the current multi-modal large model if the first annotation result is not consistent with the second annotation result, and return to the step of processing the sample image data using the current multi-modal large model to obtain the second annotation result based on the current multi-modal large model with adjusted parameters until the first annotation result is consistent with the second annotation result, and obtain a preset multi-modal large model.

[0019] In an embodiment, the extraction module further includes: A first extraction unit is configured to extract a driver drowsiness video from a driver drowsiness video library in response to a data set construction instruction. A second extraction unit is configured to perform an image frame extraction operation on the driver drowsiness video based on a preset image frame extraction rate to obtain continuous multiple frames of driver driving images.

[0020] In an embodiment, the pre-annotation module further includes: A third determination unit is configured to determine a region of interest in the driver driving image based on prior knowledge to obtain an initial region of interest. A correction unit is configured to perform a correction operation on the initial region of interest based on a preset region of interest correction strategy to obtain a target region of interest. A pre-annotation unit is configured to automatically pre-annotate an image corresponding to the target region of interest using a preset multi-modal large model to obtain a pre-annotation result.

[0021] In addition, to achieve the above-mentioned purpose, the present application also provides a driver drowsiness detection data set construction device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the driver drowsiness detection data set construction method as described above.

[0022] In addition, in order to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program realizes the steps of the driver drowsiness detection dataset construction method when executed by a processor.

[0023] In addition, in order to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, and the computer program realizes the steps of the driver drowsiness detection dataset construction method when executed by a processor.

[0024] The one or more technical solutions provided by the present application have at least the following technical effects: The present application provides a driver drowsiness detection dataset construction method and related equipment, which relates to the image processing technical field. Compared with the related art, the existing public dataset (such as the RLDD dataset of the University of Texas at Arlington) has diversity, but does not provide frame-by-frame high-precision face bounding box annotation. Researchers need to perform secondary annotation to apply to target detection or deep learning training. This process not only consumes time and effort, but also is prone to annotation bias, leading to inconsistencies in annotation standards among different research teams, affecting the comparability of experiments. In the present application, first, in response to a dataset construction instruction, a plurality of continuous driver driving images are extracted from a driver drowsiness video library. Then, a preset multi-modal large model is used to automatically pre-annotate the driver driving images to obtain a pre-annotation result. Further, the pre-annotation result is sent to an annotation checking device for checking and correcting the pre-annotation result to obtain a target annotation result meeting the annotation requirements. Finally, based on the target annotation result and the driver driving images, a dataset meeting the YOLO model training format is constructed, and the dataset is used to train a YOLO model, and the YOLO model is used for real-time driver drowsiness detection.

[0025] It can be understood that, based on the synergy between the large model pre-annotation and the manual checking / automatic checking by the machine device, the workload of the staff is reduced, and the accuracy of the annotation result is improved, and thus the driver drowsiness detection dataset is efficiently and accurately constructed. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments or the related art will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0028] Figure 1 The flowchart provided by the first embodiment of the method for constructing a driver drowsiness detection data set of the present application; Figure 2 The flowchart provided by the second embodiment of the method for constructing a driver drowsiness detection data set of the present application; Figure 3 The flowchart provided by the third embodiment of the method for constructing a driver drowsiness detection data set of the present application; Figure 4 The module structure diagram of the device for constructing a driver drowsiness detection data set of the embodiment of the present application; Figure 5 The device structure diagram of the hardware running environment involved in the method for constructing a driver drowsiness detection data set in the embodiment of the present application.

[0029] The object implementation, functional features and advantages of the present application will be further described with reference to the drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0030] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0031] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the drawings in the specification and specific embodiments.

[0032] The main solution of the embodiment of the present application is: In the present embodiment, for the convenience of description, the following takes the driver drowsiness detection data set construction device as the execution subject for elaboration.

[0033] Due to the related art: although the existing public data set (such as the RLDD data set of the University of Texas at Arlington) has diversity, it does not provide frame-by-frame high-precision face bounding box annotation, researchers need to perform secondary annotation before applying it to object detection or deep learning training. This process not only consumes time and effort, but also is prone to annotation deviation, leading to inconsistency in annotation standards among different research teams, affecting the comparability of experiments.

[0034] The application provides a solution, which first extracts continuous multiple frames of driver driving images from a driver drowsiness video library in response to a data set construction instruction, then uses a preset multi-modal large model to automatically pre-label the driver driving images to obtain a pre-labeling result, further sends the pre-labeling result to a labeling checking device for checking and correcting the pre-labeling result to obtain a target labeling result meeting labeling requirements, and finally constructs a data set meeting a YOLO model training format based on the target labeling result and the driver driving images, where the data set is used to train a YOLO model, and the YOLO model is used for real-time driver drowsiness detection.

[0035] It can be understood that, based on the synergy between the large model pre-labeling and the manual checking and automatic checking by a machine device, the workload of the staff is reduced, and the accuracy of the labeling result is improved, and thus the driver drowsiness detection data set is efficiently and accurately constructed.

[0036] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a driver drowsiness detection data set construction device, etc. The following takes the driver drowsiness detection data set construction device as an example to describe the embodiment and the following embodiments.

[0037] Based on this, the embodiment of the application provides a driver drowsiness detection data set construction method, which refers to Figure 1 , Figure 1 The figure is a flowchart of the first embodiment of the driver drowsiness detection data set construction method of the application.

[0038] In the embodiment, the driver drowsiness detection data set construction method includes steps S10-S40: Step S10, in response to a data set construction instruction, extracting continuous multiple frames of driver driving images from a driver drowsiness video library; It should be noted that the data set construction instruction is an instruction triggering an operation, which is used to start the data extraction and processing process. The source of the data set construction instruction can be an automatic system (the system automatically triggers the instruction at a preset time point or when a condition is met) or a user operation (the user manually triggers the instruction through an interface or a command). In the embodiment, the process of extracting images from the video library is started through the data set construction instruction.

[0039] It should be noted that the driver drowsiness video library is a database storing driver driving videos, which can contain records of driver drowsiness states.

[0040] It should be noted that the continuous multiple frames of images refer to continuous multiple frames of images extracted from the video for analyzing and researching the behavior state of the driver.

[0041] In this embodiment, the specific extraction process includes selecting a video (selecting a specific video according to an instruction) and extracting frames (extracting continuous multiple frames of images from the video, which can be used for further analysis).

[0042] It can be understood that this process ensures that the construction of the data set can accurately obtain the required image data, providing a basis for subsequent analysis and model training.

[0043] Specifically, the step of extracting continuous multiple frames of driver driving images from the driver drowsiness video library in response to the data set construction instruction further includes steps S11-S12: Step S11, in response to the data set construction instruction, extracting a driver drowsiness video from the driver drowsiness video library; Extracting a driver drowsiness video refers to selecting corresponding video files from the video library according to parameters (such as a video ID list) in the data set construction instruction.

[0044] For example, the instruction requires extracting videos with video IDs 123, 456, and 789. The system finds and extracts these video files from the video library: video_123.mp4, video_456.mp4, and video_789.mp4.

[0045] Step S12, performing an image frame extraction operation on the driver drowsiness video based on a preset image frame extraction rate to obtain continuous multiple frames of driver driving images.

[0046] It should be noted that the preset image frame extraction rate is a preset parameter for specifying the frequency of extracting image frames from the video. In the extraction process, the frame rate and time range also need to be considered.

[0047] The frame rate refers to how many frames of images are extracted per second, for example, 10 frames / second means 10 images are extracted per second. The time range refers to which time period of the video the images are extracted from. For example, from 00:00:00 to 00:01:00. By controlling the number of images extracted from the video and the time range, it is ensured that the extracted images can reflect the behavior changes of the driver.

[0048] It can be understood that these two steps together ensure that the videos extracted from the video library can be further processed, providing basic data for subsequent data set construction and model training.

[0049] Step S20, using a preset multi-modal large model to automatically pre-label the driver driving images to obtain a pre-labeling result; It should be noted that the preset multi-modal large model is a pre-trained machine learning model that can process multiple modal data (such as images, text, audio, etc.). For example, in the scenario of driver driving images, the model may process image data and related text descriptions (such as driver state descriptions) simultaneously.

[0050] It should be noted that automated pre-labeling refers to using a machine learning model to automatically generate preliminary labeling information for images.

[0051] In this embodiment, the specific pre-labeling process is as follows: First, input the driver driving image into the preset multi-modal large model. Then, the model generates labeling information for each frame of image based on its pre-trained knowledge and feature extraction capability.

[0052] It should be noted that the labeling information may include the driver's state (such as drowsy, alert, distracted, etc.), action (such as yawning, closing eyes, etc.) or other related information.

[0053] It should be noted that the pre-labeling result is the preliminary labeling information generated by the model, which is used for further processing or manual review. The pre-labeling result includes: state labeling, action labeling, face region frame labeling, and confidence. Among them, the state labeling can be "state":"drowsy" (drowsy) or "state":"alert" (alert). Action labeling: for example, "action":"yawning" (yawning) or "action":"eye_closure" (eye closure).

[0054] It should be noted that the labeling result usually comes with a confidence score, indicating the model's confidence in the labeling.

[0055] For example, suppose there is an autonomous driving safety system that needs to pre-label driver driving images for subsequent training of a driver drowsiness detection model. The following is the specific process: First, select a pre-trained multi-modal large model, such as the Qwen2.5VL-32B model. Then, load the model into the system and prepare for pre-labeling.

[0056] Further, extract continuous multiple frames of images from the driver drowsiness video library, and input each frame of image into the preset multi-modal large model. Finally, the model generates labeling information for each frame of image, and stores the pre-labeling information into a database or file for subsequent use.

[0057] It can be understood that this process greatly reduces the workload of manual labeling through automated pre-labeling, while providing preliminary labeling data for subsequent data processing and model training.

[0058] Specifically, the step of automatically pre-annotating the driver's driving image using a preset multimodal large model to obtain the pre-annotation result further includes steps A1 to A3: Step A1: Based on prior knowledge, determine the region of interest in the driver's driving image to obtain the initial region of interest; In this embodiment, the system utilizes pre-accumulated rules or experience (such as "the driver's face is usually located in the upper half of the image" and "the steering wheel area is in a specific position") to quickly delineate preliminary candidate regions that may contain key targets in the input image, similar to "coarse localization," to quickly eliminate obviously irrelevant regions and reduce the amount of subsequent computation.

[0059] For example, based on the camera's installation location, the driver's face area is pre-defined to be in the upper 1 / 3 of the frame, and this area is directly cropped as the initial ROI.

[0060] Step A2: Based on the preset region of interest correction strategy, perform a correction operation on the initial region of interest to obtain the target region of interest; In this embodiment, the rough selection result from step A1 is finely adjusted. By using preset rules (such as scaling, edge detection, and shape constraints), the position, size, and shape of the ROI are optimized to ensure it closely matches the actual target boundary. "Fine trimming" addresses the issues of the initial ROI being too large, too small, or offset.

[0061] For example, if the initial face bounding box detects that it contains part of the seat background, the system automatically shrinks the boundary to precisely align with the facial contour.

[0062] Step A3: Use a preset multimodal large model to automatically pre-annotate the image corresponding to the target region of interest to obtain the pre-annotation result.

[0063] In this embodiment, the high-quality ROI image optimized in step A2 is fed into a multimodal AI model (such as a visual language model that integrates image and text understanding) to automatically identify and label specific behaviors or states within the region, thereby achieving semantic-level intelligent understanding and generating labeled data.

[0064] Understandably, this design balances computational efficiency with recognition accuracy, making it particularly suitable for autonomous driving testing or fleet management scenarios that require batch processing of driving monitoring videos.

[0065] Step S30: Send the pre-annotation results to the annotation inspection device for verification and correction to obtain target annotation results that meet the annotation requirements; In this embodiment, the pre-labeling results are sent to a labeling inspection device for manual or machine inspection, verification, and correction. Specifically, the device constructing the driver drowsiness detection dataset sends the pre-labeling results to the labeling inspection device via a system interface or file transfer. The labeling inspection device is typically equipped with specialized labeling tools to facilitate automatic viewing and modification of the labeling results by personnel or machines.

[0066] For example, the device for building a driver drowsiness detection dataset stores the pre-labeled results as a JSON file and sends it to the labeling platform.

[0067] On the interface provided by the annotation platform, staff can view each frame of image and its pre-annotation results.

[0068] Furthermore, staff checked the pre-annotation results to verify their accuracy and corrected any inaccurate parts, ultimately obtaining target annotation results that met the annotation requirements.

[0069] Understandably, this process ensures the accuracy and reliability of the annotation results, providing a guarantee for the construction of high-quality datasets.

[0070] Step S40: Based on the target annotation results and the driver's driving image, construct a dataset that conforms to the YOLO model training format. The dataset is used to train the YOLO model, and the YOLO model is used for real-time driver drowsiness detection.

[0071] In this embodiment, the target annotation results and the driver's driving image are combined and converted into the format required for YOLO model training.

[0072] After constructing the dataset, the YOLO model is trained using the constructed dataset to obtain the trained YOLO model, which is used to detect the driver's drowsiness in real time.

[0073] Understandably, this process ensures a complete workflow from data annotation to model training and real-time detection, providing reliable technical support for autonomous driving safety systems.

[0074] This application proposes a method and related equipment for constructing a driver drowsiness detection dataset, relating to the field of image processing technology. While existing public datasets (such as the RLDD dataset from the University of Texas at Arlington) possess diversity, they do not provide frame-by-frame high-precision face bounding box annotations. Researchers need to perform secondary annotations before applying them to object detection or deep learning training. This process is not only time-consuming and laborious but also prone to annotation bias, leading to inconsistencies in annotation standards among different research teams and affecting the comparability of experiments. In contrast, this application first extracts multiple consecutive frames of driver driving images from a driver drowsiness video library in response to dataset construction instructions. Then, it automatically pre-annotates the driver driving images using a pre-set multimodal large model to obtain pre-annotation results. Further, the pre-annotation results are sent to an annotation checking device for verification and correction, resulting in target annotation results that meet the annotation requirements. Finally, based on the target annotation results and the driver driving images, a dataset conforming to the YOLO model training format is constructed. This dataset is used to train the YOLO model, which is then used for real-time driver drowsiness detection.

[0075] Understandably, this application leverages the synergy between large model pre-annotation and manual / automatic verification by machines to reduce the workload of staff and improve the accuracy of annotation results, thereby efficiently and accurately constructing a driver drowsiness detection dataset.

[0076] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Following step S40, the method for constructing the driver drowsiness detection dataset further includes steps S50-S70: Step S50: Based on the dataset, train the YOLO model to obtain a target YOLO model that meets the user's needs; It should be noted that YOLO (You Only Look Once) is a popular real-time object detection model that can quickly detect target objects in images.

[0077] Specifically, the model training process is as follows: First, the dataset is divided into training, validation, and test sets. A YOLO model version (e.g., YOLOv5, YOLOv8) is selected, and hyperparameters (such as learning rate and batch size) are configured. Then, the model is trained using the training set, and the hyperparameters are tuned using the validation set to ensure model performance. Finally, the model's performance is evaluated on the test set to ensure its accuracy and generalization ability. The trained and optimized YOLO model should meet user needs, such as high accuracy and low false positive rate.

[0078] It should be noted that user requirements refer to specific performance requirements that users may have for the model, such as detection speed and accuracy. For example, user requirements could be: real-time detection with an accuracy rate higher than 90%.

[0079] Step S60: Based on the target YOLO model, annotate the real-time acquired driver driving image to obtain the annotation result; It should be noted that the real-time captured driver driving image refers to the driver's driving image obtained from a camera in real time. The camera can be a vehicle-mounted camera or a surveillance camera.

[0080] It should be noted that the annotation results refer to the annotation information generated by the target YOLO model after detecting real-time acquired images. The annotation results include the driver's state (e.g., drowsy, alert). The annotation format of the annotation results typically includes category ID, confidence score, bounding box coordinates, etc.

[0081] Step S70: Based on the annotation results corresponding to the driver's driving image collected in real time for N consecutive frames and the preset label setting strategy, driver drowsiness detection is performed to obtain the driver drowsiness detection result. The value of N is preset.

[0082] As can be understood, in this embodiment, N frames of images are continuously captured from the real-time video stream. The value of N refers to a pre-set number of frames used to assess the driver's state. For example, N=10 means 10 frames are captured continuously.

[0083] It should be noted that the preset label setting strategy refers to the strategy for determining the driver's drowsiness state based on the labeling results. In this embodiment, the preset label setting strategy may be: if more than a certain proportion of N frames are labeled as "drowsy", then the driver is determined to be drowsy; only labels with a confidence level higher than a certain threshold are considered. For example, if more than 6 out of 10 frames are labeled as "drowsy", then the driver is determined to be drowsy, and only labels with a confidence level higher than 0.85 are considered.

[0084] The driver drowsiness detection result refers to the final determination of the driver's drowsiness state based on a preset strategy. If drowsiness is detected, an alarm mechanism is triggered.

[0085] Understandably, in this application, the process, from dataset construction to model training and real-time detection, ensures the efficiency and reliability of the entire system, providing strong support for autonomous driving safety.

[0086] Specifically, the step of detecting driver drowsiness based on the annotation results corresponding to the driver's driving image acquired in real time for N consecutive frames and the preset label setting strategy, and obtaining the driver drowsiness detection result, further includes steps S71~S72: Step S71: Based on the annotation results corresponding to the driver driving images collected in real time for N consecutive frames, determine the closed-eye frequency and open-eye frequency corresponding to the driver driving images collected in real time for N consecutive frames. It should be noted that the eye-closing frequency and eye-opening frequency refer to the frequency at which the driver closes and opens their eyes in N consecutive frames of images. The calculation method is as follows: Eye-closing frequency = Number of frames with eyes closed / Total number of frames Eye-opening frequency = Number of eye-opening frames / Total number of frames For example, if the number of frames with eyes closed is 3 and the number of frames with eyes open is 7, then the frequency of closing eyes = 3 / 10 = 0.3 and the frequency of opening eyes = 7 / 10 = 0.7.

[0087] Step S72: Based on the eye-closing frequency, the eye-opening frequency, and the preset label setting strategy, perform driver drowsiness detection to obtain driver drowsiness detection results, which include wakefulness and drowsiness.

[0088] It should be noted that in this embodiment, a strategy for determining driver drowsiness is set based on the frequency of eye closing and the frequency of eye opening. The strategy refers to setting thresholds for the frequency of eye closing and the frequency of eye opening to determine whether the driver is drowsy. For example, if the frequency of eye closing is >0.5, the driver is determined to be drowsy; if the frequency of eye opening is >0.8, the driver is determined to be awake.

[0089] Understandably, these two steps, by calculating the frequency of eye closing and opening, and combining them with preset strategies, effectively detect the driver's drowsiness, providing important technical support for the safety of autonomous driving.

[0090] Specifically, if the driver drowsiness detection result is "awake", after the step of performing driver drowsiness detection based on the eye-closing frequency, the eye-opening frequency, and the preset label setting strategy to obtain the driver drowsiness detection result, the method further includes steps S80~S110: Step S80: Determine the changing trend of the eye-opening frequency and the eye-closing frequency; In this embodiment, the trend refers to the change in the frequency of eye opening and closing over time. Methods for determining the trend include sliding window and time series analysis. Sliding window refers to using a sliding window to calculate the frequency of eye opening and closing within each time period. Time series analysis refers to analyzing the trend of frequency changes using statistical methods (such as moving average, differencing, etc.).

[0091] Step S90: Based on the changing trend and the pre-built prediction model, predict the driver's eye-opening frequency and eye-closing frequency in future frames; A predictive model is a pre-built model used to predict the future frequency of eye opening and closing based on current and historical data. Predictive models can be linear regression models, time series models, and machine learning models.

[0092] Step S100: Based on the driver's eye-opening frequency and eye-closing frequency in the future frame, determine the driver drowsiness detection result in the future frame. In this embodiment, the driver's drowsy state in future frames is determined based on the predicted frequency of eye opening and closing.

[0093] Step S110: If the driver's drowsiness detection result is drowsy in the future frame, a drowsiness warning is issued.

[0094] Understandably, if the system predicts that the driver will be drowsy in a future frame, it will issue a warning signal. Warning methods include audible alerts, visual alerts, and vibration alerts. Audible alerts involve sounding an alarm to remind the driver; visual alerts involve displaying a warning message on the instrument panel or screen; and vibration alerts involve vibrating the seat or steering wheel to remind the driver.

[0095] Understandably, this process provides important technical support for the safety of autonomous driving by analyzing and predicting the frequency of the driver's eye opening and closing, thus giving early warning of the driver's drowsiness.

[0096] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Before step 20, the method for constructing the driver drowsiness detection dataset further includes steps S21 to S24: Step S21: Obtain sample image data, wherein the annotation result corresponding to the sample image data is the first annotation result; It should be noted that the sample image data is used for training and validating the model. The sample image data is typically collected from real-world application scenarios and is representative and diverse.

[0097] It should be noted that the first annotation result refers to the annotation result corresponding to the sample image data, which is usually the accurate result of manual annotation. The annotation information may include the object category, location (such as bounding box coordinates), etc.

[0098] Step S22: Process the sample image data using the current multimodal large model to obtain the second annotation result; It should be noted that the current multimodal large model refers to the multimodal large model that is currently being trained and optimized. The current multimodal large model can process data of multiple modalities (such as images, text, etc.) and generate labeled results.

[0099] The specific process of processing the sample image data using the current multimodal large model is as follows: First, input sample image data, and then the annotation results generated by the model are called the second annotation results.

[0100] Step S23: Determine whether the first annotation result is consistent with the second annotation result; In this embodiment, the consistency between the first annotation result (manual annotation) and the second annotation result (model generation) is compared. The consistency judgment methods include: category matching() to check whether the category IDs are consistent, location matching (calculating the intersection-union ratio of the bounding boxes, i.e., IoU, to determine whether the locations are consistent), and confidence threshold (checking whether the confidence of the model generation is higher than a certain threshold).

[0101] Step S24: If there is a discrepancy, adjust the parameters of the current multimodal large model. Based on the current multimodal large model with adjusted parameters, return to the step of using the current multimodal large model to process the sample image data and obtain the second annotation result, until the first annotation result is consistent with the second annotation result, and obtain the preset multimodal large model.

[0102] In this embodiment, the model parameters are adjusted based on the consistency judgment result to improve model performance. The parameter adjustment method can be backpropagation or an optimizer. Backpropagation refers to calculating the loss function (such as cross-entropy loss, IoU loss, etc.) and updating the model parameters through backpropagation.

[0103] An optimizer is a tool that uses optimization algorithms (such as SGD, Adam, etc.) to adjust parameters.

[0104] Furthermore, the sample image data is reprocessed using the model with adjusted parameters to generate new second annotation results. The sample image data is then re-inputted into the model to generate new second annotation results, and consistency is checked again. Parameter adjustment stops when the first and second annotation results are consistent.

[0105] Understandably, this process continuously adjusts model parameters to ensure that the model's output matches the actual annotations, thereby improving the model's accuracy and reliability.

[0106] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method of constructing the driver drowsiness detection dataset of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0107] It should be noted that all user-related data involved in this application (such as user attribute data, user behavior data, and user geographical location, etc., the data types here should be modified according to the adaptability of the solution content) were obtained with the user's permission or consent; that is to say, when this application is applied to specific products or technologies, user permission is required to obtain and process the relevant data, and the processing of the relevant data must comply with the relevant laws, regulations and regulatory standards of the relevant countries and regions.

[0108] For example, when it is necessary to obtain a user's current geographical location, a location acquisition prompt can be displayed on the user's terminal. After receiving confirmation from the user regarding the location acquisition prompt, the terminal can obtain the user's current geographical location.

[0109] This application also provides an apparatus for constructing a driver drowsiness detection dataset, please refer to... Figure 4 The apparatus for constructing the driver drowsiness detection dataset includes: Extraction module 10, which is used to extract multiple consecutive frames of driver driving images from a driver drowsiness video library in response to a dataset construction instruction; Pre-annotation module 20, the pre-annotation module is used to automatically pre-annotate the driver's driving image using a preset multimodal large model to obtain pre-annotation results; Verification and correction module 30, the verification and correction module is used to send the pre-annotation result to the annotation inspection device for verification and correction of the pre-annotation result, so as to obtain the target annotation result that meets the annotation requirements; The construction module 40 is used to construct a dataset that conforms to the YOLO model training format based on the target annotation results and the driver's driving image. The dataset is used to train the YOLO model, and the YOLO model is used to perform real-time driver drowsiness detection.

[0110] In one embodiment, the apparatus for constructing the driver drowsiness detection dataset further includes a detection module, the detection module comprising: The model training unit is used to train the YOLO model based on the dataset to obtain a target YOLO model that meets the user's needs. The annotation unit is used to annotate the real-time acquired driver driving image based on the target YOLO model to obtain the annotation result; The first detection unit is used to detect driver drowsiness based on the annotation results corresponding to the driver's driving image collected in real time for N consecutive frames and the preset label setting strategy, and to obtain the driver drowsiness detection result. The value of N is preset.

[0111] In one embodiment, the detection module further includes: The first determining unit is used to determine the closed-eye frequency and open-eye frequency corresponding to the driver's driving image collected in real time for N consecutive frames based on the annotation results corresponding to the driver's driving image collected in real time for N consecutive frames. The second detection unit is used to perform driver drowsiness detection based on the eye-closing frequency, the eye-opening frequency and the preset label setting strategy, and to obtain driver drowsiness detection results, which include wakefulness and drowsiness.

[0112] In one embodiment, if the driver drowsiness detection result is that the driver is awake, the detection module further includes: The second determining unit is used to determine the changing trend of the eye-opening frequency and the eye-closing frequency; The prediction unit is used to predict the driver's eye-opening frequency and eye-closing frequency in future frames based on the changing trend and a pre-built prediction model. The first judgment unit is used to judge the driver drowsiness detection result of the driver in the future frame based on the driver's eye-opening frequency and the eye-closing frequency corresponding to the future frame. The warning unit is used to issue a drowsiness warning if the driver's drowsiness detection result is drowsy in the future frame.

[0113] In one embodiment, the detection module further includes: The acquisition unit is used to acquire sample image data, wherein the annotation result corresponding to the sample image data is the first annotation result; The data processing unit is used to process the sample image data using the current multimodal large model to obtain the second annotation result; The second judgment unit is used to determine whether the first annotation result and the second annotation result are consistent; The iterative training unit is used to adjust the parameters of the current multimodal large model if there is a discrepancy. Based on the current multimodal large model with adjusted parameters, it returns to the step of processing the sample image data using the current multimodal large model to obtain the second annotation result, until the first annotation result is consistent with the second annotation result, thus obtaining the preset multimodal large model.

[0114] In one embodiment, the extraction module further includes: The first extraction unit is used to extract driver drowsiness videos from the driver drowsiness video library in response to the dataset construction instruction; The second extraction unit is used to perform image frame extraction operations on the driver drowsiness video based on a preset image frame extraction rate to obtain a series of driver driving images.

[0115] In one embodiment, the pre-annotation module further includes: The third determining unit is used to determine the region of interest in the driver's driving image based on prior knowledge, and obtain the initial region of interest; The correction unit is used to perform a correction operation on the initial region of interest based on a preset region of interest correction strategy to obtain the target region of interest; The pre-annotation unit is used to automatically pre-annotate the image corresponding to the target region of interest using a preset multimodal large model, and obtain the pre-annotation result.

[0116] The driver drowsiness detection dataset construction apparatus provided in this application adopts the driver drowsiness detection dataset construction method in the above embodiments, and can solve the technical problem of dataset construction. Compared with related technologies, the beneficial effects of the driver drowsiness detection dataset construction apparatus provided in this application are the same as the beneficial effects of the driver drowsiness detection dataset construction method provided in the above embodiments, and other technical features in the driver drowsiness detection dataset construction apparatus are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0117] This application provides a device for constructing a driver drowsiness detection dataset. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the driver drowsiness detection dataset construction method in Embodiment 1 above.

[0118] The following is for reference. Figure 5The diagram illustrates a structural schematic of a device suitable for constructing a driver drowsiness detection dataset to implement embodiments of this application. The device for constructing the driver drowsiness detection dataset in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The device for constructing the driver drowsiness detection dataset shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0119] like Figure 5 As shown, the device for constructing a driver drowsiness detection dataset may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the device for constructing the driver drowsiness detection dataset. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the device for constructing the driver drowsiness detection dataset to communicate wirelessly or wiredly with other devices to exchange data. Although a device for constructing a driver drowsiness detection dataset with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0120] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0121] The driver drowsiness detection dataset construction device provided in this application, employing the driver drowsiness detection dataset construction method in the above embodiments, can solve the technical problem. Compared with related technologies, the beneficial effects of the driver drowsiness detection dataset construction device provided in this application are the same as those of the driver drowsiness detection dataset construction method provided in the above embodiments, and other technical features in the driver drowsiness detection dataset construction device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0122] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0124] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the method for constructing the driver drowsiness detection dataset in the above embodiments.

[0125] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0126] The aforementioned computer-readable storage medium may be included in the device for constructing the driver drowsiness detection dataset; or it may exist independently and not be assembled into the device for constructing the driver drowsiness detection dataset.

[0127] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the device for constructing the driver drowsiness detection dataset, cause the device for constructing the driver drowsiness detection dataset to: In response to a dataset construction instruction, extract consecutive multi-frame driver driving images from a driver drowsiness video library; The driver's driving images are automatically pre-annotated using a preset multimodal large model to obtain pre-annotation results; The pre-annotation results are sent to the annotation inspection equipment for verification and correction to obtain target annotation results that meet the annotation requirements; Based on the target annotation results and the driver's driving image, a dataset conforming to the YOLO model training format is constructed. The dataset is used to train the YOLO model, which is used for real-time driver drowsiness detection.

[0128] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0131] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for constructing the driver drowsiness detection dataset, thereby solving the technical problem of dataset construction. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the driver drowsiness detection dataset construction method provided in the above embodiments, and will not be repeated here.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for constructing a driver drowsiness detection dataset as described above.

[0133] The computer program product provided in this application can solve the technical problem of dataset construction. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the driver drowsiness detection dataset construction method provided in the above embodiments, and will not be repeated here.

[0134] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for constructing a driver drowsiness detection dataset, characterized in that, The method for constructing the driver drowsiness detection dataset includes: In response to a dataset construction instruction, extract consecutive multi-frame driver driving images from a driver drowsiness video library; The driver's driving images are automatically pre-annotated using a preset multimodal large model to obtain pre-annotation results; The pre-annotation results are sent to the annotation inspection equipment for verification and correction to obtain target annotation results that meet the annotation requirements; Based on the target annotation results and the driver's driving image, a dataset conforming to the YOLO model training format is constructed. The dataset is used to train the YOLO model, which is used for real-time driver drowsiness detection.

2. The method for constructing a driver drowsiness detection dataset as described in claim 1, characterized in that, After the step of constructing a dataset conforming to the YOLO model training format based on the target annotation results and the driver's driving image, the method further includes: Based on the dataset, the YOLO model is trained to obtain the target YOLO model that meets the user's needs; Based on the target YOLO model, the real-time collected driver driving images are labeled to obtain the labeling results; Based on the annotation results corresponding to the driver's driving image acquired in real time for N consecutive frames and the preset label setting strategy, driver drowsiness is detected to obtain the driver drowsiness detection result, where the value of N is preset.

3. The method for constructing a driver drowsiness detection dataset as described in claim 2, characterized in that, The step of detecting driver drowsiness based on the annotation results corresponding to the driver's driving image acquired in real time based on N consecutive frames and the preset label setting strategy, and obtaining the driver drowsiness detection result, further includes: Based on the annotation results corresponding to the driver driving images acquired in real time for N consecutive frames, the closed-eye frequency and open-eye frequency corresponding to the driver driving images acquired in real time for N consecutive frames are determined. Based on the eye-closing frequency, the eye-opening frequency, and the preset label setting strategy, driver drowsiness is detected to obtain driver drowsiness detection results, which include wakefulness and drowsiness.

4. The method for constructing a driver drowsiness detection dataset as described in claim 2, characterized in that, If the driver drowsiness detection result is "awake," after the step of performing driver drowsiness detection based on the eye-closing frequency, the eye-opening frequency, and the preset label setting strategy to obtain the driver drowsiness detection result, the method further includes: Determine the changing trend of the eye-opening frequency and the eye-closing frequency; Based on the changing trend and the pre-built prediction model, the driver's eye-opening frequency and eye-closing frequency in future frames are predicted. Based on the driver's eye-opening frequency and eye-closing frequency in the future frame, determine the driver drowsiness detection result in the future frame; If the driver's drowsiness detection result is drowsy in the future frame, a drowsiness warning will be issued.

5. The method for constructing a driver drowsiness detection dataset as described in claim 1, characterized in that, Before the step of automatically pre-annotating the driver's driving image using a preset multimodal large model to obtain the pre-annotation result, the method further includes: Acquire sample image data, and the annotation result corresponding to the sample image data is the first annotation result; The sample image data is processed using the current multimodal large model to obtain the second annotation result; Determine whether the first annotation result is consistent with the second annotation result; If they are inconsistent, adjust the parameters of the current multimodal large model. Based on the current multimodal large model with adjusted parameters, return to the step of using the current multimodal large model to process the sample image data and obtain the second annotation result, until the first annotation result is consistent with the second annotation result, and obtain the preset multimodal large model.

6. The method for constructing a driver drowsiness detection dataset as described in claim 1, characterized in that, The step of extracting consecutive multi-frame driver driving images from the driver drowsiness video library in response to the dataset construction instruction further includes: In response to the dataset construction instruction, extract driver drowsiness videos from the driver drowsiness video library; Based on a preset image frame extraction rate, an image frame extraction operation is performed on the driver drowsy video to obtain a series of consecutive driver driving images.

7. The method for constructing a driver drowsiness detection dataset as described in claim 1, characterized in that, The step of automatically pre-annotating the driver's driving image using a preset multimodal large model to obtain the pre-annotation result further includes: Based on prior knowledge, the region of interest in the driver's driving image is determined to obtain the initial region of interest; Based on a preset region of interest correction strategy, a correction operation is performed on the initial region of interest to obtain the target region of interest; The image corresponding to the target region of interest is automatically pre-annotated using a preset multimodal large model to obtain the pre-annotation result.

8. A device for constructing a driver drowsiness detection dataset, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for constructing a driver drowsiness detection dataset as claimed in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for constructing the driver drowsiness detection dataset as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method for constructing a driver drowsiness detection dataset as described in any one of claims 1 to 7.

Images