Driver posture detection method and device and computer equipment
By constructing the object detection model and using feature extraction, fusion and prediction networks, the problem of low driver attitude detection efficiency and accuracy is solved, and efficient and accurate attitude detection is achieved.
Patent Information
- Application Number
- CN202510651621.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2025-08-15
AI Technical Summary
The existing driver's attitude detection methods are low in efficiency and accuracy, especially in areas with smooth surfaces or lack of texture features, which leads to confusion in detection.
Build an object detection model, including feature extraction network, feature fusion network and pose prediction network, feature extraction and fusion of input images through training a complete model, and efficient feature fusion is used to use the Ghost network to reduce the computational amount and generate rich feature maps.
It improves the efficiency and accuracy of driver attitude detection, achieves a balance between calculation accuracy and processing efficiency, and enhances the model's adaptability to complex scenarios.
Smart Images
Figure CN120496038A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automobile safety technology, and in particular to a driver posture detection method, device, and computer equipment. Background Art
[0002] Whether a driver's driving behavior is standardized directly affects driving safety. For example, fatigue driving or using a mobile phone often distracts the driver, affecting the safety of the vehicle and passengers.
[0003] To address this issue, existing driver posture detection methods typically extract keypoint features of the target object from captured driver images. These keypoint features are then compared and analyzed with a pre-set 3D model to establish a corresponding relationship between 2D image features and 3D model features, further inferring the driver's posture. However, this method not only requires labor to define keypoints, but also struggles to distinguish between keypoints in areas with smooth surfaces or lack of texture features. This can lead to confusion in keypoint estimation, resulting in low efficiency and accuracy in driver posture detection.
[0004] There is currently no effective solution to the problem of low efficiency and accuracy in driver posture detection in related technologies. Summary of the Invention
[0005] In this embodiment, a driver posture detection method, apparatus, and computer device are provided to solve the problem of low efficiency and accuracy of driver posture detection in related technologies.
[0006] In a first aspect, a driver posture detection method is provided in this embodiment, including:
[0007] Constructing a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network;
[0008] By training the complete target detection model, the posture of the target object in the input image is detected to obtain the actual posture of the target object; wherein the feature extraction network includes a first network and a second network, which are used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature map to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
[0009] In some embodiments, the first network and the second network each include three convolution blocks; and extracting features from the input image using the first network and the second network to obtain a multi-branch feature map includes:
[0010] Performing feature extraction on the input image through each of the convolution blocks in the first network to obtain a first feature map output by each of the convolution blocks in the first network;
[0011] Performing feature extraction on the input image through each of the convolution blocks in the second network to obtain a second feature map output by each of the convolution blocks in the second network;
[0012] Output the multi-branch feature map; the multi-branch feature map includes each of the first feature maps extracted by the first network and each of the second feature maps extracted by the second network.
[0013] In some embodiments, the convolution layer in each convolution block adopts a Ghost network; processing the input image through the convolution layer includes:
[0014] Performing channel dimensionality reduction on an initial feature map obtained by convolution processing the input image;
[0015] Performing depth-wise separable convolution on the initial feature map after dimensionality reduction to obtain corresponding multiple intermediate feature maps;
[0016] The initial feature map and each of the intermediate feature maps are superimposed.
[0017] In some embodiments, fusing the multi-branch feature maps to obtain a target fused feature map includes:
[0018] generating corresponding fusion information based on each of the first feature maps and the corresponding second feature map;
[0019] Based on each of the fusion information, the target fusion feature map is determined.
[0020] In some embodiments, generating corresponding fusion information based on each first feature map and the corresponding second feature map includes:
[0021] Performing feature fusion on the preprocessed first feature map and the corresponding second feature map to obtain a first fused feature map;
[0022] Performing preliminary pose prediction on each of the first feature maps and the corresponding second feature map;
[0023] Based on the preliminary posture prediction result, performing feature fusion on the first feature map and the corresponding second feature map to obtain a second fused feature map;
[0024] Based on the first fusion feature map and the second fusion feature map, the corresponding fusion information is generated.
[0025] In some embodiments, predicting the actual posture of the target object based on the target fusion feature map includes:
[0026] Performing prediction analysis on the target fusion feature map to obtain a plurality of prediction parameters; the prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle intervals, and the dynamic index range;
[0027] Based on the prediction parameters, the actual posture of the target object is determined.
[0028] In some embodiments, after building the target detection model, the method further includes:
[0029] Preprocessing the first sample images of different postures to obtain second sample images corresponding to each of the first sample images; the preprocessing includes random transformation;
[0030] Randomly dividing the labeled second sample images into a training set, a validation set, and a test set according to a preset ratio;
[0031] The object detection model is trained based on the training set, the validation set, and the test set.
[0032] In some embodiments, preprocessing the first sample images of different postures to obtain a second sample image corresponding to each of the first sample images includes:
[0033] Generate a corresponding affine matrix based on a preset rotation center and a preset rotation angle;
[0034] Based on the affine matrix, an affine transformation is performed on each of the first sample images in different postures to obtain the corresponding second sample image.
[0035] In a second aspect, a driver posture detection device is provided in this embodiment, comprising:
[0036] A construction module for constructing a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network;
[0037] The detection module is used to perform posture detection on the target object in the input image by training the complete target detection model to obtain the actual posture of the target object; wherein the feature extraction network includes a first network and a second network, and is used to extract features from the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
[0038] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the driver posture detection method described in the first aspect when executing the computer program.
[0039] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the driver posture detection method described in the first aspect is implemented.
[0040] Compared with the related art, the driver posture detection method, device and computer equipment provided in this embodiment construct a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network; by training a complete target detection model, the posture of the target object in the input image is detected to obtain the actual posture of the target object; wherein, the feature extraction network includes a first network and a second network, which are used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map, which solves the problem of low efficiency and accuracy of driver posture detection and achieves improved efficiency and accuracy of driver posture detection.
[0041] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0043] Figure 1 This is a hardware structure block diagram of a terminal device for a driver posture detection method provided by an embodiment of the present application;
[0044] Figure 2 is a flow chart of a driver posture detection method provided by an embodiment of the present application;
[0045] Figure 3 is a schematic diagram of a feature extraction and feature fusion method provided in one embodiment of the present application;
[0046] Figure 4 is a schematic diagram of a feature fusion method provided in one embodiment of the present application;
[0047] Figure 5 A flowchart of a driver posture detection method provided by a preferred embodiment of the present application;
[0048] Figure 6 This is a structural block diagram of a driver posture detection device provided in one embodiment of the present application.
[0049] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, construction module; 20, detection module. DETAILED DESCRIPTION
[0050] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0051] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0052] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a block diagram of the hardware structure of the terminal of the driver posture detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 The processor 102 (only one is shown) and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input / output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.
[0053] Memory 104 can be used to store computer programs, such as software programs and modules for application software, such as the computer program corresponding to the driver posture detection method in this embodiment. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the aforementioned method. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and such remote memory may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0054] Transmission device 106 is used to receive or transmit data via a network. This network may include a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0055] In this embodiment, a driver posture detection method is provided. Figure 2 is a flow chart of the driver posture detection method of this embodiment, as shown in FIG. Figure 2 As shown, the process includes the following steps:
[0056] Step S210, constructing a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network;
[0057] Step S220, by training a complete target detection model, the target object in the input image is subjected to posture detection to obtain the actual posture of the target object; wherein the feature extraction network includes a first network and a second network, which are used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
[0058] Specifically, the driver's posture is monitored using in-vehicle monitoring equipment. For example, the monitoring equipment's camera is placed at the upper left edge of the vehicle's windshield, with the camera's field of view covering the driver's head. This allows the capture of first sample images of the driver in different postures under different scenarios. A corresponding training dataset is constructed based on each first sample image, and each captured first sample image is preprocessed. Preprocessing operations include image resizing and random transformation. For example, each first sample image is resized to 1152×768 and then randomly rotated and translated to enhance the robustness of the trained model to changes in target position.
[0059] Furthermore, an SSRNet target detection model is constructed, which includes a feature extraction network, a feature fusion network, and a posture prediction network. In the target detection model, the feature extraction network consists of two heterogeneous data streams, specifically a heterogeneous first network and a second network. The first network and the second network use different activation functions and pooling layers, such as ReLU and Tanh activation functions, or maximum pooling layers and average pooling layers, to extract different feature information. The first network and the second network respectively extract features from the input image and output a multi-branch feature map. The multi-branch feature map includes the first feature map extracted by the first network and the second feature map extracted by the second network. Subsequently, the feature maps extracted by the first network and the second network are fused through the feature fusion network, and the fused feature maps are predicted and analyzed by the posture prediction network to obtain multiple prediction parameters. The actual posture of the target object in the input image is determined based on each prediction parameter.
[0060] Among them, the prediction parameters obtained by the posture prediction network analysis include the probability P of the target object in different posture angle intervals, the dynamic range of the posture angle interval and dynamic index ranges Finally, the actual posture of the target object is calculated based on the probability of the target object being in different postures, the dynamic range of the posture parameters, and the dynamic index range. It should be noted that posture prediction is based on the three degrees of freedom angle for posture estimation, specifically including the yaw angle, pitch angle, and roll angle. The rotation angle range of each degree of freedom [0, 90°] is divided into multiple intervals, such as 0 to 30°, 30° to 60°, and 60° to 90°, to determine the probability of the target object being in different posture angle intervals. In actual situations, the probability of the target object appearing in different angle ranges may be different. The dynamic range is used to determine the length of the angle interval. For angle ranges with smaller predicted probabilities, a larger interval length is set. At the same time, the angle ranges with larger predicted probabilities are finely divided, which helps save computing resources. In addition, the dynamic index range is used to shift the angle interval. For example, if the probability of predicting that the target object is in the angle range of 10° to 20° is too small, and the probability of predicting that the target object is in the angle range of 20° to 30° is too large, then the interval of 10° to 20° is shifted to the range of 20° to 30°, and the two intervals are merged or partially overlapped, so that the division of the angle interval is more consistent with the actual distribution of the target object posture.
[0061] Existing driver posture detection methods typically extract keypoint features of the target object from captured driver images. These keypoint features are then compared and analyzed with a pre-set 3D model to establish a correspondence between 2D image features and 3D model features, further inferring the driver's posture. However, these methods not only require labor to define keypoints, but also have difficulty distinguishing keypoints in areas with smooth surfaces or lack of texture features. This can lead to confusion in keypoint estimation, resulting in low efficiency and accuracy in driver posture detection.
[0062] Compared with the existing technology, the present application constructs a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network; by training a complete target detection model, the posture of the target object in the input image is detected to obtain the actual posture of the target object; wherein, the feature extraction network includes a first network and a second network, which are used to extract features from the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map. Based on this, by adopting a dual-backbone network for feature extraction to achieve efficient fusion of features at different levels, the calculation accuracy of the model is improved while the amount of calculation is effectively controlled, so that the model achieves a better balance between calculation accuracy and processing efficiency, solves the problem of low efficiency and accuracy in driver posture detection, and achieves improved efficiency and accuracy in driver posture detection.
[0063] In some embodiments, the first network and the second network each include three convolution blocks; and extracting features from the input image using the first network and the second network to obtain a multi-branch feature map includes the following steps:
[0064] Performing feature extraction on the input image through each convolution block in the first network to obtain a first feature map output by each convolution block in the first network;
[0065] Performing feature extraction on the input image through each convolution block in the second network to obtain a second feature map output by each convolution block in the second network;
[0066] Output a multi-branch feature map; the multi-branch feature map includes each first feature map extracted by the first network and each second feature map extracted by the second network.
[0067] Specifically, such as Figure 3 As shown in the figure, the first network and the second network are composed of three layers, each including three convolution blocks. Each convolution block includes a convolution layer, a batch normalization (BN) layer, an activation function and a pooling layer, which are used for information extraction, information filtering and information compression. The three-layer design realizes coarse-to-fine information extraction. Shallow features usually contain more edge structure information in the input image, while deep structures contain high-level semantic information.
[0068] Furthermore, feature extraction is performed on the input image through each convolution block in the first network to obtain a first feature map output by each convolution block in the first network. At the same time, feature extraction is performed on the input image through each convolution block in the second network to obtain a second feature map output by each convolution block in the second network. Finally, a multi-branch feature map is output, which includes the first feature maps extracted by the first network and the second feature maps extracted by the second network.
[0069] Through this embodiment, feature extraction is performed on the input image through each convolution block in the first network to obtain a first feature map output by each convolution block in the first network, and feature extraction is performed on the input image through each convolution block in the second network to obtain a second feature map output by each convolution block in the second network, and a multi-branch feature map is output. The multi-branch feature map includes the first feature maps extracted by the first network and the second feature maps extracted by the second network. In this way, a dual-trunk multi-stage network is adopted to reduce model training time and inference time, which helps to improve detection efficiency.
[0070] In some embodiments, the convolutional layer in each convolutional block adopts a Ghost network; processing the input image through the convolutional layer includes the following steps:
[0071] Perform channel dimensionality reduction on the initial feature map obtained by convolution processing of the input image;
[0072] Perform depth-wise separable convolution on the initial feature map after dimensionality reduction to obtain multiple corresponding intermediate feature maps;
[0073] The initial feature map and each intermediate feature map are superimposed.
[0074] In this embodiment, the convolution layer in each convolution block adopts the Ghost network, which is a lightweight model structure for convolutional neural networks. It can generate a small number of original feature maps and use linear transformation to generate a large number of feature map pairs, thereby increasing the number of feature maps without increasing the complexity of the model, which helps to reduce the computational complexity of the convolution operation.
[0075] Specifically, the output channels are reduced in dimensionality. A small number of initial feature maps are obtained through preliminary convolution of the input image. Depthwise separable convolution is then performed on these initial feature maps. This involves extracting features using a 3x3 convolution kernel at each layer of the feature map, and then fusing the channels using a 1x1 convolution kernel to obtain multiple intermediate feature maps. The initial feature map and the intermediate feature maps are then superimposed to obtain a feature map with richer features.
[0076] Through this embodiment, the initial feature map obtained by convolution processing of the input image is subjected to channel dimensionality reduction, and the initial feature map after dimensionality reduction is subjected to depthwise separable convolution to obtain corresponding multiple intermediate feature maps. The initial feature map and the intermediate feature maps are superimposed. In this way, on the basis of adopting a lightweight network architecture, the Ghost network is further adopted to replace the traditional convolution module, thereby reducing the amount of model calculation and improving the model inference speed while ensuring that rich feature maps can be generated.
[0077] In some embodiments, fusing multiple branch feature maps to obtain a target fused feature map includes the following steps:
[0078] Based on each first feature map and the corresponding second feature map, generating corresponding fusion information;
[0079] Based on each fusion information, a target fusion feature map is determined.
[0080] Specifically, the multi-branch feature map includes the first feature map output by each convolutional block in the first network and the second feature map output by each convolutional block in the second network. The corresponding second feature map for each first feature map is determined, and corresponding fusion information is generated based on the first feature map and the corresponding second feature map. The fusion results of each pair of feature maps are then finally fused to obtain the target fused feature map. Fusion methods include splicing and weighted fusion.
[0081] Exemplarily, the first network includes convolution block 1, convolution block 2, and convolution block 3 in sequence, and the second network includes convolution block 4, convolution block 5, and convolution block 6 in sequence. The activation function and pooling layer of the convolution blocks in the first network are different from those in the second network. The first feature map output by convolution block 1 and the second feature map output by convolution block 4 are fused to obtain the corresponding fusion result a, the first feature map output by convolution block 2 and the second feature map output by convolution block 5 are fused to obtain the corresponding fusion result b, and the first feature map output by convolution block 3 and the second feature map output by convolution block 6 are fused to obtain the corresponding fusion result c. Based on the fusion results a, b, and c, the target fusion feature map is fused.
[0082] Through this embodiment, corresponding fusion information is generated based on each first feature map and the corresponding second feature map, and the target fusion feature map is determined based on each fusion information, so that the fused feature map has information extracted by the two feature extraction networks, which helps to improve the accuracy of subsequent posture detection.
[0083] In some embodiments, generating corresponding fusion information based on each first feature map and the corresponding second feature map includes the following steps:
[0084] Perform feature fusion on the preprocessed first feature map and the corresponding second feature map to obtain a first fused feature map;
[0085] Perform preliminary pose prediction on each first feature map and the corresponding second feature map;
[0086] Based on the preliminary posture prediction result, the first feature map and the corresponding second feature map are subjected to feature fusion to obtain a second fused feature map;
[0087] Based on the first fused feature map and the second fused feature map, corresponding fusion information is generated.
[0088] Specifically, such as Figure 3 and Figure 4 As shown in the figure, each first feature map and the corresponding second feature map are fused through a 1*1 convolution channel to reduce the complexity of the feature map, and the activation function is used for nonlinear activation and pooling processing, using maximum pooling and average pooling respectively. Then, the first feature map and the second feature map after dimensionality reduction are fused by multiplying the corresponding elements one by one to obtain the first fused feature map.
[0089] Furthermore, the first and second feature maps, after nonlinear activation and pooling, are processed through dropout, a fully connected layer (FC), and a ReLU activation function. Preliminary pose prediction (PB) is performed on each of the processed first and second feature maps, obtaining preliminary pose prediction results for each feature map. The preliminary pose prediction results include preliminary estimated probabilities of the target object in different poses, the dynamic range of pose parameters, and the dynamic index range, thereby obtaining preliminary estimated information about the target object's pose. Based on the preliminary pose prediction results, the first feature map and the corresponding second feature map are fused to obtain a second fused feature map. Corresponding fusion information is generated based on the first and second fused feature maps.
[0090] Through this embodiment, the preprocessed first feature map and the corresponding second feature map are feature fused to obtain a first fused feature map, and each first feature map and the corresponding second feature map are subjected to preliminary posture prediction. Based on the preliminary posture prediction result, the first feature map and the corresponding second feature map are feature fused to obtain a second fused feature map. Based on the first fused feature map and the second fused feature map, corresponding fusion information is generated to obtain the fusion result of the pairwise feature maps, so that the fused feature map has the information extracted by the two feature extraction networks.
[0091] In some embodiments, predicting the actual posture of the target object based on the target fusion feature map includes the following steps:
[0092] Perform prediction analysis on the target fusion feature map to obtain multiple prediction parameters; the prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle interval, and the dynamic index range;
[0093] Based on the predicted parameters, the actual pose of the target object is determined.
[0094] Specifically, the target fusion feature map is predicted and analyzed through the posture prediction network, and the fusion information related to the first fusion feature map in the target fusion feature map is processed by the full connection layer (FC) and the activation function Tanh to output the dynamic range , the fusion information related to the second fusion feature map in the target fusion feature map is processed by the fully connected layer (FC) and the activation function Tanh, and the dynamic index range is output , and the fusion information related to the second fusion feature map in the target fusion feature map is processed by the fully connected layer (FC) and the activation function ReLU, outputting the probability P of the target object in different posture angle intervals. Afterwards, the actual posture of the target object is calculated based on the probability of the target object in different posture angle intervals, the dynamic range of the posture angle interval, and the dynamic index range.
[0095] It should be noted that attitude prediction is based on attitude estimation based on the angles of three degrees of freedom, including yaw, pitch and roll. The rotation angle of each degree of freedom is divided into multiple intervals to determine the probability of the target object being in different attitude angle intervals; the dynamic range is used to determine the length of the angle interval; the dynamic index range is used to translate the angle interval so that the division of the angle interval is more consistent with the actual distribution of the target object's attitude.
[0096] Through this embodiment, the target fusion feature map is predictively analyzed to obtain multiple prediction parameters, which include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle intervals, and the dynamic index range. Based on each prediction parameter, the actual posture of the target object is determined to achieve accurate posture prediction.
[0097] In some embodiments, after building the target detection model, the following steps are further included:
[0098] Preprocessing the first sample images of different postures to obtain second sample images corresponding to each first sample image; the preprocessing includes random transformation;
[0099] The labeled second sample images are randomly divided into a training set, a validation set, and a test set according to a preset ratio;
[0100] The object detection model is trained based on the training set, validation set, and test set.
[0101] Specifically, first sample images of drivers in different scenes and postures are collected, and a corresponding training dataset is constructed based on each first sample image. Each first sample image in the training dataset is preprocessed to obtain a corresponding second sample image. Preprocessing operations include image resizing and random transformation. For example, each first sample image is resized to 1152×768 and then randomly rotated and translated to enhance the robustness of the trained model to changes in target position.
[0102] Furthermore, each second sample image is labeled, and each labeled second sample image is randomly divided into a training set, a validation set, and a test set according to a preset ratio. The preset ratio is usually 8:1:1, and then the target detection model is trained based on the training set, the validation set, and the test set.
[0103] Through this embodiment, the first sample images of different postures are preprocessed to obtain second sample images corresponding to each first sample image. The preprocessing includes random transformation, and the labeled second sample images are randomly divided into a training set, a validation set, and a test set according to a preset ratio. The target detection model is trained based on the training set, the validation set, and the test set. By constructing a data set of different driver postures, the adaptability of the trained model in complex situations is enhanced. Random transformation operations are used to enhance the model's adaptability to spatial transformations in complex working conditions, thereby improving the model's generalization performance and accuracy.
[0104] In some embodiments, preprocessing the first sample images of different postures to obtain a second sample image corresponding to each first sample image includes the following steps:
[0105] Generate a corresponding affine matrix based on a preset rotation center and a preset rotation angle;
[0106] Based on the affine matrix, an affine transformation is performed on each first sample image of different postures to obtain a corresponding second sample image.
[0107] Specifically, based on the preset rotation center and the preset rotation angle, a corresponding affine matrix is generated. For example, the function for obtaining a two-dimensional rotation matrix in the OpenCV library is used to obtain the affine matrix. Based on the affine matrix, an affine transformation is performed on the first sample image of each different posture to obtain the corresponding second sample image. The specific formula of the affine transformation is as follows:
[0108] (1)
[0109] In formula (1), m11, m12, m21 and m22 are linear transformation parameters used to describe the rotation relationship between images; m13 and m23 are translation parameters used to describe the translation relationship between images; x and y are the pixel coordinates of the first sample image; x' and y' are the pixel coordinates of the second sample image after transformation.
[0110] Through this embodiment, a corresponding affine matrix is generated based on a preset rotation center and a preset rotation angle, and based on the affine matrix, an affine transformation is performed on the first sample image of each different posture to obtain a corresponding second sample image, so as to train a target detection model based on the enhanced image, which helps to enhance the robustness of the model to changes in target position.
[0111] The present embodiment is described and illustrated below through preferred embodiments.
[0112] Figure 5 Flowchart of the driver posture detection method of the preferred embodiment is as follows: Figure 5As shown, the driver posture detection method includes the following steps:
[0113] Step S510: constructing a target detection model, which includes a feature extraction network, a feature fusion network, and a pose prediction network; preprocessing the first sample images of different poses to obtain second sample images corresponding to each first sample image; the preprocessing includes random transformation; randomly dividing the labeled second sample images into a training set, a validation set, and a test set according to a preset ratio; and training the target detection model based on the training set, the validation set, and the test set.
[0114] Step S520: Performing posture detection on the target object in the input image by training a complete target detection model. The feature extraction network is used to extract features from the input image through the first network and the second network to obtain a multi-branch feature map; the multi-branch feature map includes a first feature map output by each convolution block in the first network and a second feature map output by each convolution block in the second network.
[0115] In step S530, the feature fusion network is used to perform feature fusion on the preprocessed first feature map and the corresponding second feature map to obtain a first fused feature map; perform preliminary pose prediction on each first feature map and the corresponding second feature map; based on the preliminary pose prediction result, perform feature fusion on the first feature map and the corresponding second feature map to obtain a second fused feature map; generate corresponding fusion information based on the first fused feature map and the second fused feature map; and determine a target fused feature map based on each fusion information;
[0116] In step S540, the posture prediction network is used to perform prediction analysis on the target fusion feature map to obtain multiple prediction parameters; the prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle intervals, and the dynamic index range; based on each prediction parameter, the actual posture of the target object is determined.
[0117] Through this embodiment, a target detection model is constructed, which includes a feature extraction network, a feature fusion network, and a pose prediction network. First sample images of different poses are preprocessed to obtain second sample images corresponding to each first sample image. The preprocessing includes random transformation. Each labeled second sample image is randomly divided into a training set, a validation set, and a test set according to a preset ratio. The target detection model is then trained based on the training set, the validation set, and the test set.
[0118] Furthermore, by training a complete target detection model, the posture of the target object in the input image is detected. In the target detection model, the input image is feature extracted through the first network and the second network to obtain a multi-branch feature map, which includes the first feature map output by each convolution block in the first network and the second feature map output by each convolution block in the second network. The preprocessed first feature map and the corresponding second feature map are feature fused to obtain a first fused feature map. A preliminary posture prediction is performed on each first feature map and the corresponding second feature map. Based on the preliminary posture prediction result, the first feature map and the corresponding second feature map are feature fused to obtain a second fused feature map. Based on the first fused feature map and the second fused feature map, corresponding fusion information is generated. Based on each fusion information, a target fused feature map is determined. The target fused feature map is predictively analyzed to obtain multiple prediction parameters. The prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle interval, and the dynamic index range. Based on each prediction parameter, the actual posture of the target object is determined, thereby solving the problem of low efficiency and accuracy in driver posture detection and improving the efficiency and accuracy of driver posture detection.
[0119] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0120] This embodiment also provides a driver posture detection device for implementing the aforementioned embodiments and preferred implementations. Details already described will not be repeated. Terms such as "module," "unit," and "subunit" used below may refer to a combination of software and / or hardware that implements a predetermined function. While the devices described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.
[0121] Figure 6 is a structural block diagram of the driver posture detection device of this embodiment, as shown in FIG. Figure 6 As shown, the device includes:
[0122] Construction module 10 is used to construct a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network;
[0123] The detection module 20 is used to perform posture detection on the target object in the input image by training a complete target detection model to obtain the actual posture of the target object; wherein, the feature extraction network includes a first network and a second network, which is used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
[0124] Through the device provided in this embodiment, a target detection model is constructed; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network; by training a complete target detection model, the posture of the target object in the input image is detected to obtain the actual posture of the target object; wherein, the feature extraction network includes a first network and a second network, which are used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map, which solves the problem of low efficiency and accuracy in driver posture detection and achieves improved efficiency and accuracy in driver posture detection.
[0125] In some embodiments, the detection module 20 is further used to perform feature extraction on the input image through each convolution block in the first network to obtain a first feature map output by each convolution block in the first network; perform feature extraction on the input image through each convolution block in the second network to obtain a second feature map output by each convolution block in the second network; and output a multi-branch feature map; the multi-branch feature map includes each first feature map extracted by the first network and each second feature map extracted by the second network.
[0126] In some of the embodiments, the detection module 20 is further used to perform channel dimensionality reduction on the initial feature map obtained by convolution processing of the input image; perform depth-wise separable convolution on the initial feature map after dimensionality reduction to obtain corresponding multiple intermediate feature maps; and superimpose the initial feature map and each intermediate feature map.
[0127] In some embodiments, the detection module 20 is further configured to generate corresponding fusion information based on each first feature map and the corresponding second feature map; and determine a target fusion feature map based on each fusion information.
[0128] In some embodiments, the detection module 20 is further used to perform feature fusion on the preprocessed first feature map and the corresponding second feature map to obtain a first fused feature map; perform preliminary posture prediction on each first feature map and the corresponding second feature map; based on the preliminary posture prediction result, perform feature fusion on the first feature map and the corresponding second feature map to obtain a second fused feature map; and generate corresponding fusion information based on the first fused feature map and the second fused feature map.
[0129] In some embodiments, the detection module 20 is also used to perform predictive analysis on the target fusion feature map to obtain multiple prediction parameters; the prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle intervals, and the dynamic index range; based on each prediction parameter, the actual posture of the target object is determined.
[0130] In some of these embodiments, Figure 6 On the basis of this, the device also includes a preprocessing module for preprocessing the first sample images of each different posture to obtain a second sample image corresponding to each first sample image; the preprocessing includes random transformation; the labeled second sample images are randomly divided into a training set, a validation set and a test set according to a preset ratio; the target detection model is trained based on the training set, the validation set and the test set.
[0131] In some embodiments, the preprocessing module is further used to generate a corresponding affine matrix based on a preset rotation center and a preset rotation angle; based on the affine matrix, perform an affine transformation on each first sample image of different posture to obtain a corresponding second sample image.
[0132] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0133] This embodiment further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0134] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0135] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:
[0136] S1, build a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network;
[0137] S2, by training a complete target detection model, the target object in the input image is detected in posture to obtain the actual posture of the target object; wherein, the feature extraction network includes a first network and a second network, which is used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
[0138] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.
[0139] In addition, in conjunction with the driver posture detection method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program that, when executed by a processor, implements any of the driver posture detection methods in the above embodiments.
[0140] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0141] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.
[0142] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.
[0143] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A driver posture detection method, characterized in that: include: Build an object detection model; The target detection model includes a feature extraction network, a feature fusion network and a posture prediction network; By training the complete target detection model, the posture of the target object in the input image is detected to obtain the actual posture of the target object; wherein the feature extraction network includes a first network and a second network, which are used to extract features of the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature map to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
2. The driver posture detection method according to claim 1, characterized in that: The first network and the second network each include three convolution blocks; the feature extraction of the input image by the first network and the second network to obtain a multi-branch feature map includes: Performing feature extraction on the input image through each of the convolution blocks in the first network to obtain a first feature map output by each of the convolution blocks in the first network; Performing feature extraction on the input image through each of the convolution blocks in the second network to obtain a second feature map output by each of the convolution blocks in the second network; Output the multi-branch feature map; the multi-branch feature map includes each of the first feature maps extracted by the first network and each of the second feature maps extracted by the second network.
3. The driver posture detection method according to claim 2, characterized in that: The convolutional layer in each convolutional block adopts the Ghost network; Processing the input image through the convolutional layer includes: Performing channel dimensionality reduction on an initial feature map obtained by convolution processing the input image; Performing depth-wise separable convolution on the initial feature map after dimensionality reduction to obtain corresponding multiple intermediate feature maps; The initial feature map and each of the intermediate feature maps are superimposed.
4. The driver posture detection method according to claim 2, characterized in that: The fusing the multi-branch feature maps to obtain a target fused feature map includes: generating corresponding fusion information based on each of the first feature maps and the corresponding second feature map; Based on each of the fusion information, the target fusion feature map is determined.
5. The driver posture detection method according to claim 4, characterized in that: The generating corresponding fusion information based on each of the first feature maps and the corresponding second feature map includes: Performing feature fusion on the preprocessed first feature map and the corresponding second feature map to obtain a first fused feature map; Performing preliminary pose prediction on each of the first feature maps and the corresponding second feature map; Based on the preliminary posture prediction result, performing feature fusion on the first feature map and the corresponding second feature map to obtain a second fused feature map; Based on the first fusion feature map and the second fusion feature map, the corresponding fusion information is generated.
6. The driver posture detection method according to claim 1, characterized in that: The predicting the actual posture of the target object based on the target fusion feature map includes: Performing prediction analysis on the target fusion feature map to obtain a plurality of prediction parameters; the prediction parameters include the probability of the target object being in different posture angle intervals, the dynamic range of the posture angle intervals, and the dynamic index range; Based on the prediction parameters, the actual posture of the target object is determined.
7. The driver posture detection method according to claim 1, characterized in that: After building the target detection model, the following steps are also included: Preprocessing the first sample images of different postures to obtain second sample images corresponding to each of the first sample images; the preprocessing includes random transformation; Randomly dividing the labeled second sample images into a training set, a validation set, and a test set according to a preset ratio; The object detection model is trained based on the training set, the validation set, and the test set.
8. The driver posture detection method according to claim 7, characterized in that: The preprocessing of the first sample images of different postures to obtain a second sample image corresponding to each of the first sample images includes: Generate a corresponding affine matrix based on a preset rotation center and a preset rotation angle; Based on the affine matrix, an affine transformation is performed on each of the first sample images in different postures to obtain the corresponding second sample image.
9. A driver posture detection device, characterized in that: include: A construction module for constructing a target detection model; the target detection model includes a feature extraction network, a feature fusion network and a posture prediction network; The detection module is used to perform posture detection on the target object in the input image by training the complete target detection model to obtain the actual posture of the target object; wherein the feature extraction network includes a first network and a second network, and is used to extract features from the input image through the first network and the second network to obtain a multi-branch feature map; the feature fusion network is used to fuse the multi-branch feature maps to obtain a target fusion feature map; the posture prediction network is used to predict the actual posture of the target object based on the target fusion feature map.
10. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of the driver posture detection method according to any one of claims 1 to 8.