Face position detection method and device, electronic equipment and computer program product

Abstract

The invention belongs to the technical field of vehicle-mounted equipment, and mainly provides a face position detection method and device, electronic equipment and a computer program product.The face position detection method comprises the steps that firstly, an initial region of interest in an image collected by a camera is calculated based on driver features and environment features; and optimizing and dynamically adjusting the initial region of interest through a target face position detection model obtained through pre-training to obtain a target region of interest, namely, a face position region, so that the purpose of accurately detecting the face position in the image collected by the camera is achieved. Compared with the prior art, the initial region-of-interest and the target region-of-interest are generated based on external variables, namely driver characteristics and environment characteristic dynamic states, and are not fixed regions-of-interest, so that deviation caused by the fixed regions-of-interest can be avoided, the accuracy of face position detection is improved, and the occurrence of false detection conditions is reduced.

Description

Technical Field

[0001] The present application belongs to the technical field of vehicle-mounted equipment, and in particular, relates to a face position detection method, device, electronic equipment and computer program product. Background Art

[0002] In the automotive field, face recognition technology has been widely used in scenarios such as driver identity verification and fatigue driving detection. Usually, the camera is fixed near the driver's seat and the driver's identity and driving conditions are confirmed by detecting and identifying the face position of the entire screen.

[0003] However, in traditional face recognition solutions, false detections often occur when detecting the face position, and the accuracy needs to be improved. Summary of the invention

[0004] The present application provides a face position detection method, device, electronic device and computer program product, which can solve the technical problems of low accuracy and easy false detection of face position detection.

[0005] A first aspect of an embodiment of the present application provides a method for detecting a face position, the method comprising:

[0006] An acquisition unit, used to acquire driver characteristics and environmental characteristics; the driver characteristics include the driver's height and seat position, and the environmental characteristics include the installation position of the camera;

[0007] A calculation unit, configured to calculate an initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environmental characteristics;

[0008] The detection unit is used to input the initial region of interest, the driver's characteristics and the environmental characteristics into a pre-trained target face position detection model, and obtain a target region of interest of the driver's face in the image captured by the camera output by the target face position detection model.

[0009] A second aspect of the embodiment of the present application further provides a face position detection device, the face position detection device comprising:

[0010] An acquisition unit, used to acquire driver characteristics and environmental characteristics; the driver characteristics include the driver's height and seat position, and the environmental characteristics include the installation position of the camera;

[0011] A calculation unit, configured to calculate an initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environmental characteristics;

[0012] The detection unit is used to input the initial region of interest, the driver's characteristics and the environmental characteristics into a pre-trained target face position detection model, and obtain a target region of interest of the driver's face in the image captured by the camera output by the target face position detection model.

[0013] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the face position detection method described in the first aspect are implemented.

[0014] A fourth aspect of an embodiment of the present application provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the steps of the face position detection method described in the first aspect are implemented.

[0015] In an embodiment of the present application, the initial region of interest of the driver's face in the image captured by the camera is obtained by acquiring the driver's characteristics and the environmental characteristics, and a stable starting point is provided for the face detection model. Then, the initial region of interest, the driver's characteristics and the environmental characteristics are input into a pre-trained target face position detection model, and the target face position detection model outputs the target region of interest of the driver's face in the image captured by the camera, that is, the face position area, thereby achieving the purpose of accurately detecting the face position in the image captured by the camera. Compared with the prior art, the initial region of interest and the target region of interest in the present application are generated based on the external variable of the driver's characteristics and the dynamic environmental characteristics, and are not fixed regions of interest. Therefore, the deviation caused by the fixed region of interest can be avoided, and the accuracy of face position detection is improved. In addition, the present application obtains the initial region of interest by preliminarily calculating first, and then optimizes and dynamically adjusts the initial region of interest based on the driver's characteristics and the environmental characteristics through the target face position detection model, thereby improving the adaptability and accuracy of face position detection in a dynamic environment, ensuring that the target region of interest is the best recognition area for the face, improving the accuracy of face position detection, reducing the occurrence of false detection, and, without the need to search for faces in the entire screen, reducing the consumption of computing resources and time costs, and improving the efficiency of face position detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the first implementation flow of the face position detection method provided in an embodiment of the present application.

[0017] Figure 2 This is a schematic diagram of the implementation flow of step 101 of the face position detection method provided in an embodiment of the present application.

[0018] Figure 3 This is a schematic diagram of the implementation flow of step 102 of the face position detection method provided in an embodiment of the present application.

[0019] Figure 4 This is another implementation flowchart of steps 101 and 102 of the face position detection method provided in an embodiment of the present application.

[0020] Figure 5 A schematic diagram of the structure of a face position detection device provided in an embodiment of the present application.

[0021] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] It should be understood that in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0024] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0026] In the automotive field, face recognition technology has been widely used in scenarios such as driver identity verification and fatigue driving detection. Usually, the camera is fixed near the driver's seat, and face recognition is performed after detecting the face position of the entire screen to confirm the driver's identity and driving conditions. However, due to factors such as the driver's height, seat adjustment position, sitting habits, etc., the positions of different drivers' faces in the pictures collected by the camera may vary significantly. If a fixed region of interest (ROI) is used to detect the face position, the detection may fail or the accuracy may decrease due to deviation. In addition, the face search process of the entire picture will also consume more computing resources and time, especially when there are multiple faces in the picture (such as in a taxi or bus, where there are other passengers in the car in addition to the driver), the risk of false detection also increases.

[0027] Based on this, the embodiments of the present application provide a face position detection method, device, electronic device and computer program product, which can solve the technical problems of low accuracy and easy false detection of face position detection in existing solutions.

[0028] In an embodiment of the present application, the above-mentioned face position detection method can be executed by an on-board computer (on-board equipment) on a vehicle, and the vehicle can be a taxi, bus or other type of vehicle. The present application does not limit the specific type of vehicle.

[0029] like Figure 1 As shown, it is a schematic diagram of a first implementation flow of a face position detection method provided in an embodiment of the present application. The face position detection method can be implemented by the following steps 101 to 103.

[0030] Step 101, obtaining driver characteristics and environmental characteristics.

[0031] In the embodiment of the present application, the driver characteristics may include the height and seat position of the driver. The environmental characteristics may include the installation position of the camera.

[0032] It should be noted that the examples provided here are merely for illustration of driver characteristics and environmental characteristics, and are not intended to limit the scope of protection of this application.

[0033] In some embodiments of the present application, the driver characteristics may also include one or more characteristics of the driver's gender, the driver's age, the driver's face width, and the driver's face height, and the environmental characteristics may also include one or more characteristics of the image acquisition time period, the vehicle speed, the vehicle acceleration, and the image acquisition light intensity. Among them, the image acquisition time period can be divided into two time periods of day and night or other more time periods, and the present application does not limit this.

[0034] In some embodiments of the present application, the seat position may include a seat fore-aft position adjustment value s horizontal and seat up and down position adjustment value s vertical ; The installation position of the camera may include the installation angle θ of the camera.

[0035] Step 102 : calculating an initial region of interest of the driver's face in the image captured by the camera based on the driver's features and the environment features.

[0036] As mentioned above, the driver characteristics may include the height and seat position of the driver. The environmental characteristics may include the installation position of the camera. In addition, the driver characteristics may also include one or more characteristics of the driver's gender, the driver's age, the driver's face width and the driver's face height, and the environmental characteristics may also include one or more characteristics of the image acquisition time period, the vehicle speed, the vehicle acceleration and the image acquisition light intensity.

[0037] Therefore, in the embodiment of the present application, in the process of calculating the initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environmental characteristics, the initial region of interest of the driver's face in the image captured by the camera can be calculated based on the driver's height and seat position in the driver's characteristics and the camera's installation position in the environmental characteristics, or the initial region of interest can be calculated based on the driver's height and seat position in the driver's characteristics and one or more of the following features: one or more of the driver's gender, the driver's age, the driver's face width and the driver's face height; the camera's installation position in the environmental characteristics, and one or more of the following features: one or more of the image capture time period, the vehicle speed, the vehicle acceleration and the image capture light intensity. The present application does not limit this.

[0038] In one embodiment of the present application, in the above step 102, in the process of calculating the initial area of ​​interest of the driver's face in the image captured by the camera based on the height and seat position of the driver in the driver characteristics and the installation position of the camera in the environmental characteristics, the initial area of ​​interest of the driver's face in the image captured by the camera can be specifically based on s horizontal 、s vertical , θ, h, α, β, δ, ζ, γ, H, ξ, λ, W calculate the upper boundary, lower boundary, left boundary and right boundary of the initial region of interest of the driver's face in the image captured by the camera. Among them, the rectangular area defined by the upper boundary, lower boundary, left boundary and right boundary is the initial region of interest of the driver's face in the image captured by the camera.

[0039] In the embodiment of the present application, h is the height of the driver, α, β, δ, ζ, and γ are fitting parameters obtained based on experimental data fitting, H is the preset facial height, ξ is the coefficient of the influence of the camera installation angle on the horizontal position, λ is the preset pixel starting value, and W is the preset facial width.

[0040] It should be noted that when the driver's features include the driver's face width and the driver's face height features, H is the driver's face height and W is the driver's face width. The experimental data used to fit α, β, δ, ζ, and γ can be a large number of different driver features and environmental feature data and their corresponding data of the region of interest of the driver's face in the image captured by the camera.

[0041] In one embodiment of the present application, Figure 2 As shown, the above is based on s horizontal 、s vertical , θ, h, α, β, δ, ζ, γ, H, ξ, λ, W calculate the upper boundary, lower boundary, left boundary and right boundary of the initial area of ​​interest of the driver's face in the image captured by the camera, which can be specifically implemented by the following steps 201 to 204.

[0042] Step 201, based on the formula ROI top =α×h+β×s horizontal +δ×s vertical +ζ×θ+γ calculates the upper boundary ROI of the initial region of interest of the driver's face in the image captured by the camera top .

[0043] In the embodiment of the present application, it is assumed that the height h of the driver is 175 cm, the seat is adjusted back 10 cm, upward 5 cm, the camera installation angle is 20 degrees, and the fitting parameters obtained based on the experimental data fitting are: α = 0.6, β = -0.2, δ = 1.0, ζ = -1.5, γ = 50 pixels, then substituting into the formula, we can get: ROI top ==0.6×175+(-0.2)×10+1.0×5+(-1.5)×20+50=128 pixels.

[0044] Step 202, based on the formula ROI bottom =ROI top +H calculates the lower boundary ROI of the initial region of interest of the driver's face in the image captured by the camera bottom .

[0045] In the embodiment of the present application, assuming that the preset face height is 120 pixels, the calculated ROI lower boundary is: ROI bottom =128+120=248 pixels.

[0046] Step 203, based on the formula ROI left =λ+ξ×θ Calculate the left boundary ROI of the initial region of interest of the driver's face in the image captured by the camera left .

[0047] In the embodiment of the present application, it is assumed that the camera is installed at an angle of 20 degrees, and through experiments it is found that ξ = 1.5 pixels / degree, λ = 80 pixels, and the formula is substituted to obtain: ROI left =80+1.5×20=110 pixels.

[0048] Step 204, based on the formula ROI right =ROI left +W calculates the right boundary ROI of the initial region of interest of the driver's face in the image captured by the camera right .

[0049] In the embodiment of the present application, assuming that the preset face width is 80 pixels, substituting into the formula, we can get: ROI right =110+80=190 pixels.

[0050] Based on the above assumptions, the calculated position of the ROI in the image is finally a rectangular area from (110, 128) to (190, 248).

[0051] In the embodiment of the present application, the above steps 201 to 204 are merely examples of how to calculate the initial region of interest, and are not intended to limit the scope of protection of the present application. In some embodiments of the present application, more driver characteristics and / or environmental characteristics may be added to calculate the initial region of interest, which will cause the formulas in steps 201 to 204 to add more parameters. For example, in one embodiment of the present application, the vehicle speed and vehicle acceleration corresponding to the image captured by the camera may also be used as parameters affecting the initial region of interest, and incorporated into the formula for calculating the initial region of interest, which will not be described in detail here.

[0052] Step 103, inputting the initial region of interest, the driver's features and the environmental features into a pre-trained target face detection model, and the target face detection model outputs the target region of interest of the driver's face in the image captured by the camera.

[0053] In an embodiment of the present application, the driver features and environmental features input into the target face detection model may be of the same feature type as the driver features and environmental features used to calculate the initial region of interest, for example, both only include the driver's height and seat position, and the camera installation position in the environmental features. Alternatively, the driver features and environmental features input into the target face detection model may have more feature types than the driver features and environmental features used to calculate the initial region of interest. For example, the driver features and environmental features used to calculate the initial region of interest only include the driver's height, seat position, and camera installation position; while the driver features and environmental features input into the target face detection model, in addition to this, also include one or more features of the driver's gender, driver's age, driver's face width, driver's face height, image acquisition time period, vehicle speed, vehicle acceleration, and image acquisition light intensity; which is related to the target face detection model training process, and will be described in step 301 below.

[0054] In the embodiment of the present application, the preliminary region of interest is first calculated based on a formula to provide a stable starting point. The preliminary region of interest calculated by the formula and the driver characteristics (such as the driver's height, seat position, driver's gender, etc.) and environmental characteristics (such as image acquisition time period, vehicle speed, vehicle acceleration and image acquisition light intensity, camera angle, etc.) are then input into the neural network model (i.e., the target face detection model), so that the neural network model dynamically adjusts the preliminary calculated ROI based on real-time monitoring of the environment and driver behavior to optimize the ROI calculated by the formula, so as to improve the adaptability and accuracy of face position detection in a dynamic environment, ensure that the target region of interest is the best recognition area for the face, improve the accuracy of face position detection, reduce the occurrence of false detection, and indirectly improve the efficiency and accuracy of face recognition.

[0055] Compared with the prior art, the initial region of interest in this application is generated based on the external variables of the driver's characteristics and the dynamics of environmental characteristics, rather than a fixed region of interest, thus avoiding the deviation caused by a fixed region of interest and improving the accuracy of face position detection. In addition, there is no need to search for faces in the entire screen, which reduces the consumption of computing resources and time costs and improves the efficiency of face position detection.

[0056] In one embodiment of the present application, after obtaining the target area of ​​interest of the driver's face in the image captured by the camera output by the target face position detection model in step 103, the camera can be driven to rotate in real time when the target area of ​​interest deviates from the preset area of ​​the image captured by the camera, so that the target area of ​​interest of the driver's face in the image captured by the camera is located in the preset area of ​​the image captured by the camera.

[0057] For example, when the target area of ​​interest deviates from the central area of ​​the image captured by the camera, the camera can be driven to rotate so that the target area of ​​interest of the driver's face in the image captured by the camera is located in the central area of ​​the image captured by the camera, so as to improve the accuracy of driver face recognition in real time.

[0058] In the implementation of this application, the above-mentioned preset area, in addition to the central area, can also be any area set according to user needs, for example, the area located in the upper third of the image, or the area located in the left third of the image, or the area located in the right third of the image. This application does not impose any restrictions on this.

[0059] In one embodiment of the present application, Figure 3 As shown, the target face position detection model in the above step 103 can be implemented by the following steps 301 to 303.

[0060] Step 301: Obtain a driving sample data set and a region of interest label value corresponding to each set of driving sample data in the driving sample data set.

[0061] In an embodiment of the present application, the type of data contained in each group of driving sample data in the above-mentioned driving sample data set is consistent with the data type of the data input into the target face position detection model in the aforementioned step 103, including: sample initial area of ​​interest, sample driver characteristics and sample environment characteristics.

[0062] Wherein, when the driver characteristics input into the target face position detection model in the aforementioned step 103 include the driver's height, seat position, gender, age, face width and face height, the sample driver characteristics also include sample height, sample seat position, sample gender, sample age, sample face width and sample face height. When the environmental characteristics input into the target face position detection model in the aforementioned step 103 include the camera's installation position, image acquisition time period, vehicle speed, vehicle acceleration and image acquisition light intensity, the sample environmental characteristics also include the camera's sample installation position, sample image acquisition time period, sample vehicle speed, sample vehicle acceleration and sample image acquisition light intensity.

[0063] For example, a large amount of driving video data is collected through the vehicle camera, which covers different drivers, various seat positions, camera installation angles, and different driving conditions (such as day, night, vehicle speed, vehicle acceleration, etc.). Each video clip is marked with the best ROI (the area corresponding to the driver's face).

[0064] Step 302: input each set of driving sample data in the driving sample data set into an initial face position detection model to be trained, so that the initial face position detection model outputs a predicted value of the region of interest corresponding to each set of driving sample data.

[0065] In the embodiment of the present application, the initial face position detection model to be trained can be a pre-built lightweight neural network model (such as MobileNet or EfficientNet), which learns the optimal ROI configuration in various scenarios through a large amount of labeled driving sample data. Among them, MobileNet has the characteristics of efficient computing performance and low computing resource requirements, and is suitable for deployment on vehicle-mounted equipment.

[0066] In one embodiment of the present application, the initial face position detection model may include an input layer, a convolutional layer, and a fully connected layer. Among them, the input layer is used to input the standardized sample initial region of interest, sample driver characteristics, and sample environment characteristics (i.e., driving sample data). The convolutional layer is used to extract complex associations in the input features and capture the relationship between different features, such as the impact of camera angles on facial position. The fully connected layer is used to integrate the high-dimensional features output by the convolutional layer and generate ROI parameters (including the positions of the upper boundary, lower boundary, left boundary, and right boundary, i.e., the predicted value of the region of interest corresponding to each set of driving sample data).

[0067] Step 303, adjusting the parameters of the initial face position detection model based on the difference between the region of interest label value corresponding to each group of driving sample data and the region of interest prediction value corresponding to each group of driving sample data, until the training of the initial face position detection model is completed and the target face position detection model is obtained.

[0068] In the embodiment of the present application, in the process of adjusting the parameters of the initial face position detection model based on the difference between the region of interest label value corresponding to each set of driving sample data and the region of interest prediction value corresponding to each set of driving sample data, the mean square error function (MSE) can be used as the loss function to calculate the difference between the region of interest label value corresponding to each set of driving sample data and the region of interest prediction value corresponding to each set of driving sample data. y i is the label value of the region of interest corresponding to the i-th group of driving sample data, is the predicted value of the region of interest corresponding to the i-th group of driving sample data output by the initial face position detection model, and n is the total number of groups of driving sample data.

[0069] In the embodiment of the present application, the Adam optimizer can be used to train the initial face position detection model in steps 302 to 303. The Adam optimizer dynamically adjusts the learning rate of each parameter by calculating the first-order moment estimate and the second-order moment estimate of the gradient, thereby accelerating the convergence speed and improving the training efficiency of the initial face position detection model. In the embodiment of the present application, the initial learning rate of the Adam optimizer can be set to 0.001, and dynamically adjusted according to the performance of the validation set during the training process.

[0070] In order to improve the generalization ability of the model, in one embodiment of the present application, after the driving sample data set and the region of interest label value corresponding to each group of driving sample data in the driving sample data set are obtained in the above step 301, it can also include: performing data enhancement on each group of driving sample data in the driving sample data set to obtain an expanded sample data set, and annotating each group of driving sample data in the expanded sample data set to obtain the region of interest label value corresponding to each group of driving sample data in the expanded sample data set, and then, performing steps 302 and 303 using the region of interest label value corresponding to each group of driving sample data in the expanded sample data set to train the initial face position detection model and obtain the target face position detection model.

[0071] For example, 200,000 annotated image frames are extracted from 1,000 hours of driving videos, and each frame corresponds to an annotated ROI (i.e., region of interest label value). Each frame is randomly rotated (±10 degrees) and / or scaled (±20%) and / or contrast and brightness adjusted (±30%) and / or translated, etc., to generate about 600,000 enhanced sample image samples, and the corresponding driving sample data are determined based on the rotation, scaling and other operations used in image enhancement, such as sample height, sample seat position, sample face width and sample face height, sample installation position of the camera, sample image acquisition time period, sample vehicle speed, sample vehicle acceleration and sample image acquisition light intensity, thereby obtaining an expanded sample data set containing 600,000 sets of driving sample data to simulate various scenarios that may be encountered in actual driving, so that the trained target face position detection model can maintain stable performance under different conditions, thereby improving the generalization ability of the target face position detection model.

[0072] In order to make the target region of interest output by the target face position detection model more accurate, in one embodiment of the present application, in step 103, after obtaining the target region of interest of the driver's face in the image captured by the camera output by the target face position detection model, the target face position detection model can also be locally optimized based on the following steps 401 to 402.

[0073] Step 401, upload the target area of ​​interest and the image captured by the camera to the cloud server, and receive the evaluation result of whether the face position detection is accurate or not fed back by the cloud server, and when the evaluation result is that the face position detection is wrong, the cloud server feeds back the correct face position.

[0074] In one embodiment of the present application, after the vehicle-mounted device calculates the target region of interest in the image currently captured by the camera, the vehicle-mounted device can upload the image and the target region of interest corresponding to the image to the cloud server for storage. When performing model optimization, the backend maintenance personnel can evaluate the image uploaded by the vehicle-mounted device and the target region of interest corresponding to the image to obtain an evaluation result of whether the face position detection is accurate. When the evaluation result is that there is an error in the face position detection, the correct face position is fed back to the vehicle-mounted device through the cloud server.

[0075] In another embodiment of the present application, after calculating the target region of interest in the image currently captured by the camera, the vehicle-mounted device can upload the image and the target region of interest corresponding to the image to the cloud server, and the large model deployed in the cloud server performs face position detection on the image currently captured by the camera to obtain the face region of interest output by the large model, and compares the face region of interest output by the large model with the target region of interest to obtain an evaluation result of whether the face position detection is accurate. Then, when the evaluation result shows that there is an error in the face position detection, the correct face position (i.e., the face region of interest output by the large model) is fed back to the vehicle-mounted device through the cloud server.

[0076] In the embodiment of the present application, the above-mentioned large model refers to a face position detection model with a large number of parameters and high performance, which is suitable for automatic image labeling and is not suitable for deployment on vehicle-mounted equipment. The present application compares the face region of interest output by the large model with the target region of interest output by the target face position detection model to evaluate whether the target region of interest output by the target face position detection model is accurate.

[0077] It should be noted that the above is merely an example of a method for evaluating whether the target region of interest output by the target face position detection model is accurate, and does not constitute a limitation on the scope of protection of the present application. In other embodiments of the present application, other methods may be used to evaluate whether the target region of interest output by the target face position detection model is accurate, and the present application does not impose any restrictions on this.

[0078] Step 402: incrementally train the target face position detection model based on the correct face position fed back by the cloud server to obtain an optimized target face position detection model.

[0079] In an embodiment of the present application, the vehicle-mounted device fine-tunes the model by incrementally training the target face position detection model based on the correct face position feedback from the cloud server, thereby achieving gradual optimization of the target face position detection model, thereby making the target area of ​​interest output by the target face position detection model more accurate.

[0080] In order to adapt to the individual differences and driving habits of different drivers and improve the compatibility of the model with a single driver, in one embodiment of the present application, the above steps 401 to 402 can locally optimize the target face position detection model for different drivers.

[0081] Specifically, when the vehicle-mounted device receives the evaluation result of whether the face position detection is accurate from the cloud server, and when the cloud server feeds back the correct face position when the evaluation result is that there is an error in the face position detection, the correct face position and the driver characteristics, environmental characteristics, initial area of ​​interest, driver's identity information and other data corresponding to the correct face position can be used as training data for the driver and stored in a folder corresponding to the driver.

[0082] For example, when a driver logs in to the on-board device for the first time or is identified by the on-board device, the on-board device can create an independent profile for each driver based on their identity information (such as driver ID) to store training data and model parameters related to the driver.

[0083] When the on-board equipment is in an idle state (such as when the vehicle is stopped or running at low load), the on-board equipment uses the data independently stored by each driver to perform incremental training and model fine-tuning to obtain an optimized target face position detection model. The optimized target face position detection model gradually adapts to the behavior and environmental changes of the specific driver, thereby making the target area of ​​interest output by the target face position detection model more accurate.

[0084] For example, when the vehicle-mounted device recognizes driver A's boarding system, it loads his historical data and uses this data to train and fine-tune the target face position detection model when the vehicle-mounted device is idle. Similarly, when driver B is driving the vehicle, the vehicle-mounted device switches to B's historical data and uses this data to optimize B's target face position detection model. The incremental training results of each driver are saved in their corresponding configuration files. Through continuous local training, the model recognition accuracy of each driver is gradually improved.

[0085] For example, the optimized model parameters of driver A are saved in the / scard / driver_A / model_params / directory. When the device recognizes driver A again, the vehicle device will automatically load its latest model parameters. After driver B logs in, the vehicle device will record its corresponding data in the / sdcard / driver_B / directory to ensure that the data of different drivers are stored independently.

[0086] In one embodiment of the present application, the vehicle-mounted device can also synchronize the model parameters corresponding to different drivers to the cloud server, so that the driver can achieve accurate face position detection in any vehicle equipped with the vehicle-mounted device for executing the above-mentioned face position detection method.

[0087] For example, the model parameters of driver A can be synchronized to the cloud server after being updated. When driver A drives another vehicle that is also equipped with on-board equipment for executing the above-mentioned face position detection method, he can obtain the model parameters of the corresponding optimized target face position detection model from the cloud server, thereby providing a consistent recognition experience.

[0088] In the above embodiment of the present application, by deploying the trained target face position detection model on the vehicle, when the vehicle starts, firstly, the initial region of interest is quickly calculated based on the formula, and then, the target face position detection model dynamically adjusts the initial region of interest based on the real-time monitoring environment and driver behavior to obtain the target region of interest, that is, the face position region, thereby achieving the purpose of accurately detecting the face position in the image captured by the camera. In addition, by optimizing the target face position detection model corresponding to each driver locally and in a targeted manner, the model is made more in line with the habits of each driver, so that the target region of interest output by the target face position detection model is more accurate.

[0089] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, and in some embodiments of the present application, certain steps may be performed in other orders.

[0090] The present application also provides a face position detection device. Figure 5 As shown, the face position detection device 500 can be configured on a vehicle and executed by an onboard device on the vehicle. The face position detection device may include:

[0091] The acquisition unit 501 is used to acquire driver characteristics and environmental characteristics; the driver characteristics include the height and seat position of the driver, and the environmental characteristics include the installation position of the camera;

[0092] A calculation unit 502, configured to calculate an initial region of interest of the driver's face in the image captured by the camera based on the driver's features and the environment features;

[0093] The detection unit 503 is used to input the initial region of interest, driver characteristics and environmental characteristics into a pre-trained target face position detection model to obtain a target region of interest output by the target face position detection model of the driver's face in the image captured by the camera.

[0094] It should be noted that, for the convenience and simplicity of description, the specific working process of the face position detection device 500 described above can refer to the above Figures 1 to 4 The corresponding process of the method described in will not be repeated here.

[0095] like Figure 6 As shown, the embodiment of the present application also provides an electronic device. The electronic device may be a vehicle. Figure 6 As shown, the electronic device 6 may include: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the above-mentioned various face position detection method embodiments are implemented, for example, Figure 1 Steps 101 to 103 are shown.

[0096] The processor 60 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0097] The memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or a memory. The memory 61 may also be an external storage device for the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 61 may also include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 is used to store the above-mentioned computer program and other programs and data required by the electronic device.

[0098] The computer program may be divided into one or more units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the process of the computer program executing the face position detection method in the electronic device.

[0099] For example, the above computer program can be divided into: an acquisition unit, a calculation unit and a detection unit, and the specific functions are as follows:

[0100] An acquisition unit, used to acquire driver characteristics and environmental characteristics; the driver characteristics include the driver's height and seat position, and the environmental characteristics include the installation position of the camera;

[0101] A calculation unit, used for calculating an initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environment characteristics;

[0102] The detection unit is used to input the initial region of interest, driver features and environmental features into a pre-trained target face position detection model to obtain a target region of interest output by the target face position detection model, which is the driver's face in the image captured by the camera.

[0103] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0104] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0106] In the embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, systems or units, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer program product, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or recording medium that can carry computer program code, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0110] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting a face position, characterized in that: The face position detection method comprises: Acquire driver characteristics and environmental characteristics; the driver characteristics include the driver's height and seat position, and the environmental characteristics include the installation position of the camera; Calculate an initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environmental characteristics; The initial region of interest, the driver features and the environmental features are input into a pre-trained target face position detection model to obtain a target region of interest output by the target face position detection model of the driver's face in the image captured by the camera.

2. The face position detection method according to claim 1, characterized in that: The seat position includes the seat front and rear position adjustment value s horizontal and seat up and down position adjustment value s vertical ; The installation position of the camera includes the installation angle θ of the camera; The calculating, based on the driver characteristics and the environment characteristics, an initial region of interest of the driver's face in the image captured by the camera comprises: Based on s horizontal 、s vertical , θ, h, α, β, δ, ζ, γ, H, ξ, λ, W calculate the upper boundary, lower boundary, left boundary and right boundary of the initial interest area of ​​the driver's face in the image captured by the camera; Among them, h is the height of the driver, α, β, δ, ζ, γ are fitting parameters obtained based on experimental data fitting, H is the preset face height, ξ is the coefficient of the influence of the installation angle of the camera on the horizontal position, λ is the preset pixel starting value, and W is the preset face width.

3. The face position detection method according to claim 2, characterized in that: The based horizontal 、s vertical , θ, h, α, β, δ, ζ, γ, H, ξ, λ, W calculate the driver's face in the camera The upper boundary, lower boundary, left boundary and right boundary of the initial region of interest in the image captured by the camera, including: ROI based on formula top =α×h+β×s horizontal +δ×s vertical +ζ×θ+γ calculates the upper boundary ROI of the initial region of interest of the driver's face in the image captured by the camera top ; ROI based on formula bottom =ROI top +H calculates the lower boundary ROI of the initial region of interest of the driver's face in the image captured by the camera bottom ; ROI based on formula left =λ+ξ×θ Calculate the left boundary ROI of the initial region of interest of the driver's face in the image captured by the camera left ; ROI based on formula right =ROI left +W calculates the right boundary ROI of the initial region of interest of the driver's face in the image captured by the camera right .

4. The face position detection method according to any one of claims 1 to 3, characterized in that: After obtaining the target region of interest of the driver's face in the image captured by the camera output by the target face position detection model, the face position detection method further includes: When the target area of ​​interest deviates from the preset area of ​​the image captured by the camera, the camera is driven to rotate so that the target area of ​​interest of the driver's face in the image captured by the camera is located in the preset area of ​​the image captured by the camera.

5. The face position detection method according to any one of claims 1 to 3, characterized in that: After inputting the initial region of interest, the driver features, and the environmental features into a pre-trained target face position detection model to obtain a target region of interest of the driver's face in the image captured by the camera output by the target face position detection model, the face position detection method further includes: Uploading the target area of ​​interest and the image captured by the camera to a cloud server, and receiving an evaluation result of whether the face position detection is accurate from the cloud server, and when the evaluation result is that there is an error in the face position detection, the cloud server feedbacks the correct face position; The target face position detection model is incrementally trained based on the correct face position fed back by the cloud server to obtain an optimized target face position detection model.

6. The face position detection method according to any one of claims 1 to 3, characterized in that: The training steps of the target face position detection model include: Acquire a driving sample data set and a region of interest label value corresponding to each group of driving sample data in the driving sample data set; Inputting each set of driving sample data in the driving sample data set into the initial face position detection model to be trained, so that the initial face position detection model outputs a predicted value of the region of interest corresponding to each set of driving sample data; Based on the difference between the region of interest label value corresponding to each group of driving sample data and the region of interest prediction value corresponding to each group of driving sample data, the parameters of the initial face position detection model are adjusted until the training of the initial face position detection model is completed and the target face position detection model is obtained.

7. The face position detection method according to claim 6, characterized in that: After obtaining the driving sample data set and the label value of the region of interest corresponding to each group of driving sample data in the driving sample data set, the method further includes: Data enhancement is performed on each group of driving sample data in the driving sample data set to obtain an expanded sample data set, and each group of driving sample data in the expanded sample data set is labeled to obtain a region of interest label value corresponding to each group of driving sample data in the expanded sample data set.

8. A face position detection device, characterized in that: The face position detection device comprises: An acquisition unit, used to acquire driver characteristics and environmental characteristics; the driver characteristics include the driver's height and seat position, and the environmental characteristics include the installation position of the camera; A calculation unit, configured to calculate an initial region of interest of the driver's face in the image captured by the camera based on the driver's characteristics and the environmental characteristics; The detection unit is used to input the initial region of interest, the driver's characteristics and the environmental characteristics into a pre-trained target face position detection model, and obtain a target region of interest of the driver's face in the image captured by the camera output by the target face position detection model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the face position detection method as described in any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that It comprises a computer program, which, when executed, implements the steps of the face position detection method according to any one of claims 1 to 7.

Images