Driver head position determination method, device and medium based on image recognition

The driver's head position is determined through image recognition technology and the airbag deployment parameters are adjusted, which solves the situation where the existing technology cannot effectively protect different drivers, and achieves a safer airbag protection effect.

CN118628558BActive Publication Date: 2024-11-05CATARC AUTOMOTIVE TEST CENT TIANJIN CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411120965.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-05
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

The existing airbag deployment parameter calibration method cannot effectively protect drivers of different body shapes, genders and riding habits, resulting in impact damage to the driver during collision.

Method used

Using an image recognition-based method, the driver's head image is collected through frontal, side and top-view positioning cameras, the pre-trained key point recognition model is used to determine the driver's head key point position, and the airbag deployment parameters are adjusted according to these positions. .

Benefits of technology

Accurate identification of the driver's head position and targeted adjustment of the airbag deployment parameters are achieved to ensure that the airbag can effectively protect the driver and reduce head damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118628558B_ABST
    Figure CN118628558B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of vehicle passive safety technology, and discloses a method, device and medium for determining the position of a driver's head based on image recognition. The method obtains the reference point coordinates of the reference points of each positioning camera in the vehicle in the whole vehicle coordinate system, and then in response to detecting that the vehicle has a collision risk, collects the driver's head image of the vehicle through each positioning camera respectively, inputs it into the corresponding pre-trained key point recognition model, and obtains the position of the head key point, thereby combining the position of the head key point in each driver's head image with the position of the reference point of the positioning camera and the reference point coordinates of each positioning camera, and determining the key point coordinates of each head key point in the whole vehicle coordinate system, and finally determining the driver's head position information, so as to determine the deployment parameters of the airbag through the driver's head position information, and realize timely and targeted adjustment of the deployment parameters of the airbag, so as to ensure that the airbag can be effectively deployed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle passive safety, and in particular to a method, device and medium for determining a driver's head position based on image recognition. Background Art

[0002] As an important device of automobile safety technology, airbags reduce the driver's head injury to a certain extent. However, the reliability and effectiveness of airbags are directly affected by the driver's head position. The existing method for calibrating airbag deployment parameters is to place the driver's seat in a fixed position, install a certain type of dummy in a fixed posture, collect the damage value of the collision test, and calibrate the airbag parameters.

[0003] However, in actual driving situations, drivers of different body shapes, genders, and riding habits are likely to be out of the effective protection area of ​​the airbag. In addition, the airbag deploys very quickly and has a small effective area. If the driver's head is in a dangerous position during a collision, the airbag will not only fail to effectively reduce the damage, but will also likely cause impact damage to the driver.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method, device and medium for determining the driver's head position based on image recognition, so as to achieve accurate identification of the driver's head position, and then deploy the airbag in a targeted manner according to the driver's head position, thereby avoiding the situation where the airbag cannot effectively protect the driver's head due to changes in the driver's posture.

[0006] An embodiment of the present invention provides a method for determining a driver's head position based on image recognition, the method comprising:

[0007] Obtaining the coordinates of the reference points of the reference points of each positioning camera in the vehicle in the vehicle coordinate system, wherein the positioning camera is placed according to the driver's position in the vehicle, and each positioning camera includes a front-view positioning camera, a side-view positioning camera, and a top-view positioning camera;

[0008] In response to detecting that the vehicle has a collision risk, collecting a head image of a driver in the vehicle based on each positioning camera;

[0009] For each driver head image captured by the positioning camera, the driver head image is input into a corresponding trained key point recognition model to determine the position of the head key points in the driver head image;

[0010] Determine the key point coordinates of each head key point in the vehicle coordinate system based on the position of the head key point in each driver's head image, the position of the reference point of the positioning camera in each driver's head image, and the reference point coordinates of each positioning camera;

[0011] The position information of the driver's head of the vehicle is determined according to the coordinates of each of the key points, so as to determine the deployment parameters of the airbag in the vehicle based on the driver's head position information.

[0012] An embodiment of the present invention provides an electronic device, the electronic device comprising:

[0013] Processor and memory;

[0014] The processor is used to execute the steps of the method for determining the driver's head position based on image recognition described in any embodiment by calling the program or instructions stored in the memory.

[0015] An embodiment of the present invention provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method for determining the driver's head position based on image recognition as described in any embodiment.

[0016] The embodiments of the present invention have the following technical effects:

[0017] By obtaining the reference point coordinates of the reference points of each positioning camera in the vehicle in the whole vehicle coordinate system, and then in response to detecting that the vehicle has a collision risk, each positioning camera is used to collect the driver's head image of the vehicle, and for each driver's head image, it is input into the corresponding pre-trained key point recognition model to obtain the position of the head key point, so as to determine the key point coordinates of each head key point in the whole vehicle coordinate system according to the position of the head key point in each driver's head image and the position of the reference point of the positioning camera, as well as the reference point coordinates of each positioning camera, and finally determine the driver's head position information according to the coordinates of each key point, so as to determine the deployment parameters of the airbag through the driver's head position information, realize timely and targeted adjustment of the deployment parameters of the airbag, ensure that the airbag can be effectively deployed, avoid the situation where the driver's head cannot be effectively protected by the airbag due to changes in the driver's posture, and reduce the driver's head injury. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 is a flow chart of a method for determining a driver's head position based on image recognition provided by an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of placing a positioning camera provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a whole vehicle coordinate system provided by an embodiment of the present invention;

[0022] Figure 4 is a collection schematic diagram of a front-view positioning camera provided by an embodiment of the present invention;

[0023] Figure 5 is a collection schematic diagram of a side-view positioning camera provided by an embodiment of the present invention;

[0024] Figure 6 is a collection schematic diagram of a top-down positioning camera provided by an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram of a key point recognition model provided by an embodiment of the present invention;

[0026] Figure 8 is a schematic diagram of a first convolutional network unit provided for an embodiment of the present invention;

[0027] Fig. 9 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a front-view positioning camera provided by an embodiment of the present invention;

[0028] Fig.10 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a side-view positioning camera provided by an embodiment of the present invention;

[0029] Fig.11 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a bird's-eye view positioning camera provided by an embodiment of the present invention;

[0030] Fig.12 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0032] The method for determining the driver's head position based on image recognition provided by the embodiment of the present invention is mainly applicable to identifying the head position of the driver in the vehicle, so as to facilitate adjusting the deployment parameters of the airbag in combination with the head position. The method for determining the driver's head position based on image recognition provided by the embodiment of the present invention can be executed by an electronic device integrated in the vehicle, such as an airbag system controller or an electronic control unit integrated in the vehicle.

[0033] Figure 1 is a flow chart of a method for determining the position of a driver's head based on image recognition provided by an embodiment of the present invention. Figure 1 , the method for determining the driver's head position based on image recognition specifically includes:

[0034] S110, obtaining the coordinates of the reference points of the reference points of each positioning camera in the vehicle in the vehicle coordinate system.

[0035] Among them, the positioning cameras are placed according to the driver's position in the vehicle, and each positioning camera includes a front-view positioning camera, a side-view positioning camera and a top-view positioning camera.

[0036] Specifically, the lens of the front-view positioning camera is perpendicular to the plumb plane where the top-view positioning camera and the side-view positioning camera are located; the lens of the top-view positioning camera is perpendicular to the horizontal plane where the front-view positioning camera and the side-view positioning camera are located; the lens of the side-view positioning camera is perpendicular to the plumb plane where the front-view positioning camera and the top-view positioning camera are located.

[0037] For example, Figure 2 This is a schematic diagram of the placement of a positioning camera provided by an embodiment of the present invention, wherein the front-view positioning camera can be understood as a camera facing the front of the main driving seat, the top-view positioning camera can be understood as a camera facing the top of the main driving seat, and the side-view positioning camera can be understood as a camera facing the side of the main driving seat. To facilitate the subsequent calculation of the key point coordinates of the head, the side-view positioning camera can be set to be located within the negative half axis of the Y axis of the vehicle coordinate system.

[0038] In this embodiment, the reference point coordinates of the reference points of the front-view positioning camera, the side-view positioning camera and the top-view positioning camera in the whole vehicle coordinate system can be determined according to the established whole vehicle coordinate system of the vehicle.

[0039] The process of establishing the vehicle coordinate system is as follows: the X-axis points from the rear of the vehicle to the front of the vehicle, and is located on the left-right central symmetry plane of the vehicle. The origin of the X-axis is the most prominent point of the vehicle on its left-right central symmetry plane; the Y-axis points from the left side of the vehicle to the right side, that is, the driver's side points to the co-driver's side, and is located on a horizontal plane perpendicular to the left-right central symmetry plane of the vehicle and passing through the origin of the X-axis. The origin of the Y-axis is the same as the origin of the X-axis. The Z-axis points from the origin to the roof of the vehicle, that is, the direction from the origin to the roof of the vehicle is the positive direction of the Z-axis. For example, Figure 3 It is a schematic diagram of a whole vehicle coordinate system provided by an embodiment of the present invention, which shows the positive directions of the X-axis and the Y-axis, and the Z-axis starts from the origin of the X-axis and the Y-axis and is perpendicular to the X-axis and the Y-axis and points to the roof.

[0040] After the vehicle coordinate system is established, the reference point coordinates of the reference points of each positioning camera in the vehicle coordinate system can be determined according to the placement position of each positioning camera relative to the origin of the vehicle coordinate system.

[0041] The reference point of the positioning camera can be a pre-set mark point in the positioning camera, for example, the reference point can be the center point of the lens of the positioning camera. The reference points of the front-view positioning camera, the side-view positioning camera and the top-view positioning camera are respectively located in the interval of the negative half axis of the X axis, the negative half axis of the Y axis and the positive half axis of the Z axis of the vehicle coordinate system.

[0042] It should be noted that for each positioning camera, in addition to ensuring that the positioning camera can capture the corresponding key points of the driver's head, it is also necessary to ensure that the positioning camera can capture the other two positioning cameras at the same time, so that the captured image contains the reference points of the other two positioning cameras, which is convenient for the subsequent calculation of the coordinates of the head key points in the image in the vehicle coordinate system.

[0043] Figure 4 is a collection schematic diagram of a front-view positioning camera provided by an embodiment of the present invention, Figure 4 It can be seen that the image captured by the front-view positioning camera, in addition to the selected key points of the driver's head (at least one key point can be selected), also contains the reference points of the top-view positioning camera and the side-view positioning camera. The coordinates of the reference points in the vehicle coordinate system are: (x2, y2, z2) and (x3, y3, z3) respectively.

[0044] Figure 5 is a collection schematic diagram of a side-view positioning camera provided by an embodiment of the present invention, comprising Figure 5 It can be seen that the image captured by the side-view positioning camera, in addition to the selected key points of the driver's head (at least one key point can be selected), also contains the reference points of the front-view positioning camera and the top-view positioning camera. The coordinates of the reference points in the vehicle coordinate system are: (x1, y1, z1) and (x2, y2, z2) respectively.

[0045] Figure 6 is a schematic diagram of a collection of a top-down positioning camera provided by an embodiment of the present invention, comprising Figure 6 It can be seen that the image captured by the overhead positioning camera, in addition to the selected key points of the driver's head (at least one key point can be selected), also contains the reference points of the front-view positioning camera and the side-view positioning camera. The coordinates of the reference points in the vehicle coordinate system are: (x1, y1, z1) and (x3, y3, z3) respectively. Figure 4-Figure 6 The vehicle coordinate systems in the figure are schematic, used to indicate the viewing angle of each positioning camera and the direction of the collected image in the vehicle coordinate system.

[0046] S120: In response to detecting that there is a risk of collision with the vehicle, respectively collect a head image of the driver in the vehicle based on each positioning camera.

[0047] Specifically, the obstacle information of obstacles (such as surrounding vehicles) can be obtained through the perception system installed on the vehicle, such as the speed, distance, acceleration, etc. of the surrounding vehicles, and then combined with the obstacle information of the obstacles to predict whether the vehicle has a collision risk.

[0048] For example, the minimum collision time can be calculated based on the speed of the vehicle and the speed, distance and acceleration of the obstacle, and the vehicle can be judged whether there is a collision risk by the minimum collision time. Alternatively, the obstacle's driving trajectory can be predicted based on the obstacle's speed, distance and acceleration, and then the vehicle can be judged whether there is a collision risk based on the predicted obstacle's trajectory and the vehicle's driving trajectory.

[0049] In this embodiment, if a collision risk is detected in the vehicle, in order to adjust the deployment parameters of the airbag before the airbag is deployed, all positioning cameras can respectively capture the driver's head image in the vehicle to identify the driver's head position.

[0050] Specifically, the front-view positioning camera, the side-view positioning camera and the top-view positioning camera can be controlled to respectively collect images to obtain the driver's head images collected by the front-view positioning camera, the side-view positioning camera and the top-view positioning camera.

[0051] S130. For each driver head image captured by the positioning camera, the driver head image is input into a corresponding trained key point recognition model to determine the position of the head key points in the driver head image.

[0052] Among them, the key point recognition model can be used to identify the position of the head key points in the input image. The head key points can be pre-set feature points for identifying the position of the driver's head. Taking the driver's head image captured by the front-view positioning camera as an example, the head key points can specifically be feature points in the face, such as the head key points can include the left eye, the right eye, the tip of the nose, the left lip corner and the right lip corner.

[0053] In a specific implementation, the training step of the key point recognition model corresponding to the positioning camera includes:

[0054] Step 11, obtaining an initial sample set and constructing a model to be trained, wherein the initial sample set includes sample head images of the driver in different postures collected by the positioning camera and the corresponding actual key point positions, and the model to be trained includes a first convolutional network unit, a second convolutional network unit, and a third convolutional network unit;

[0055] Step 12: training the first convolutional network unit based on the initial sample set;

[0056] Step 13: After the first convolutional network unit is trained, based on the predicted key point positions corresponding to each sample head image output by the first convolutional network unit, a first local frame image is selected from each sample head image to construct a first local sample set, and a second convolutional network unit is trained based on the first local sample set;

[0057] Step 14: after the training of the second convolutional network unit is completed, based on the predicted key point positions corresponding to the first local frame images output by the second convolutional network unit, a second local frame image is selected from the first local frame images to construct a second local sample set, and a third convolutional network unit is trained based on the second local sample set;

[0058] Step 15: After the training of the third convolutional network unit is completed, all convolutional network units in the training model are trained based on the initial sample set to obtain a key point recognition model.

[0059] Among them, in step 11, each positioning camera can be used to collect sample head images of the driver in different postures, and the position of each key point in the sample head image is determined to obtain the corresponding actual key point position as a sample label. For example, images of the driver's head in different posture spaces can be collected, and the division of the posture space can be based on actual needs. By adjusting the density of the posture space division, detailed images of the key points of the head in a certain spatial area can be collected to facilitate enhanced recognition accuracy.

[0060] In addition, a model to be trained including the first convolutional network unit, the second convolutional network unit and the third convolutional network unit may be constructed. For example, the model to be trained may be a DCNN (Deep Convolutional Neural Network) model, which can achieve more accurate key point detection by using the powerful feature extraction capability of the convolutional neural network.

[0061] Among them, the first convolutional network unit, the second convolutional network unit and the third convolutional network unit are three-level cascaded convolutional neural network units. The first convolutional network unit is connected to the second convolutional network unit, that is, the output of the first convolutional network unit will be input to the second convolutional network unit; the second convolutional network unit is connected to the third convolutional network unit, that is, the output of the second convolutional network unit will be input to the third convolutional network unit.

[0062] Furthermore, in the above step 12, the first convolutional network unit can be trained by the initial sample set, that is, each sample head image in the initial sample set is input into the first convolutional network unit, and the loss value is calculated according to the corresponding predicted key point position and the actual key point position output by the first convolutional network unit, and then the parameters in the first convolutional network unit are adjusted according to the loss value until the loss converges, thereby completing the training of the first convolutional network unit.

[0063] In an embodiment of the present invention, in order to further improve the accuracy of detection, the first convolutional network unit can also be composed of multiple convolutional layers, and the key points identified by each convolutional layer may not be exactly the same, so that a more accurate prediction of the key point position can be obtained by averaging the outputs of all convolutional layers.

[0064] With respect to the above step 12, optionally, the first convolutional network unit includes a first convolutional layer, a second convolutional layer, and a third convolutional layer, and training the first convolutional network unit based on the initial sample set includes the following steps:

[0065] Step 121, inputting the sample head images in the initial sample set into the first convolution layer, the second convolution layer and the third convolution layer respectively;

[0066] Step 122: determining predicted key point positions corresponding to the sample head image based on all key point positions output by the first convolutional layer, some key point positions output by the second convolutional layer, and some key point positions output by the third convolutional layer;

[0067] Step 123: determine the loss value according to the actual key point position and the predicted key point position corresponding to each sample head image, and adjust the parameters of the first convolution layer, the second convolution layer, and the third convolution layer based on the loss value until the loss value converges.

[0068] Among them, the three convolutional layers in the first convolutional network unit are in parallel structure. In step 121, the sample head images in the initial sample set are output to the first convolutional layer, the second convolutional layer and the third convolutional layer respectively.

[0069] Furthermore, the first convolution layer can output the positions of all key points of the sample head image. Taking the sample head image captured by the front-view positioning camera as an example, the first convolution layer can output the positions of 5 key points on the face; the second convolution layer can output the positions of some key points of the sample head image. Taking the sample head image captured by the front-view positioning camera as an example, the second convolution layer can output the positions of 3 key points on the face (left eye, right eye, and nose tip); the third convolution layer can output the positions of some key points of the sample head image. Taking the sample head image captured by the front-view positioning camera as an example, the third convolution layer can output the positions of 3 key points on the face (nose tip, left lip corner, and right lip corner).

[0070] Specifically, in step 122, the outputs of the first convolutional layer, the second convolutional layer, and the third convolutional layer may be averaged to obtain the predicted key point position output by the first convolutional network unit.

[0071] Furthermore, in step 123, the loss value can be calculated according to the actual key point position (i.e., sample label) and the predicted key point position (i.e., prediction result) corresponding to each sample head image, and the parameters of the three convolutional layers can be reversely adjusted according to the loss value. This process is repeated until the loss value converges, for example, the rate of change of the loss value is lower than the preset rate of change threshold.

[0072] In the above steps 121 to 123, by designing three convolutional layers in the first convolutional network unit, and the key points identified by each convolutional layer are not exactly the same, the reliability and accuracy of key point detection can be further improved.

[0073] Considering that the result output by the first convolutional network unit may still not be accurate enough, in order to obtain a more precise key point position, the second and third convolutional network units can be trained after the first convolutional network unit is trained to fine-tune the coarse positioning of the first convolutional network unit to obtain a more precise key point position. Among them, the first convolutional network unit can use a deeper network to estimate the position of the key point, and the second and third convolutional network units can share a shallower network to achieve high-precision position recognition.

[0074] After the training of the first convolutional network unit is completed, specifically, in the above step 13, the second convolutional network unit can be trained. Wherein, for the training of the second convolutional network unit, it can be based on the predicted key point positions corresponding to each sample head image output after the training of the first convolutional network unit, to construct a first local sample set required for training the second convolutional network unit, and train the second convolutional network unit through the first local sample set.

[0075] For the above step 13, optionally, based on the predicted key point position corresponding to each sample head image output by the first convolutional network unit, a first local frame image is selected from each sample head image to construct a first local sample set, including:

[0076] For each head key point, taking the position of the head key point in the predicted key point position output by the first convolutional network unit as the center, determine the corresponding first local box in the sample head image according to two preset first sizes, and move each first local box in the sample head image to obtain the first local box images corresponding to the two preset first sizes, and construct two first local sample sets for the head key points.

[0077] The preset first size refers to the size of the local frame; the two preset first sizes are different. For example, the two preset first sizes are (2*0.18*W, 2*0.18*H) and (2*0.16*W, 2*0.16*H), where W and H are the width and height of the face frame, 0.18 and 0.16 are the scale factors, and 2 is the magnification factor. It should be noted that the magnification factor can be dynamically adjusted according to actual needs. For example, if it is found that the local frame is too small to miss context information, the magnification factor can be appropriately increased.

[0078] Specifically, the predicted key point positions output by the trained first convolutional network unit can be used to locate two first local boxes in the sample head image according to two preset first sizes with the head key points as the center, and the first local boxes are randomly translated slightly to make the positions of the head key points in the first local boxes random, and then the first local box images corresponding to the two preset first sizes are cropped.

[0079] Each group of predicted key point positions output by the first convolutional network unit is processed in the above manner to obtain two first local frame images of each head key point in the group of predicted key point positions. For each head key point, a first local sample set is constructed according to each first local frame image corresponding to one of the preset first sizes, and another first local sample set is constructed according to each first local frame image corresponding to another preset first size, so that each head key point corresponds to two first local sample sets.

[0080] After obtaining the first local sample set, the second convolutional network unit can be trained using the first local sample set. In one example, the second convolutional network unit includes a convolutional layer combination corresponding to each head key point, the convolutional layer combination is composed of two convolutional layers, and training the second convolutional network unit based on the first local sample set includes the following steps:

[0081] Step 131: for each convolutional layer combination in the second convolutional network unit, the first local frame images in the two first local sample sets corresponding to the head key points are respectively input into the two convolutional layers of the convolutional layer combination;

[0082] Step 132: determining the predicted key point position corresponding to the first local frame image based on the mean value between the positions of the head key points output by the two convolutional layers;

[0083] Step 133: determine a loss value according to the actual key point positions and the predicted key point positions corresponding to each first local frame image, and adjust the parameters of the convolutional layer combination based on the loss value until the loss value converges.

[0084] Among them, taking the sample head image collected by the front-view positioning camera as an example, where the number of head key points is 5, the second convolutional network unit includes 5 convolutional layer combinations, each convolutional layer combination is composed of two parallel convolutional layers, and each convolutional layer combination processes the first local sample set of the corresponding head key points respectively.

[0085] Specifically, in step 131, for each convolution layer combination, the first local frame image in one of the first local sample sets corresponding to the head key point can be input into one convolution layer, and the first local frame image in another first local sample set corresponding to the head key point can be input into another convolution layer.

[0086] Furthermore, in step 132, for each convolutional layer combination, the outputs of the two convolutional layers may be averaged to obtain the predicted key point position of the convolutional layer combination output.

[0087] Furthermore, in step 133, the loss value can be calculated according to the actual key point position and the predicted key point position, and the parameters in the convolutional layer combination can be adjusted according to the loss value. This process is repeated until the loss converges, and the training of the second convolutional network unit is completed.

[0088] After completing the training of the second convolutional network unit, further, in the above step 14, the third convolutional network unit can continue to be trained. For the training of the third convolutional network unit, the second local sample set required for training the third convolutional network unit can be constructed based on the predicted key point positions output by the second convolutional network unit after training, and the third convolutional network unit can be trained by the second local sample set.

[0089] Specifically, the predicted key point positions corresponding to the first local frame images output by the second convolutional network unit can be combined with two preset second sizes to determine the corresponding second local frames in each first local frame image, and each second local frame can be moved in the first local frame image to obtain the second local frame images corresponding to the two preset second sizes, and construct two second local sample sets for the key points of the head. The two preset second sizes can be: (2*0.11*W, 2*0.11*H), (2*0.12*W, 2*0.12*H).

[0090] The third convolutional network unit includes a convolutional layer combination corresponding to each head key point, and the convolutional layer combination is composed of two convolutional layers. Taking the sample head image collected by the front-view positioning camera as an example, where the number of head key points is 5, the third convolutional network unit includes 5 convolutional layer combinations, each convolutional layer combination is composed of two parallel convolutional layers, and each convolutional layer combination processes the second local sample set of the corresponding head key point.

[0091] Furthermore, two second local sample sets of each head key point can be used and respectively input into each convolution layer combination in the third convolutional network unit. The training process can refer to the second convolutional neural unit and will not be repeated here.

[0092] After completing the training of the third convolutional network unit, further, in the above step 15, all the convolutional network units in the training model can be trained as a whole through the initial sample set to optimize the parameters in each convolutional network unit again to obtain a key point recognition model.

[0093] It should be noted that, in an embodiment of the present invention, the input size of the first convolutional network unit is larger than the input size of the second convolutional network unit, and the input size of the second convolutional network unit is larger than that of the third convolutional network unit. The reason is that because the relative position of the driver is not fixed, there will be large changes in the horizontal and vertical directions. Therefore, the relative position of the bounding box of face detection needs to vary over a large range. In addition, the driver's facial posture will also change, which will lead to the diversity of the input image. Therefore, the first convolutional network unit can have a large enough input size to better handle diverse input images. The second and third convolutional network units can fine-tune the coarse positioning output by the first convolutional network unit to obtain a more precise key point position.

[0094] Through the above steps 11 to 15, the key point recognition models corresponding to each positioning camera can be trained. By constructing a model including multiple convolutional network units and according to the cascade processing logic between the convolutional network units, the key point recognition model finally obtained can extract more precise key point positions, further ensuring the recognition accuracy of the head position.

[0095] In an embodiment of the present invention, the driver's head image captured by each positioning camera can be respectively input into the corresponding key point recognition model to output coarse positioning information through the first convolutional network unit in the key point recognition model, and the coarse positioning information includes the first position of each head key point, and then the second position of each head key point is output respectively through the combination of each convolutional layer in the second convolutional network unit of the key point recognition model, and finally the third position of each head key point is output respectively through the combination of each convolutional layer in the second convolutional network unit of the key point recognition model as the position of the head key point in the driver's head image.

[0096] Taking the driver's head image captured by the front-view positioning camera as an example, the process of determining the positions of the key points of the head in the driver's head image based on the key point recognition model is explained. Figure 7 is a schematic diagram of a key point recognition model provided by an embodiment of the present invention, Figure 8 This is a schematic diagram of a first convolutional network unit provided for an embodiment of the present invention.

[0097] like Figure 7 As shown in Figure 2, the key point recognition model consists of three cascaded convolutional network units. The first convolutional network unit consists of three convolutional layers, the second convolutional network unit consists of a combination of five convolutional layers (i.e., 10 convolutional layers), and the third convolutional network unit consists of a combination of five convolutional layers (i.e., 10 convolutional layers).

[0098] Among them, the three convolutional layers of the first convolutional network unit are used to identify the positions of 5 key points (left eye, right eye, nose tip, left lip corner and right lip corner), 3 key points (left eye, right eye, nose tip) and 3 key points (nose tip, left lip corner and right lip corner). The input size of the first convolutional layer is 39*39, the input size of the second convolutional layer is 39*31, and the input size of the third convolutional layer is 39*31. The output of the first convolutional network unit is the average of the outputs of the three convolutional layers.

[0099] The five key points obtained by the first convolutional layer in the first convolutional network unit can be named le1-1, re1-1, n1-1, lm1-1, and rm1-1, respectively representing the left eye, right eye, nose tip, left lip corner, and right lip corner outputted by the first convolutional layer for overall facial recognition; the three key points obtained by the second convolutional layer are named le1-2, re1-2, and n1-2, representing the left eye, right eye, and nose tip outputted by eye and nose recognition; the three key points obtained by the third convolutional layer are named n1-3, lm1-3, and rm1-3, representing the nose tip, left lip corner, and right lip corner outputted by nose and mouth recognition. For the positions of the same type of head key points, the average value is calculated and input to the second convolutional network unit.

[0100] like Figure 8 As shown in the figure, the input layer of the first convolutional network unit uses a 39*39 single-channel gray image, which passes through multiple convolutional layers with pooling layers in the figure, and finally passes through a fully connected layer to output a result with a dimension of 10, representing the coordinate values ​​of 5 key points. The last layer is the Euclidean loss layer, which calculates the accumulation of the mean error between the predicted position and the actual position (both are relative values).

[0101] After the first convolutional network unit outputs the rough positioning information of the driver's head image, further, the first position of each head key point can be used to crop a local image in the driver's head image, and then input into each convolutional layer combination in the second convolutional network unit. For example, the input size of the second convolutional network unit is 15*15, and one convolutional layer combination predicts one head key point, and the prediction results are averaged.

[0102] like Figure 7 As shown, the convolutional layer LE2_1 and the convolutional layer LE2_2 are a combination, and the outputs thereof are averaged to obtain the second position of the corresponding head key point, and the convolutional layer RM2_1 and the convolutional layer RM2_2 are a combination, and the outputs thereof are averaged to obtain the second position of the corresponding head key point.

[0103] Furthermore, after the second convolutional network unit outputs the second position of each head key point in the driver's head image, the second position can be used in the local image cropped in the previous process, and the local image is cropped again, so as to be input into each convolutional layer combination in the third convolutional network unit. For example, the input size of the third convolutional network unit is 15*15, and one convolutional layer combination predicts one head key point, and the prediction results are averaged.

[0104] like Figure 7As shown, the convolutional layer LE3_1 and the convolutional layer LE3_2 are a combination, and the outputs thereof are averaged to obtain the third position of the corresponding head key point, and the convolutional layer RM3_1 and the convolutional layer RM3_2 are a combination, and the outputs thereof are averaged to obtain the third position of the corresponding head key point.

[0105] Finally, according to the third positions of all head key points output by the third convolutional network unit, the positions of the head key points in the driver's head image can be obtained.

[0106] In an embodiment of the present invention, through the above method, the positions of the key points of the head in the driver's head image captured by the front-view positioning camera, the positions of the key points of the head in the driver's head image captured by the side-view positioning camera, and the positions of the key points of the head in the driver's head image captured by the overhead positioning camera can be obtained.

[0107] S140, based on the position of the head key points in each driver's head image, the position of the reference point of the positioning camera in each driver's head image, and the reference point coordinates of each positioning camera, determine the key point coordinates of each head key point in the vehicle coordinate system.

[0108] After obtaining the position of the key point of the head in each driver's head image, the position of the key point of the head in each driver's head image can be converted into a whole vehicle coordinate system to obtain the key point coordinates of the head key point in the whole vehicle coordinate system.

[0109] For coordinate conversion, it is necessary to use the position of the reference point of the positioning camera in each driver's head image and the reference point coordinates of each positioning camera in the vehicle coordinate system.

[0110] In a specific implementation, based on the position of the key points of the head in each driver's head image, the position of the reference point of the positioning camera in each driver's head image, and the reference point coordinates of each positioning camera, the key point coordinates of each head key point in the vehicle coordinate system are determined, including the following steps:

[0111] Step 21: for each driver head image captured by the positioning camera, obtain a pixelated image after the driver head image is processed by the corresponding key point recognition model, and perform pixel decomposition on the pixelated image until the reference point of the positioning camera in the pixelated image occupies a full pixel grid;

[0112] Step 22: for each pixelated image, based on the position of the head key point in the pixelated image and the position of the reference point of each positioning camera, determine the pixel distance between the head key point and the reference point of each positioning camera, and the pixel distance between the reference points of each positioning camera;

[0113] Step 23: Based on the pixel distances and the reference point coordinates of the positioning cameras, determine the key point coordinates of each head key point in the vehicle coordinate system.

[0114] Among them, in step 21, for each driver's head image captured by the positioning camera, a pixelated image processed by the key point recognition model in the process of identifying the position of the head key point can be obtained, and then considering the difference in the size of the pixel grid, in order to achieve accurate coordinate transformation, the pixelated image can be decomposed into pixels until the reference point in the pixelated image can occupy a pixel grid, so that the distance between the head key point and the reference point in the pixelated image can be accurately measured through the pixel grid later.

[0115] Furthermore, in step 22, the pixel distance between the head key point and the reference point of each positioning camera, and the pixel distance between the reference points of each positioning camera can be measured by the position of the head key point in the pixelized image and the position of the reference point of each positioning camera, and the pixel distance between the reference points of each positioning camera. The pixel distance can be represented by the number of pixel points spaced in each direction.

[0116] Furthermore, in step 23, the coordinates of each key point of the head in the driver's head image can be transformed through each pixel distance and the reference point coordinates of each positioning camera to obtain the key point coordinates of each key point of the head in the vehicle coordinate system.

[0117] For the above step 23, optionally, based on the pixel distances and the reference point coordinates of each positioning camera, determining the key point coordinates of each head key point in the vehicle coordinate system includes the following steps:

[0118] Step 231: for the pixelated image corresponding to the front-view positioning camera, based on the pixel distance between the key point of the head and the reference point of the side-view positioning camera or the top-view positioning camera, the pixel distance between the reference point of the side-view positioning camera and the reference point of the top-view positioning camera, and the coordinates of the reference points of the side-view positioning camera and the top-view positioning camera, determine the first coordinate in the Y direction and the first coordinate in the Z direction of the key point of the head in the vehicle coordinate system;

[0119] Step 232: for the pixelated image corresponding to the side-view positioning camera, based on the pixel distance between the key point of the head and the reference point of the front-view positioning camera or the top-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the reference point of the top-view positioning camera, and the coordinates of the reference points of the front-view positioning camera and the top-view positioning camera, determine the first coordinate in the X direction and the second coordinate in the Z direction of the key point of the head in the vehicle coordinate system;

[0120] Step 233: for the pixelated image corresponding to the overhead positioning camera, based on the pixel distance between the head key point in the pixelated image and the reference point of the front-view positioning camera or the side-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the reference point of the side-view positioning camera, and the coordinates of the reference points of the front-view positioning camera and the side-view positioning camera, determine the second coordinate in the X direction and the second coordinate in the Y direction of the head key point in the vehicle coordinate system.

[0121] In step 231, for the pixelated image corresponding to the front-view positioning camera, the first coordinate in the Y direction and the first coordinate in the Z direction of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the side-view positioning camera, the pixel distance between the reference point of the side-view positioning camera and the reference point of the top-view positioning camera, and the reference point coordinates of the side-view positioning camera and the top-view positioning camera. As shown in the following formula:

[0122] ;

[0123] ;

[0124] In the above formula, is the first Y coordinate of the head key point in the vehicle coordinate system, , They are the Y direction values ​​of the reference point coordinates of the side-view positioning camera and the top-view positioning camera, is the number of pixels in the Y direction between the head key point and the reference point of the side-view positioning camera in the pixelated image, It is the number of pixels in the Y direction between the reference points of the side-view positioning camera and the top-view positioning camera in the pixelated image. is the first coordinate of the head key point in the Z direction in the vehicle coordinate system, , are the Z direction values ​​of the reference point coordinates of the side-view positioning camera and the top-view positioning camera, respectively. is the number of pixels in the Z direction between the head key point and the reference point of the side-view positioning camera in the pixelated image, It is the number of pixels in the Z direction between the reference points of the side-view positioning camera and the top-view positioning camera in the pixelated image.

[0125] Alternatively, the first Y-direction coordinate and the first Z-direction coordinate of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the top-view positioning camera, the pixel distance between the reference point of the side-view positioning camera and the top-view positioning camera, and the reference point coordinates of the side-view positioning camera and the top-view positioning camera. As shown in the following formula:

[0126] ;

[0127] ;

[0128] In the formula, The number of pixels in the Y direction between the head key point and the reference point of the top-down positioning camera in the pixelated image. is the number of pixels in the Z direction between the head key point in the pixelated image and the reference point of the top-down positioning camera. Fig. 9 As shown, Fig. 9 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a front-view positioning camera provided by an embodiment of the present invention, wherein the coordinate values ​​of the feature points are the coordinates of the key points of the head in the coordinate system of the entire vehicle.

[0129] In step 232, for the pixelated image corresponding to the side-view positioning camera, the first coordinate in the X direction and the second coordinate in the Z direction of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the front-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the reference point of the top-view positioning camera, and the coordinates of the reference points of the front-view positioning camera and the top-view positioning camera. As shown in the following formula:

[0130] ;

[0131] ;

[0132] In the formula, is the first coordinate of the head key point in the X direction in the vehicle coordinate system, , They are the X-direction values ​​of the reference point coordinates of the front-view positioning camera and the top-view positioning camera, is the number of pixels in the X direction between the head key points in the pixelated image and the reference points of the front-view positioning camera, It is the number of pixels in the X direction between the reference points of the front-view positioning camera and the top-view positioning camera in the pixelated image. is the second coordinate of the head key point in the Z direction in the vehicle coordinate system, , They are the Z direction values ​​of the reference point coordinates of the front-view positioning camera and the top-view positioning camera, is the number of pixels in the Z direction between the head key point and the reference point of the front-view positioning camera in the pixelated image, It is the number of pixels in the Z direction between the reference points of the front-view positioning camera and the top-view positioning camera in the pixelated image.

[0133] Alternatively, the first coordinate in the X direction and the second coordinate in the Z direction of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the top-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the top-view positioning camera, and the reference point coordinates of the front-view positioning camera and the top-view positioning camera. As shown in the following formula:

[0134] ;

[0135] ;

[0136] In the formula, The number of pixels in the X direction between the head key point and the reference point of the top-down positioning camera in the pixelated image. is the number of pixels in the Z direction between the head key point in the pixelated image and the reference point of the top-down positioning camera. Fig.10 As shown, Fig.10 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a side-view positioning camera provided by an embodiment of the present invention, wherein the coordinate values ​​of the feature points are the coordinates of the key points of the head in the coordinate system of the entire vehicle.

[0137] In step 233, for the pixelated image corresponding to the top-view positioning camera, the second coordinate in the X direction and the second coordinate in the Y direction of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the side-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the side-view positioning camera, and the reference point coordinates of the front-view positioning camera and the side-view positioning camera. As shown in the following formula:

[0138] ;

[0139] ;

[0140] In the formula, is the second coordinate of the head key point in the X direction in the vehicle coordinate system, , are the X-direction values ​​of the reference point coordinates of the front-view positioning camera and the side-view positioning camera, is the number of pixels in the X direction between the head key point and the reference point of the side-view positioning camera in the pixelated image, is the number of pixels in the X direction between the reference points of the front-view positioning camera and the side-view positioning camera in the pixelated image. is the second coordinate of the head key point in the Y direction in the vehicle coordinate system, , They are the Y direction values ​​of the reference point coordinates of the front-view positioning camera and the side-view positioning camera, is the number of pixels in the Y direction between the head key point and the reference point of the side-view positioning camera in the pixelated image, It is the number of pixels in the Y direction between the reference points of the front-view positioning camera and the side-view positioning camera in the pixelated image.

[0141] Alternatively, the second coordinate in the X direction and the second coordinate in the Y direction of each head key point in the vehicle coordinate system can be calculated by the pixel distance between the head key point and the reference point of the front view positioning camera, the pixel distance between the reference point of the front view positioning camera and the side view positioning camera, and the reference point coordinates of the front view positioning camera and the side view positioning camera. As shown in the following formula:

[0142] ;

[0143] ;

[0144] In the formula, The number of pixels in the X direction between the head key points in the pixelated image and the reference point of the front-view positioning camera. is the number of pixels in the Y direction between the head key points in the pixelated image and the reference point of the front-view positioning camera. Fig.11 As shown, Fig.11 It is a schematic diagram of calculating the coordinate values ​​of feature points in an image captured by a bird's-eye view positioning camera provided by an embodiment of the present invention, wherein the coordinate values ​​of the feature points are the coordinates of the key points of the head in the coordinate system of the entire vehicle.

[0145] Through the above steps 231 to 233, for each head key point, its two coordinates in the X direction, two coordinates in the Y direction, and two coordinates in the Z direction of the vehicle coordinate system can be obtained, thereby obtaining the key point coordinates of the head key point in the vehicle coordinate system. By using the positional relationship between pixel points in the image and combining the reference point coordinates of the reference point in the vehicle coordinate system, the coordinates of the head key point are transformed, which can ensure the accuracy of the head key point position recognition, and thus ensure the accuracy of subsequent head position recognition.

[0146] S150. Determine the driver's head position information of the vehicle according to the coordinates of each key point, so as to determine the deployment parameters of the airbag in the vehicle based on the driver's head position information.

[0147] Specifically, after obtaining the coordinates of each key point, the position of the driver's head can be further identified to obtain the driver's head position information. In an embodiment of the present invention, the driver's head position information can be represented by the distance between the driver's head and various parts of the cabin; for example, the distance between the driver's head and the window glass, the distance between the center of the steering wheel, and the distance between the driver's head and the roof.

[0148] Considering that in the previous step, the coordinates of each head key point in each direction can be calculated twice, in order to further improve the accuracy of head position recognition, more accurate coordinates can be selected according to actual conditions to determine the driver's head position information.

[0149] Optionally, determining the position information of the driver's head of the vehicle according to the coordinates of each key point includes the following steps:

[0150] Step 31: if it is detected that the front-view positioning camera is not disturbed by the environment, the distance between the driver's head and face and the window glass is determined according to the first coordinate in the Y direction; otherwise, the distance between the driver's head and face and the window glass is determined according to the second coordinate in the Y direction;

[0151] Step 32: if it is detected that the side-view positioning camera is not disturbed by the environment, the distance between the top of the driver's head and the roof is determined according to the second coordinate in the Z direction; otherwise, the distance between the top of the driver's head and the roof is determined according to the first coordinate in the Z direction;

[0152] Step 33: if it is detected that the overhead positioning camera is not disturbed by the environment, the distance between the driver's head and face and the center of the steering wheel is determined according to the second coordinate in the X direction; otherwise, the distance between the driver's head and face and the center of the steering wheel is determined according to the first coordinate in the X direction;

[0153] Step 34: Determine the distance between the driver's head surface and the vehicle window glass, the distance between the top of the head and the roof, and the distance between the head surface and the center of the steering wheel as the driver's head position information.

[0154] Among them, in step 31, if the front-view positioning camera is not affected by environmental interference (such as dim light, the camera is blocked, or the camera is damaged, etc.), it means that the image collected by the front-view positioning camera has a high reliability. At this time, the distance between the driver's head and face and the vehicle window glass can be determined based on the first coordinate in the Y direction determined based on the image collected by the front-view positioning camera; if the front-view positioning camera is affected by environmental interference, it means that the image collected by the front-view positioning camera has a low reliability. The distance between the driver's head and face and the vehicle window glass can be determined based on the second coordinate in the Y direction determined based on the image collected by the overhead positioning camera.

[0155] In step 32, if the side-view positioning camera is not affected by environmental interference (such as dim light, the camera is blocked, or the camera is damaged, etc.), it means that the image collected by the side-view positioning camera has a high reliability. At this time, the distance between the top of the driver's head and the roof can be determined based on the second coordinate in the Z direction determined based on the image collected by the side-view positioning camera; if the side-view positioning camera is affected by environmental interference, it means that the image collected by the side-view positioning camera has a low reliability. At this time, the distance between the top of the driver's head and the roof can be determined based on the first coordinate in the Z direction determined based on the image collected by the front-view positioning camera.

[0156] In step 33, if the overhead positioning camera is not disturbed by the environment (such as dim light, the camera is blocked, or the camera is damaged, etc.), it means that the image collected by the overhead positioning camera has high reliability. At this time, the distance between the driver's head and face and the center of the steering wheel can be determined based on the second coordinate in the X direction determined based on the image collected by the overhead positioning camera; if the overhead positioning camera is disturbed by the environment, it means that the image collected by the overhead positioning camera has low reliability. At this time, the distance between the driver's head and face and the center of the steering wheel can be determined based on the first coordinate in the X direction determined based on the image collected by the side positioning camera.

[0157] Further, in step 34, the distance between the driver's head surface and the window glass, the distance between the top of the head and the roof, and the distance between the head surface and the center of the steering wheel can be used to describe the driver's head position information.

[0158] Through the above steps 31 to 34, more reliable coordinates can be selected in combination with actual conditions to determine the driver's head position information, thereby further ensuring the accuracy and reliability of the driver's head position recognition.

[0159] In the embodiment of the present invention, after the driver's head position information is determined, the deployment parameters of the airbag can be determined according to the driver's head position information.

[0160] For example, at least one of the detonation time and detonation area of ​​the airbag can be determined based on the driver's head position information and the collision type. The detonation time can describe the time at which the airbag is controlled to be deployed, and the detonation area can describe which part of the airbag is controlled to be deployed. The collision type can be determined based on the predicted collision point between the vehicle and the obstacle, such as a frontal collision, a side collision, a rear-end collision, a 25% offset collision, etc.

[0161] Through the method provided by the embodiment of the present invention, the head key points and camera reference points of the driver under three perspectives can be collected simultaneously. The positions of the head key points can be obtained by identifying the collected images through the model, thereby realizing real-time, accurate and fast recognition of the driver's head key points. Furthermore, after the collected images are pixelated, the coordinates of the head key points in the whole vehicle coordinate system can be obtained by proportional conversion based on the coordinates of the reference points, which is convenient for measuring the position of the driver's head in the cockpit, providing a basis for the correct and effective deployment of the airbag, ensuring the accurate and effective deployment of the airbag, and reducing the driver's head injury.

[0162] The present invention has the following technical effects: by acquiring the reference point coordinates of the reference point of each positioning camera in the vehicle in the whole vehicle coordinate system, and then in response to detecting that the vehicle has a collision risk, each positioning camera respectively collects the driver's head image of the vehicle, and for each driver's head image, it is input into the corresponding pre-trained key point recognition model to obtain the position of the head key point, so as to determine the key point coordinates of each head key point in the whole vehicle coordinate system according to the position of the head key point in each driver's head image and the position of the reference point of the positioning camera, as well as the reference point coordinates of each positioning camera, and finally determine the driver's head position information according to the coordinates of each key point, so as to determine the deployment parameters of the airbag through the driver's head position information, realize timely and targeted adjustment of the deployment parameters of the airbag, ensure that the airbag can be effectively deployed, avoid the situation where the driver's head cannot be effectively protected by the airbag due to changes in the driver's posture, and reduce the driver's head injury.

[0163] Fig.12 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig.12 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0164] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0165] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may run the program instructions to implement the driver head position determination method based on image recognition and / or other desired functions of any embodiment of the present invention described above. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0166] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including early warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0167] Of course, to simplify, Fig.12 Only some of the components related to the present invention in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0168] In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the driver head position determination method based on image recognition provided by any embodiment of the present invention.

[0169] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0170] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for determining the driver's head position based on image recognition provided by any embodiment of the present invention.

[0171] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0172] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0173] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the driver's head position based on image recognition, It is characterized in that include: Obtaining the coordinates of the reference points of the reference points of each positioning camera in the vehicle in the vehicle coordinate system, wherein the positioning camera is placed according to the driver's position in the vehicle, and each positioning camera includes a front-view positioning camera, a side-view positioning camera, and a top-view positioning camera; In response to detecting that the vehicle has a collision risk, collecting a head image of a driver in the vehicle based on each positioning camera; For each driver head image captured by the positioning camera, the driver head image is input into a corresponding trained key point recognition model to determine the position of the head key points in the driver head image; For each driver head image captured by the positioning camera, a pixelated image after the driver head image is processed by the corresponding key point recognition model is obtained, and pixel decomposition is performed on the pixelated image until the reference point of the positioning camera in the pixelated image occupies a full pixel grid; For each pixelated image, based on the position of the head key point in the pixelated image and the position of the reference point of each positioning camera, determine the pixel distance between the head key point and the reference point of each positioning camera, and the pixel distance between the reference points of each positioning camera; For the pixelated image corresponding to the front-view positioning camera, based on the pixel distance between the key point of the head and the reference point of the side-view positioning camera or the top-view positioning camera, the pixel distance between the reference point of the side-view positioning camera and the reference point of the top-view positioning camera, and the coordinates of the reference points of the side-view positioning camera and the top-view positioning camera, determine the first coordinate in the Y direction and the first coordinate in the Z direction of the key point of the head in the vehicle coordinate system; For the pixelated image corresponding to the side-view positioning camera, based on the pixel distance between the key point of the head and the reference point of the front-view positioning camera or the top-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the reference point of the top-view positioning camera, and the coordinates of the reference points of the front-view positioning camera and the top-view positioning camera, determine the first coordinate in the X direction and the second coordinate in the Z direction of the key point of the head in the vehicle coordinate system; For the pixelated image corresponding to the top-view positioning camera, based on the pixel distance between the key point of the head and the reference point of the front-view positioning camera or the side-view positioning camera, the pixel distance between the reference point of the front-view positioning camera and the reference point of the side-view positioning camera, and the coordinates of the reference points of the front-view positioning camera and the side-view positioning camera, determine the second coordinate in the X direction and the second coordinate in the Y direction of the key point of the head in the vehicle coordinate system; Determining the driver's head position information of the vehicle according to the coordinates of each of the key points, so as to determine the deployment parameters of the airbag in the vehicle based on the driver's head position information; The training step of the key point recognition model corresponding to the positioning camera includes: Acquire an initial sample set and construct a model to be trained, wherein the initial sample set includes sample head images of the driver in different postures collected by a positioning camera and corresponding actual key point positions, and the model to be trained includes a first convolutional network unit, a second convolutional network unit, and a third convolutional network unit; Training the first convolutional network unit based on the initial sample set; After the first convolutional network unit is trained, for each head key point, taking the position of the head key point in the predicted key point position output by the first convolutional network unit as the center, determining a corresponding first local frame in the sample head image according to two preset first sizes, and moving each first local frame in the sample head image to obtain first local frame images corresponding to two preset first sizes, constructing two first local sample sets for the head key point, and training the second convolutional network unit based on the first local sample sets; After the training of the second convolutional network unit is completed, based on the predicted key point positions corresponding to the first local frame images output by the second convolutional network unit, a second local frame image is selected from each of the first local frame images to construct a second local sample set, and the third convolutional network unit is trained based on the second local sample set; After the training of the third convolutional network unit is completed, all convolutional network units in the model to be trained are trained based on the initial sample set to obtain a key point recognition model.

2. The method according to claim 1, It is characterized in that The first convolutional network unit includes a first convolutional layer, a second convolutional layer and a third convolutional layer, and training the first convolutional network unit based on the initial sample set includes: Inputting the sample head images in the initial sample set into the first convolution layer, the second convolution layer and the third convolution layer respectively; Determine the predicted key point positions corresponding to the sample head image based on all key point positions output by the first convolution layer, some key point positions output by the second convolution layer, and some key point positions output by the third convolution layer; A loss value is determined according to the actual key point position and the predicted key point position corresponding to each of the sample head images, and parameters of the first convolution layer, the second convolution layer, and the third convolution layer are adjusted based on the loss value until the loss value converges.

3. The method according to claim 1, It is characterized in that The second convolutional network unit includes a convolutional layer combination corresponding to each head key point, the convolutional layer combination is composed of two convolutional layers, and the second convolutional network unit is trained based on the first local sample set, including: For each convolutional layer combination in the second convolutional network unit, first local frame images in two first local sample sets corresponding to head key points are respectively input into two convolutional layers of the convolutional layer combination; Determine a predicted key point position corresponding to the first local frame image based on a mean value between the positions of the head key points output by the two convolutional layers; A loss value is determined according to the actual key point position and the predicted key point position corresponding to each of the first local frame images, and the parameters of the convolutional layer combination are adjusted based on the loss value until the loss value converges.

4. The method according to claim 1, It is characterized in that Determining the driver's head position information of the vehicle according to the coordinates of each key point includes: If it is detected that the front-view positioning camera is not disturbed by the environment, the distance between the driver's head and face and the vehicle window glass is determined according to the first coordinate in the Y direction; otherwise, the distance between the driver's head and face and the vehicle window glass is determined according to the second coordinate in the Y direction; If it is detected that the side-view positioning camera is not disturbed by the environment, the distance between the top of the driver's head and the roof is determined according to the second coordinate in the Z direction; otherwise, the distance between the top of the driver's head and the roof is determined according to the first coordinate in the Z direction; If it is detected that the overhead positioning camera is not disturbed by the environment, the distance between the driver's head and face and the center of the steering wheel is determined according to the second coordinate in the X direction; otherwise, the distance between the driver's head and face and the center of the steering wheel is determined according to the first coordinate in the X direction; The distance between the driver's head surface and the window glass, the distance between the top of the head and the roof, and the distance between the head surface and the center of the steering wheel are determined as the driver's head position information.

5. An electronic device, It is characterized in that The electronic device comprises: Processor and memory; The processor is used to execute the steps of the method for determining the driver's head position based on image recognition as described in any one of claims 1 to 4 by calling the program or instructions stored in the memory.

6. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the method for determining the driver's head position based on image recognition as described in any one of claims 1 to 4.