Face-based in-vehicle control method, system, device and storage medium thereof

Abstract

The application provides a face-based vehicle control method, comprising: obtaining first face information through a face detection network; obtaining age information through an age recognition network, and determining the identity of a passenger according to the age information; after determining that the identity of the passenger is a baby, collecting monitoring data of the baby in real time, obtaining second face information according to the monitoring data; detecting the second face information through the face detection network to obtain third face information, detecting the third face information through a key point detection network to obtain a baby posture; and controlling a car player or a car component according to the baby posture to stabilize the mood and state of the baby. The baby posture can be monitored and detected in real time, and the car player or the car component can be controlled according to the baby posture, so that the mood and state of the baby are stabilized, and the safety of the baby and driving is improved. The application also provides a system, a device and a storage medium for implementing the face-based vehicle control method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and in particular to a face-based vehicle control method, system, device and storage medium. BACKGROUND

[0002] With the increase of cars, the safety problem of babies in cars is getting more and more attention.

[0003] Chinese patent CN 208248020 U discloses a new rotating mechanism for an automobile baby safety seat, comprising: a locking mechanism and a second welding group; the locking mechanism is correspondingly provided in parallel with two groups; the locking mechanism is connected through the second welding group; the locking mechanism comprises a right shell, a left shell, a tension spring, a left slide plate, a right slide plate, a button spring, a release lever, a lock hook torsional spring and a lock hook; a slide button is arranged on the left slide plate; a lock button indicator is attached to the slide button; the left slide plate and the right slide plate are wrapped around the two sides of the button spring; the lock hook torsional spring is connected with the lock hook; the lock hook and the lock hook torsional spring are arranged on one side of the left slide plate; the release lever is arranged on the side of the lock hook away from the left slide plate; the tension spring is arranged on the right shell; the left shell and the right shell are wrapped around the two sides of the tension spring, the left slide plate, the right slide plate, the button spring, the release lever, the lock hook torsional spring and the lock hook. The overall stability and safety of the automobile baby safety seat are improved. However, the posture and emotion of the baby are not considered, and when the baby's emotion or posture is unstable, not only the safety of the baby in the car is affected, but also the driver may be indirectly affected.

[0004] Therefore, it is necessary to provide a face-based vehicle control method, system, device and storage medium to solve the problems existing in the prior art. SUMMARY

[0005] The purpose of the present application is to provide a face-based vehicle control method, system, device and storage medium to solve the safety problem caused by the unstable emotion or posture of the baby in the car.

[0006] To achieve the above purpose, the face-based vehicle control method of the present application comprises the following steps:

[0007] Detecting passenger face information through a face detection network to obtain first face information, and detecting the first face information through an age recognition network to obtain age information;

[0008] Determining the identity of the passenger according to the age information;

[0009] After determining that the passenger is a baby, real-time monitoring data of the baby is collected, and second face information is obtained according to the monitoring data;

[0010] detecting the second face information through the face detection network to obtain third face information, and detecting the third face information through a key point detection network to obtain a baby posture;

[0011] controlling a car player or a car component according to the baby posture to stabilize the mood and state of the baby.

[0012] The face-based vehicle control method has the following advantages:

[0013] detecting passenger face information through a face detection network to obtain first face information, detecting the first face information through an age recognition network to obtain age information, determining the identity of the passenger according to the age information, collecting monitoring data of the baby in real time after determining that the passenger is a baby, realizing monitoring of the baby, obtaining second face information according to the monitoring data of the baby, detecting the second face information through the face detection network to obtain third face information, and detecting the third face information through a key point detection network to obtain a baby posture; controlling a car player or a car component according to the baby posture to stabilize the mood and state of the baby; the monitoring data of the baby can be collected in real time and the baby posture can be obtained, and the car player or the car component is controlled according to the baby posture, without manual adjustment of the player and the car component by the driver, saving the time and effort of manual control by the driver, ensuring the driving safety of the driver, realizing monitoring and adjustment of the mood and state of the baby, and ensuring the safety of the baby and driving.

[0014] Optionally, the step of detecting passenger face information through a face detection network to obtain first face information includes:

[0015] adjusting the resolution of the passenger face information to obtain a first image, and feeding the first image to the face detection network; the face detection network outputs first face classification information, first face frame regression information and first key point regression information after detecting the first image.

[0016] cropping the first face frame regression information to obtain a first face image, and adjusting the resolution of the first face image to obtain the first face information.

[0017] Optionally, the step of detecting the second face information through the face detection network to obtain third face information includes:

[0018] adjusting the resolution of the second face information to obtain a second image, and feeding the second image to the face detection network; the face detection network outputs second face classification information, second face frame regression information and second key point regression information after detecting the second image.

[0019] cropping the second face frame regression information to obtain a second face image, and adjusting a resolution of the second face image to obtain the third face information.

[0020] Optionally, the step of obtaining the second face information according to the monitoring data comprises:

[0021] cropping a last frame of the monitoring data to obtain the second face information.

[0022] Optionally, the step of determining the identity of the passenger according to the age information comprises:

[0023] when the age information of the passenger does not exceed the infant age threshold, determining that the identity of the passenger is an infant.

[0024] Optionally, the step of detecting the third face information by the key point detection network to obtain the infant posture comprises:

[0025] the key point detection network outputs face key point information and head posture angle information after detecting the third face information;

[0026] obtaining the posture of the infant according to any one of the face key point information and the head posture angle information. Its beneficial effect lies in that the key point detection network outputs face key point information and head posture angle information after detecting the third face information, and the posture of the infant is obtained according to any one of the face key point information and the head posture angle information, which provides multiple ways for obtaining the posture of the infant, facilitates the detection of multiple infant postures, improves the accuracy of obtaining the posture of the infant, and improves the applicability of the application.

[0027] Optionally, the face key point information comprises a first key point located in a left eyebrow region, a second key point located in a right eyebrow region, a third key point located in a left eye upper eyelid region, a fourth key point located in a left eye lower eyelid region, a fifth key point located in a right eye upper eyelid region, and a sixth key point located in a right eye lower eyelid region.

[0028] Optionally, the step of obtaining the posture of the infant according to any one of the face key point information and the head posture angle information comprises:

[0029] performing distance calculation on the first key point and the second key point to obtain a first distance;

[0030] determining the state of the infant according to a comparison of the first distance with a preset first distance threshold. Its beneficial effect lies in that the comparison of the distance between the first key point located in the left eyebrow region and the second key point located in the right eyebrow region with the first distance threshold realizes the determination of the state of the infant.

[0031] Optionally, the step of determining the state of the baby according to the comparison of the first distance with a preset first distance threshold comprises:

[0032] When the first distance is greater than or equal to the first distance threshold, it is determined that the baby posture is in a stable state. Its beneficial effect lies in that the determination of the stable state of the baby is realized.

[0033] Optionally, the step of determining the state of the baby according to the comparison of the first distance with a preset first distance threshold comprises:

[0034] When the first distance is less than the first distance threshold, a duration that the first distance is less than the first distance threshold is recorded to obtain a first duration;

[0035] When the first duration is less than a preset first time threshold, it is determined that the baby posture is in a stable state. Its beneficial effect lies in that the determination of the stable state of the baby is realized.

[0036] Optionally, the step of determining the state of the baby according to the comparison of the first distance with a preset first distance threshold further comprises:

[0037] When the first duration is greater than or equal to the first time threshold, it is determined that the baby posture is in a fussy state. Its beneficial effect lies in that the determination of the fussy state of the baby is realized.

[0038] Optionally, the step of determining the state of the baby according to any one of the face key point information and the head posture angle information comprises:

[0039] A distance calculation is performed on the third key point and the fourth key point to obtain a second distance, and a distance calculation is performed on the fifth key point and the sixth key point to obtain a third distance;

[0040] The state of the baby is determined according to the comparison of the second distance and the third distance with a preset second distance threshold. Its beneficial effect lies in that the determination of the state of the baby is realized through the comparison of the second distance and the third distance with the second distance threshold.

[0041] Optionally, the step of determining the state of the baby according to the comparison of the second distance and the third distance with a preset second distance threshold comprises:

[0042] When any one of the second distance and the third distance is greater than or equal to the second distance threshold, it is determined that the baby posture is in a non-sleep state. Its beneficial effect lies in that the determination of the non-sleep state of the baby is realized.

[0043] Optionally, the step of determining the state of the baby according to the comparison of the second distance and the third distance with a preset second distance threshold comprises:

[0044] when the second distance and the third distance are both less than the second distance threshold, recording a duration that the second distance and the third distance are both less than or equal to the second distance threshold to obtain a second duration;

[0045] when the second duration is less than a preset second time threshold, determining that the baby posture is in a non-sleep state. The beneficial effect is that the non-sleep state of the baby is determined.

[0046] Optionally, the step of determining the state of the baby according to the comparison of the second distance and the third distance with a preset second distance threshold comprises:

[0047] when the second duration is greater than or equal to the second time threshold, determining that the baby posture is in a sleep state. The beneficial effect is that the sleep state of the baby is determined.

[0048] Optionally, the step of determining the state of the baby according to any one of the face key point information and the head posture angle information comprises:

[0049] obtaining a head pitch angle according to the head posture angle information;

[0050] when the head pitch angle is less than a preset posture angle threshold, recording a duration that the head pitch angle is less than the posture angle threshold to obtain a third duration;

[0051] when the third duration is greater than or equal to a preset third time threshold, determining that the baby posture is in a sleep state. The beneficial effect is that the state of the baby is determined by comparing the head pitch angle and the posture angle threshold and comparing the third duration with the third time threshold, and the sleep state of the baby is detected and confirmed.

[0052] Optionally, the step of controlling the car player or car components according to the baby posture comprises:

[0053] when it is determined that the baby posture is in a fretful state, performing a play control step to make the player play a video or music;

[0054] when it is determined that the baby posture is in a sleep state, performing a stop control step to make the player stop playing a video or music;

[0055] When it is determined that the infant posture is the stable state, the duration of the stable state is recorded to obtain a stable duration, and when the stable duration is greater than or equal to a preset stable time threshold, a stop control step is performed to stop the player from playing the video or the music. Its beneficial effect lies in that, after the infant posture is determined to be the restlessness state and the wakeful state, the video music is played to make the infant relax and stabilize the mood of the infant; after the infant posture is determined to be the sleep state, the playing of the video or the music is stopped, so as to avoid disturbing the infant and keep the infant posture stable, thereby realizing the stabilization of the mood and state of the infant in various infant postures; after the infant posture is determined to be the stable state and the stable duration is greater than the stable time threshold, the playing of the video or the music is stopped, thereby realizing the stabilizing effect on the state and mood of the infant.

[0056] Optionally, the step of controlling the player or the component of the automobile according to the infant posture further comprises:

[0057] When the infant posture is the restlessness state, a seat adjustment step is performed to adjust the seat of the infant until the infant posture becomes the stable state. Its beneficial effect lies in that, the restlessness state is relieved by adjusting the seat of the infant, the comfort of the infant is improved on the premise of ensuring the safety of the infant, and the stabilization of the mood and state of the infant is further realized.

[0058] Optionally, after the identity of the passenger is determined to be an infant, the following step is further performed:

[0059] A reminding sign step is performed to display that there is an infant in the vehicle. Its beneficial effect lies in that, after there is an infant passenger in the vehicle, the person outside the vehicle is reminded that there is an infant in the vehicle, the possibility of disturbing the infant by the person outside the vehicle is reduced, and the safety of the infant is improved.

[0060] Optionally, after the infant posture is obtained by detecting the third facial information through the key point detection network, the following step is further performed:

[0061] When the infant posture changes, the monitoring data of the infant of the infant is displayed on the display screen of the driver's cabin. Its beneficial effect lies in that, the state of the infant is obtained by displaying the monitoring data of the infant of the infant in time, and the driver can understand the state of the infant in time to take corresponding measures to protect the safety of the infant.

[0062] The application further provides a vehicle-mounted control system based on a face, comprising:

[0063] A first acquisition module is configured to acquire passenger facial information.

[0064] A face detection network is configured to detect the passenger facial information to obtain first facial information, and detect second facial information to obtain third facial information after determining that the identity of the passenger is an infant.

[0065] An age recognition network is configured to detect the first face information to obtain age information.

[0066] An identity recognition module is configured to determine the identity of the passenger according to the age information.

[0067] A second acquisition module is configured to acquire monitoring data of the baby in real time after determining that the identity of the passenger is a baby, and obtain second face information according to the monitoring data.

[0068] A key point detection network is configured to detect the third face information to obtain a baby posture.

[0069] A control module is configured to control a car player or a car component according to the baby posture, so as to stabilize the mood and state of the baby.

[0070] The face-based vehicle control system has the following advantages:

[0071] The first acquisition module is configured to acquire passenger face information, and the face detection network is configured to detect the passenger face information to obtain first face information; the age recognition network is configured to detect the first face information to obtain age information; the identity recognition module is configured to determine the identity of the passenger; the face image acquisition module is configured to acquire monitoring data of the baby in real time to monitor the state of the baby in real time; the key point detection network is configured to obtain a baby posture; and the control module is configured to control a car player or a car component to stabilize the mood and state of the baby, so as to ensure the safety of the baby, stabilize the mood and state of the baby, and improve the comfort and safety of the baby when riding in a car.

[0072] The face-based vehicle control method is also provided as a program stored in a memory and executable on a processor.

[0073] The face-based vehicle control method is also provided as a computer program stored in a storage medium and executable on a processor. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 A flowchart of the face-based vehicle control method is shown.

[0075] Figure 2 A position diagram of face key points output by the posture detection network is shown.

[0076] Figure 3 A flowchart of a first method for determining a baby posture is shown.

[0077] Figure 4 A second method flow chart for determining the posture of the baby according to an embodiment of the present application;

[0078] Figure 5 A third method flow chart for determining the posture of the baby according to an embodiment of the present application;

[0079] Figure 6 A structure schematic diagram of the vehicle-mounted control system based on the face according to an embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the common meanings thereof by those of ordinary skill in the art. The similar words such as “comprise” used herein mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, and do not exclude other elements or objects.

[0081] In view of the problems in the prior art, the embodiments of the present application provide a vehicle-mounted control method based on a face, Figure 1 A flow chart of the vehicle-mounted control method based on the face according to an embodiment of the present application.

[0082] Reference Figure 1 , the vehicle-mounted control method based on the face comprises the steps of:

[0083] S1: detecting passenger face information through a face detection network to obtain first face information, and detecting the first face information through an age recognition network to obtain age information;

[0084] S2: determining the identity of the passenger according to the age information;

[0085] S3: after determining that the identity of the passenger is a baby, collecting monitoring data of the baby in real time, and obtaining second face information according to the monitoring data;

[0086] S4: detecting the second face information through the face detection network to obtain third face information, and detecting the third face information through a key point detection network to obtain a baby posture;

[0087] S5: controlling a car player or a car component according to the baby posture to stabilize the mood and state of the baby.

[0088] The face-based vehicle control method has the advantages that: the passenger face information is detected by the face detection network to obtain first face information, the first face information is detected by the age recognition network to obtain age information, and the identity of the passenger is determined according to the age information; after the identity of the passenger is determined as a baby, monitoring data of the baby is collected in real time, and monitoring of the baby is realized; second face information is obtained according to the monitoring data of the baby, the second face information is detected by the face detection network to obtain third face information, and the third face information is detected by the key point detection network to obtain a baby posture; the car player or the car part is controlled according to the baby posture, so as to stabilize the mood and state of the baby; the monitoring data of the baby can be collected in real time and the baby posture can be obtained, and the car player or the car part is controlled according to the baby posture, so that the driver does not need to manually adjust the player and the car part, the time and effort of manual control of the driver are saved, the driving safety of the driver is ensured, the monitoring and adjustment of the mood and state of the baby are realized, and the safety of the driver and the baby in the car is ensured.

[0089] In some embodiments, a surround view camera is arranged in the car, and the face of the passenger is photographed online by the surround view camera to obtain the passenger face information and the monitoring data of the baby.

[0090] In some embodiments, before the step S1, the method further comprises a step S0:

[0091] The to-be-trained base network is trained in batches to obtain the face detection network, the age recognition network and the key point detection network, specifically including the following two steps:

[0092] (1) training a first base network to obtain the face detection network:

[0093] The face detection Mnet network is provided as the first base network, and the face detection Mnet network is an improved version based on the face recognition Retinaface network. The difference between the face detection Mnet network and the face recognition Retinaface network is that the backbone network of the face detection Mnet network uses the mobilenet to make the model lightweight, and an SSH detection module composed of an SSH (SingleStage Headless) context module is added at the end of the detection head position; the size of the input of the face detection Mnet network is a 3-channel RGB image, and the output of the face detection Mnet network includes three parts: face classification information, face frame regression information and a plurality of face key point regression information;

[0094] In the caffe framework, the face detection Mnet network is trained: 204,000 images are input into the face detection Mnet network as a training data set and 3,500 images are input into the face detection Mnet network as a verification set, and the face detection Mnet network is trained by using a Stochastic Gradient Descent (SGD) optimizer, and the training parameters are set as follows: the momentum of the SGD optimizer is 0.9, the weights decay is 0.0005, the batch size is 8x4, the initial learning rate is 0.001, the learning rate is modified to 0.01 after 5 epochs, and finally the modified rate is modified to 0.001 at the 55th epoch and the 68th epoch, and the face detection Mnet network after training is the face detection network.

[0095] It can be explained that when all the training data sets are input into the network model, a complete training is performed, and this process is called an epoch.

[0096] (2) Training the second basic network to obtain the key point detection network:

[0097] The face key point detection landmark network is used as the second basic network, and the main structure of the face key point detection landmark network is composed of a depthwise network and a resnet network structure, and the parameter amount of the network is reduced;

[0098] The first face image output by the first network is adjusted to have an image resolution of 112x112, and a second face image is obtained, and the second face image is input into the face key point detection landmark network for training from the beginning, and the training parameters include:

[0099] 25.8 million face data are used as a training data set, and 4.2 thousand data are used as a verification data set; the face key point detection landmark network is trained by using an SGD optimizer, the momentum of the SGD optimizer is 0.9, the weights decay is 0.0005, the batch size is 16x4, the initial learning rate is 0.001, the learning rate is modified to 0.01 after 5 epochs, and finally the modified rate is modified to 0.001 at the 38th and 56th epochs;

[0100] The face key point detection landmark network after training is the key point detection network in the embodiment of the application;

[0101] The output of the key point detection network includes two parts: 98 key point information on the face and head pose angle information, the head pose angle information including yaw, pitch and roll, wherein the yaw can represent the angle of the left and right head of the face, the pitch can represent the angle of the low head or the raised head of the face, and the roll can represent the angle of the twisted head of the face.

[0102] It can be explained that the training step of the age recognition network of the present application is similar to the training step of the key point detection network, and those skilled in the art can train the to-be-trained basic network according to the above training step to obtain the age recognition network, which will not be described here.

[0103] In some embodiments, in the step S1, the step of detecting the passenger face information by the face detection network to obtain the first face information includes: adjusting the resolution of the passenger face information to obtain a first image, inputting the first image into the face detection network, and outputting first face classification information, first face frame regression information and first key point regression information after the face detection network detects the first image; performing cropping processing on the first face frame regression information to obtain a first face image, and adjusting the resolution of the first face image to obtain the first face information.

[0104] In some specific embodiments, in the step S1, the steps of detecting the passenger face information by the face detection network to obtain the first face information and detecting the first face information by the age recognition network to obtain the age information include:

[0105] The current frame image of the passenger is captured by the surround view camera, the current frame image of the passenger is adjusted to an image with a resolution of 320*320 and input into the face detection network, and the face detection network outputs a face frame rectbox and a plurality of face key points;

[0106] The first face image is cropped from the face frame rectbox, and the first face image is adjusted to a second face image faceimg with a resolution of 112*112, and the second face image faceimg is input into the age recognition network, and the age recognition network detects the second face image faceimg to obtain the age information of the passenger.

[0107] As an optional embodiment of the present application, in the step S2, the step of determining the identity of the passenger according to the age information includes:

[0108] When it is judged that the age information of the passenger does not exceed the infant age threshold, it is determined that the identity of the passenger is an infant.

[0109] When it is determined that the age information of the passenger is greater than the infant age threshold, it is determined that the passenger is not an infant, and the steps S3 to S5 are ended.

[0110] In some embodiments, the infant age threshold is 2-6 years old.

[0111] As an optional embodiment of the present application, after it is determined in step S3 that the passenger is an infant, the following steps are further performed:

[0112] The reminding identifier step is performed to display that there is an infant in the vehicle. The advantage is that the person outside the vehicle is reminded that there is an infant in the vehicle, the possibility of disturbing the infant by the person outside the vehicle is reduced, and the safety of the infant is improved to a certain extent.

[0113] In some embodiments, a reminding identifier module is arranged on the outside of the vehicle, such as the rear windshield of the vehicle, and the reminding identifier module can be any one of an LED font lamp, an LED display screen, or other display devices. After it is determined that the passenger is an infant, the reminding identifier module displays an infant word identifier or a graphic identifier to remind the person behind the vehicle that there is an infant in the vehicle, to remind the person behind the vehicle to slow down or try to reduce honking, to reduce the possibility of disturbing or danger to the infant, and to improve the safety and comfort of the infant in the vehicle.

[0114] As an optional embodiment of the present application, in step S3, the step of obtaining second face information according to the monitoring data comprises:

[0115] The last frame image of the monitoring data is intercepted to obtain the second face information. Here, the interception time can be set, such as intercepting the last frame image of the monitoring data every 3s-10s to update the second face information, so as to timely detect the state of the infant. Since intercepting the last frame image of the video is a conventional technique in the art, it will not be described here.

[0116] As an optional embodiment of the present application, in step S4, the step of detecting the second face information by the face detection network to obtain third face information comprises:

[0117] The resolution of the second face information is adjusted to obtain a second image, and the second image is fed to the face detection network. After the face detection network detects the second image, second face classification information, second face frame regression information, and second key point regression information are output.

[0118] The second face frame regression information is cropped to obtain a second face image, and the resolution of the second face image is adjusted to obtain the third face information.

[0119] As an optional implementation of the present application, in the step S4, the step of detecting the third face information by the key point detection network to obtain the infant posture comprises:

[0120] The key point detection network outputs face key point information and head posture angle information after detecting the third face information;

[0121] The infant posture is obtained according to any one of the face key point information and the head posture angle information. Its advantage lies in that the key point detection network outputs face key point information and head posture angle information after detecting the third face information, and the infant posture is obtained according to any one of the face key point information and the head posture angle information, which provides multiple ways for obtaining the infant posture, facilitates the detection of multiple infant postures, improves the accuracy of obtaining the infant posture, and improves the applicability of the present application.

[0122] In some embodiments, the step of outputting face key point information and head posture angle information by the key point detection network after detecting the third face information comprises:

[0123] The monitoring data of the infant is captured by the surround view camera, and the second face information in the monitoring data is obtained. The second face information is adjusted to a first image with a resolution of 320x320 and input into the face detection network. The face detection network outputs a face frame rectbox and a plurality of face key points;

[0124] A second image is cropped from the face frame rectbox, and the second image is adjusted to a third image faceimg with a resolution of 112x112. The fourth face image faceimg is input into the key point detection network. The key point detection network outputs 98 face key point information and head posture angle information.

[0125] Figure 2 The position diagram of the face key point output by the posture detection network of the embodiment of the present application is shown in the following figure, in which the positions of part of the 98 face key points output by the posture detection network are shown as Figure 2 .

[0126] As an optional implementation of the present application, the face key point information comprises a first key point in the left eyebrow region, a second key point in the right eyebrow region, a third key point in the left eye upper eyelid region, a fourth key point in the left eye lower eyelid region, a fifth key point in the right eye upper eyelid region, and a sixth key point in the right eye lower eyelid region.

[0127] Figure 3 The flow chart of the first method for determining the infant posture of the embodiment of the present application is shown in the following figure.

[0128] Referring toFigure 3 In the first method for determining the posture of the baby in the embodiment of the present application, the step of obtaining the posture of the baby according to any one of the face key point information and the head posture angle information comprises:

[0129] performing distance calculation on the first key point and the second key point to obtain a first distance, and determining the state of the baby according to comparison of the first distance with a preset first distance threshold value, and the specific determination steps are as follows:

[0130] when the first distance is greater than or equal to the first distance threshold value, determining that the posture of the baby is a stable state;

[0131] when the first distance is less than the first distance threshold value, recording a duration for which the first distance is less than the first distance threshold value to obtain a first duration, and when the first duration is less than a preset first time threshold value, determining that the posture of the baby is a stable state;

[0132] when the first duration is greater than or equal to the first time threshold value, determining that the posture of the baby is a fretful state.

[0133] Figure 3 a flowchart of the first method for determining the posture of the baby in the embodiment of the present application.

[0134] In some specific embodiments, with reference to Figure 3 In the second method for determining the posture of the baby in the embodiment of the present application, the step of determining the posture of the baby according to any one of the face key point information and the head posture angle information comprises:

[0135] S411: performing distance calculation on the first key point and the second key point to obtain a first distance;

[0136] S412: determining whether the first distance is less than the first distance threshold value;

[0137] if not, performing step S413: determining that the posture of the baby is a stable state, and ending the detection;

[0138] if yes, performing step S414: recording a duration for which the first distance is less than the first distance threshold value to obtain a first duration;

[0139] S415: determining whether the first duration is greater than or equal to the first time threshold value;

[0140] if not, performing step S413;

[0141] if yes, performing step S416: determining that the posture of the baby is a fretful state, and ending the detection.

[0142] The advantages of the steps S411 to S416 are that the infant posture is determined to be in the state of fussing by comparing the distance between the first key point of the left eyebrow region and the second key point of the right eyebrow region with the first distance threshold value, and comparing the first duration with the first time threshold value, so that the determination of the infant posture is efficient and accurate.

[0143] In some embodiments, the first distance threshold value is 0.2-1.0 cm, and the first time threshold value is 2-10 s.

[0144] In some specific embodiments, the steps S411 to S416 specifically include:

[0145] The nose bridge center point 100 is determined, and the distance between the first key point 101 of the left eyebrow region and the second key point 102 of the right eyebrow region is calculated to obtain the first distance. The first key point 101 and the second key point 102 are symmetric about the nose bridge center point 100 left and right, and the line connecting the first key point 101 and the second key point 102 is parallel to the horizontal axis of the coordinate system, so as to ensure the accuracy of the first distance calculation. When the first distance is less than 0.3 cm for a first duration t1≥2 s, it is determined that the infant posture is in the state of fussing; when the first duration t1<2 s or the first distance is greater than 0.3 cm, it is determined that the infant posture is in the state of calm.

[0146] As an optional embodiment of the present application, the step of determining the state of the infant according to the comparison of the first distance with the preset first distance threshold value further includes:

[0147] The distance between the third key point and the fourth key point is calculated to obtain a second distance, and the distance between the fifth key point and the sixth key point is calculated to obtain a third distance. The state of the infant is determined according to the comparison of the second distance and the third distance with a preset second distance threshold value, and the specific determination steps are:

[0148] When any one of the second distance and the third distance is greater than or equal to the second distance threshold value, it is determined that the infant posture is in the state of non-sleeping;

[0149] When the second distance and the third distance are both less than the second distance threshold value, the duration during which the second distance and the third distance are both less than or equal to the second distance threshold value is recorded to obtain a second duration. When the second duration is less than a preset second time threshold value, it is determined that the infant posture is in the state of non-sleeping;

[0150] When the second duration is greater than or equal to the second time threshold value, it is determined that the infant posture is in the state of sleeping.

[0151] Figure 4 A second method for determining the baby posture according to an embodiment of the present application.

[0152] In some embodiments, the face key point information is obtained by referring to Figure 4 In the second method for determining the baby posture according to an embodiment of the present application, the step of determining the baby posture according to any one of the face key point information and the head posture angle information comprises:

[0153] S421: performing distance calculation on the third key point and the fourth key point to obtain a second distance, and performing distance calculation on the fifth key point and the sixth key point to obtain a third distance;

[0154] S422: determining whether the second distance and the third distance are both less than the second distance threshold value;

[0155] If not, step S423 is performed: determining that the baby posture is in a non-sleep state, and ending the detection;

[0156] If yes, step S424 is performed: recording a second duration in which the second distance and the third distance are both less than or equal to the second distance threshold value to obtain a second duration;

[0157] S425: determining whether the second duration is greater than or equal to the second time threshold value;

[0158] If not, step S423 is performed;

[0159] If yes, step S426 is performed: determining that the baby posture is in a sleep state, and ending the detection.

[0160] The steps S421 to S426 have the advantage that the baby posture is calculated by comparing the second distance and the third distance with the second distance threshold value and comparing the second duration with the second time threshold value, and the baby posture is determined to be in a sleep state, so that the efficiency and accuracy of determining the baby posture are high.

[0161] In some embodiments, the second distance threshold value is 0.02-0.10 cm, and the second time threshold value is 3-10 s.

[0162] In some embodiments, the steps S421 to S426 are as follows:

[0163] In some embodiments, the face key point information is obtained by referring to Figure 2, the third key point 103 is left-right symmetrical to the fifth key point 105 about the bridge of the nose center point 100, and the fourth key point 104 is left-right symmetrical to the sixth key point 106 about the bridge of the nose center point 100. A second distance is calculated by calculating the distance between the third key point 103 and the fourth key point 104, and a third distance is calculated by calculating the distance between the fifth key point 105 and the sixth key point 106; when the second distance and the third distance are both less than 0.025 for a second duration t2≥3s, it is determined that the infant posture is in a sleep state; when the second duration t2<3s or either of the second distance and the third distance is greater than or equal to 0.025, it is determined that the infant posture is in an awakening state.

[0164] Figure 5 A third method for determining an infant posture according to an embodiment of the present application is shown in the flowchart.

[0165] In some embodiments, the step of determining the infant posture according to any one of the face key point information and the head posture angle information comprises: Figure 5 In the third method for determining an infant posture according to an embodiment of the present application, the step of determining the infant posture according to any one of the face key point information and the head posture angle information comprises:

[0166] S431: obtaining a head pitch angle according to the head posture angle information;

[0167] S432: when the head pitch angle is less than a preset posture angle threshold, recording a duration for which the head pitch angle is less than the posture angle threshold to obtain a third duration;

[0168] S433: when the head pitch angle is less than the preset posture angle threshold, performing a third timing step to record a duration for which the head pitch angle is less than the preset posture angle threshold to obtain a third duration;

[0169] S434: when the third duration is greater than or equal to a preset third time threshold, determining that the infant posture is in a sleep state. The steps S331 to S334 have the advantage that the state of the infant is determined by comparing the head pitch angle and the posture angle threshold and comparing the third duration and the third time threshold, the infant posture is determined without calculating the distance, and the efficiency of determining the infant posture is improved.

[0170] In some embodiments, the posture angle threshold has a value of -0.4-0.6, and the third time threshold has a value of 4-8s.

[0171] In some embodiments, the steps S431 to S434 comprise:

[0172] When the pitch angle is less than -0.48, a third timing step is performed to obtain a third duration t3 of the pitch angle being less than -0.48, and when the third duration t3 is greater than or equal to 5s, the infant posture is determined as the sleep state; when the third duration t3 is less than 5s or the pitch angle is greater than -0.48, the infant posture is determined as the wake state.

[0173] Additionally, the three methods for determining the infant posture can be used to determine the infant posture, and the fault tolerance of determining the infant posture is improved.

[0174] In some embodiments, when the head pitch angle is greater than a preset posture angle threshold or the third duration is less than the third time threshold, the infant posture can be determined by any one of the steps S11 to S416 and the steps S21 to S426, and the accuracy of determining the infant posture is improved.

[0175] In some embodiments, when the key points of the eyebrows or the eyelids of the infant cannot be detected, the infant posture can be determined by the steps S331 to S334.

[0176] As an optional embodiment of the present application, in the step S4, after the infant posture is determined by detecting the third face information by the key point detection network, the following steps are further performed:

[0177] When the infant posture changes, the monitoring data of the infant is displayed on the display screen of the driver's cabin, and the duration of displaying the monitoring data of the infant is 10-60s, so as to avoid long-time display and delay the driving of the driver. The monitoring data of the infant is displayed in time to obtain the state of the infant, so that the driver can understand the state of the infant in time and take corresponding measures to protect the safety of the infant and ensure the safety and emotional stability of the infant.

[0178] As an optional embodiment of the present application, in the step S5, the step of controlling the car player or the car component according to the infant posture includes:

[0179] S501: When the infant posture is the fretful state, a playing control step is performed to make the player play a video or music;

[0180] In some embodiments, the player can be installed in front of the back seat of the car, such as the back side of the back of the driver's seat, so as to facilitate the infant to watch the video or listen to the music. The playing control step includes turning on the player, adjusting the volume of the playing program, and switching the video program or the music program.

[0181] S502: When the baby posture is in the sleep state, a closing control step is performed to stop the player from playing the video or music.

[0182] S503: When it is determined that the baby posture is in the stable state, the duration of the stable state is recorded to obtain a stable duration, and when the stable duration is greater than or equal to a preset stable time threshold, a stopping control step is performed to stop the player from playing the video or music.

[0183] In some embodiments, the stable time threshold is 2-10 min.

[0184] The steps S501 to S503 have the advantages that, after the baby posture is determined to be in the fretful state or the wake-up state, the video or music is played to relax the baby and stabilize the baby's mood; and after the baby posture is determined to be in the sleep state or the baby posture is determined to be in the stable state for a period of time, the playing of the video or music is stopped, so as to avoid disturbing the baby and keep the baby posture stable, thereby achieving the stabilization of the baby's mood and state in various baby postures.

[0185] As an optional embodiment of the present application, the step of controlling the player or the automobile component according to the baby posture further comprises:

[0186] When the baby posture is in the fretful state, a seat adjustment step is performed to adjust the seat of the baby until the baby posture changes to the stable state. This has the advantages that the fretful state is relieved by adjusting the seat of the baby, the experience of the baby is improved on the premise of improving the safety of the baby, and the stabilization of the baby's mood and state is further achieved.

[0187] In some embodiments, when the baby is in the fretful state, the baby may be in an incorrect sitting posture or the seat angle is uncomfortable for the baby. When the baby posture is determined to be in the fretful state, a control seat adjustment instruction is sent by the vehicle control system, and the seat adjustment step is performed by the control execution module to adjust the seat of the baby according to the control seat adjustment instruction, such as adjusting the angle, height or distance from the front seat of the seat, so as to accurately and safely and comfortably adjust the sitting or sleeping posture of the baby, automatically adjust the seat, and stabilize the mood and state of the baby while ensuring the safety of the baby.

[0188] Figure 6 A structure diagram of the face-based vehicle control system according to an embodiment of the present application.

[0189] The present application also provides a face-based vehicle control system, which will be described below with reference to Figure 6 The face-based vehicle control system comprises:

[0190] A first acquisition module 1 is configured to acquire passenger face information.

[0191] a face detection network 21 configured to detect the passenger face information to obtain first face information, and detect second face information to obtain third face information after determining that the passenger is a baby;

[0192] an age recognition network 22 configured to detect the first face information to obtain age information;

[0193] an identity recognition module 3 configured to determine the identity of the passenger according to the age information;

[0194] a second acquisition module 4 configured to acquire monitoring data of the baby in real time after determining that the passenger is a baby, and obtain second face information according to the monitoring data;

[0195] a key point detection network 23 configured to detect the third face information to obtain a baby posture;

[0196] In some embodiments, a network module 2 configured to store neural networks is arranged to store the face detection network 21, the age recognition network 22, and the key point detection network 23.

[0197] a control module 5 configured to control a car player or a car component according to the baby posture to stabilize the mood and state of the baby.

[0198] The face-based vehicle control system has the following advantages: the first acquisition module 1 is arranged to acquire passenger face information, the face detection network 21 is arranged to detect the passenger face information to obtain first face information, the age recognition network 22 is arranged to detect the first face information to obtain age information, the identity recognition module 3 is arranged to determine the identity of the passenger, the face image acquisition module is arranged to acquire monitoring data of the baby in real time to monitor the state of the baby in real time, the key point detection network 23 is arranged to obtain a baby posture, and the control module 5 is arranged to control a car player or a car component to stabilize the mood and state of the baby, thereby ensuring the safety of the baby, stabilizing the mood and state of the baby, and improving the comfort and safety of the baby when the baby is in a car.

[0199] In some embodiments, the face-based vehicle control system further comprises a face image acquisition module 2 configured to acquire monitoring data of the baby in real time to monitor the state of the baby in real time. Figure 6The first acquisition module 1 and the second acquisition module 4 each comprise a surround view camera, and the surround view camera is internally provided with a frame cutting device. The surround view camera is installed on the inside of the car, such as the top of the inner wall of the car, and the shooting direction of the camera is towards the seat of the passenger, so as to acquire the passenger face information and the monitoring data of the baby of the passenger. The surround view camera can automatically adjust the monitoring angle and lock the visual angle to monitor the posture of the baby, so as to obtain the complete face information. The frame cutting device is used to cut the last frame image of the monitoring data, so as to obtain the second face information. Since the surround view camera is a common device in the prior art, it will not be described here. The frame cutting device is used to cut the last frame image of the video to obtain the second face information, and any video frame cutting means in the prior art can be used, such as any one of the frame cutting hardware or frame cutting software, and the frame cutting device will not be limited here.

[0200] In some embodiments, after the passenger face information is acquired by the first acquisition module 1, the passenger face information is detected by the face detection network 21 to obtain the first face information, and the specific steps are as follows:

[0201] The resolution of the passenger face information is adjusted to obtain a first image, and the first image is sent to the face detection network 21. After the face detection network 21 detects the first image, first face classification information, first face frame regression information and first key point regression information are output.

[0202] The first face frame regression information is cropped to obtain a first face image, and the resolution of the first face image is adjusted to obtain the first face information.

[0203] In some embodiments, the step of determining the identity of the passenger according to the age information by the identity recognition module 3 comprises:

[0204] After the age information of the passenger is acquired by the age recognition network 22, the identity recognition module 3 receives the age information of the passenger and compares the age information of the passenger with a baby age threshold value.

[0205] When the age information of the passenger does not exceed the baby age threshold value, the identity recognition module 3 determines that the identity of the passenger is a baby.

[0206] When the age information of the passenger is greater than the baby age threshold value, the identity recognition module 3 determines that the identity of the passenger is not a baby.

[0207] In some embodiments, the baby age threshold value is 2-6 years old.

[0208] In some embodiments, after determining that the passenger is a baby, the second face information is detected by the face detection network 21 to obtain third face information, and the specific steps are as follows:

[0209] The resolution of the second face information is adjusted to obtain a second image, and the second image is input into the face detection network 21. After the face detection network 21 detects the second image, second face classification information, second face frame regression information, and second key point regression information are output.

[0210] The second face frame regression information is cropped to obtain a second face image, and the resolution of the second face image is adjusted to obtain the third face information.

[0211] In some embodiments, the key point detection network 23 detects the third face information to obtain a baby posture, and the specific steps are as follows:

[0212] The third face information is input into the key point detection network 23. After the key point detection network 23 detects the third face information, face key point information and head posture angle information are output.

[0213] In some embodiments, the face key point information and the head posture angle information are output after the second network detects the third image, and the baby posture is determined by any one of the following three implementation manners:

[0214] In the first implementation manner of the present application, the steps of confirming the baby posture are as follows:

[0215] The first key point in the left eyebrow region and the second key point in the right eyebrow region in the face key point information are obtained.

[0216] The distance between the first key point and the second key point is calculated to obtain a first distance.

[0217] When the first distance is less than the first distance threshold, the duration that the first distance is less than the first distance threshold is recorded to obtain a first duration. When the first duration is greater than or equal to the first time threshold, it is determined that the baby posture is a state of restlessness.

[0218] When the first distance is greater than or equal to a preset first distance threshold or the first duration is less than a preset first time threshold, it is determined that the baby posture is a state of calmness.

[0219] In some embodiments, the second distance threshold is 0.02-0.10 cm, and the first time threshold is 2-10 s.

[0220] In the second embodiment of the present application, the step of confirming the infant posture is:

[0221] obtaining a third key point of an upper eyelid region of the left eye, a fourth key point of a lower eyelid region of the left eye, a fifth key point of an upper eyelid region of the right eye, and a sixth key point of a lower eyelid region of the right eye in the face key point information;

[0222] performing distance calculation on the third key point and the fourth key point to obtain a second distance, and performing distance calculation on the fifth key point and the sixth key point to obtain a third distance;

[0223] when both the second distance and the third distance are less than the second distance threshold, recording a duration in which both the second distance and the third distance are less than or equal to the second distance threshold to obtain a second duration, and determining that the infant posture is in a sleep state when the second duration is greater than or equal to a second time threshold;

[0224] when either the second distance or the third distance is greater than or equal to a preset second distance threshold, or when the second duration is less than a preset second time threshold, determining that the infant posture is in a non-sleep state.

[0225] In some embodiments, the second distance threshold has a value of 0.02-0.10 cm, and the second time threshold has a value of 3-10 s.

[0226] In the third embodiment of the present application, the step of confirming the infant posture is:

[0227] obtaining a head pitch angle according to the head posture angle information;

[0228] when the head pitch angle is less than a preset posture angle threshold, recording a duration in which the head pitch angle is less than the posture angle threshold to obtain a third duration;

[0229] when the third duration is greater than or equal to a preset third time threshold, determining that the infant posture is in a sleep state.

[0230] In some embodiments, the posture angle threshold has a value of -0.4-0.6, and the third time threshold has a value of 4-8 s.

[0231] In some embodiments, reference is made to Figure 6The face-based vehicle-mounted control system further comprises a reminding identification module 6. After determining that the age stage information of the passenger is a baby, the control module 5 sends a first control instruction, and the reminding identification module 6 executes a reminding identification step to display that there is a baby in the vehicle after receiving the first control instruction. The advantage is that the reminding identification module 6 executes the reminding identification step to display that there is a baby in the vehicle, so as to remind the people outside the vehicle that there is a baby in the vehicle, and reduce the possibility that the baby is disturbed, thereby improving the safety of the baby.

[0232] In some specific embodiments, the reminding identification module 6 is installed on the outside of the vehicle, such as the rear windshield of the vehicle, and the reminding identification device comprises any one of an electronic display screen and an LED font. When there is a baby passenger in the vehicle, the reminding identification module 6 executes the reminding identification step to remind the people behind the vehicle that there is a baby in the vehicle, so as to remind the people behind the vehicle to press the horn less or slow down, thereby reducing the possibility that the baby is disturbed or in danger.

[0233] In some embodiments, referring to Figure 6 The face-based vehicle-mounted control system further comprises a display module 7. When the baby posture changes, the control module 5 sends a second control instruction, and the display module 7 displays the monitoring data of the baby after receiving the second control instruction, so that the driver knows the posture and state of the baby. The advantage is that the display module 7 displays the monitoring data of the baby, so that the driver knows the posture and state of the baby in time, thereby facilitating the driver to take corresponding measures to stabilize the mood and state of the baby and ensure the safety of the baby.

[0234] In some specific embodiments, the display module 7 is arranged in the driver's cabin, and the display module 7 comprises an intelligent display screen arranged in the central control area of the driver's cabin. When the baby posture changes, such as when the baby changes from a sleep state to a wake-up state, or from a wake-up state to a sleep state, or from a fretful state to a stable state, or from a fretful state, the display module 7 starts timing from the change of the baby posture, displays the online video of the baby for 10-60 seconds, so as to facilitate the driver to observe the mood and state of the baby in time and take corresponding measures, such as slowing down or stopping the vehicle, reducing or turning off the music player, to stabilize the mood and state of the baby. After displaying the online video of the baby for 30-60 seconds, if the baby posture changes to a sleep state or a stable state, the display module 7 is turned off, thereby reducing the duration of the driver observing the display screen, so as to make the driver feel at ease while driving and improve the safety of the driver driving the vehicle.

[0235] In some embodiments, referring to Figure 6The face-based vehicle control system further comprises a seat adjustment module 8, and when the baby posture is in the fussy state, the control module 5 sends a third control instruction to the seat adjustment module 8, so that the seat adjustment module 8 performs a seat adjustment step to adjust the seat of the baby until the baby posture changes to the stable state.

[0236] In some embodiments, the face detection network 21 is used to detect passenger face information in real time to obtain first face information, and age information is obtained according to the first face information

[0237] In some specific embodiments, with reference to Figure 6 The working steps of the face-based vehicle control system are as follows:

[0238] (1) After the passenger enters the camera range of the ring camera, the first acquisition module 1 captures a face photo and outputs passenger face information to the face detection network 21, the face detection network 21 detects the passenger face information, and outputs first face information to the age recognition network 22, and the age recognition network 22 detects the first face information to obtain age information;

[0239] (2) The identity recognition module 3 determines the identity of the passenger according to the age information, and when it is determined that the passenger is not a baby, the monitoring of the ring camera is suspended or the player is not processed;

[0240] When it is determined that the passenger is a baby, the control module 5 sends a control instruction to control the second acquisition module 4 to lock the face of the baby to obtain baby monitoring data of the baby; at the same time, the control module 5 sends a first control instruction, and the reminding sign module 6 receives the first control instruction and performs a reminding sign step to display that there is a baby in the vehicle;

[0241] (3) The baby posture is detected by the key point detection network 23, and when it is determined that the baby posture is in a fussy state, the control module 5 performs a playing control step to control the player to play a video or music; at the same time, the control module 5 sends a left adjustment instruction to control the seat adjustment module 8 to adjust the seat of the baby to stabilize the mood and state of the baby;

[0242] When it is determined that the baby posture is in a sleep state, the control module 5 performs a stop control step to control the player to stop playing a video or music;

[0243] When it is determined that the baby posture is in a stable state, the duration of the stable state is recorded to obtain a stable duration, and when the stable duration is greater than or equal to a preset stable time threshold, the control module 5 performs a stop control step to make the player stop playing a video or music.

[0244] The application also provides an apparatus, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the program is executed by the processor to implement the face-based vehicle control method.

[0245] The application also provides a storage medium, wherein a computer program is stored in the storage medium, and the computer program is executed by a processor to implement the face-based vehicle control method.

[0246] Although the embodiments of the present application have been described in detail above, it is obvious to those skilled in the art that various modifications and changes can be made to the embodiments without departing from the scope and spirit of the present application as described in the following claims. Moreover, the present application described herein can have other embodiments and be practiced or implemented in various ways.

Claims

1. A face-based in-vehicle control method, characterized by, The method comprises the steps of: detecting passenger face information through a face detection network to obtain first face information, and detecting the first face information through an age recognition network to obtain age information; determining the identity of the passenger according to the age information; after determining that the passenger is a baby, collecting monitoring data of the baby in real time, and obtaining second face information according to the monitoring data; detecting the second face information through the face detection network to obtain third face information, and detecting the third face information through a key point detection network to obtain a baby posture; controlling a car player or a baby seat according to the baby posture to stabilize the mood and state of the baby; wherein the step of detecting the third face information through the key point detection network to obtain the baby posture comprises: the key point detection network outputs face key point information and head posture angle information after detecting the third face information; and the baby posture is obtained according to any one of the face key point information and the head posture angle information; the face key point information comprises a first key point in a left eyebrow area, a second key point in a right eyebrow area, a third key point in a left eye upper eyelid area, a fourth key point in a left eye lower eyelid area, a fifth key point in a right eye upper eyelid area, and a sixth key point in a right eye lower eyelid area; the step of detecting passenger face information through a face detection network to obtain first face information comprises: adjusting the resolution of the passenger face information to obtain a first image, and inputting the first image into the face detection network; the face detection network outputs first face classification information, first face frame regression information, and first key point regression information after detecting the first image; cropping the first face frame regression information to obtain a first face image, and adjusting the resolution of the first face image to obtain the first face information; the step of determining the baby posture according to any one of the face key point information and the head posture angle information comprises: obtaining a head pitch angle according to the head posture angle information; when the head pitch angle is less than a preset posture angle threshold, recording the duration for which the head pitch angle is less than the posture angle threshold to obtain a third duration; when the third duration is greater than or equal to a preset third time threshold, determining that the baby posture is a sleep state. 2.The face-based in-vehicle control method of claim 1, wherein, the step of detecting the second face information through the face detection network to obtain third face information comprises: adjusting the resolution of the second face information to obtain a second image, and inputting the second image into the face detection network; the face detection network outputs second face classification information, second face frame regression information, and second key point regression information after detecting the second image; cropping the second face frame regression information to obtain a second face image, and adjusting the resolution of the second face image to obtain the third face information. 3.The face-based in-vehicle control method of claim 1, wherein, the step of obtaining second face information according to the monitoring data comprises: obtaining the second face information by intercepting the last frame of image of the monitoring data. 4.The face-based in-vehicle control method of claim 1, wherein, the step of determining the identity of the passenger according to the age information comprises: When the age information of the passenger does not exceed the infant age threshold, the identity of the passenger is determined as an infant. 5.The face-based in-vehicle control method of claim 1, wherein, The step of determining the infant posture according to any one of the face key point information and the head posture angle information comprises: performing distance calculation on the first key point and the second key point to obtain a first distance; determining the state of the infant according to comparison of the first distance with a preset first distance threshold. 6.The face-based in-vehicle control method of claim 5, wherein, The step of determining the state of the infant according to comparison of the first distance with a preset first distance threshold comprises: when the first distance is greater than or equal to the first distance threshold, the infant posture is determined as a calm state. 7.The face-based in-vehicle control method of claim 5, wherein, The step of determining the state of the infant according to comparison of the first distance with a preset first distance threshold comprises: when the first distance is less than the first distance threshold, a duration that the first distance is less than the first distance threshold is recorded to obtain a first duration; when the first duration is less than a preset first time threshold, the infant posture is determined as a calm state. 8.The face-based in-vehicle control method of claim 7, wherein, Further comprising: when the first duration is greater than or equal to the first time threshold, the infant posture is determined as a fussy state. 9.The face-based in-vehicle control method of claim 1, wherein, The step of determining the infant posture according to any one of the face key point information and the head posture angle information comprises: performing distance calculation on the third key point and the fourth key point to obtain a second distance, and performing distance calculation on the fifth key point and the sixth key point to obtain a third distance; determining the state of the infant according to comparison of the second distance and the third distance with a preset second distance threshold. 10.The face-based in-vehicle control method of claim 9, wherein, The step of determining the state of the infant according to comparison of the second distance and the third distance with a preset second distance threshold comprises: when any one of the second distance and the third distance is greater than or equal to the second distance threshold, the infant posture is determined as a non-sleep state. 11.The face-based in-vehicle control method of claim 9, wherein, The step of determining the state of the infant according to comparison of the second distance and the third distance with a preset second distance threshold comprises: when the second distance and the third distance are both less than the second distance threshold, a duration that the second distance and the third distance are both less than or equal to the second distance threshold is recorded to obtain a second duration; when the second duration is less than a preset second time threshold, the infant posture is determined as a non-sleep state. 12.The face-based in-vehicle control method of claim 11, wherein, Further comprising: when the second duration is greater than or equal to the second time threshold, the infant posture is determined as a sleep state. 13.The face-based in-vehicle control method of claim 1, wherein, The step of controlling the car player or the infant seat according to the infant posture comprises: when the infant posture is determined as a fussy state, a playing control step is performed to make the player play a video or music; when the infant posture is determined as a sleep state, a stopping control step is performed to make the player stop playing a video or music; when the infant posture is determined as a calm state, a duration of the calm state is recorded to obtain a calm duration, and when the calm duration is greater than or equal to a preset calm time threshold, the stopping control step is performed to make the player stop playing a video or music. 14.The face-based in-vehicle control method of claim 13, wherein, The step of controlling the car player or the baby seat according to the baby posture further comprises: When the baby posture is the fussy state, a seat adjustment step is performed to adjust the seat of the baby until the baby posture becomes the stable state. 15.The face-based in-vehicle control method of claim 1, wherein, After determining that the passenger is a baby, the following steps are further performed: A reminding identification step is performed to display that there is a baby in the car. 16.The face-based in-vehicle control method of claim 1, wherein, After detecting the third face information through the key point detection network to obtain the baby posture, the following steps are further performed: When the baby posture changes, the monitoring data of the baby is displayed on the display screen in the driver's cabin.

17. A face-based in-vehicle control system, characterized by, Comprise: A first acquisition module is configured to acquire passenger face information; A face detection network is configured to detect the passenger face information to obtain first face information; After determining that the passenger is a baby, the second face information is detected to obtain third face information; An age recognition network is configured to detect the first face information to obtain age information; An identity recognition module is configured to determine the identity of the passenger according to the age information; A second acquisition module is configured to acquire real-time monitoring data of the baby after determining that the passenger is a baby, and to acquire second face information according to the monitoring data; A key point detection network is configured to detect the third face information to obtain a baby posture, including that the key point detection network outputs face key point information and head posture angle information after detecting the third face information; the baby posture is obtained according to any one of the face key point information and the head posture angle information; wherein the face key point information comprises a first key point in a left eyebrow area, a second key point in a right eyebrow area, a third key point in a left eye upper eyelid area, a fourth key point in a left eye lower eyelid area, a fifth key point in a right eye upper eyelid area, and a sixth key point in a right eye lower eyelid area; Detecting the passenger face information through the face detection network to obtain the first face information comprises: Adjusting the resolution of the passenger face information to obtain a first image, and feeding the first image to the face detection network, wherein the face detection network outputs first face classification information, first face frame regression information, and first key point regression information after detecting the first image; The first face frame regression information is cropped to obtain a first face image, and the resolution of the first face image is adjusted to obtain the first face information; Determining the baby posture according to any one of the face key point information and the head posture angle information comprises: Obtaining a head pitch angle according to the head posture angle information; When the head pitch angle is less than a preset posture angle threshold, recording the duration that the head pitch angle is less than the posture angle threshold to obtain a third duration; When the third duration is greater than or equal to a preset third time threshold, determining that the baby posture is a sleep state; A control module is configured to control a car player or a baby seat according to the baby posture to stabilize the mood and state of the baby.

18. An apparatus comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein, The program is executed by the processor to implement the face-based vehicle control method according to any one of claims 1-16.

19. A storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the face-based in-vehicle control method according to any one of claims 1-16.

Images