Vehicle door safety control method, device and equipment based on face rotation angle

Abstract

Embodiments of the present disclosure provide a face rotation angle-based vehicle door safety control method, device and equipment, which are applied to the technical field of driving safety control. The method comprises: acquiring a first real-time face image of a user and a second real-time face image of the user based on a first camera and a second camera arranged in a cab, wherein the second camera is arranged at a preset included angle with the first camera in the cab; determining a face rotation trend parameter based on the first real-time face image of the user and the second real-time face image of the user; and sending a vehicle door unlocking command if the rotation trend parameter meets a preset condition. The present disclosure acquires the face image of the user by using the cameras arranged at the preset included angle in the cab, and determines the face rotation angle. When the rotation angle and the rotation time meet the preset condition, the vehicle door is unlocked, which ensures that the user can carefully observe the oncoming vehicles and pedestrians behind when opening the door, thereby reducing the occurrence of accidents.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle safety control technology, and in particular to a method, device, equipment and storage medium for vehicle door safety control based on the angle of facial rotation. Background Technology

[0002] During the use of vehicles, many users lack safety awareness and open car doors without checking the rearview mirror or turning their heads to confirm the situation behind them, which leads to many accidents.

[0003] Existing technical solutions include blind spot monitoring systems to monitor other vehicles or pedestrians in the user's blind spot. When a vehicle or pedestrian enters the driver's blind spot, the system generates a warning image to remind the user. However, the system cannot guarantee that the user will notice the warning information and not open the door. In other words, even if the system reminds the user, the user can still ignore the warning and open the door. The opened door may affect oncoming vehicles and pedestrians, leading to an accident.

[0004] Therefore, existing technical solutions cannot reduce the probability of accidents when opening doors by ensuring that users have noticed oncoming vehicles and pedestrians. Summary of the Invention

[0005] This disclosure provides a method, device, equipment, and storage medium for vehicle door safety control based on the angle of facial rotation.

[0006] According to a first aspect of this disclosure, a vehicle door safety control method based on face rotation angle is provided, the method comprising:

[0007] The user's first real-time facial image is acquired using a first camera positioned inside the driver's cab;

[0008] The user's second face real-time image is obtained based on a second camera arranged in the driver's cab, wherein the second camera and the first camera are arranged at a preset angle in the driver's cab;

[0009] Determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image;

[0010] If the rotation trend parameters meet the preset conditions, a door unlock command will be sent.

[0011] In some implementations of the first aspect, if the rotation trend parameter meets preset conditions, a door unlocking command is sent, including:

[0012] If the rotation trend parameters meet the preset conditions, and the vehicle is in a parked state and receives the user's door unlock trigger signal, then a door unlock command is sent.

[0013] In some implementations of the first aspect, the face rotation trend parameters include rotation angle, rotation direction, and rotation duration;

[0014] The face rotation trend parameters are determined based on the user's first real-time face image and the user's second real-time face image, including:

[0015] Identify individual facial feature points in the user's first real-time face image and the user's second real-time face image;

[0016] The first horizontal rotation angle, the first horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's first real-time face image.

[0017] The second horizontal rotation angle, the second horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's real-time second face image.

[0018] If the rotation trend parameters meet the preset conditions, a door unlock command is sent, including:

[0019] If the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first horizontal rotation angle and the second horizontal rotation angle is greater than the angle threshold, the horizontal rotation direction of the face determined based on the first horizontal rotation direction and the second horizontal rotation direction is a preset direction, and the rotation duration is greater than the time threshold, then a door unlocking command is sent to unlock the door.

[0020] In some implementations of the first aspect, the method also includes:

[0021] The user's head stereo model is determined based on each facial feature point in the user's first and second real-time face images, and the distance from each feature to the horizontal rotation center of the face is determined based on the constructed user head stereo model.

[0022] Based on the total offset of each facial feature point in the user's first and second real-time face images, and the distance of each feature to the face's horizontal rotation center, the first horizontal rotation angle and the first horizontal rotation direction, the second horizontal rotation angle and the second horizontal rotation direction are determined, and the rotation duration corresponding to the first horizontal rotation angle in the first horizontal rotation direction and the rotation duration corresponding to the second horizontal rotation angle in the second horizontal rotation direction are determined.

[0023] In some implementations of the first aspect, the method also includes:

[0024] Based on the constructed 3D model of the user's head, the distance from each feature to the vertical rotation center of the face is determined.

[0025] The vertical rotation angle and direction are determined based on the total offset of each facial feature point in the user's first real-time face image and the distance to the vertical rotation center of the face.

[0026] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction.

[0027] If the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first and second horizontal rotation angles is greater than an angle threshold, the horizontal rotation direction of the face determined based on the first and second horizontal rotation directions is a preset direction, and the rotation duration is greater than a time threshold, then a door unlocking command is sent to unlock the door, including:

[0028] If the first horizontal rotation angle matches the second horizontal rotation angle, and the first horizontal rotation direction matches the second horizontal rotation direction, then the horizontal rotation angle to be adjusted for the face is determined based on the first horizontal rotation angle and the second horizontal rotation angle.

[0029] Based on the visual adjustment parameters, the value of the horizontal rotation angle of the face to be adjusted is adjusted to obtain the horizontal rotation angle of the face.

[0030] If the horizontal rotation angle of the face is greater than the angle threshold, the rotation direction is a preset direction, and the rotation duration is greater than the time threshold, then a door unlock command is sent to unlock the car door.

[0031] In some implementations of the first aspect, the method also includes:

[0032] Identify the pupils in the user's first real-time face image and determine the duration of the pupils in various directions within the eye socket;

[0033] The pupil fixation parameters are obtained by summing the products of the weights of the pupils in different directions within the eye socket and the duration of fixation.

[0034] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction, including:

[0035] Based on the pupil annotation parameters, combined with the vertical rotation angle and vertical rotation direction, the visual adjustment parameters are determined.

[0036] In some implementations of the first aspect, the method also includes:

[0037] The pupil is classified according to the duration of its position in the eye socket, and the directions corresponding to the first, second, third and fourth durations are determined.

[0038] Based on the directions corresponding to the first duration, the second duration, the third duration, and the fourth duration, the correlation coefficients of the pupil in each direction within the eye socket are determined.

[0039] The fixation effect achieved by the pupil in each direction within the eye socket is determined based on the correlation coefficients in each direction.

[0040] The weights of the pupil in each direction within the eye socket are determined based on the gaze effect achieved by the pupil in each direction.

[0041] According to a second aspect of this disclosure, a vehicle door safety control device based on the facial rotation angle is provided, the device comprising:

[0042] The acquisition module is used to acquire a real-time image of the user's first face based on a first camera installed in the driver's cab;

[0043] The acquisition module is also used to acquire a real-time image of the user's second face based on a second camera arranged in the driver's cab, wherein the second camera is arranged in a different position than the first camera in the driver's cab;

[0044] The processing module is used to determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image;

[0045] The sending module is used to send a door unlocking command if the rotation trend parameters meet preset conditions.

[0046] According to a third aspect of this disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the program to implement the method described above.

[0047] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0048] This disclosure uses a camera positioned at a preset angle inside the driver's cab to acquire a user's facial image and determine the user's facial rotation angle. When the rotation angle and rotation time meet preset conditions, the door is unlocked, thereby ensuring that the user can carefully observe oncoming vehicles and pedestrians when opening the door, reducing the occurrence of accidents.

[0049] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0050] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0051] Figure 1 A schematic flowchart of a vehicle door safety control method based on face rotation angle according to an embodiment of the present disclosure is shown.

[0052] Figure 2 A block diagram of a door safety control device based on face rotation angle according to an embodiment of the present disclosure is shown;

[0053] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0055] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0056] In order to solve the above-mentioned technical problems, this disclosure provides a method, device, and equipment for vehicle door safety control based on face rotation angle. The method involves acquiring a first real-time image of a user's face using a first camera positioned in the driver's cab; acquiring a second real-time image of the user's face using a second camera positioned in the driver's cab, wherein the second camera and the first camera are arranged at a preset angle within the driver's cab; determining face rotation trend parameters based on the first and second real-time face images; and sending a door unlock command if the rotation trend parameters meet preset conditions. This disclosure acquires the user's face image using a camera positioned at a preset angle within the driver's cab and determines the face rotation angle. When the rotation angle and rotation time meet preset conditions, the door unlocks, ensuring that the user can carefully observe oncoming vehicles and pedestrians when opening the door, thereby reducing the occurrence of accidents.

[0057] Figure 1 A flowchart illustrating a vehicle door safety control method based on face rotation angle according to an embodiment of this disclosure is shown, as follows: Figure 1 As shown, the door safety control method 100 based on the face rotation angle may include:

[0058] S101, acquires the user's first real-time facial image based on the first camera installed in the driver's cab;

[0059] S102, acquire a real-time image of the user's second face based on a second camera arranged in the driver's cab, wherein the second camera and the first camera are arranged at a preset angle in the driver's cab;

[0060] S103, determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image;

[0061] S104, if the rotation trend parameter meets the preset conditions, send a door unlock command.

[0062] During processes S101-S104, the user's facial image is acquired by using a camera positioned at a preset angle inside the driver's cab, and the rotation angle is determined. When the rotation angle and rotation time meet preset conditions, the door is unlocked, ensuring that the user can carefully observe oncoming vehicles and pedestrians when opening the door, thereby reducing the occurrence of accidents.

[0063] It should be noted that, for safety control of the driver, the first camera can be installed in the center console area directly in front of the driver's seat, and the second camera can be installed on the A-pillar of the vehicle on the driver's side. For safety control of the front passenger, the first camera can be installed in the center console area directly in front of the front passenger seat, and the second camera can be installed on the A-pillar of the vehicle on the front passenger side. Furthermore, for safety control of rear passengers, the first camera can be installed on the B-pillar of the vehicle on the driver's side, and the second camera can be installed on the B-pillar of the vehicle on the front passenger side. Moreover, for safety control of both the driver and front passenger, the first camera can also be installed in the middle area of ​​the center console; there are no restrictions on this.

[0064] Considering that unlocking the doors is only safe for driving when the vehicle stops and a user's door unlocking trigger signal is received, in some embodiments, sending a door unlocking command in S104 if the rotation trend parameter meets a preset condition may include:

[0065] If the rotation trend parameters meet the preset conditions, and the vehicle is in a parked state and receives the user's door unlock trigger signal, then a door unlock command is sent. The user's door unlock trigger signal can be an electrical signal generated by the user pulling the door handle, or it can be other door unlock trigger signals from the user. Specifically, the door unlock command can be a command sent to the door locking component to unlock.

[0066] In the above embodiments, by sending a door unlocking command only when the rotation trend parameter meets preset conditions when the vehicle stops and the user's door unlocking trigger signal is received, the vehicle's own driving safety is ensured and the user can carefully observe the vehicles and pedestrians behind when opening the door, thereby reducing the occurrence of accidents.

[0067] In some embodiments, the face rotation trend parameters may include rotation angle, rotation direction, and rotation duration;

[0068] Determining face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image can include:

[0069] Identify individual facial feature points in the user's first real-time face image and the user's second real-time face image;

[0070] The first horizontal rotation angle, the first horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's first real-time face image.

[0071] The second horizontal rotation angle, the second horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's real-time second face image.

[0072] If the rotation trend parameters meet the preset conditions, a door unlock command is sent, including:

[0073] If the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first horizontal rotation angle and the second horizontal rotation angle is greater than the angle threshold, the horizontal rotation direction of the face determined based on the first horizontal rotation direction and the second horizontal rotation direction is a preset direction, and the rotation duration is greater than the time threshold, then a door unlocking command is sent to unlock the door.

[0074] Among them, facial feature points can include the left corner of the eye, the right corner of the eye, the tip of the nose, the left corner of the mouth, the right corner of the mouth, and the jaw features.

[0075] Specifically, for the driver's safety control, the preset direction can be to the left; for the front passenger's safety control, the preset direction can be to the right; for the rear passenger's safety control, when the rear passenger is behind the driver, the preset direction can be to the left, and when the rear passenger is behind the front passenger, the preset direction can be to the right.

[0076] When the first and second horizontal rotation angles and the first and second horizontal rotation directions are matched, for safety control of the driver, the data obtained by the first camera can be a face rotating 45 degrees to the left from 0 degrees, and the data obtained by the second camera can be a face rotating 45 degrees to the right from 0 degrees to the left, satisfying the matching of the first and second horizontal rotation directions. The horizontal rotation direction of the face determined based on the first and second horizontal rotation directions is to the left, and the horizontal rotation angle of the face determined based on the first and second horizontal rotation angles is 45 degrees. The same principle applies to safety control of the front passenger and rear passengers.

[0077] In addition, the time threshold can be 10 seconds and the angle threshold can be 44 degrees, but both the time threshold and the angle threshold can be adjusted according to the actual situation.

[0078] In the above embodiments, by recognizing facial feature points and determining the rotation angle based on the offset of the facial feature points, the door is unlocked when the rotation angle and rotation time meet preset conditions, ensuring that the user can carefully observe the vehicles and pedestrians behind when opening the door, thereby reducing the occurrence of accidents.

[0079] To accurately determine the first horizontal rotation angle and the first horizontal rotation direction, the second horizontal rotation angle and the second horizontal rotation direction, and to determine the rotation duration corresponding to the first horizontal rotation angle in the first horizontal rotation direction and the rotation duration corresponding to the second horizontal rotation angle in the second horizontal rotation direction, thereby enabling subsequent door unlocking control based on accurate data, in some embodiments, the method may further include:

[0080] The user's head stereo model is determined based on each facial feature point in the user's first and second real-time face images, and the distance from each feature to the horizontal rotation center of the face is determined based on the constructed user head stereo model.

[0081] Based on the total offset of each facial feature point in the user's first and second real-time face images, and the distance of each feature to the face's horizontal rotation center, the first horizontal rotation angle and the first horizontal rotation direction, the second horizontal rotation angle and the second horizontal rotation direction are determined, and the rotation duration corresponding to the first horizontal rotation angle in the first horizontal rotation direction and the rotation duration corresponding to the second horizontal rotation angle in the second horizontal rotation direction are determined.

[0082] Considering that the user's head will rotate not only in the horizontal direction but also in the vertical direction, and the vertical rotation will also affect the horizontal angle and thus affect the judgment process, in some embodiments, the distance from each feature to the vertical rotation center of the face can be determined based on the constructed three-dimensional model of the user's head.

[0083] The vertical rotation angle and direction are determined based on the total offset of each facial feature point in the user's first real-time face image and the distance to the vertical rotation center of the face.

[0084] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction.

[0085] If the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first and second horizontal rotation angles is greater than an angle threshold, the horizontal rotation direction of the face determined based on the first and second horizontal rotation directions is a preset direction, and the rotation duration is greater than a time threshold, then a door unlocking command is sent to unlock the door, including:

[0086] If the first horizontal rotation angle matches the second horizontal rotation angle, and the first horizontal rotation direction matches the second horizontal rotation direction, then the horizontal rotation angle to be adjusted for the face is determined based on the first horizontal rotation angle and the second horizontal rotation angle.

[0087] Based on the visual adjustment parameters, the value of the horizontal rotation angle of the face to be adjusted is adjusted to obtain the horizontal rotation angle of the face.

[0088] If the horizontal rotation angle of the face is greater than the angle threshold, the rotation direction is a preset direction, and the rotation duration is greater than the time threshold, then a door unlock command is sent to unlock the car door.

[0089] In the above embodiment, by combining the user's head rotation in the vertical direction, the horizontal rotation angle of the face is accurately determined, thereby enabling subsequent door unlocking control based on accurate data. This ensures that the user can carefully observe oncoming vehicles and pedestrians when opening the door, reducing the occurrence of accidents.

[0090] In addition, considering that the user's gaze can affect the user's attention to pedestrians and oncoming vehicles outside the vehicle, in some embodiments, this disclosure can also identify the pupils in the user's first real-time face image and determine the duration of the pupils in various directions within the eye socket.

[0091] The pupil fixation parameters are obtained by summing the products of the weights of the pupils in different directions within the eye socket and the duration of fixation.

[0092] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction, including:

[0093] Based on the pupil annotation parameters, combined with the vertical rotation angle and vertical rotation direction, the visual adjustment parameters are determined.

[0094] In the above embodiments, by recognizing the pupil and combining the vertical rotation angle and vertical rotation direction, the aforementioned visual adjustment parameters are accurately determined. Subsequently, door unlocking control can be performed based on accurate data, ensuring that the user can carefully observe oncoming vehicles and pedestrians when opening the door, thereby reducing the occurrence of accidents.

[0095] In some embodiments, considering that the duration of gaze in different directions will directly affect the user's observation of the outside world, the pupil in the eye socket can be classified according to the duration based on the user's position in the driver's cab, and the direction corresponding to the first duration, the second duration, the third duration, and the fourth duration can be determined.

[0096] Based on the directions corresponding to the first duration, the second duration, the third duration, and the fourth duration, the correlation coefficients of the pupil in each direction within the eye socket are determined.

[0097] The fixation effect achieved by the pupil in each direction within the eye socket is determined based on the correlation coefficients in each direction.

[0098] The weights of the pupil in each direction within the eye socket are determined based on the gaze effect achieved by the pupil in each direction.

[0099] In the above embodiments, considering that the directions corresponding to different durations will affect the actual observation, a correlation coefficient is determined for the actual gaze effect of each direction by the pupil direction of different durations. Then, the gaze effect achieved by the pupil in each direction in the eye socket is determined according to the correlation coefficient, and the weight corresponding to each direction is accurately determined. Subsequently, the door unlocking control can be performed based on accurate data to ensure that the user can carefully observe the vehicles and pedestrians behind when opening the door, thereby reducing the occurrence of accidents.

[0100] The above is an introduction to the method embodiments. The following describes the present disclosure further through device embodiments.

[0101] Figure 2 A block diagram of a door safety control device based on the face rotation angle according to an embodiment of the present disclosure is shown.

[0102] like Figure 2 As shown, the door safety control device 200 based on the face rotation angle may include:

[0103] The acquisition module 201 is used to acquire a real-time image of the user's first face based on a first camera installed in the driver's cab;

[0104] The acquisition module 201 is also used to acquire a real-time image of the user's second face based on a second camera arranged in the driver's cab, wherein the second camera and the first camera are arranged at a preset angle in the driver's cab;

[0105] Processing module 202 is used to determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image;

[0106] The sending module 203 is used to send a door unlocking command if the rotation trend parameter meets the preset conditions.

[0107] In some embodiments, the sending module 203 is further configured to send a door unlocking command if the rotation trend parameter meets a preset condition, and the vehicle is in a parked state and receives a door unlocking trigger signal from the user.

[0108] In some embodiments, the face rotation trend parameters include rotation angle, rotation direction, and rotation duration;

[0109] The processing module 202 is also used to identify each facial feature point in the user's first real-time face image and the user's second real-time face image;

[0110] The first horizontal rotation angle, the first horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's first real-time face image.

[0111] The second horizontal rotation angle, the second horizontal rotation direction, and the corresponding rotation duration are determined based on the total offset of each facial feature point in the user's real-time second face image.

[0112] The sending module 203 is further configured to send a door unlocking command to unlock the car door if the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first horizontal rotation angle and the second horizontal rotation angle is greater than an angle threshold, the horizontal rotation direction of the face determined based on the first horizontal rotation direction and the second horizontal rotation direction is a preset direction, and the rotation duration is greater than a time threshold.

[0113] In some embodiments, the processing module 202 is further configured to determine a stereoscopic model of the user's head based on each facial feature point in the user's first real-time face image and second real-time face image, and to determine the distance from each feature to the horizontal rotation center of the face based on the constructed stereoscopic model of the user's head.

[0114] Based on the total offset of each facial feature point in the user's first and second real-time face images, and the distance of each feature to the face's horizontal rotation center, the first horizontal rotation angle and the first horizontal rotation direction, the second horizontal rotation angle and the second horizontal rotation direction are determined, and the rotation duration corresponding to the first horizontal rotation angle in the first horizontal rotation direction and the rotation duration corresponding to the second horizontal rotation angle in the second horizontal rotation direction are determined.

[0115] In some embodiments, the processing module 202 is further configured to determine the distance from each feature to the vertical rotation center of the face based on the constructed stereoscopic model of the user's head;

[0116] The vertical rotation angle and direction are determined based on the total offset of each facial feature point in the user's first real-time face image and the distance to the vertical rotation center of the face.

[0117] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction.

[0118] If the first horizontal rotation angle matches the second horizontal rotation angle, and the first horizontal rotation direction matches the second horizontal rotation direction, then the horizontal rotation angle to be adjusted for the face is determined based on the first horizontal rotation angle and the second horizontal rotation angle.

[0119] Based on the visual adjustment parameters, the value of the horizontal rotation angle of the face to be adjusted is adjusted to obtain the horizontal rotation angle of the face.

[0120] The sending module 203 is also used to send a door unlocking command to unlock the car door if the horizontal rotation angle of the face is greater than an angle threshold, the rotation direction is a preset direction, and the rotation duration is greater than a time threshold.

[0121] In some embodiments, the processing module 202 is further configured to identify the pupils in the user's first real-time face image and determine the duration of the pupils in various directions within the eye socket;

[0122] The pupil fixation parameters are obtained by summing the products of the weights of the pupils in different directions within the eye socket and the duration of fixation.

[0123] Visual adjustment parameters are determined based on the vertical rotation angle and vertical rotation direction, including:

[0124] Based on the pupil annotation parameters, combined with the vertical rotation angle and vertical rotation direction, the visual adjustment parameters are determined.

[0125] In some embodiments, the processing module 202 is further configured to classify the pupil in various directions in the eye socket according to the duration, and determine the direction corresponding to the first duration, the direction corresponding to the second duration, the direction corresponding to the third duration, and the direction corresponding to the fourth duration.

[0126] Based on the directions corresponding to the first duration, the second duration, the third duration, and the fourth duration, the correlation coefficients of the pupil in each direction within the eye socket are determined.

[0127] The fixation effect achieved by the pupil in each direction within the eye socket is determined based on the correlation coefficients in each direction.

[0128] The weights of the pupil in each direction within the eye socket are determined based on the gaze effect achieved by the pupil in each direction.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0130] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] Figure 3A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] Device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0134] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of method 100 described above may be performed.

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A vehicle door safety control method based on face rotation angle, characterized in that, The method includes: The user's first real-time facial image is acquired using a first camera positioned inside the driver's cab; The user's second face real-time image is obtained based on a second camera arranged in the driver's cab, wherein the second camera and the first camera are arranged at a preset angle in the driver's cab; Determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image; If the rotation trend parameter meets the preset conditions, then send a door unlock command; The face rotation trend parameters include rotation angle, rotation direction, and rotation duration; The step of determining the face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image includes: identifying each facial feature point in the user's first real-time face image and the user's second real-time face image; determining a first horizontal rotation angle, a first horizontal rotation direction, and a corresponding rotation duration based on the total offset corresponding to each facial feature point in the user's first real-time face image; determining a second horizontal rotation angle, a second horizontal rotation direction, and a corresponding rotation duration based on the total offset corresponding to each facial feature point in the user's second real-time face image; and sending a door unlock command if the rotation trend parameters meet preset conditions includes: if the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the face horizontal rotation angle determined based on the first horizontal rotation angle and the second horizontal rotation angle is greater than an angle threshold, the face horizontal rotation direction determined based on the first horizontal rotation direction and the second horizontal rotation direction is a preset direction, and the rotation duration is greater than a time threshold, then a door unlock command is sent to unlock the door; The method further includes: determining a stereoscopic model of the user's head based on each facial feature point in the user's first real-time face image and second real-time face image; determining the distance from each feature to the horizontal rotation center of the face based on the constructed stereoscopic model of the user's head; determining a first horizontal rotation angle and a first horizontal rotation direction, a second horizontal rotation angle and a second horizontal rotation direction based on the total offset corresponding to each facial feature point in the user's first real-time face image and the distance from each feature to the horizontal rotation center of the face; and determining the rotation duration corresponding to the first horizontal rotation angle in the first horizontal rotation direction and the rotation duration corresponding to the second horizontal rotation angle in the second horizontal rotation direction. The method further includes: determining the distance from each feature to the vertical rotation center of the face based on the constructed stereoscopic model of the user's head; determining the vertical rotation angle and vertical rotation direction based on the total offset of each facial feature point in the user's first real-time face image and the distance to the vertical rotation center of the face; and determining visual adjustment parameters based on the vertical rotation angle and vertical rotation direction. If the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, and the horizontal rotation angle of the face determined based on the first horizontal rotation angle and the second horizontal rotation angle is greater than an angle threshold, the horizontal rotation direction of the face determined based on the first horizontal rotation direction and the second horizontal rotation direction is a preset direction, and the rotation duration is greater than a time threshold, then a door unlocking command is sent to unlock the door. This includes: if the first horizontal rotation angle matches the second horizontal rotation angle, the first horizontal rotation direction matches the second horizontal rotation direction, then determining the horizontal rotation angle of the face to be adjusted based on the first horizontal rotation angle and the second horizontal rotation angle; adjusting the value of the horizontal rotation angle of the face to be adjusted based on visual adjustment parameters to obtain the horizontal rotation angle of the face; if the horizontal rotation angle of the face is greater than an angle threshold, the rotation direction is a preset direction, and the rotation duration is greater than a time threshold, then a door unlocking command is sent to unlock the door. The method further includes: identifying the pupils in the user's first real-time face image, determining the duration of the pupils in each direction within the eye socket; and summing the products of the weights corresponding to the pupils in each direction within the eye socket and the duration to obtain pupil fixation parameters. The method of determining visual adjustment parameters based on vertical rotation angle and vertical rotation direction includes: determining visual adjustment parameters based on pupil annotation parameters, combined with vertical rotation angle and vertical rotation direction.

2. The method according to claim 1, characterized in that, If the rotation trend parameter meets the preset conditions, then a door unlocking command is sent, including: If the rotation trend parameter meets the preset conditions, and the vehicle is in a parked state and receives the user's door unlock trigger signal, then a door unlock command is sent.

3. The method according to claim 1, characterized in that, The method further includes: The pupil is classified according to the duration of its position in the eye socket, and the directions corresponding to the first, second, third and fourth durations are determined. Based on the directions corresponding to the first duration, the second duration, the third duration, and the fourth duration, the correlation coefficients of the pupil in each direction within the eye socket are determined. The fixation effect achieved by the pupil in each direction within the eye socket is determined based on the correlation coefficients in each direction. The weights of the pupil in each direction within the eye socket are determined based on the gaze effect achieved by the pupil in each direction.

4. A vehicle door safety control device based on the angle of facial rotation, characterized in that, The apparatus for implementing the method according to any one of claims 1-3 comprises: The acquisition module is used to acquire a real-time image of the user's first face based on a first camera installed in the driver's cab; The acquisition module is further configured to acquire a real-time image of the user's second face based on a second camera arranged in the driver's cab, wherein the second camera is arranged in a different position than the first camera in the driver's cab. The processing module is used to determine face rotation trend parameters based on the user's first real-time face image and the user's second real-time face image; The sending module is used to send a door unlocking command if the rotation trend parameter meets a preset condition.

5. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory is characterized in that it stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method according to any one of claims 1-3.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3.

Images