Automobile automatic door opening method and system based on face recognition and gestures

Abstract

The invention relates to the technical field of automobiles, and discloses an automatic automobile door opening method and system based on face recognition and gestures. According to the method, functions are achieved through an all-dimensional camera outside an automobile, a personnel monitoring camera inside the automobile, a central control screen, a central controller, a stepping motor, a door lock mechanism, a gear sensor, an automobile door lock controller, an automobile door driving motor controller, an automobile lamp controller, a horn controller and an automobile wire harness, and the specific operation steps include the information presetting stage; a user selects a face and a gesture for unlocking and door opening through the central control screen; in the face information collection stage, a camera captures face information; in the face comparison and analysis stage, the central processing unit carries out comparison; in the vehicle door unlocking / locking execution stage, a door lock is controlled according to the comparison result; in the gesture information collection stage, gesture actions are captured; in the vehicle door opening / closing information analysis stage, gestures are recognized, and opening or closing of a vehicle door is judged; and in the vehicle door automatic opening / closing execution stage, the driving motor controls the vehicle door to act.

Description

Technical Field

[0001] The present invention relates to the technical field of automobiles, and particularly to an automatic car door opening method and system based on face recognition and gestures. Background Art

[0002] With the continuous and vigorous development of the automobile industry, people have put forward increasingly stringent requirements for the experience, emotional value, convenience, cool feeling, safety and intelligence level carried by automobiles. As a very common operation in the daily use scenario of automobiles, the opening of car doors has a high usage frequency. Against the background of the continuous improvement of people's material living standards, the demand for car door opening methods is becoming more diverse and the standards are more stringent.

[0003] Currently, the mainstream car door opening methods on the market can be roughly classified into three types. One is the traditional manual opening method. Although this method performs stably and reliably in terms of safety guarantee, it requires the driver or passenger to directly touch the car door for opening operation. Not only are there many inconveniences in the actual use process, it lacks intelligence, but also it is difficult to create a cool car use experience, and it cannot give users additional emotional value.

[0004] The second is the voice, pressing, and key induction opening method. Among them, although voice opening has a certain sense of technology, it is easily interfered by environmental noise, resulting in limited recognition accuracy, and the effective control distance is short. In some specific occasions, it may not be elegant enough because users need to shout out commands loudly; although the pressing opening method has certain guarantees in terms of safety and convenience, it also relies on hand contact operation, with relatively low efficiency, and it cannot meet the user's demand for emotional value. Especially for ladies with manicures, pressing operations may cause inconvenience to them.

[0005] The third is the key and mobile phone APP control opening method. This method requires users to manipulate the key or mobile phone in advance, which is not convenient and efficient in actual operation, and it is difficult to achieve a truly instant and non-intrusive door opening experience.

[0006] In view of the fact that existing various car door opening methods fail to provide effective interaction between people and cars and cannot fully meet the user's demand for diversified door opening methods, there is an urgent need for a new door opening method to fill this market gap. An automatic car door opening method based on face recognition and gestures of the present invention emerges as the times require, aiming to provide users with a new car door opening solution that is cool, convenient, safe and intelligent, so as to meet the high-quality pursuit of modern consumers for car use. Summary of the Invention

[0007] In view of the deficiencies of the prior art, the present invention provides an automatic car door opening method based on face recognition and gestures, which has the advantages of safety, convenience, intelligence and personalization, and solves the problems of inconvenient operation, lack of intelligence and inability to meet the diverse needs of users in traditional door opening methods.

[0008] To achieve the above object, the present invention provides the following technical solution: an automatic car door opening method based on face recognition and gestures. The door opening method relies on an omni-directional camera outside the car, an in-vehicle personnel monitoring camera, a central control screen, a central controller, a stepping motor, a door lock mechanism, a gear sensor, a car door lock controller, a car door drive motor controller, a headlight controller, a horn controller, and a car wiring harness to implement the automatic car door opening function based on face recognition and gestures. The specific operation steps are as follows: Set the distance and time options for camera face and gesture capture in the car function options, and provide common gestures for customers to choose. Set the face and gesture recognition functions in the car central processor. Customers can select suitable parameters through the central control screen and input legal face images. When the vehicle gear is in the P gear, the doors are closed and there are no occupants in the car, the outside camera can capture faces when a person enters the camera coverage area; otherwise, it will not capture. The in-vehicle central processor quickly compares the face captured by the outside camera with the preset face. If the match is successful, it will issue an unlocking instruction to the car door lock controller; if the match fails, there will be no feedback. When no gesture information is received after unlocking within the timeout period, a locking instruction will be issued. After the face match is successful, within the preset time, the outside camera will continuously capture gesture actions. When the car door is unlocked, the vehicle gear is in the P gear and there are no people in the car, the central processor analyzes the relationship between the person and the vehicle's position by comparing the gesture action with the preset gesture to determine whether to open the car door. If the match is successful, it will issue an opening instruction to the car door drive motor controller; otherwise, there will be no instruction. When the car door is in the open state, a locking instruction will be issued when specific conditions are met; otherwise, there will be no instruction. After receiving the instruction, the car door drive motor controller controls the car door drive motor to push the car door to open or close automatically.

[0009] Preferably, the time option specifically includes: the effective time from the successful comparison of the camera face to the issuance of the gesture instruction. If the time is exceeded, the car door will be automatically locked.

[0010] Preferably, the customer can, according to their own needs, select through the central control screen the suitable distances and effective times for car door unlocking and automatic car door opening, and select or customize their favorite door opening gestures.

[0011] Preferably, the face information collection stage includes: the gear sensor of the vehicle continuously monitors the gear state, the door state sensor monitors whether the doors are closed, and the in-vehicle personnel monitoring camera detects whether there are occupants in the car.

[0012] Preferably, the face information collection stage includes: only when the three conditions of the gear being in the P gear, the door being closed, and there being no occupants in the vehicle are simultaneously met, the front, left, right, and rear surround cameras outside the vehicle will activate the face capture function. At the same time, the cameras collect images of the covered area at a preset frame rate. When a face is detected entering the frame, the face information is immediately locked and captured.

[0013] Preferably, the face comparison and analysis stage includes: the central processing unit has a built-in face recognition algorithm. When the face image is captured by the outside vehicle camera, it is quickly converted into a feature vector and compared with the legal face feature vectors pre-stored in the system. The comparison process uses a fast matching algorithm.

[0014] Preferably, the door unlocking / locking execution stage includes: after the door lock controller receives the signal sent by the central processing unit, the microcontroller inside analyzes the signal. When it is an unlocking instruction, the microcontroller drives the stepping motor, and through the gear transmission mechanism, drives the locking tongue in the door lock mechanism to retract, realizing door unlocking. When it is a locking instruction, the stepping motor is driven to reverse, causing the locking tongue to protrude, completing the door locking action.

[0015] Preferably, in the gesture information collection stage: when the central processing unit determines that the face matching is successful, it sends a continuous working instruction to the outside vehicle camera. The outside vehicle camera takes video of the area where the person is located at a high frame rate within a preset time, uses image recognition technology to detect the gesture actions of the person in real time, and transmits the video stream data to the central processing unit through a wire harness.

[0016] Preferably, in the door opening / closing information analysis stage: the central processing unit uses a gesture recognition algorithm to analyze the video stream transmitted by the outside vehicle camera, extracts the gesture features and compares them with the gesture features preset in the system. At the same time, according to the position where the gesture appears in the video streams transmitted by different cameras and the installation position parameters of the cameras, the position of the person relative to the vehicle is calculated.

[0017] Preferably, in the door automatic opening / closing execution stage: after the door drive motor controller receives the instruction from the central processing unit, the power amplifier inside it amplifies the instruction signal and drives the door drive motor to operate.

[0018] An automatic car door opening system based on face recognition and gestures, including a preset module, an identification and capture module, a matching module, and a control module; Preset module: Set the distance and time options for camera face and gesture capture in the car function options, provide common gestures for customers to choose, set the face and gesture recognition functions in the car central processor, and customers can select suitable parameters through the central control screen and input legal face images; Recognition and capture module: When the vehicle gear is in P gear, the door is closed and there are no passengers in the car, the external camera can capture the face when a person enters the camera coverage area, otherwise it will not capture; Matching module: The in-car central processor quickly compares the face captured by the external camera with the preset face. After successful matching, it issues an unlocking instruction to the door lock controller. If the matching fails, there is no feedback. When no gesture information is received after unlocking within the timeout period, it issues a locking instruction; after successful face matching, within the preset time, the external camera will continuously capture gesture actions; Control module: When the door is unlocked, the vehicle gear is in P gear and there is no one in the car, the central processor analyzes the relationship between the person and the vehicle's position by comparing the gesture action with the preset gesture, and determines to open the door. If the matching is successful, it issues an opening instruction to the door drive motor controller, otherwise there is no instruction; when the door is in the open state, it issues a locking instruction when specific conditions are met, otherwise there is no instruction; after receiving the instruction, the door drive motor controller controls the door drive motor to push the door to open or close automatically.

[0019] Compared with the prior art, the present invention provides an automatic car door opening method based on face recognition and gestures, which has the following beneficial effects: An automatic car door opening method based on face recognition and gestures of the present invention realizes the automatic unlocking and opening functions of the car door by collecting and comparing face information, and then combining with the preset gesture actions for comparison and analysis. This method can not only improve the user experience, but also enhance the safety and convenience of customers using the car. Finally, the method of the present invention achieves the beneficial effects of cool, convenient, safe and intelligent automatic door opening in actual use.

[0020] In the method of the present invention, users can choose or customize the unlocking and opening distances, effective times and gesture forms according to their own needs, so that the car internal system can adapt to the usage habits and preferences of different users, thereby enhancing the usage experience of car users. Therefore, through the settings of the in-car camera and the central control screen, the present invention can realize the user's personalized customized door opening method.

[0021] The present invention through the joint cooperation of the in-car and external cameras of the car helps to capture and process face and gesture information in real time to ensure that the central processor can accurately judge and issue corresponding instructions, so as to realize efficient and accurate door control. Therefore, through the collaborative work of multiple cameras, it can ensure the accurate implementation of face recognition and gesture control in the car internal system. Description of the Drawings

[0022] Figure 1 This is a schematic flow chart of the automatic car door opening method based on face recognition and gestures in the specific implementation manner of the present invention.

[0023] Figure 2 This is a schematic diagram of the control principle in the specific implementation manner of the present invention. Specific implementation manner Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Please refer to Figure 1 , an automatic car door opening method based on face recognition and gestures. The door opening method relies on a front surround camera (installed above the front windshield), a left surround camera (installed on the left fender or the base of the outside rearview mirror), a right surround camera (installed on the right fender or the base of the outside rearview mirror), a left side camera (installed at a suitable position on the left B-pillar), a right side camera (installed at a suitable position on the right B-pillar), a rear surround camera (installed at the rear end of the vehicle tailgate / trunk lid), an in-vehicle personnel monitoring camera, a central control screen, a central controller, a stepping motor, a door lock mechanism, a gear position sensor, a door lock controller, a door drive motor controller, a headlight controller, a horn controller, and several automotive wiring harnesses to jointly achieve. The specific operation steps are as follows: Information preset stage: During the development of the car head unit, set the distance and time options for camera face and gesture capture in the car function options (in the development link of the head unit system, engineers need to build adjustable parameter settings in the system, such as the distance range for the camera to capture the face and gestures (for example, 1 - 5 meters), the effective time from the successful face comparison by the camera to the initiation of the gesture command (such as 5 - 30 seconds). At the same time, provide multiple common gesture templates, such as making a fist, waving, etc. for customers to choose), and provide common gestures for customers to choose. Set the face and gesture recognition functions in the car central processor, and customers can select suitable parameters through the central control screen and input legal face images; Face information collection stage: When the vehicle gear is in the P gear, the doors are closed and there are no occupants in the vehicle, the out-of-vehicle cameras can capture the face only when the person enters the area covered by the cameras, otherwise, no capture is performed; Face comparison and analysis stage: The in-vehicle central processor quickly compares the face captured by the out-of-vehicle cameras with the preset face. After successful matching, an unlocking command is sent to the door lock controller. If the matching fails, there is no feedback. When the gesture information is not received after unlocking and the time limit is exceeded, a locking command is sent; Door unlocking / locking execution stage: The door lock controller drives the door lock mechanism according to the instruction to achieve the unlocking or locking function; Gesture information collection stage: After successful face matching, within a preset time, the external camera will continuously capture gesture actions; Door opening / closing information analysis stage: When the door is unlocked, the vehicle gear is in P gear, and there is no one in the vehicle, the central processing unit compares the gesture action with the preset gesture, analyzes the position relationship between the person and the vehicle to determine to open the door, and issues an opening command to the door drive motor controller if the match is successful, otherwise no command; When the door is in the open state, a locking command is issued when specific conditions are met (such as no operation within the preset time or a closing gesture is captured and the match is successful), otherwise no command.

[0025] Door automatic opening / closing execution stage: After receiving the command, the door drive motor controller controls the door drive motor to push the door to open or close automatically.

[0026] Specifically, the time options include: the effective time from successful face comparison by the camera to initiating the gesture command. If the time is exceeded, the door will automatically lock.

[0027] Specifically, customers can, according to their own needs, through the central control screen, select the distance and effective time suitable for themselves for door unlocking and automatic door opening, select or customize their favorite door opening gestures, and input legal face images through the in-vehicle camera and the vehicle head unit for the system to compare and analyze (customers operate the central control screen, select appropriate distance and time parameters according to their own needs. In terms of customizing gestures, the system needs to provide operation guidance for customers to record gestures through the external camera. After recording, the system automatically generates feature data for storage. When the face image is input, the in-vehicle camera is activated, and customers complete multi-angle face collection according to the prompts. The system stores it as a feature vector for subsequent comparison).

[0028] Specifically, the face information collection stage includes: The gear sensor of the vehicle monitors the gear state in real time, the door state sensor monitors whether the door is closed, and the in-vehicle occupant monitoring camera detects whether there are occupants in the vehicle.

[0029] Specifically, the face information collection stage includes: Only when the three conditions of the gear being in P gear, the door being closed, and no occupants in the vehicle are simultaneously met, the front panoramic, left panoramic, right panoramic, and rear panoramic cameras outside the vehicle will activate the face capture function. At the same time, the cameras collect images of the covered area at a preset frame rate (such as 30 - 60 frames per second). When a face is detected entering the picture, it immediately locks and captures the face information.

[0030] Specifically, the face comparison and analysis stage includes: The central processing unit (CPU) has a dedicated face recognition algorithm built-in. When the external camera captures a face image, it quickly converts it into a feature vector and compares it with the legal face feature vectors pre-stored in the system. The comparison process uses a fast matching algorithm to shorten the response time (e.g., complete the comparison within 1 second).

[0031] Specifically, the middle door unlocking / locking execution stage includes: After the door lock controller receives the signal sent by the central processing unit, the microcontroller inside it analyzes the signal. If the door shows an unlocking instruction, the microcontroller drives the stepper motor, and through the gear transmission mechanism, drives the locking tongue in the door lock mechanism to retract, realizing door unlocking; if the door shows a locking instruction, it drives the stepper motor to reverse, making the locking tongue protrude to complete the door locking action.

[0032] Specifically, when the central processing unit determines that the face match is successful, it sends a continuous working instruction to the external camera. The external camera takes a video of the area where the person is located at a high frame rate (e.g., 60 frames per second) within a preset time (e.g., 10 seconds), uses image recognition technology to detect the gesture actions of the person in real time, and transmits the video stream data to the central processing unit through the wire harness.

[0033] Specifically, the central processing unit uses a gesture recognition algorithm to analyze the video stream transmitted by the external camera, extracts the gesture features and compares them with the gesture features preset in the system. At the same time, according to the position where the gesture appears in the video stream transmitted by different cameras and the installation position parameters of the cameras, it calculates the position of the person relative to the vehicle. For example, if the gesture captured by the left panoramic camera matches successfully and the person's position is close to the left door, it is determined to open the left door. When the door is in the open state, the in-vehicle personnel monitoring camera continuously monitors the situation inside the vehicle. If there is no personnel entering or leaving and no operation within a preset time (e.g., 30 seconds), or the external camera captures a closing gesture and it matches successfully, the central processing unit sends a closing instruction (low-level signal) to the door drive motor controller; otherwise, no instruction is sent.

[0034] Specifically, after the door drive motor controller receives the instruction from the central processing unit, the power amplifier inside it amplifies the instruction signal, drives the door drive motor to operate. The motor pushes the door to slide along the guide rail through a transmission device (such as a chain or a connecting rod mechanism) to realize automatic opening or closing of the door. During the movement of the door, the door drive motor controller monitors the motor current and position feedback signals in real time to ensure the smooth operation of the door. When an abnormal increase in the motor current occurs due to an obstacle, the motor operation is immediately stopped to prevent damage to the door and related components.

[0035] In a specific embodiment of the present invention, an automatic car door opening system based on face recognition and gestures is provided, which includes a presetting module, a recognition and capture module, a matching module, and a control module; The presetting module sets the distance and time options for camera face and gesture capture in the car function options, provides common gestures for customers to choose, sets the face and gesture recognition functions in the car central processor, and customers can select suitable parameters through the central control screen and input legal face images; The recognition and capture module can capture the face by the external camera of the car only when the vehicle gear is in the P gear, the car door is closed and there is no occupant in the car, and when a person enters the coverage area of the camera. Otherwise, it will not capture; The matching module quickly compares the face captured by the external camera of the car with the preset face by the in-car central processor. After successful matching, an unlocking instruction will be sent to the car door lock controller. If the matching fails, there will be no feedback. When no gesture information is received after unlocking within the timeout period, a locking instruction will be sent; after successful face matching, within the preset time, the external camera of the car will continuously capture gesture actions; The control module, when the car door is unlocked, the vehicle gear is in the P gear and there is no one in the car, the central processor compares the gesture action with the preset gesture, analyzes the position relationship between the person and the vehicle to determine to open the car door, and sends an opening instruction to the car door drive motor controller after successful matching, otherwise there is no instruction; when the car door is in the open state, a locking instruction will be sent when specific conditions are met, otherwise there is no instruction; after receiving the instruction, the car door drive motor controller controls the car door drive motor to push the car door to open or close automatically.

[0036] Embodiment 1 After shopping, the user walks towards the vehicle with both hands full of items. The camera of the vehicle captures the face set by the user. After successful comparison by the central processor, the car door is unlocked. The user makes an opening gesture, and the central processor determines to open the trunk door according to the gesture information and the position of the external camera of the car. The car door drive motor controller drives the trunk door to open automatically, facilitating the user to place items.

[0037] Embodiment 2 The car owner and family members walk to the car. The vehicle camera captures the face information of the car owner and family members. After successful comparison by the central processor, the car door is unlocked. The car owner's child makes an opening gesture outside the car, and the central processor determines to open the rear row car door to facilitate the child to get in the car. After the family gets in the car, within the preset time, the in-car personnel monitoring camera monitors that there are people in the car, and the car door will not close automatically. After everyone is seated, the car owner closes the car door through in-car operations.

[0038] Embodiment 3 After finishing a day's work, the car owner walks towards his / her car. When the vehicle detects that the car owner enters the area covered by the camera, all the external cameras (front surround camera, left surround camera, and right surround camera) start capturing the face information. The central processing unit quickly compares the captured face with the preset face in the system. After successful matching, the central processing unit issues an unlocking instruction to the car door lock controller. The car owner makes a pre-set door-opening gesture within the preset time. After the external camera captures the gesture, the central processing unit analyzes and judges to open the driver's door according to the gesture information. The car door drive motor controller receives the door-opening instruction and controls the car door drive motor to push the car door open automatically.

[0039] Comparative Example 1 For current existing vehicles, when the car owner walks to the car and tries to open the car door through voice commands, but the surrounding environmental noise is relatively high and the voice recognition is not successful. So the car owner presses the button on the car door handle to unlock the car door and then pulls it open to get in the car. When getting out of the car, the car owner presses the button again to lock the car.

[0040] Comparative Example 2 For current existing vehicles, when the car owner is about to drive, he / she takes out the car key in advance and presses the unlock button, or opens the mobile APP and operates to unlock the car door on the APP. After walking to the car, the car owner pulls open the car door to get in. After getting out of the car, the car owner locks the car through the key or the APP.

[0041] Comparative Example 3 For current existing vehicles, when the car owner walks to the car, takes out the car key, manually inserts the key into the keyhole or presses the unlock button on the key to unlock the car door, and then pulls open the car door to get in. When getting out of the car, the car owner manually closes the car door and locks it.

[0042] The following table lists the data values after comparing the embodiments with the comparative examples. The table compares the performance of the door-opening methods in different scenarios in terms of convenience, degree of intelligence, emotional value, coolness, safety, and applicable scenarios, and quantifies them with numerical values (1 - 5 points, 1 point is the lowest and 5 points is the highest). The comparison data is as follows:

[0043] The following information is obtained from the above table: Convenience: In Embodiments 1 / 2 / 3, there is no need for manual operation, and the door is directly opened through face recognition and gestures, which is especially suitable for the situation where both hands are occupied and is very convenient (5 points); in Comparative Example 1, the voice recognition method is used, which is greatly affected by the environment and the pressing operation efficiency is relatively low (3 points); in Comparative Example 2, it is necessary to operate the key or APP in advance, which is not convenient enough (2 points); in Comparative Example 3, the manual operation is cumbersome, especially when both hands are occupied (1 point).

[0044] Intelligence: In Examples 1 / 2 / 3, face recognition and gesture recognition are achieved through cameras and central processors, with a high degree of intelligence (5 points); in Comparative Example 1, voice and pressing operations have a certain degree of intelligence, but are limited by the environment and operation methods (3 points); in Comparative Example 2, it relies on keys or APPs, with a low degree of intelligence (2 points); in Comparative Example 3, it is completely manual operation without intelligence (1 point).

[0045] Emotional value: Examples 1 / 2 / 3 can all provide a cool door-opening experience, enhancing the emotional value of users (4 points); in Comparative Example 1, the operation is relatively cumbersome, with a low emotional value (2 points); in Comparative Example 2, the operation is ordinary without emotional value (1 point); in Comparative Example 3, the traditional manual operation has no emotional value (1 point).

[0046] Cool feeling: In Examples 1 / 2 / 3, face recognition and gesture door-opening bring a full sense of technology and coolness (5 points); in Comparative Example 1, voice and pressing operations are relatively ordinary (2 points); in Comparative Example 2, key or APP operations have no cool feeling (1 point); in Comparative Example 3, traditional manual operations have no cool feeling (1 point).

[0047] Security: In Examples 1 / 2 / 3, face recognition and gesture recognition require dual verification, with high security (5 points); in Comparative Example 1, voice recognition is affected by the environment, and the security of pressing operations is (3 points); in Comparative Example 2, the security of key or APP operations is (3 points); in Comparative Example 3, the traditional key operation has relatively high security but is easy to lose (4 points).

[0048] Applicable scenarios: Examples 1 / 2 / 3 are applicable to a variety of scenarios, especially when both hands are occupied or quick access to the vehicle is needed (4 points); in Comparative Example 1, it is affected by environmental noise, with limited applicable scenarios (2 points); in Comparative Example 2, it requires prior operation, with applicable scenarios (3 points); in Comparative Example 3, it is only applicable to situations where manual operation is convenient (2 points).

[0049] Overall score: Examples 1 / 2 / 3 have excellent comprehensive performance, with an average score of 4.8 points; Comparative Example 1 has poor comprehensive performance, with a score of 2.3 points; Comparative Example 2 has poor comprehensive performance, with a score of 2.0 points; Comparative Example 3 has the worst comprehensive performance, with a score of 1.4 points.

[0050] In summary, through the information comparison between the above examples and comparative examples, it can be seen that the door-opening method based on face recognition and gesture recognition is superior to the traditional door-opening method in multiple dimensions and can better meet users' needs for convenience, intelligence, emotional value, and security.

[0051] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic car door opening method based on face recognition and gestures, characterized in that, Specifically, it includes the following steps: Set the distance and time options for camera face and gesture capture in the vehicle function options, and provide common gestures for customers to choose. Set the face and gesture recognition functions in the vehicle central processor. Customers can select suitable parameters through the central control screen and input legal face images; When the vehicle gear is in P gear, the door is closed and there are no passengers in the vehicle, the external camera can capture the face when a person enters the camera coverage area, otherwise it will not capture; The in-vehicle central processor quickly compares the face captured by the external camera with the preset face. After successful matching, it issues an unlocking instruction to the door lock controller. If the matching fails, there is no feedback. When no gesture information is received after unlocking and the time limit is exceeded, a locking instruction is issued; after successful face matching, within the preset time, the external camera will continuously capture gesture actions; When the door is unlocked, the vehicle gear is in P gear and there is no one in the vehicle, the central processor compares the gesture action with the preset gesture, analyzes the position relationship between the person and the vehicle to determine to open the door. After successful matching, it issues an opening instruction to the door drive motor controller, otherwise there is no instruction; when the door is in the open state, a locking instruction is issued when specific conditions are met, otherwise there is no instruction; after receiving the instruction, the door drive motor controller controls the door drive motor to push the door to open or close automatically.

2. The method for automatically opening a car door based on face recognition and gestures according to claim 1, wherein: The specific time option includes: the effective time from successful face comparison by the camera to initiating the gesture instruction. If the time is exceeded, the door will be automatically locked.

3. The automatic car door opening method based on face recognition and gestures according to claim 1, characterized in that: The customer can, according to their own needs, through the central control screen, select the distance and effective time suitable for themselves for door unlocking and automatic door opening, and select or customize their favorite door opening gestures.

4. The automatic vehicle door opening method based on face recognition and gestures according to claim 1, wherein: The face information collection stage includes: the gear sensor of the vehicle monitors the gear state in real time, the door state sensor monitors whether the door is closed, and the in-vehicle personnel monitoring camera detects whether there are passengers in the vehicle.

5. A method for automatically opening a car door based on face recognition and gestures according to claim 1 or 4, characterized in that: The face information collection stage includes: only when the three conditions of the gear being in P gear, the door being closed and there being no passengers in the vehicle are simultaneously met, the front panoramic, left panoramic, right panoramic and rear panoramic cameras outside the vehicle will start the face capture function. At the same time, the camera collects images of the coverage area at the preset frame rate. When a face is detected entering the picture, it immediately locks and captures the face information.

6. The automatic car door opening method based on face recognition and gestures according to claim 1, characterized in that: The face comparison and analysis stage includes: the central processor has a built-in face recognition algorithm. When the external camera captures a face image, it quickly converts it into a feature vector and compares it with the legal face feature vector pre-stored in the system. The comparison process uses a fast matching algorithm.

7. A method for automatically opening a car door based on face recognition and gestures according to claim 1, characterized in that: The door unlocking / locking execution stage includes: after the door lock controller receives the signal sent by the central processor, the microcontroller inside analyzes the signal. When it is an unlocking instruction, the microcontroller drives the stepping motor, and through the gear transmission mechanism, drives the lock tongue in the door lock mechanism to retract, realizing door unlocking. When it is a locking instruction, it drives the stepping motor to reverse, making the lock tongue protrude to complete the door locking action.

8. The automatic car door opening method based on face recognition and gestures according to claim 1, characterized in that: In the gesture information collection stage: After the central processing unit determines that the face match is successful, it sends a continuous working instruction to the external camera. The external camera takes video of the area where the person is located at a high frame rate within a preset time, uses image recognition technology to detect the gesture actions of the person in real time, and transmits the video stream data to the central processing unit through a wiring harness.

9. The method for automatically opening a car door based on face recognition and gestures according to claim 1, wherein: In the door opening / closing information analysis stage: The central processing unit uses a gesture recognition algorithm to analyze the video stream transmitted by the external camera, extracts the gesture features and compares them with the preset gesture features in the system. At the same time, according to the position where the gesture appears in the video stream transmitted by different cameras and the installation position parameters of the cameras, it calculates the position of the person relative to the vehicle. In the door automatic opening / closing execution stage: After the door drive motor controller receives the instruction from the central processing unit, the power amplifier inside it amplifies the instruction signal and drives the door drive motor to operate.

10. An automatic car door opening system based on face recognition and gestures, characterized in that: It includes a presetting module, an identification and capture module, a matching module, and a control module; The presetting module sets the distance and time options for camera face and gesture capture in the vehicle function options, provides common gestures for customers to choose, sets the face and gesture recognition functions in the vehicle central processing unit, and customers can select suitable parameters through the central control screen and input legal face images; The identification and capture module: When the vehicle gear is in the P gear, the door is closed, and there are no passengers in the vehicle, the external camera can capture the face only when the person enters the camera coverage area, otherwise it does not capture; The matching module: The in-vehicle central processing unit quickly compares the face captured by the external camera with the preset face. After a successful match, it issues an unlocking instruction to the door lock controller. If the match fails, there is no feedback. When no gesture information is received after unlocking and the time limit is exceeded, a locking instruction is issued; After the face match is successful, the external camera will continuously capture gesture actions within a preset time; The control module: When the door is unlocked, the vehicle gear is in the P gear, and there is no one in the vehicle, the central processing unit compares the gesture action with the preset gesture, analyzes the position relationship between the person and the vehicle to determine whether to open the door. If the match is successful, it issues an opening instruction to the door drive motor controller, otherwise there is no instruction; When the door is in the open state, a locking instruction is issued when specific conditions are met, otherwise there is no instruction; After receiving the instruction, the door drive motor controller controls the door drive motor to push the door to open or close automatically.

Images