Tailgate recognition method, apparatus and vehicle door unlocking method, system, and vehicle

Abstract

This invention discloses a tailgating detection method, device, vehicle door unlocking method, system, and vehicle, relating to the field of vehicle technology. The tailgating detection method includes: in response to detecting a vehicle owner approaching the vehicle, identifying the location of the vehicle owner's sound source based on audio data surrounding the vehicle; and identifying whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's sound source and point cloud data surrounding the vehicle. This effectively identifies whether the vehicle owner is at risk of being tailgated, thereby improving the security and intelligence level of the vehicle door unlocking process. Therefore, by using intelligent connected vehicle technology, at least two modalities of information—audio localization and point cloud perception—are integrated, and the sound source direction guides the point cloud data for precise analysis within a specific spatial area, effectively identifying whether the vehicle owner is at risk of being tailgated, thus contributing to improving the security and intelligence level of the vehicle door unlocking process.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a tailgating identification method, device, door unlocking method, system, and vehicle. Background Technology

[0002] With the development of automotive intelligence, identity verification, as the entry point for user interaction with the vehicle, directly impacts the user experience. Current mainstream solutions still rely on a "key-based" mechanism. Whether it's a mechanical key, electronic remote key, or NFC (Near Field Communication) digital key, all have the inherent flaw of "recognizing objects but not people." If the key is lost or attacked, vehicle control is completely lost. While some high-end models have introduced single biometric recognition methods such as facial recognition and fingerprints, these have poor environmental adaptability, with recognition rates significantly decreasing in low light or obstructed conditions, and are vulnerable to forgery attacks such as 3D-printed masks.

[0003] Therefore, judging solely by whether the key is present or whether a single biometric feature matches cannot identify whether a stranger is following the owner when unlocking the car door, posing a security risk of a follower forcibly getting into the car and hijacking it. Summary of the Invention

[0004] The purpose of this invention is to provide a tailgating identification method, device, vehicle door unlocking method, system, and vehicle to effectively identify whether a vehicle owner is at risk of being tailgated, thereby helping to improve the security and intelligence level of the vehicle door unlocking process.

[0005] In a first aspect, embodiments of the present invention propose a tailgating identification method, comprising: in response to identifying that a vehicle owner is approaching the vehicle, identifying the location of the vehicle owner's sound source based on audio data around the vehicle; and identifying whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's sound source and point cloud data around the vehicle.

[0006] In some embodiments, the vehicle owner is identified as being near the vehicle by at least one of the following methods: identifying whether the vehicle owner is near the vehicle based on image data around the vehicle; or identifying whether the vehicle owner is near the vehicle based on a trigger signal generated when a microswitch of the vehicle is triggered.

[0007] In some embodiments, identifying whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's voice source and point cloud data around the vehicle includes: performing three-dimensional face recognition on the point cloud data to identify whether the face at the location of the vehicle owner's voice source matches the vehicle owner; if a match is found and the number of faces is 1, then it is determined that the vehicle owner is not at risk of being tailgated; if a match is found and the number of faces is greater than 1, then the first location information of the vehicle owner and at least one second location information of a non-vehicle owner are obtained based on the three-dimensional face recognition result, and the vehicle owner is identified as being at risk of being tailgated based on the first location information and the second location information.

[0008] In some embodiments, identifying whether the vehicle owner is at risk of being tailgated based on the first location information and the second location information includes: calculating the distance between the vehicle owner and each of the non-vehicle owners based on the first location information and the second location information; if at least one of the distances is less than a preset distance threshold, then determining that the vehicle owner is at risk of being tailgated; if all the distances are greater than or equal to the preset distance threshold, then determining that the vehicle owner is not at risk of being tailgated.

[0009] In some embodiments, after determining that the vehicle owner is at risk of being followed, the method further includes: issuing a tailgating warning message; if the vehicle owner confirms that they are not being followed by the tailgating warning message, the method determines that the vehicle owner is not at risk of being followed.

[0010] Secondly, embodiments of the present invention provide a method for unlocking a car door, comprising: performing tailgating identification using the tailgating identification method described in the first aspect embodiment; and performing a car door unlocking operation in response to the identification that the car owner is not at risk of being tailgated.

[0011] Thirdly, embodiments of the present invention propose a tailgating detection device, comprising: an image acquisition module for acquiring image data around a vehicle; an audio acquisition module for acquiring audio data around the vehicle; a point cloud acquisition module for acquiring point cloud data around the vehicle; and a controller connected to the image acquisition module, the audio acquisition module, and the point cloud acquisition module, respectively. The controller is configured to: identify whether a vehicle owner is approaching the vehicle based on the image data; and when the vehicle owner is identified as approaching the vehicle, identify the location of the vehicle owner's sound source based on the audio data around the vehicle; and identify whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's sound source and the point cloud data around the vehicle.

[0012] In some embodiments, the image acquisition module, the audio acquisition module, and the point cloud acquisition module are all mounted on the B-pillar on the driver's side of the vehicle, and are arranged vertically from top to bottom as follows: the point cloud acquisition module, the audio acquisition module, and the image acquisition module.

[0013] Fourthly, embodiments of the present invention provide a vehicle door unlocking system, including: a vehicle door; and the tailgating identification device described in the third aspect embodiment; wherein, the controller in the tailgating identification device is further configured to: perform a vehicle door unlocking operation when it is detected that the vehicle owner is not at risk of being tailgated.

[0014] Fifthly, embodiments of the present invention provide a vehicle, including: the door unlocking system described in the fourth aspect embodiment.

[0015] The tailgating identification method, device, door unlocking method, system, and vehicle of this invention, when performing tailgating identification, respond to the detection of a vehicle owner approaching the vehicle, identify the location of the vehicle owner's sound source based on audio data around the vehicle; and identify whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's sound source and point cloud data around the vehicle. Therefore, by fusing at least two modalities of information—audio localization and point cloud perception—and using the direction of the sound source to guide the point cloud data for precise analysis within a specific spatial area, the risk of a vehicle owner being tailgated can be effectively identified, thereby helping to improve the security and intelligence level of the door unlocking process. Attached Figure Description

[0016] Figure 1 This is a flowchart of the tail-following identification method according to an embodiment of the present invention; Figure 2 This is a structural diagram of an embodiment of the tail-following identification method of the present invention; Figure 3 This is a structural diagram of another embodiment of the tail-following identification method of the present invention; Figure 4 This is a schematic diagram illustrating the configuration of each acquisition module in an example of the present invention; Figure 5 This is a schematic diagram illustrating a scenario of integrated authentication, as exemplified by the present invention. Figure 6 This is a flowchart of a car door unlocking method according to an embodiment of the present invention; Figure 7 This is a flowchart of a specific embodiment of a car door unlocking method according to the present invention; Figure 8 This is a structural block diagram of the tail-following identification device according to an embodiment of the present invention; Figure 9 This is a structural block diagram of the vehicle door unlocking system according to an embodiment of the present invention; Figure 10 This is a structural block diagram of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0017] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0018] The following description, with reference to the accompanying drawings, describes a tailgating identification method, device, door unlocking method, system, and vehicle according to embodiments of the present invention.

[0019] Figure 1 This is a flowchart of a tailgating identification method according to an embodiment of the present invention. The tailgating identification method can be executed by an on-board electronic device, which can be a vehicle domain controller or an on-board central computing platform. It can also be a SoC (System on Chip) or MCU (Microcontroller Unit) working together to realize the tailgating identification function, or it can be an independent microprocessor, DSP (Digital Signal Processor) or ASIC (Application-Specific Integrated Circuit).

[0020] like Figure 1 As shown, the tail-following identification method includes the following steps: S11, in response to recognizing that the car owner is approaching the vehicle, identifies the location of the car owner's sound source based on audio data around the vehicle.

[0021] In this embodiment, whether the vehicle owner is near the vehicle can be identified by at least one of the following methods: Based on image data around the vehicle, it can identify whether the vehicle owner is approaching the vehicle; The system identifies whether the vehicle owner is near the vehicle based on the trigger signal generated when the vehicle's microswitch is activated.

[0022] Specifically, the image data can be in RGB (Red, Green, Blue) format, YUV (luminance and chrominance separation) format, or RGB-D (Red, Green, Blue Depth) format, and can be acquired by an image acquisition module located on the exterior of the vehicle. Taking RGB format as an example, the image acquisition module may include a low-power RGB camera, which can be mounted on the B-pillar (e.g., driver's side), exterior rearview mirror, or door handle. A microswitch can be located inside the driver's side door handle to generate a trigger signal in response to pulling or pressing the door handle.

[0023] In some examples, when a pedestrian enters the FOV (Field of View) of the image acquisition module, target tracking technology is used to determine if a pedestrian is approaching the vehicle. If a pedestrian is approaching, gait pre-identification is performed: if the gait feature pre-identification result indicates that the pedestrian is a registered vehicle owner, the fusion authentication function is activated; if the gait feature pre-identification result does not identify the registered vehicle owner, the fusion authentication function is activated in response to the pressing of a microswitch. The fusion authentication function refers to the function of fusing audio data and point cloud data for vehicle owner authentication.

[0024] Optionally, the fusion authentication function can be activated by gait identity pre-identification alone, or by the pressing of a microswitch alone.

[0025] Audio data can be collected by an audio acquisition module located on the exterior of the vehicle. This module may include a microphone array, which can be mounted on the B-pillar (e.g., driver's side), exterior rearview mirror, or door area. The microphone array is used to collect ambient sounds around the vehicle, including the owner's voice commands, footsteps, or sounds produced by approaching actions. Based on the sound signals received by the microphone array, voice recognition (e.g., verifying preset passwords) and voiceprint recognition can be performed to identify the owner. Furthermore, based on the time difference, phase difference, or energy difference of the sound received by each element of the microphone array when the owner speaks, the spatial azimuth angle of the owner's sound source relative to the vehicle can be calculated, thereby determining the location of the owner's sound source.

[0026] Therefore, by triggering signals or image data through micro-switches, the system can quickly sense the driver's intention to approach and trigger the tailgating recognition process. Then, by accurately locating the driver's voice source through audio data, it provides spatial guidance for the targeted analysis of subsequent point cloud data, thereby reducing the computational load of point cloud data processing and improving the real-time performance and accuracy of tailgating recognition.

[0027] S12 identifies whether the driver is at risk of being followed based on the location of the driver's voice source and point cloud data around the vehicle.

[0028] The point cloud data can be a set of three-dimensional spatial coordinate points, acquired by a point cloud acquisition module. This point cloud acquisition module may include a 3D ToF (Time of Flight) module. The 3D ToF module can be installed on the B-pillar of the vehicle (e.g., on the driver's side), below the exterior rearview mirror, or at the door handle. The acquisition principle is as follows: The ToF camera in the 3D ToF module sends a beam of light to the target object; the wavelength of this light is usually infrared. When this beam of light is reflected back by the target object, the PD (Photodiode) array of the ToF camera receives the reflected light and measures the time of flight of the light. Based on the ToF measurement technology, the time difference between the light traveling from the ToF camera to the target object and back to the ToF camera is calculated. Combined with the principle of the constancy of the speed of light, the distance between the object and the camera is accurately determined, ultimately generating point cloud data containing the object's three-dimensional spatial position information.

[0029] In this embodiment, based on the location of the vehicle owner's voice source identified in S11, spatial filtering is performed on the point cloud data around the vehicle to extract point cloud information within the corresponding spatial area. The extracted point cloud information is analyzed to identify human targets within that area. If the identified human target does not match the vehicle owner's identity, it is determined that there is no legitimate vehicle owner in that area, and the vehicle can be kept locked and an alarm can be issued. If the identified human target matches the vehicle owner's identity, the point cloud data in other areas besides the vehicle owner's voice source location is further analyzed to identify whether other human targets exist. If they do, the distance between the other human targets and the vehicle owner's point cloud information is calculated, and the risk of the vehicle owner being followed is determined based on the distance.

[0030] Therefore, by guiding point cloud data to a specific spatial area through the direction of the sound source for targeted analysis, computational redundancy caused by full-range point cloud processing is avoided, effectively improving recognition efficiency and real-time performance. This enables accurate detection of tailgating individuals, thereby effectively ensuring security when unlocking the car doors. Furthermore, the point cloud acquisition module proactively acquires suspicious tailgating risks within the field of view, eliminating the need for users to manually identify risks, thus offering a higher level of intelligence and security compared to passive confirmation methods.

[0031] Taking the collaborative work of SoC and MCU to implement tailgating recognition function as an example, such as Figure 2 As shown: The image acquisition module transmits the acquired image data to the first deserializer for serial data deserialization and data reconstruction. The reconstructed image data can be transmitted to the SoC via MIPI (Mobile Industry Processor Interface) for vehicle occupancy detection. The point cloud acquisition module transmits the acquired point cloud data to the second deserializer for serial data deserialization and format conversion. The converted point cloud data can be transmitted to the SoC via MIPI. The audio acquisition module transmits the acquired audio data to the SoC via PDM (Pulse Density Modulation) data stream, and combines it with the point cloud data for fusion authentication processing.

[0032] After the SoC completes the fusion authentication process, it sends the tailgating identification result to the MCU (Microcontroller Unit) via ETH (Ethernet) for anti-tailgating verification.

[0033] After the MCU completes the anti-tailgating verification, it generates corresponding control commands based on the verification results and executes the commands on different mechanical structures through the CAN (Controller Area Network) / LIN (Local Interconnect Network) bus. For example, when it is determined that there is no risk of being tailgated, it sends an unlock command to the door lock actuator to unlock the door, and can also send a welcome light turn-on command to control the welcome light to turn on, and can also send a seat adjustment command to the seat adjustment motor to adjust the seat; when it is determined that there is a risk of being tailgated, it sends an alarm command to the alarm device and keeps the door locked.

[0034] Thus, through the division of labor and collaboration between SoC and MCU, SoC focuses on the efficient fusion computing of multimodal data such as images, point clouds, and audio, while MCU is responsible for security decisions and low-level execution control, thereby achieving balanced computing load and improved system reliability.

[0035] For example, the SoC may include an identity pre-identification module and a converged authentication module, and the MCU may include an anti-tailgating authentication module. A schematic diagram of the interaction between the modules is shown below. Figure 3 As shown.

[0036] In practice, personnel (including users, drivers, and vehicle owners) send different types of input information to the vehicle, including visual information (collected by the image acquisition module and the point cloud acquisition module) and voice information (collected by the audio acquisition module), to trigger and participate in the door unlocking process. The vehicle is equipped with an image acquisition module, a point cloud acquisition module, and an audio acquisition module, used to collect pedestrian gait features, facial features (point cloud), and voice features, respectively. The identity pre-identification module pre-identifies the person's identity by acquiring their gait features to determine if a vehicle owner is approaching the vehicle, and sends a wake-up command to the fusion authentication module when a vehicle owner is detected. The fusion authentication module simultaneously acquires point cloud feature data (such as 3D point cloud) and audio information, and uses multimodal fusion verification to authenticate the person in the area where the vehicle owner's voice source is located. The anti-tailgating verification module calculates the probability of tailgating risk based on the observable personnel point cloud data within the field of view in front of the B-pillar, and proactively alerts the vehicle owner when a risk exists; if safety is confirmed, it sends an unlocking command to the unlocking module. The unlocking module receives the unlocking command and executes the door unlocking action, while also controlling the welcome light module to start relevant welcome services.

[0037] It should be noted that before performing multimodal data fusion processing, the data collected by different types of sensors, such as the image acquisition module, audio acquisition module, and point cloud acquisition module, can be preprocessed according to ambient light and ambient noise information to generate a unified data format for subsequent identity recognition and tailgating risk analysis.

[0038] In some embodiments of the present invention, the image acquisition module, point cloud acquisition module, and audio acquisition module are all arranged on the B-pillar on the driver's side of the vehicle. Integrating the above sensors into the B-pillar allows the driver to directly enter the driver's seat after identity verification, while ensuring that each sensor obtains a large effective field of view.

[0039] For example, the point cloud acquisition module is positioned near the upper part of the B-pillar, which is beneficial for capturing facial depth information of people near the B-pillar, thereby improving the accuracy of subsequent face recognition and 3D spatial positioning. The audio acquisition module is positioned in the middle of the B-pillar, which can collect audio information from different sound sources as much as possible, which is beneficial for improving the accuracy of voiceprint recognition and the precision of sound source localization. The image acquisition module uses a low-power wide-angle camera and is positioned near the lower part of the B-pillar. It is mainly responsible for collecting gait feature information of people approaching the vehicle, capturing gait movement details, thereby improving the accuracy of gait recognition.

[0040] In some embodiments of the present invention, identifying whether a vehicle owner is at risk of being tailgated based on the location of the vehicle owner's voice source and point cloud data around the vehicle includes: performing three-dimensional face recognition on the point cloud data to identify whether the face at the location of the vehicle owner's voice source matches the vehicle owner; if a match is found and the number of faces is 1, then it is determined that the vehicle owner is not at risk of being tailgated; if a match is found and the number of faces is greater than 1, then the vehicle owner's first location information and at least one non-vehicle owner's second location information are obtained based on the three-dimensional face recognition results, and the vehicle owner is identified as being at risk of being tailgated based on the first location information and the second location information.

[0041] Specifically, such as Figure 5 As shown, when the fusion authentication module is activated, the audio acquisition module collects and recognizes all audio data within its receiving range. If a registered vehicle owner's identity is identified, the corresponding location information is obtained through sound source localization technology; otherwise, it enters a sleep state. After obtaining the vehicle owner's sound source location, the point cloud acquisition module selectively crops the corresponding facial point cloud from the collected point cloud data, performs data preprocessing and recognition, and obtains the number of faces within the point cloud acquisition module's field of view. If the recognition result is a registered vehicle owner, a verification success signal is sent, and the number of faces is sent to the anti-tailgating verification module.

[0042] After receiving a successful verification signal, the anti-tailgating verification module determines whether the number of faces is no more than 1: if the number of faces is no more than 1, it determines that there is no risk of being tailgated, unlocks the car door and starts the relevant welcoming service; otherwise (i.e., the number of faces is greater than 1), it obtains the first location information of the car owner and at least one second location information of a non-car owner based on the 3D face recognition results, and identifies whether the car owner is at risk of being tailgated based on the first location information and the second location information.

[0043] This method integrates voiceprint recognition, sound source localization, and 3D face recognition technologies, and has the following technical advantages: 1) Multimodal fusion authentication improves identity recognition accuracy By combining voiceprint recognition technology with point cloud facial recognition technology, dual verification of the vehicle owner's identity is achieved, thereby ensuring high accuracy and reliability of recognition and effectively resisting the risk of single-modal forgery attacks.

[0044] 2) Multimodal assisted verification reduces computing power consumption By acquiring and recognizing 3D point cloud facial data from the location of the vehicle owner's voice source after successful voiceprint recognition, the computational cost of cropping all facial point clouds from the point cloud acquisition module and performing recognition can be significantly reduced. This method significantly reduces the overall computational cost of 3D point cloud facial recognition by only slightly increasing the computational cost of voiceprint recognition, which has a lower unit computational requirement, thus achieving optimized allocation of computing resources.

[0045] In some examples, identifying whether a vehicle owner is at risk of being tailgated based on first and second location information includes: calculating the distance between the vehicle owner and each non-vehicle owner based on the first and second location information; if at least one distance is less than a preset distance threshold, then it is determined that the vehicle owner is at risk of being tailgated; if all distances are greater than or equal to the preset distance threshold, then it is determined that the vehicle owner is not at risk of being tailgated.

[0046] Specifically, it can calculate the facial point clouds of non-vehicle owner individuals within the collected point cloud data. (i is a positive integer) Midpoint and the midpoint of the car owner's face point cloud European distance :

[0047] in, , , for The corresponding number of points in the point cloud for The three-dimensional coordinates of the j-th point in the point cloud.

[0048] like ( If a preset distance threshold is used, it is determined that the car owner is not at risk of being followed. At this point, the car doors can be unlocked and the relevant welcome functions can be activated. This represents the total number of distances less than a preset distance threshold. This means that all distances are greater than or equal to a preset distance threshold. Otherwise, it is determined that the car owner is at risk of being followed.

[0049] In some embodiments of the present invention, after determining that the vehicle owner is at risk of being followed, the method further includes: issuing a tailgating warning message; if the vehicle owner confirms that there is no tailgating in response to the tailgating warning message, it is determined that the vehicle owner is not at risk of being followed.

[0050] Specifically, after determining that the car owner is at risk of being tailgated, a tailgating warning message is issued, for example, by prompting the user through voice broadcast that there is a risk of being tailgated, and requesting the user to confirm via voice (such as replying "no tailgating" or "safety confirmed"). If the user's confirmation instruction is received, it is determined that there is no risk of being tailgated, and the car door is unlocked; if no confirmation instruction is received within a certain period of time (e.g., 2 minutes), it enters a sleep state.

[0051] Therefore, by using a tailgating warning and user confirmation mechanism, users are given the authority to make their own judgments and intervene when a potential tailgating risk is detected. This not only avoids the impact on user experience caused by false detections but also ensures security when there is a real risk, achieving a balance between security and convenience.

[0052] Figure 6 This is a flowchart of a vehicle door unlocking method according to an embodiment of the present invention. This vehicle door unlocking method can also be executed by the electronic device described above for performing the tailgating identification method.

[0053] like Figure 6 As shown, the methods for unlocking the car doors include: S21, tail-following identification is performed using a tail-following identification method.

[0054] The tail-following identification method is the tail-following identification method described in the above embodiments.

[0055] S22, in response to the recognition that there is no risk of the vehicle owner being followed, performs the door unlocking operation.

[0056] Specifically, after completing the tailgating detection, if the detection result indicates that the vehicle owner is not at risk of being tailgated, the electronic device can send an unlocking command to the door lock actuator via the CAN bus or LIN bus to unlock the door. In addition, at least one of the following welcome operations can be performed according to preset rules: activating the exterior welcome lights, unfolding the exterior rearview mirrors, activating the interior ambient lighting, playing a welcome prompt sound, and adjusting the driver's seat to the user's preset welcome position.

[0057] If the identification result indicates that the vehicle owner is at risk of being followed, the door unlocking operation will be refused, and at least one of the following safety responses may be triggered: issuing a voice warning through the external speaker (such as "Please be aware of people following behind"), sending a safety reminder to the user's mobile phone through the in-vehicle APP, activating the panoramic imaging system to record the surrounding environment, or entering a safety lock state to wait for the user's subsequent confirmation.

[0058] Therefore, by linking tailgating detection with door unlocking control, door unlocking is only performed when safety is confirmed, effectively preventing tailgators from forcibly entering and hijacking the vehicle the moment the owner unlocks the door, thus improving the security of the door unlocking process. Simultaneously, the simultaneous activation of the welcome service optimizes the user experience, achieving a harmonious balance between security and convenience.

[0059] The following is combined Figure 7 This invention describes a specific embodiment of a vehicle door unlocking method. For example... Figure 7 As shown, the methods for unlocking the car doors include: Once a pedestrian enters the gait pre-recognition range, the identity pre-recognition module determines whether the pedestrian is approaching a vehicle. The gait pre-recognition range is the field of view of the image acquisition module, within which the module can capture image data including the pedestrian. If a pedestrian approaches the vehicle, gait identity pre-recognition is performed to determine if the vehicle owner is present. If the vehicle owner is present, the fusion identity verification module is activated. If no pedestrian is approaching the vehicle, or if no vehicle owner is present, the fusion identity verification module can be activated by pressing a microswitch.

[0060] When the car owner opens the door using a preset voice command, the identity recognition module first performs voiceprint recognition on all audio data within the audio acquisition module's field of view. After confirming the car owner's identity, it locates the owner's voice source. Then, based on the owner's voice source location, it extracts a 3D facial point cloud and performs identity verification. Simultaneously, it records the number of faces (Nfaces) within the field of view of the point cloud acquisition module. If identity verification is successful, the next step, anti-tailgating logic judgment, is performed; otherwise, identity verification continues for a first preset time (e.g., 2 minutes).

[0061] If Nface is not greater than 1, the car door is unlocked directly, and the relevant welcome service is activated. Otherwise, the distance between the midpoint of the non-owner's face point cloud and the midpoint of the owner's face point cloud in the collected point cloud data is calculated. If the distance is less than or equal to a preset distance threshold, the car door is unlocked directly, and the relevant welcome function is activated. Otherwise, a voice prompt is given to the user indicating a risk of being followed, and the user needs to confirm via voice that there is no tailgating before unlocking the car door; otherwise, the system waits for a second preset time (e.g., 2 minutes) before entering sleep mode.

[0062] Figure 8 This is a structural block diagram of the tail-following identification device according to an embodiment of the present invention.

[0063] like Figure 8 As shown, the tailgating detection device 100 includes: an image acquisition module 10, an audio acquisition module 20, and a point cloud acquisition module 30. The image acquisition module 10 is used to acquire image data around the vehicle; the audio acquisition module 20 is used to acquire audio data around the vehicle; and the point cloud acquisition module 30 is used to acquire point cloud data around the vehicle. A controller 40 is connected to the image acquisition module 10, the audio acquisition module 20, and the point cloud acquisition module 30. The controller 40 is used to: identify whether the vehicle owner is approaching the vehicle based on the image data; and when the vehicle owner is detected to be approaching the vehicle, identify the location of the vehicle owner's sound source based on the audio data around the vehicle; and identify whether the vehicle owner is at risk of being tailgated based on the location of the vehicle owner's sound source and the point cloud data around the vehicle.

[0064] In some embodiments of the present invention, the image acquisition module 10, the audio acquisition module 20 and the point cloud acquisition module 30 are all installed on the B-pillar on the driver's side of the vehicle, and are arranged vertically from top to bottom as follows: point cloud acquisition module 30, audio acquisition module 20 and image acquisition module 10.

[0065] It should be noted that the controller 40 may be the electronic device in the above embodiments. For other specific embodiments of the tail-following identification device 100 of the present invention, please refer to the specific embodiments of the tail-following identification method in the above embodiments.

[0066] Figure 9This is a structural block diagram of the vehicle door unlocking system according to an embodiment of the present invention.

[0067] like Figure 9 As shown, the vehicle door unlocking system 1000 includes: a vehicle door 200 and a tailgating identification device 100 as described in the above embodiment.

[0068] The controller 40 in the tailgating identification device 100 is also used to: unlock the car door when it is detected that the car owner is not at risk of being tailgated.

[0069] Figure 10 This is a structural block diagram of a vehicle according to an embodiment of the present invention.

[0070] like Figure 10 As shown, vehicle 10000 includes: the door unlocking system 1000 of the above embodiment.

[0071] The present invention also proposes a computer-readable storage medium. This computer-readable storage medium stores a computer program, which, when executed by a processor, implements at least one of the tailgating identification method and the vehicle door unlocking method described in the above embodiments.

[0072] The present invention also proposes an electronic device. This electronic device is configured to perform at least one of the tailgating identification method and the vehicle door unlocking method described in the above embodiments.

[0073] In summary, the tailgating identification method, device, door unlocking method, system, and vehicle of the present invention can achieve the following beneficial effects: 1) Improved recognition accuracy and robustness By combining ToF point cloud, RGB images, and audio information, the system enables identity recognition and monitoring of the entire process of unlocking and getting into the vehicle. Through multimodal fusion technology (including gait pre-recognition, voiceprint recognition, and 3D face recognition), it overcomes the shortcomings of single biometric features in complex environments such as low light, occlusion, and noise, thereby improving the accuracy of identity recognition and the stability of the system.

[0074] 2) Enhanced security By calculating the spatial distance between suspicious persons and car owners within the field of vision, it can determine whether there is a tailing situation and actively trigger an early warning when the risk of being tailed is detected. This can effectively prevent car owners from unlocking their vehicles without their knowledge of being tailed, prevent tailers from forcibly getting into the car and hijacking it, and comprehensively protect the personal and property safety of users.

[0075] 3) User experience optimization It achieves a completely contactless owner authentication and vehicle unlocking experience, allowing owners to complete identity verification and unlock the doors without any additional operations (such as taking out a key or pressing a fingerprint). At the same time, it can automatically identify potential tailgating risks without user intervention and unlock the doors after confirming safety, achieving a perfect balance between security and convenience.

[0076] 4) Optimization of computing power consumption By setting up identity recognition technologies in stages that are suitable for the characteristics of data at each stage: low-power gait recognition is used for pre-identification at long distances, and sound source direction is used to guide the directional acquisition and recognition of point cloud data at close distances. This avoids the computational power loss caused by full-range point cloud data processing, and achieves a significant reduction in the overall computational power of 3D point cloud face recognition with a lower unit computational power expenditure, thus realizing the optimized allocation of computing resources.

[0077] 5) Low power consumption, suitable for new energy vehicles Taking into full account the low power consumption requirements of new energy vehicles under normal operating conditions, a low power wide-angle RGB module is adopted to monitor the gait of approaching pedestrians, so as to achieve active anti-tailgating while effectively controlling the system power consumption, thus meeting the stringent requirements of new energy vehicles for vehicle static power consumption.

[0078] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer system, a system including a processor, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0079] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0080] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0081] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0082] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0083] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0084] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A tail identification method characterized by, include: In response to detecting that a vehicle owner is approaching the vehicle, the location of the vehicle owner's sound source is identified based on audio data around the vehicle; Based on the location of the driver's voice source and the point cloud data around the vehicle, it is determined whether the driver is at risk of being followed.

2. The tail identification method according to claim 1, characterized by, The vehicle owner is identified as being near the vehicle by at least one of the following methods: Based on image data around the vehicle, it can be determined whether the vehicle owner is near the vehicle; The system identifies whether the vehicle owner is near the vehicle based on the trigger signal generated when the vehicle's microswitch is activated.

3. The tail identification method of claim 1, wherein, The step of identifying whether the driver is at risk of being followed based on the location of the driver's voice source and point cloud data around the vehicle includes: Perform 3D face recognition on the point cloud data to identify whether the face at the location of the vehicle owner's voice source matches the vehicle owner's face; If a match is found and the number of faces is 1, then it is determined that the car owner is not at risk of being tailgated. If a match is found and the number of faces is greater than 1, the first location information of the vehicle owner and at least one second location information of a non-vehicle owner are obtained based on the 3D face recognition result. Based on the first location information and the second location information, it is determined whether the vehicle owner is at risk of being tailgated.

4. The tail identification method according to claim 3, characterized by, The step of identifying whether the vehicle owner is at risk of being followed based on the first location information and the second location information includes: Based on the first location information and the second location information, calculate the distance between the vehicle owner and each of the non-vehicle owners; If at least one of the distances is less than a preset distance threshold, it is determined that the vehicle owner is at risk of being tailgated. If all the distances are greater than or equal to the preset distance threshold, then it is determined that the car owner is not at risk of being tailgated.

5. The tail identification method according to any one of claims 1 to 4, characterized in that, After determining that the vehicle owner is at risk of being followed, the method further includes: Issue a tailgating warning; If a confirmation message from the vehicle owner confirming that they are not being followed is received in response to the tailgating warning, it is determined that the vehicle owner is not at risk of being tailgated.

6. A vehicle door unlocking method characterized by comprising: include: Tail-following identification is performed using the tail-following identification method as described in any one of claims 1 to 5; In response to the detection that there is no risk of the car owner being followed, the door unlocking operation is performed.

7. A tail identification device, characterized by, include: Image acquisition module, used to acquire image data around the vehicle; An audio acquisition module is used to acquire audio data around the vehicle; A point cloud acquisition module is used to acquire point cloud data around the vehicle; The controller is connected to the image acquisition module, the audio acquisition module, and the point cloud acquisition module respectively. The controller is used to: identify whether the car owner is approaching the vehicle based on the image data; when the car owner is identified as approaching the vehicle, identify the location of the car owner's sound source based on the audio data around the vehicle; and identify whether the car owner is at risk of being tailgated based on the location of the car owner's sound source and the point cloud data around the vehicle.

8. The following identification device of claim 7, wherein, The image acquisition module, the audio acquisition module, and the point cloud acquisition module are all installed on the B-pillar on the driver's side of the vehicle, and are arranged vertically from top to bottom as follows: the point cloud acquisition module, the audio acquisition module, and the image acquisition module.

9. A vehicle door unlocking system, characterized in that, include: Car door; as well as The tailgating identification device as described in claim 7 or 8; The controller in the tailgating detection device is also used to: unlock the car door when it is detected that the car owner is not at risk of being tailgated.

10. A vehicle, characterized in that, include: The door unlocking system as described in claim 9.

Images