Method and system for authentication of vehicle-to-vehicle communication

Abstract

Systems, methods, and apparatus for authenticating vehicle-to-vehicle communications are disclosed. A method includes receiving sensor data from a first vehicle and receiving auxiliary sensor data from a second vehicle. The method includes extracting, based on the sensor data and the auxiliary sensor data, an authentication including one or more of: proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data. The method includes determining whether the authentication satisfies a trust threshold of the first vehicle.

Description

Technical Field

[0001] The present disclosure relates to performing authenticated vehicle-to-vehicle communications. Background Art

[0002] Automobiles provide a large portion of transportation for commercial, government, and private entities. Autonomous vehicles and driver assistance systems are currently being developed and deployed to provide safety, reduce the amount of user input required, or even completely eliminate user involvement. For example, some driver assistance systems (such as collision avoidance systems) can monitor the driving, position and speed of the vehicle and other objects while a person is driving. When the system detects an impending collision or impact, the collision avoidance system can intervene and apply braking, steer the vehicle, or perform other avoidance or safety maneuvers. As another example, an autonomous vehicle can drive and navigate the vehicle with little or no user input. Autonomous vehicles can also communicate with other autonomous vehicles to help with collision avoidance and safety operations. Effective authentication of vehicle identity can be beneficial to communication between two or more autonomous vehicles.

[0003] Advances in vehicle technology include vehicle-to-vehicle (V2V) communications, which may require authorization to ensure the safety and security of the vehicle owner or driver. It is known that inter-vehicle communications are used in many other applications to improve the security features of each vehicle. In V2V communications, the authenticity of the vehicle's identity is verified by its digital certificate. Considering the security-critical meaning of digital certificates in many V2V applications, the validity of digital certificates can be guaranteed by a public key infrastructure (PKI) system. However, in some implementations, it is almost impossible to completely prevent any security vulnerabilities in PKI. As seen in the past few decades, a certificate authority (CA) may be compromised to issue an unauthorized certificate, or even a benign CA may mistakenly issue a valid certificate to an unauthorized party. In addition, if the certificate is not properly stored in a secure place, the signing key of the certificate may be directly captured. Summary of the invention

[0004] According to one aspect, a method for authenticating vehicle-to-vehicle communications is disclosed. The method includes receiving sensor data from a first vehicle and receiving auxiliary sensor data from a second vehicle. The method includes extracting an authentication based on the sensor data from the first vehicle and the auxiliary sensor data from the second vehicle. The authentication includes one or more of: proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle. The method also includes determining whether the authentication satisfies a trust threshold of the first vehicle to prove the authentication of the second vehicle.

[0005] According to another aspect of the present disclosure, the authentication includes equal detected ground speeds of a common object identified by the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle.

[0006] According to another aspect of the present disclosure, the sensor data includes an image received from a camera of the first vehicle. The sensor data includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle. The auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle. The auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle. The method also includes extracting an authentication point from the image received from the camera of the first vehicle. The authentication point includes one or more of: a distance of the second vehicle relative to the first vehicle; an azimuth between the second vehicle and the first vehicle; or a vehicle identification of the second vehicle. The method also includes verifying the authentication point based on the ranging data. Verifying the authentication point includes extracting a ranging authentication point from the ranging data. The ranging authentication point includes one or more of: the distance of the second vehicle relative to the first vehicle; or the azimuth between the second vehicle and the first vehicle. The method also includes cross-checking the authentication point extracted from the image with a corresponding ranging authentication point. The method also includes verifying that the authentication point is equal to the corresponding ranging authentication point within an error bound.

[0007] According to another aspect of the present disclosure, the method further includes exchanging a heading angle of the first vehicle or the second vehicle to compensate for an angle difference between the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle.

[0008] According to another aspect of the present disclosure, the method also includes receiving an object graph captured by one or more sensors of the first vehicle. The method also includes receiving an auxiliary object graph captured by one or more sensors of the second vehicle and generating an overlapping object graph by aligning the auxiliary object graph with the object graph by rotating and / or translating the object graph or one of the auxiliary object graphs. The method also includes determining whether one or more common objects are identified in the object graph and the auxiliary object graph based on the overlapping object graph. The method also includes determining whether the one or more common objects meet a trust threshold of an application of the first vehicle and authenticating communication between the application of the first vehicle and the second vehicle if the one or more common objects meet the trust threshold of the application. The method also includes rejecting communication between the application of the first vehicle and the second vehicle if the one or more common objects do not meet the trust threshold of the application.

[0009] According to another aspect of the present disclosure, determining whether the authentication satisfies the trust threshold of the first vehicle includes verifying one or more of: the identity of the second vehicle; whether the second vehicle is in the vicinity of the first vehicle; the presence of one or more common objects sensed by the sensor data of the first vehicle and the sensor data of the second vehicle; or determining ground speed based on equality of one or more common objects of the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle. The method may also include permitting communication between the first vehicle and the second vehicle if the verification satisfies the trust threshold of the first vehicle.

[0010] According to one aspect, a system for authenticating vehicle-to-vehicle communications is disclosed. The system includes a first vehicle, the first vehicle including one or more sensors that provide sensor data. The system includes a vehicle controller that communicates with the one or more sensors of the first vehicle. The vehicle controller is configured to receive auxiliary sensor data from a second vehicle. The vehicle controller includes a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to receive sensor data from the first vehicle and auxiliary sensor data from the second vehicle. The instructions also cause the one or more processors to extract authentication based on the sensor data from the first vehicle and the auxiliary sensor data from the second vehicle. The authentication includes one or more of: proximity of the second vehicle to the first vehicle; a common object identified by the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle; or equal detected ground speeds of a common object identified by the sensor data of the first vehicle and the auxiliary sensor data of the second vehicle. The instructions also cause the one or more processors to determine whether the authentication satisfies a trust threshold of the first vehicle to prove authentication of the second vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to facilitate an understanding of the advantages of the present disclosure, a more particular description of the present disclosure briefly described above will be presented by reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only typical embodiments of the present disclosure and, therefore, should not be considered to limit its scope. The present disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0012] Figure 1 is a schematic block diagram of an autonomous vehicle or assisted driving system according to the teachings and principles of the present disclosure;

[0013] Figure 2 is a schematic block diagram of an exemplary computing device suitable for implementing the method according to the teachings and principles of the present disclosure;

[0014] Figure 3 is a schematic diagram showing the relative positions of vehicles performing authentication according to the teachings and principles of the present disclosure;

[0015] Figure 4A , Figure 4B , Figure 5A and Figure 5B is an illustration of an image that may be processed according to the teachings and principles of the present disclosure;

[0016] Figure 6 is a diagram showing distances and angles between vehicles performing authentication in accordance with the teachings and principles of the present disclosure;

[0017] Fig. 7A is a diagram showing distances and angles measured from a vehicle camera in accordance with the teachings and principles of the present disclosure;

[0018] Figure 7B is a diagram showing the location of a vehicle license plate in an image in accordance with the teachings and principles of the present disclosure;

[0019] Figure 8 is a schematic block diagram illustrating a process flow for authenticating a vehicle according to the teachings and principles of the present disclosure;

[0020] Fig. 9 is a plan view showing the position of the vehicle when the image was captured in accordance with the teachings and principles of the present disclosure;

[0021] Fig.10 is a plan view showing an object map captured by a nearby vehicle in accordance with the teachings and principles of the present disclosure;

[0022] Fig.11 is a plan view showing translation and rotation used to determine whether object images captured by nearby vehicles are similar in accordance with the teachings and principles of the present disclosure;

[0023] Fig.12 is a plan view showing a velocity vector diagram of nearby vehicles for use in authentication in accordance with the teachings and principles of the present disclosure;

[0024] Fig.13 is a schematic flow chart illustrating a method for authenticating a vehicle according to the teachings and principles of the present disclosure;

[0025] Fig.14 is a schematic flow chart illustrating a method for authenticating a vehicle according to the teachings and principles of the present disclosure; and

[0026] Fig.15 is a schematic flow chart illustrating a method for authenticating a vehicle according to the teachings and principles of the present disclosure. DETAILED DESCRIPTION

[0027] Advances in vehicle technology include vehicle-to-vehicle (V2V) communications, which may require authorization to ensure the safety and security of the vehicle owner or driver. It is known that inter-vehicle communications are used in many other applications to improve the security features of each vehicle. In V2V communications, the authenticity of the vehicle's identity is verified by its digital certificate. Considering the security-critical meaning of digital certificates in many V2V applications, the validity of digital certificates can be guaranteed by a public key infrastructure (PKI) system. However, in some implementations, it is almost impossible to completely prevent any security vulnerabilities in PKI. As seen in the past few decades, a certificate authority (CA) may be compromised to issue an unauthorized certificate, or even a benign CA may mistakenly issue a valid certificate to an unauthorized party. In addition, if the certificate is not properly stored in a secure place, the signing key of the certificate may be directly captured.

[0028] While V2V communication is intended to increase vehicle safety and security, it also poses potential threats to adversaries. Attackers can launch different types of attacks to benefit themselves or maliciously cause harm to the victim. For example, attackers can transmit inaccurate information to influence neighboring vehicles to divert other vehicles on the path to gain a free path or falsify their sensor information to avoid accident liability. Vehicles traveling in platoons are also vulnerable to collision-induced attacks. In addition, Sybil attacks can also be performed by using multiple non-existent identities or pseudonyms. Therefore, ensuring inter-vehicle communication is of great significance and can protect users from life-threatening attacks.

[0029] In an effort to secure V2V communications, the de facto V2V communication standard, Dedicated Short Range Communications (DSRC), utilizes a Public Key Infrastructure (PKI) system to authenticate the public keys of vehicles. While this solution is designed to provide adequate security guarantees, there are still many attack possibilities. One of the main problems is caused by location spoofing impersonation attacks. In these attacks, an internal attacker (i.e., a malicious vehicle with the correct credentials) transmits a message with a fake location. For example, an attacker creates a "ghost vehicle" by faking its location to the victim vehicle. Similarly, a malicious vehicle in a platoon can impersonate the location of another vehicle by faking its location within the platoon.

[0030] Applicants recognize that in addition to PKI authentication, supplementary mechanisms may be provided for V2V communication. Such supplementary mechanisms may utilize camera sensors that are already prevalent in many autonomous or driver-assisted vehicles. Such supplementary mechanisms may include two vehicles taking snapshots of each other, exchanging snapshots, and verifying each other's identities by, for example, extracting vehicle numbers, relative distances between vehicles, azimuths, etc. from received images.

[0031] However, an advanced attacker may still be able to impersonate the identity of a vehicle by offline preparing a dictionary of images taken from various locations and selecting images from the dictionary to impersonate another vehicle around the victim vehicle. In addition, the attacking vehicle may take a snapshot of the victim vehicle, crop the victim vehicle from the snapshot, and superimpose the victim vehicle into a suitable position to mislead the victim vehicle into believing that the image was taken by the attacking vehicle. Due to recent advances in computer vision, new camera image modification techniques can be exploited by the attacking vehicle.

[0032] Applicants herein propose systems, methods and devices for vehicle-to-vehicle authentication and communication that overcome currently known weaknesses in PKI systems and other supplementary mechanisms that may be susceptible to attack by advanced vehicle systems. In an embodiment of the present disclosure, a method for authenticating vehicle-to-vehicle communication is provided. The method includes receiving sensor data from a first vehicle and receiving auxiliary sensor data from a second vehicle. The method includes extracting an authentication based on the sensor data and the auxiliary sensor data that includes one or more of the following: the proximity of the second vehicle relative to the first vehicle, or a common object identified by the sensor data and the auxiliary sensor data. The method also includes determining whether the authentication meets a trust threshold of the first vehicle. The method may also include: if the authentication meets the trust threshold, then allowing communication between the first vehicle and the second vehicle.

[0033] It will be readily understood that the components of the present disclosure as generally described and illustrated in the drawings herein may be arranged and designed in a wide variety of different configurations. Therefore, the following more detailed description of the embodiments of the present disclosure as illustrated in the drawings is not intended to limit the scope of the present disclosure as claimed, but is merely representative of certain examples of the embodiments currently contemplated according to the present disclosure. The presently described embodiments will be best understood with reference to the drawings, wherein like parts are designated by like reference numerals in the various figures.

[0034] According to the embodiments of the present disclosure, it can be embodied as a device, method or computer program product. Therefore, the form taken by the present disclosure can be a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software aspects and hardware aspects, which embodiments can generally all be referred to as "modules" or "systems" in this article. In addition, the form taken by the present disclosure can be a computer program product embodied in any tangible expression medium with a computer-usable program code embodied in the medium.

[0035] Any combination of one or more computer-usable or computer-readable media may be utilized. For example, computer-readable media may include one or more of the following: a portable computer diskette, a hard disk, a random access memory (RAM) device, a read-only memory (ROM) device, an erasable programmable read-only memory (EPROM or flash memory) device, a portable compact disk read-only memory (CDROM), an optical storage device, and a magnetic storage device. In selected embodiments, the computer-readable medium may include any non-transitory medium that may contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0036] The computer program code for performing the operation of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" programming language or similar programming languages. The program code can be executed entirely on a computer system as an independent software package, on an independent hardware unit, partially on a remote computer separated from the computer by a certain distance, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to form a connection through the Internet).

[0037] The present disclosure is described below with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart illustration and / or block diagram and the combination of boxes in the flowchart illustration and / or block diagram can be implemented by computer program instructions or codes. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a means for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0038] These computer program instructions may also be stored in a non-transitory computer-readable medium that can direct a computer or other programmable data processing device to function in a specific manner, so that the instructions stored in the computer-readable medium produce an article of manufacture that includes instruction means for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0039] The computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0040] refer to Figure 1 , the controller 102 may be housed in a vehicle. The vehicle may include any vehicle known in the art. The vehicle may have all the structures and features of any vehicle known in the art, including wheels, a drive train coupled to the wheels, an engine coupled to the drive train, a steering system, a braking system, and other systems known in the art to be included in a vehicle.

[0041] As discussed in more detail herein, the controller 102 may perform autonomous navigation and collision avoidance. Specifically, according to an embodiment of the present disclosure, the controller 102 may perform authenticated V2V communications.

[0042] The controller 102 may be coupled to a front camera 104 and a rear camera 106. The front camera 104 may be mounted on the vehicle with a forward facing field of view, while the rear camera 106 may be mounted to the vehicle with a rearward facing field of view. The rear camera 106 may be a conventional backup camera or a separate camera with a different field of view. The cameras 104, 106 may be used to perform authentication methods as disclosed herein, and may additionally be used to perform obstacle detection.

[0043] The controller 102 may be coupled to one or more other sensing devices 108, which may include microphones or other sensors for detecting obstacles, such as radar (RADAR), laser radar (LIDAR), sonar (SONAR), ultrasound, etc. For example, a plurality of different sensors may be attached to an advanced driver assistance system (ADAS) bus or system. Any of these available sensors may be used to provide sensor data for the purpose of assisted driving, autonomous driving, and / or authenticated vehicle-to-vehicle (V2V) communication.

[0044] The controller 102 may execute a V2V module 110a. The V2V module 110a includes a position verification module 112a. The position verification module 112a verifies that another vehicle seeking to communicate with the controller 102 using V2V communication is actually a vehicle proximate to the controller 102. In particular, the position verification module 112a verifies the position of the other vehicle by exchanging images or other sensor data (e.g., frames of radar, lidar, sonar, ultrasound, or other sensor data), object maps, and / or velocity maps, as discussed in more detail below.

[0045] The V2V module 110a may also include an authentication module 112b. The authentication module 112b performs key exchange, such as using a Diffie-Hellman method, public key encryption, or some other authentication technique. The authentication module 112b may also handle performing secure communications between the controller and another vehicle. The manner in which authentication and secure communications are performed is described in more detail below.

[0046] The controller 102 may also execute an obstacle identification module 110b, a collision prediction module 110c, and a decision module 110d. The obstacle identification module 110b may analyze one or more image streams from the cameras 104, 106, or other cameras, and identify potential obstacles, including people, animals, vehicles, buildings, curbs, and other objects and structures. Additionally, the obstacle identification module 110b may identify potential obstacles from the output of the sensing device 108, such as using data from a lidar, radar, ultrasound, or other sensing system.

[0047] The collision prediction module 110c predicts which obstacle images may collide with the vehicle based on its current trajectory or current expected path. The decision module 110d may make decisions to stop, accelerate, turn, etc. to avoid obstacles. The manner in which the collision prediction module 110c predicts potential collisions and the manner in which the decision module 110d takes action to avoid potential collisions may be based on any method or system known in the field of autonomous vehicles.

[0048] The decision module 110d may control the trajectory of the vehicle by actuating one or more actuators 114 that control the direction and speed of the vehicle. For example, the actuators 114 may include a steering actuator 116a, an accelerator actuator 116b, and a brake actuator 116c. The configuration of the actuators 116a-116c may be in accordance with any implementation of such actuators known in the field of autonomous vehicles.

[0049] Figure 2 2 is a block diagram illustrating an exemplary computing device 200 . Computing device 200 may be used to execute various programs, such as those discussed herein. Controller 102 may have some or all of the attributes of computing device 200 .

[0050] Computing device 200 includes one or more processors 202, one or more memory devices 204, one or more interfaces 206, one or more mass storage devices 208, one or more input / output (I / O) devices 210, and a display device 230, all of which are coupled to a bus 212. Processor 202 includes one or more processors or controllers that execute instructions stored in memory device 204 and / or mass storage device 208. Processor 202 may also include various types of computer-readable media, such as cache memory.

[0051] Memory device 204 includes various computer-readable media, such as volatile memory (e.g., random access memory (RAM) 214) and / or nonvolatile memory (e.g., read-only memory (ROM) 216). Memory device 204 may also include a rewritable ROM, such as flash memory.

[0052] The mass storage device 208 includes various computer-readable media, such as magnetic tapes, magnetic disks, optical disks, solid-state memory (e.g., flash memory), etc. Figure 2 As shown, the particular mass storage device is a hard drive 224. Various drives may also be included in the mass storage device 208 to enable reading from and / or writing to various computer-readable media. The mass storage device 208 includes removable media 226 and / or non-removable media.

[0053] I / O devices 210 include various devices that allow data and / or other information to be entered into or retrieved from computing device 200. Exemplary I / O devices 210 include cursor control devices, keyboards, keypads, microphones, monitors or other display devices, speakers, network interface cards, modems, lenses, CCD or other image capture devices, and the like.

[0054] Display device 230 includes any type of device capable of displaying information to one or more users of computing device 200. Examples of display device 230 include a monitor, a display terminal, a video projection device, and the like.

[0055] Interfaces 206 include various interfaces that allow computing device 200 to interact with other systems, devices, or computing environments. Exemplary interfaces 206 include any number of different network interfaces 220, such as interfaces to a local area network (LAN), a wide area network (WAN), a wireless network, and the Internet. Other interfaces include a user interface 218 and a peripheral device interface 222. Interfaces 206 may also include one or more peripheral interfaces, such as interfaces for a pointing device (mouse, track pad, etc.), a keyboard, etc.

[0056] The bus 212 allows the processor 202, memory device 204, interface 206, mass storage device 208, I / O device 210, and display device 230 to communicate with each other and with other devices or components coupled to the bus 212. The bus 212 represents one or more of several types of bus structures, such as a system bus, a PCI bus, an IEEE 1394 bus, a USB bus, and the like.

[0057] For purposes of illustration, programs and other executable program components are shown herein as discrete blocks, but it is understood that such programs and components may reside at various times in different storage components of the computing device 200 and be executed by the processor 202. Alternatively, the systems and programs described herein may be implemented in hardware or a combination of hardware, software, and / or firmware. For example, one or more application specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and programs described herein.

[0058] To enhance authentication for V2V communications, the systems, methods, and apparatus disclosed herein provide cryptographic credentials with physical identity and co-occurrence components to help infer one's location. A vehicle authentication method ("VAuth") is disclosed herein, and the method provides a secure authentication key negotiation scheme that addresses the above-mentioned problems for V2V communications while the vehicle is traveling on a road. The VAuth method includes capturing visual contextual information of a car using a camera as a means of binding its physical identity and its co-occurrence to another vehicle. Specifically, two moving vehicles on a road will have a unique pair of relative distances (d) and angles (Φ) at a given point in time that cannot be experienced by other vehicles.

[0059] For example, Figure 3 An embodiment of the use of VAuth 300 is shown. In the described embodiment, both vehicle A and vehicle B take snapshots of each other simultaneously (e.g., within one second, preferably within 100 ms, more preferably within 10 ms) and exchange images to prove their relative d and Φ. Specifically, VAuth 300 utilizes a cryptographic commitment and decommitment scheme to bind the vehicle's cryptographic key to its physical identity (license plate number) and their co-occurrence (d and Φ), which helps infer location.

[0060] Due to this binding, VAuth 300 eliminates the aforementioned location spoofing impersonation attacks. Through this binding, VAuth 300 also protects against man-in-the-middle (MitM) attacks that can occur during the key negotiation step. In addition, VAuth 300 can also limit image forgery and spoofing attacks because each vehicle can use commonly observed objects (e.g., neighboring vehicles in the background, road signs, terrain, etc.) to verify the validity of the received image.

[0061] VAuth 300 enables automated key establishment between moving vehicles even in the presence of the following constraints. First, decentralized trust management may be required, making traditional approaches that rely on a remote and central trusted third party (TTP) problematic. TTPs can incur significant management costs and are susceptible to single points of failure. Second, human interaction and involvement may not be required due to the rapid dynamics of vehicles moving in traffic. Including drivers and passengers in the loop not only reduces availability, but can also significantly distract attention from the driving task. Third, it may be necessary to use available hardware in the vehicle to reduce vehicle cost.

[0062] The main goal of VAuth 300 is to prevent location spoofing impersonation attacks in V2V communications by binding the physical identity and co-presence of a pair of neighboring cars. We define "neighboring vehicles" as vehicles that are within each other's line of sight (i.e., the field of view of the camera). In this case, we enable a pair of vehicles to establish a secure channel by performing a temporary (ad-hoc) secure key negotiation while the vehicles are driving. This process is called "pairing". The key agreement protocol should be able to resist active attacks such as man-in-the-middle (MitM) and image spoofing attacks. VAuth 300 enhances the integrity and authenticity of the key agreement messages. Integrity and authenticity guarantee that the key agreement messages are not altered on the way from the declared sender.

[0063] In summary, the description of VAuth 300 included in this article discloses: (a) a secure V2V key agreement protocol that binds physical identity and presence to cryptographic keys; (b) a security analysis of the VAuth protocol to demonstrate its robustness against MitM attacks; and (c) an implementation and evaluation of VAuth using real-world vehicles.

[0064] The goal of the attacker is to break the integrity and authenticity of the key agreement scheme between two legitimate vehicles. This application considers both passive and active attackers. Passive attackers merely observe the wireless communication while attempting to launch an attack (e.g., an eavesdropping attack). Active attackers can inject, replay, modify, and delete messages in the communication channel. In this application, a method is disclosed for dealing with an attacker who is co-located with a legitimate entity (i.e., a neighboring vehicle traveling along a road).

[0065] VAuth 300 uses visual images of vehicle context information to verify authenticity during the pairing process. Any pair of neighboring vehicles and only that pair of neighboring vehicles share and experience a unique relative distance (d) and angle (Φ) at a specific time that no other vehicle experiences (where ).For example, Figure 3Vehicle A and vehicle B in share a relative distance and angle. Note that another pair of vehicles (e.g., vehicle B and vehicle C) may have their own d and Φ relative to each other, but may not have the same d and Φ as the opposing vehicle A.

[0066] The vehicles prove their authenticity by taking camera snapshots of each other to present d and Φ as evidence. The pair of vehicles identifies each other as the "target" vehicle to be paired by initiating periodic beacon messages. The two vehicles exchange beacon messages containing their identifiers (i.e. license plate numbers). If the identifiers are not found in each vehicle's local "pairing" list, then the two vehicles identify each other as the "target" vehicle to be paired.

[0067] refer to Figure 3 , in the presence of an attacker vehicle M and possibly one or more benign vehicles C, a legitimate vehicle A and a legitimate vehicle B may be paired using VAuth 300. Each vehicle may have a front camera 104A, 104B, 104M and a rear camera 106A, 106B, 106M.

[0068] Vehicle A and vehicle B may identify each other as targets for V2V communication. Subsequently, the two vehicles will take pictures of each other and exchange images via the DSRC wireless channel. Specifically, the snapshot taken by the rear camera 106A of vehicle A contains the front image of vehicle B, and similarly, the front camera 104B of vehicle B contains the rear image of vehicle A.

[0069] If the image is taken by the intended vehicle (rather than a neighboring vehicle), then the images should share the same relative distance d. Specifically, the distance d between vehicle A and vehicle B measured by vehicle A using an image including vehicle B is A should be equal to (i.e. within a certain tolerance) the distance d measured using the image received from vehicle B that includes vehicle A. B , and vice versa. Similarly, the angle Φ between vehicle A and vehicle B measured by vehicle A using an image including vehicle B is A should be equal (i.e. within a certain tolerance) to the angle Φ measured using the image received from vehicle B that includes vehicle A. B This constraint can be expressed as and ,in is the distance tolerance is the angle tolerance. If this constraint is not satisfied, the pairing process terminates.

[0070] The security of VAuth 300 depends on the distance (d A d B ) and angle (Φ A , ΦB ) is unique. However, if Figure 3 As shown, consider an attacker, vehicle M, traveling with vehicle A and vehicle B. To launch a MitM attack, vehicle M estimates the relative distance (d M~= d A ) and angle (Φ M ~=Φ A And impersonate vehicle A to vehicle B by simply impersonating images of the victim vehicle (vehicle B) using images from a pool of prepared images at varying distances and angles relative to vehicle B. For example, the attacker may prepare a "dictionary" of images offline while the victim's vehicle (vehicle B) is parked on the street.

[0071] In some embodiments, VAuth 300 prevents such attacks by taking advantage of the fact that the surroundings of both vehicles (e.g., neighboring vehicles, road signs, background objects / views, etc.) should be roughly equal. Therefore, VAuth 300 requires both vehicle A and vehicle B to capture a front-facing image ( and ) and then take the rear image ( and ), as depicted. Thus, each vehicle can compare images to check whether the images contain similar environments. For example, V AF and V BF should share common features because they point in the same direction and V AR and V BR Likewise should they share common characteristics. If this check fails, the vehicle rejects the pairing process.

[0072] The VAuth 300 protocol includes four phases: (1) synchronization; (2) snapshot; (3) key negotiation; and (4) key confirmation phase. Each phase is discussed in detail below with respect to the algorithm in Table 1 below. Definitions of the variables in the algorithm in Table 1 are included in Table 2.

[0073] Table 1. V Auth protocol.

[0074]

[0075] Table 2. Notation for the VAuth protocol.

[0076]

[0077] During the synchronization phase, each vehicle transmits periodic beacon messages in an attempt to initiate the VAuth 300 protocol. The beacon message is simply a signature of its vehicle identifier (i.e., license plate number, ID A 、ID B ) broadcast. Continue Figure 3For example, vehicle A broadcasts its beacon message BEACON A And vehicle B receives this message, as depicted in Table 1. Upon receiving BEACON A Then, vehicle B checks against its "paired" list. If the ID is not found in the list A , then vehicle B sends a request to pair with vehicle A. Similarly, vehicle A also checks vehicle B’s license plate (ID B )(Steps 2 to 4 of Table 1). If not found, then vehicle A transmits a sync message to vehicle B to initiate the sync snapshot phase so that the two vehicles recognize each other as the "target" vehicle to be paired with (Step 5 of Table 1). It should be noted that the protocol can be further modified so that if the vehicle receives multiple pairing requests, the vehicle can use other information sent with the request to prioritize the requests (e.g., GPS location information to prioritize the requests based on the proximity of the two vehicles).

[0078] In the snapshot phase, after the synchronization phase and in response to the message received during the synchronization phase, vehicle A and vehicle B simultaneously take snapshots of the front view and the rear view, as shown in step 6 of Table 1. The front image and the rear image taken by vehicle A and vehicle B are respectively marked as and ,as well as and The synchronized taking of photos may be performed by vehicle A and vehicle B coordinating the times at which the photos are to be taken.

[0079] Figure 4A and 4B Shown separately and Used for Figure 3 scenario. Figure 5A and 5B Shown separately and Used for Figure 3 scenario.

[0080] In the key negotiation phase, vehicle A and vehicle B first exchange their commitments (C A and C B ) and later reveal their commitment to release (D A and D B Each vehicle uses a commitment to bind Diffie-Hellman (DH) public parameters (g a or b ) and vehicle ID (ID A or ID B ) (the vehicle ID is the license plate number) and by physical co-occurrence of the image ( )or( ). Note that for simplicity, we have omitted the DH (Diffie-Hellman) public parameters. mod p , but you can still use mod p Steps 7-8 of Table 1 depict the exchange of commitments. Subsequently, as depicted in steps 9-10 of Table 1, the vehicles exchange releases of commitments to disclose their commitment information to each other.

[0081] After receiving the commitment release, each vehicle performs a verification. Table 3 depicts the verification of the commitment release received from vehicle A by vehicle B (D A ) logic. First, vehicle B verifies D A Is it really C? A The hash of (rows 2 to 5 of Table 3). If yes, then vehicle B finds out which image (front view or rear view) contains the license plate number of the target vehicle (rows 7 to 9 of Table 3). For example, V B Should be assigned to V BF , because the image V BF Contains the license plate (ID A ), because vehicle A is in front of vehicle B.

[0082] Table 3. Pseudocode for commitment verification

[0083]

[0084] Subsequently, vehicle B continues to verify the relative distance and angle between vehicle A and vehicle B (rows 11 to 15 of Table 3). This is done by B and V A This is done by calculating distance and angle. and is the license plate (ID A )’s relative distance and angle. Similarly, and The license plates (ID B )'s relative distance and angle. If this pair of relative distances { and } and angle { and } is not within the error bound, then vehicle B aborts the pairing process (rows 17 to 24 of Table 3). Vehicle B also detects potential image spoofing attacks (rows 26 to 31 of Table 3). and backsight = Input to the spoofingAttackDetected() function to test whether vehicle A and vehicle B actually took the snapshot at the same time. As discussed earlier, this check may involve checking the similarity of the surroundings in each image.

[0085] Once vehicle B successfully verifies that A's commitment is released (D A ), then vehicle B accepts the DH parameter g of vehicle A a , and calculate the shared symmetric key K=(g a ) b Similarly, vehicle A also verifies vehicle B's commitment release (D B ) and if verification succeeds, the shared symmetric key is calculated =(g b ) a .

[0086] In the key confirmation phase, after calculating the shared symmetric key, vehicle A and vehicle B perform key confirmation to verify that the two vehicles have indeed generated the same key. This is depicted in steps 11 to 14 of Table 1. Vehicle A transmits a randomly generated Random number ( = 256) and using the derived symmetric key Its message authentication code (MAC) calculated (note that in this example, HMAC-SHA-3 is used with a hash bit length of 256, but other hash functions can also be used.) Upon receiving the message, vehicle B first verifies the MAC using its derived symmetric key K. If successful, vehicle B also transmits its randomly generated Random number ( = 256) and through Use its symmetric key Vehicle A verifies the MAC and if successful, sends the received With key Finally, the VAuth 300 protocol ends with a successful MAC verification of vehicle B. Now, vehicle A and vehicle B use the generated shared symmetric key as their session key.

[0087] In an embodiment, the attacker's success probability is estimated starting from a random guessing attack without any prior knowledge of complex image spoofing attacks. The attacker's basic attack can be performed by a hacker who has full control over the wireless channel. The attacker attempts to impersonate vehicle A to vehicle B and forge a DH public key. In steps 7 and 9 of Table 1, vehicle A transmits its commitment and release (C A and DA ). Therefore, the attacker, vehicle M, first prevents vehicle B from receiving D A (e.g., by jamming) and sends its own fake However, because C A Bind vehicle A’s DH public key g a as well as So the probability of the attacker successfully attacking is P CarM This is equivalent to successfully finding a hash collision. Therefore, the attacker's success probability is limited by the length of the hash function (256-bit SHA-3), as shown in Equation 1, where l is the hash bit length ( l =256).

[0088]

[0089] Equation 1

[0090] In the case where a random guessing attack cannot be performed adequately, the attacker may try to launch a more complex attack. The attacker attempts to forge a DH public key ( ) and images ( ) to successfully impersonate vehicle A to vehicle B, so that = Assume that the attacker, vehicle M, first prepares a “dictionary” of images of the victim (vehicle B in this example) at different distances and angles. Vehicle M selects the prepared image of vehicle B with corresponding d and ϕ , and simply transmits a forged commitment to vehicle B ( ) and promise release ( ), the release of the commitment includes For security analysis, attacks can be divided into three cases as described below.

[0091] In the first case, the attacker does not know d and ϕ and VAuth 300 does not check for image spoofing attacks. In this case, it is assumed that the attacker does not know the relative distance and angle between vehicle A and vehicle B. For a simpler analysis, we assume that the VAuth 300 protocol does not check for image spoofing attacks (so rows 26 to 31 of Table 1 can be omitted in this case). In this case, vehicle M needs to randomly guess and To select the image, the value is within the error limit ( and ). Therefore, the attacker’s success probability is given by Equation 2, where d max is the maximum distance of visible range, which depends on the camera's capabilities.

[0092]

[0093] Equation 2

[0094] In the second case, the attacker knows d and ϕ and the VAuth 300 protocol does not check for image spoofing attacks. Assume that the attacker is traveling with vehicle A and vehicle B and is therefore able to determine the estimated distance and angle d and ϕ.

[0095] Figure 6 Describe the scenario where car M is trying to find the relative distance d AB and angle ϕ A , ϕ B Vehicle M knows its relative distance to vehicles A and B (d AM and d BM ) and relative angle (ϕ X , ϕ Y , ϕ M ). Using simple trigonometry, vehicle M calculates the distance and angle as shown in Equation 3 and Equation 4. In Equation 3 and Equation 4, d xy The distance is defined as the distance between point x and point y. Therefore, the probability of the attacker's success is always (P CarM =1).

[0096]

[0097] Equation 3

[0098]

[0099] Equation 4

[0100] In the third case, the attacker knows d and ϕ and VAuth 300 checks for image spoofing attacks. To prevent the attacker from successfully launching an attack, as shown in the second case, VAuth 300 includes a verification step for image spoofing attacks (Table 1, lines 26 to 31). Equation 5 depicts that the attacker's success probability is equal to the success probability P of the image spoofing attack being successful. spoofing .

[0101] P success =P spoofing

[0102] Equation 5

[0103] To detect image spoofing attacks (Attacker Type 1), VAuth 300 leverages common objects (e.g., license plates of cars in common proximity). and The license plate of the neighboring vehicle is searched in . If the number is less than a predefined threshold, the protocol is terminated. The similarity of other objects and their relative positions can also be used to increase the verification accuracy. For example, buildings, road signs, trees, terrain, etc. can also be used.

[0104] An exemplary implementation of VAuth 300 uses license plate recognition from images captured by a vehicle camera. In particular, VAuth 300 may use OpenALPR, an open source library for automatic license plate recognition. OpenALPR takes an input image and traverses eight stages to output the recognized license plate number, location (corners, width, and height), and confidence level (percentage). OpenALPR implements the following stages. Stage 1 ("Detection Stage") finds "potential regions" of the license plate. Subsequent stages will process all potential regions. Stage 2 ("Binarization Stage") creates multiple black and white images of the license plate area to increase recognition accuracy. Stage 3 ("Character Analysis") finds regions or blobs of the size of the license plate number / character from the license plate area. Stage 4 ("License Plate Edges") detects possible edges of the license plate by detecting Hough lines. Stage 5 ("De-skew") corrects for rotation and skew of the license plate image. Stages 6 and 7 ("Character Segmentation" and "OCR") isolate the characters of the license plate and perform character recognition and confidence level determination. Finally, stage 8 ("post-processing") outputs a list of n potential candidate license plate numbers classified with their confidence level.

[0105] In an embodiment, to calculate distances and angles from the output of OpenALPR, the systems and methods of the present disclosure utilize techniques for image rectification and perspective correction in computer vision. The algorithm exploits the ratio of real-world objects in meters (the "world plane") to pixels (the "image plane") by leveraging knowledge of the dimensions of known objects. Similarly, a calibration image V is taken. calibration , which is one meter away from the vehicle init or such as from V calibration , the height of the identified license plate box (in pixels) h init A snapshot of the vehicle license plate taken at some other known distance from the vehicle. The distance to the license plate in the other images can be calculated based on the height ratio of the recognized license plate, as shown in Equation 6.

[0106]

[0107] Equation 6

[0108] It should be noted that different vehicles may be equipped with different types of cameras, resulting in h init However, each car can init The value is included in the commit / release messages.

[0109] The angle can be calculated using the known distances from the image. Fig. 7A and Figure 7B The problem of finding the relative angle is illustrated in . Specifically, it can be solved by using two distances d in meters. image and d shiftm to derive the angle, as shown in Equation 7. Figure 7B As shown, ϕ image is the angle at which the license plate is calculated, and d image is the distance from the camera 104, 106 to the license plate, d image .

[0110]

[0111] Equation 7

[0112] Value d shift is the "imaginary" distance in meters that the car would be displaced horizontally if it were initially in line with the camera, i.e. horizontally centered in the camera's field of view. To find , use an object of known dimensions in meters and pixels to get the ratio of pixels to meters. For this object, again use a license plate. m is the height of the actual license plate in meters, which is 0.15 meters (for example, California license plates are 6" x 12" (0.15 m x 0.3 m)). px is the height of the license plate in pixels in the image. Fig. 7A , you can also find the shift distance in pixels Finally, using Equation 8, we derive .

[0113]

[0114] Equation 8

[0115] While many of the previous examples discuss the use of information in or derived from camera images for authentication, any type of sensor data may be used to authenticate a vehicle, including its relative position to a parent vehicle. In one embodiment, lidar, radar, sonar, or other data may be used to detect objects within their respective sensing ranges and determine relative positions, angles, and the like based on the data. In different embodiments, any type of sensor data may be used alone to verify location or identity, or multiple types of sensor data may be used together to robustly confirm each other and any assertions made by a potential V2V communication source. For example, lidar or radar data may not be able to detect a vehicle's license plate number or color, but they may be used to generate an object map of vehicles surrounding the parent vehicle. In addition to the object map, the velocity vector of each object relative to the parent vehicle may also be determined. These object maps and / or velocity vector maps may be compared with those obtained based on another vehicle's sensor data to determine whether the other vehicle can be trusted for V2V communication.

[0116] Figure 88 is a schematic flow chart showing a method 800 for authenticating a vehicle based on a camera image, an object map, a heading angle, and / or a velocity vector map. The method 800 starts at 802 and both vehicles agree to initiate communication. At 804, both vehicles capture a camera snapshot of the other vehicle, capture a surrounding map with a radar / lidar sensor, and capture a heading angle with a compass. At 806, the vehicle checks to see if a velocity vector map is used. If it is determined at 806 that a velocity vector map is used, then at 824 the vehicle collects a velocity vector map of surrounding objects during time t. At 808, the two vehicles exchange camera snapshots, object maps, heading angles, and / or velocity vector maps. At 810, each vehicle extracts the vehicle number, relative distance, azimuth angle from the received camera image (or other sensor data or object map), and adds the heading angle difference to the azimuth angle. At 812, each vehicle verifies whether the extracted information matches its own. If it is determined at 812 that the extracted information does not match its own, then the authentication fails 822. If it is determined at 812 that there is a match, then at 814 each vehicle counts the number of co-detected objects from its own map and the received object map from the other vehicle. At 816, the number of co-detected objects is compared to see if it meets the system trust threshold (e.g., a percentage of detected objects or an absolute integer value of detected objects). If the number of co-detected objects determined at 816 does not meet the system trust threshold, then at 822, authentication fails. If the number of co-detected objects determined at 816 does meet the system trust threshold, then at 820, authentication succeeds without using velocity vector matching at 818. If it is determined at 818 that velocity vector matching is used, then at 826, the vehicle calculates a ground vehicle velocity map by adding the vehicle velocity vector to the velocity vector of each co-detected object. At 828, the vehicles check to see if their own ground speed map is approximately equal to the received speed map. If it is determined at 828 that the vehicle's own ground speed map is approximately equal to the received speed map, then at 820, authentication succeeds. If not, then at 822, authentication fails.

[0117] Figures 9 to 12 Shows how various parameters and comparisons can be performed in order to perform Figure 8 method. Fig. 9 906 and vehicle D 908) and roadside objects (object E 910 and object F 912). A and the heading angle h of vehicle B 904 B Also shown is a camera image taken by vehicle A 902, the image having a relative distance (dA ) and the azimuth angle (a B ).

[0118] Fig.10 It is shown as Fig. 9 A plan view of a scene 1002, an ADAS object map 1004 for vehicle A 902, and an ADAS object map 1006 for vehicle B 904 are shown. The object map 1004 for vehicle A 902 includes points or locations detected for object E 910, vehicle B 904, vehicle C 906, and vehicle D 908. The object map 1004 includes an orientation angle 1012 for object A 902, an object E point or location 1020, a vehicle B point or location 1014, a vehicle C point or location 1016, and a vehicle D point or location 1018. The object map 1006 for vehicle B 904 includes points or locations detected for object F 912, vehicle A 902, vehicle C 906, and vehicle D 908. The object graph 1006 includes an orientation angle 1024 of object B 904, a vehicle C point or position 1026, a vehicle D point or position 1028, an object F point or position 1032, and a vehicle A point or position 1022. Note that the object graph 1004 for vehicle A 902 does not include object F 912. Note that the object graph 1006 for vehicle B 904 does not include object E 910. This illustrates that the object graphs 1004, 1006 may not be a perfect match.

[0119] Fig.11 1 is a plan view showing the resulting object graph and the rotation and translation of the object graph 1004 of vehicle A 902 and the object graph 1006 of vehicle B 904 in an attempt to match or overlap. The object graph 1006 of vehicle B 904 is shown rotated at 1102. A -h B At 1104, an object diagram 1006 is shown translating vehicle B 904 by d AB The result of overlaying the object map 1006 of vehicle B 904 on the object map 1004 of vehicle A 902. The object map may be rotated by the difference between the azimuth angles (eg, h A -h B ) and then translates by the distance d between the vehicle or the vehicle sensor used to create the object map AB The number or percentage of matching objects can then be determined by counting matching and non-matching objects. As long as the number or threshold exceeds the trust threshold, the discrepancy may not cause authentication to fail.

[0120] Fig.12 is a plan view showing matching based on velocity vectors. Fig.12A vehicle A relative velocity vector diagram 1202, a vehicle B relative velocity vector diagram 1204, and a ground velocity vector diagram 1206 are shown. It should be noted that for illustrative purposes, all diagrams include vector diagrams for all objects. The velocity vector of each object can be compared by adding the velocity vector of the vehicle to the velocity vector of the object and determining whether it is similar enough to the result of adding the velocity vector of another vehicle to the velocity vector of the same object. This can be done for each commonly detected object.

[0121] According to the following equation 9, when added to the ground velocity vector of the object, the velocity V of vehicle A is A 1208 is equal to the speed V of vehicle B B 1210.

[0122]

[0123] Equation 9

[0124] Ground speed vectors are shown in ground speed vector diagram 1206, including vehicle A ground speed vector 121, vehicle B ground speed vector 1214, vehicle C ground speed vector 1216, vehicle D ground speed vector 1218, and object E ground speed vector 1220, and object F ground speed vector 1222. The relative speeds of various vehicles and objects determined by vehicle A 902 are shown in dashed lines in vehicle A relative speed vector diagram 1202. The relative speeds of various vehicles and objects determined by vehicle B 904 are shown in vehicle B relative speed vector diagram 1204.

[0125] Fig.13 A schematic flow chart of a method 1300 for authenticating vehicle-to-vehicle communications is shown. The method 1300 may be performed by any suitable computing device, including, for example, a controller 102 of a vehicle. The method 1300 begins at 1302 and the computing device receives sensor data from a first vehicle. At 1304, the computing device receives auxiliary sensor data from a second vehicle. At 1306, the computing device extracts an authentication based on the sensor data and the auxiliary sensor data, including one or more of: proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data. At 1308, the computing device determines whether the authentication satisfies a trust threshold of the first vehicle.

[0126] Fig.14A schematic flow chart of a method 1400 for authenticating vehicle-to-vehicle communications is shown. The method 1400 may be performed by any suitable computing device, including, for example, a controller 102 of a vehicle. The method 1400 begins at 1402 and the computing device receives sensor data from a first vehicle, including a camera image, and ranging data including one or more of radar data, lidar data, sonar data, or ultrasonic data. At 1404, the computing device receives auxiliary sensor data from a second vehicle, including an auxiliary camera image, and auxiliary ranging data including one or more of radar data, lidar data, sonar data, or ultrasonic data. At 1406, the computing device extracts authentication points from the camera image, wherein the authentication points include one or more of: a distance of the second vehicle relative to the first vehicle; an azimuth between the second vehicle and the first vehicle; or a vehicle identification of the second vehicle. At 1408, the computing device verifies the authentication points based on the ranging data. At 1410, the computing device exchanges the heading angle of the first vehicle or the second vehicle to compensate for the angular difference between the sensor data and the auxiliary sensor data. At 1412, the computing device extracts an authentication including one or more of the following based on the authentication points, the sensor data, and the auxiliary sensor data: proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data. At 1414, the computing device determines whether the authentication satisfies a trust threshold of the first vehicle.

[0127] Fig.15 A schematic flow chart of a method 1500 for authenticating vehicle-to-vehicle communications is shown. The method 1500 may be performed by any suitable computing device, including, for example, the controller 102 of the vehicle. The method 1500 begins at 1502 and the computing device receives an object map captured by one or more sensors of a first vehicle. At 1504, the computing device receives an auxiliary object map captured by one or more sensors of a second vehicle. At 1506, the computing device generates an overlapping object map by rotating and / or translating one of the object map or the auxiliary object map to align the auxiliary object map with the object map. At 1508, the computing device determines whether one or more common objects are identified in the object map and the auxiliary object map based on the overlapping object map. At 1510, the computing device determines whether the one or more common objects meet a trust threshold of the first vehicle. At 1512, if the one or more common objects meet the trust threshold, the computing device authenticates the communication between the first vehicle and the second vehicle.

[0128] Using sensors in addition to or as an alternative to images can also help reduce potential attacks. For example, an advanced attacker can impersonate a vehicle by offline preparing a dictionary of images taken from various locations and selecting images from the dictionary to masquerade as other vehicles around the victim vehicle. Alternatively, an attacker can take a snapshot of the victim vehicle, crop the victim vehicle from the snapshot, and superimpose the victim vehicle in a position to mislead the victim into believing that the image was taken from another vehicle. Thanks to recent advances in computer vision, more camera image modification techniques are available to attackers.

[0129] To overcome potential weaknesses in previous camera image-based V2V authentication, other ADAS sensors that are rapidly becoming ubiquitous in modern vehicles can be utilized. These other ADAS sensors (including radar, lidar, and other sensors) are capable of detecting surrounding objects (stationary and / or moving) and can be used to provide enhanced authentication procedures, such as Figure 8 As shown in the processing flow 800.

[0130] Embodiments may still check the vehicle identity (vehicle number) captured from the camera snapshot. In addition, the vehicle extracts the relative distance, azimuth, and vehicle number from the camera snapshot of another vehicle. However, to prevent image modification attacks, this information is also cross-checked with information from other ADAS sensors such as radar and lidar. Although these sensors do not provide vehicle number information, they still provide relative distances and azimuths of surrounding objects, which should be exactly the same as the information from the camera snapshot within error limits. During this process, the heading angle of each vehicle can also be exchanged to compensate for the angular difference when the two vehicles are not aligned (for example, in a curved road). The surrounding objects can be stationary (for example, landmarks, trees, buildings, road signs) or moving (for example, other surrounding vehicles, and possible pedestrians).

[0131] Reference again Figure 10 to Figure 11 , a vehicle A object graph 1004 is shown, and a vehicle B object graph 1006 is shown. To determine if two vehicles are close to each other (for authentication), the algorithm rotates and / or translates one of the two object graphs (e.g. Fig.11 The vehicle B object graph 1006 is shown to determine if there is an acceptable match between any detected objects.

[0132] Due to sensor noise, different sensing capabilities, obstacles, the surrounding objects detected by two vehicles cannot be the same in practice. Therefore, if the number of objects detected in common in both vehicles is above the system trust threshold, authentication is granted for subsequent communication. The system trust threshold can be determined empirically in the surrounding environment. For example, it will be higher in urban areas with high object density compared to rural areas. Other implementations may make trust decision choices more for each application layer rather than the system layer. In this way, for a given number of objects detected in common in two vehicles, some safety-critical applications do not use a stricter trust limit to authenticate the communication, while other non-critical applications meet it.

[0133] In an implementation, the authentication system may also use the velocity vectors of the surrounding objects during a short duration. The vehicle may calculate the ground speed of the surrounding objects by adding its velocity to the velocity vector of each object. Comparing the ground speed vectors of objects detected in common in two vehicles will make it even more difficult for an attacker to impersonate a vehicle in V2V communications. Using additional ADAS sensor data makes it even more difficult for an attacker to bypass the V2V authentication system compared to authentication based on images alone. In such an embodiment, authentication may include equal detected ground speeds of common objects identified by the sensor data and the auxiliary data.

[0134] Example

[0135] The following examples relate to other embodiments.

[0136] Example 1 is a method that includes receiving sensor data from vehicle A and receiving sensor data from vehicle B. The sensor data includes one or more of radar, lidar, image, sonar, ultrasound, or other sensor data. The method includes verifying that vehicle B is close to vehicle A based on the sensor data. For example, the method may include verifying the relative position (distance and / or angle) of vehicle B relative to vehicle A based on the sensor data from vehicle A and the sensor data from vehicle B. As another example, the method may include verifying the relative position of an object in a vehicle A object map determined based on the sensor data from vehicle A and a vehicle B object map determined based on the sensor data from vehicle B. As another example, the method may include verifying the determined ground speed of a detected object based on the sensor data from vehicle A and the sensor data from vehicle B. In one embodiment, object maps or vector maps may be exchanged so that each vehicle does not have to determine the object map or vector map of the other vehicle.

[0137] In Example 2, the method of Example 1 includes verifying that a similarity between the object graph, the velocity vector, etc. can be determined to be within a desired threshold.

[0138] In Example 3, a method as described in any of Examples 1-2 includes comparing a relative position determined based on one type of sensor data (e.g., image data) with a relative position based on another type of sensor data (e.g., radar or lidar), and verifying whether the relative positions are sufficiently similar.

[0139] In Example 4, the method of any one of Examples 1-3 further includes: determining a vehicle A object graph based on the vehicle A sensor data; determining a vehicle B object graph based on the vehicle B additional sensor data; and verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar. The authenticating the vehicle B includes authenticating in response to verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar.

[0140] In Example 5, the method as described in Example 4 includes rotating and / or translating at least one of the vehicle A object map or the vehicle B object map to accommodate different orientations or positions of the vehicle A and the vehicle B.

[0141] In Example 6, verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar as described in any of Examples 4-5 includes determining that a number of commonly detected objects in the vehicle A object graph and the vehicle B object graph is above a system trust threshold.

[0142] In Example 7, the method as described in any one of Examples 1-6 includes: determining the ground speed of one or more objects near vehicle A based on the additional sensor data of vehicle A; determining the ground speed of one or more objects near vehicle B based on the additional sensor data of vehicle B; and verifying that the ground speed of one or more objects near vehicle A and the ground speed of one or more objects near vehicle B are sufficiently similar.

[0143] Example 8 is a method comprising: receiving a first image from a camera of vehicle A; receiving a second image from vehicle B; verifying that the relative position of vehicle B and vehicle A according to the first image corresponds to the relative position of vehicle A and vehicle B according to the second image; and authenticating vehicle B in response to the verification.

[0144] In Example 9, as described in Example 8, verifying that the relative position of the vehicle B to the vehicle A according to the first image corresponds to the relative position of the vehicle A to the vehicle B according to the second image includes identifying, by the controller of the vehicle A, the image of the vehicle B in the first image, and identifying, by the controller of the vehicle A, the image of the vehicle A in the second image. The method includes determining, by the controller of the vehicle A, at least one of a first distance from the vehicle A to the vehicle B and a first angle from the vehicle A to the vehicle B according to the position of the image of the vehicle B in the first image. The method also includes determining, by the controller of the vehicle A, at least one of a second distance from the vehicle B to the vehicle A and a second angle from the vehicle B to the vehicle A according to the position of the image of the vehicle A in the second image. The method also includes determining, by the controller of the vehicle A, at least one of: (a) the second distance is within a predetermined distance tolerance of the first distance; and (b) the second angle is within a predetermined angle tolerance of the first angle.

[0145] In Example 10, as in any one of Examples 8-9, at least one of the first distance from the vehicle A to the vehicle B and the first angle from the vehicle A to the vehicle B according to the position of the image of the vehicle B in the first image includes both the first distance from the vehicle A to the vehicle B and the first angle from the vehicle A to the vehicle B according to the position of the image of the vehicle B in the first image. At least one of the second distance from the vehicle B to the vehicle A and the second angle from the vehicle B to the vehicle A according to the position of the image of the vehicle A in the second image includes both the second distance from the vehicle B to the vehicle A and the second angle from the vehicle B to the vehicle A according to the position of the image of the vehicle A in the second image.

[0146] In Example 11, receiving the first image from the camera of the vehicle A as described in any of Examples 8-10 includes receiving a first front image from a front camera mounted to the vehicle A, and receiving a first rear image from a rear camera mounted to the vehicle A. The method includes receiving a second front image from the front camera mounted to the vehicle B, and receiving a second rear image from the rear camera mounted to the vehicle B.

[0147] In Example 12, identifying the image of the vehicle b in the first image as described in any of Examples 8-11 includes identifying the license plate number of the vehicle B in the first image, and wherein identifying the image of the vehicle A in the second image includes identifying the license plate number of the vehicle A in the second image.

[0148] In Example 13, the method of Example 12 further includes receiving, by the controller of the vehicle A, a message from the vehicle B including a license plate number of the vehicle B.

[0149] In Example 14, as described in any one of Examples 12-13, determining at least one of the first distance from the vehicle A to the vehicle B and the first angle from the vehicle A to the vehicle B based on the position of the image of the vehicle B in the first image includes: determining a first size (h ) in pixels of the license plate of the vehicle B in the first image. image1 ); and determining the first distance (d1) to be equal to d init *h init / h image1 , where d init is the calibration distance and h init is located at a distance d from the test camera init Determining at least one of the second distance from vehicle B to vehicle A and the second angle from vehicle B to vehicle A based on the position of the image of vehicle A in the second image comprises: determining a second size (h2) of the license plate of vehicle A in pixels in the second image. image2 ); and determining the second distance (d2) to be equal to d init *h init / h image .

[0150] In Example 15, determining at least one of the first distance from the vehicle A to the vehicle B and the first angle from the vehicle A to the vehicle B according to the position of the image of the vehicle B in the first image as described in Example 14 includes: determining a first pixel offset (d ) of the license plate of the vehicle B in the first image in pixels from the center of the first image. shift1 ); offset the first distance (d shift1m ) is determined to be equal to (h m*dshift1 / h image1 ), where h m is the measured dimension of the test license plate; and the first angle is determined to be equal to Arccos(d1 / d shift1m). Determining at least one of a second distance from vehicle B to vehicle A and a second angle from vehicle B to vehicle A based on the position of the image of vehicle A in the second image includes: determining a second pixel offset (d shift2 ); determine equal to (h m *d shift2 / h image2 ) of the second distance offset (d shift2m ); and determine that it is equal to Arccos(d2 / d shift2m )’s second angle.

[0151] In Example 16, the camera of the vehicle A as described in any of Examples 8-15 is a first camera. The method also includes authenticating the vehicle B in response to determining that one or more background objects in the second image correspond to objects in an image received from a second camera mounted to the vehicle A and facing in an opposite direction from the first camera.

[0152] In Example 17, authenticating the vehicle B as described in any of Examples 8-16 includes performing a Diffie-Hellman key exchange between the vehicle A and the vehicle B.

[0153] In Example 18, the method as described in any of Examples 8-17 also includes: receiving additional sensor data from vehicle A, including one or more of radar and lidar data; and receiving additional sensor data from vehicle B, including one or more of radar and lidar data from said vehicle B.

[0154] In Example 19, the method described in Example 18 includes verifying that the relative position of the vehicle B to the vehicle A based on the additional sensor data of the vehicle A is similar to the relative position of the vehicle A to the vehicle B based on the additional sensor data of the vehicle B, and wherein the relative position is within the error limit of the relative position determined based on the first image and the second image.

[0155] In Example 20, the method of any of Examples 18-19 includes: determining a vehicle A object graph based on the vehicle A additional sensor data; determining a vehicle B object graph based on the vehicle B additional sensor data; and verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar. The authenticating the vehicle B includes authenticating in response to verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar.

[0156] In Example 21, verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar as described in Example 20 includes rotating and / or translating at least one of the vehicle A object graph or the vehicle B object graph to accommodate different orientations or positions of the vehicle A and the vehicle B.

[0157] In Example 22, verifying that the vehicle A object graph and the vehicle B object graph are sufficiently similar as described in any of Examples 20-21 includes determining that a number of commonly detected objects in the vehicle A object graph and the vehicle B object graph is above a system trust threshold.

[0158] In Example 23, the method as described in any of Examples 18-22 also includes: determining the ground speed of one or more objects near vehicle A based on the additional sensor data of vehicle A; determining the ground speed of one or more objects near vehicle B based on the additional sensor data of vehicle B; and verifying that the ground speed of one or more objects near vehicle A and the ground speed of one or more objects near vehicle B are sufficiently similar.

[0159] Example 24 is a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to implement a method as described in any one of Examples 1-23 or to realize the system or device described therein.

[0160] Example 25 is a method for authenticating vehicle-to-vehicle communications. The method includes: receiving sensor data from a first vehicle and receiving auxiliary sensor data from a second vehicle. The method includes extracting, based on the sensor data and the auxiliary sensor data, an authentication including one or more of: proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data. The method includes determining whether the authentication satisfies a trust threshold of the first vehicle.

[0161] Example 26 is the method as in Example 25, wherein the authentication includes equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data.

[0162] Example 27 is a method as described in any of Examples 25-26, wherein: the sensor data includes an image received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; the auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0163] Example 28 is a method as described in any of Examples 25-27, which also includes extracting authentication points from the camera image, wherein the authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0164] Example 29 is a method as described in any of Examples 25-28, further comprising verifying the authentication point based on the ranging data, wherein verifying the authentication point comprises: extracting a ranging authentication point from the ranging data. The ranging authentication point comprises one or more of: a distance of the second vehicle relative to the first vehicle; or an azimuth between the second vehicle and the first vehicle. The method further comprises: cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0165] Example 30 is a method as described in any of Examples 25-29, further comprising exchanging a heading angle of the first vehicle or the second vehicle to compensate for an angular difference between the sensor data and the auxiliary sensor data.

[0166] Example 31 is a method as described in any of Examples 25-30, which also includes: receiving an object map captured by one or more sensors of the first vehicle; receiving an auxiliary object map captured by one or more sensors of the second vehicle; generating an overlapping object map by rotating and / or translating the object map or one of the auxiliary object maps to align the auxiliary object map with the object map; and determining whether there are one or more object matches between the object map and the auxiliary object map based on the overlapping object map.

[0167] Example 32 is a method as described in any of Examples 25-31, which also includes: determining whether the one or more object matches between the object graph and the auxiliary object graph satisfy a trust threshold of an application of the first vehicle; if the one or more object matches satisfy the trust threshold of the application, authenticating the communication between the application of the first vehicle and the second vehicle; and if the one or more object matches do not satisfy the trust threshold of the application, denying the communication between the application of the first vehicle and the second vehicle.

[0168] Example 33 is a method as described in any of Examples 25-32, which also includes: determining whether the one or more object matches between the object graph and the auxiliary object graph meet the global trust threshold of the first vehicle; if the one or more object matches meet the global trust threshold, then authenticating the communication between the first vehicle and the second vehicle.

[0169] Example 34 is a method as described in any of Examples 25-33, wherein determining whether the authentication satisfies the trust threshold of the first vehicle includes verifying one or more of: the identity of the second vehicle; whether the second vehicle is in proximity to the first vehicle; the presence of one or more common objects sensed by the sensor data and the auxiliary sensor data; or determining ground speed based on the equality of one or more common objects of the sensor data and the auxiliary sensor data.

[0170] Example 35 is a method as described in any of Examples 25-34, further comprising permitting communication between the first vehicle and the second vehicle if the verification satisfies the trust threshold of the first vehicle.

[0171] Example 36 is a system for authenticating vehicle-to-vehicle communications. The system includes: a first vehicle, the first vehicle including one or more sensors that provide sensor data; and a vehicle controller that communicates with the one or more sensors and is configured to receive auxiliary sensor data from a second vehicle. The system thereby causes the vehicle controller to include a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: receive sensor data from the one or more sensors of the first vehicle; receive auxiliary sensor data from the second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication that includes one or more of the following: proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data. The instructions also cause the one or more processors to determine whether the authentication satisfies a trust threshold of the first vehicle.

[0172] Example 37 is a system as described in Example 36, wherein: the sensor data includes an image received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; the auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0173] Example 38 is a system as described in any of Examples 36-37, wherein the instructions further cause the one or more processors to extract authentication points from the camera image, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0174] Example 39 is a system as described in any of Examples 36-38, wherein the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of: the distance of the second vehicle relative to the first vehicle; or the azimuth between the second vehicle and the first vehicle. The instructions also cause the one or more processors to: cross-check the authentication point extracted from the image with a corresponding ranging authentication point; and verify that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0175] Example 40 is a system as described in any of Examples 36-39, wherein the instructions further cause the one or more processors to swap the heading angle of the first vehicle or the second vehicle to compensate for the angular difference between the sensor data and the auxiliary sensor data.

[0176] Example 41 is a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: receive sensor data from one or more sensors of a first vehicle; receive auxiliary sensor data from a second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication including one or more of: proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data. The instructions also cause the one or more processors to determine whether the authentication satisfies a trust threshold of the first vehicle.

[0177] Example 42 is a non-transitory computer-readable storage medium as described in Example 41, wherein: the sensor data includes an image received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; the auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0178] Example 43 is a non-transitory computer-readable storage medium as described in any of Examples 41-42, wherein the instructions further cause the one or more processors to extract authentication points from the camera image, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0179] Example 44 is a non-transitory computer-readable storage medium as described in any of Examples 41-43, wherein the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of: the distance of the second vehicle relative to the first vehicle; or the azimuth between the second vehicle and the first vehicle. The instructions further cause the one or more processors to: cross-check the authentication point extracted from the image with a corresponding ranging authentication point; and verify that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0180] Example 45 is a system or apparatus including means for implementing the method or system or device of any one of Examples 1-44.

[0181] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments should be considered in all respects as merely illustrative and not restrictive. Therefore, the scope of the present disclosure is indicated by the appended claims rather than by the foregoing description. All changes within the equivalent meaning and scope of the claims should be included in the scope of the claims.

[0182] According to the present invention, a method is provided, which receives sensor data from a first vehicle and auxiliary sensor data from a second vehicle; based on the sensor data and the auxiliary sensor data, extracts authentication including one or more of the following: the proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data; and determines whether the authentication meets the trust threshold of the first vehicle.

[0183] According to an embodiment, said authentication comprises equal detected ground speeds of a common object identified by said sensor data and said auxiliary sensor data.

[0184] According to an embodiment, the sensor data includes an image received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor or an ultrasonic sensor of the first vehicle; the auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor or an ultrasonic sensor of the second vehicle.

[0185] According to an embodiment, the above invention is also characterized in that: authentication points are extracted from the image received by the camera of the first vehicle, wherein the authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0186] According to an embodiment, the present invention is further characterized by: verifying the authentication point based on the ranging data, wherein verifying the authentication point comprises: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point comprises one or more of the following: the distance of the second vehicle relative to the first vehicle; or

[0187] The azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0188] According to an embodiment, the above invention is further characterized by exchanging the heading angle of the first vehicle or the second vehicle to compensate for the angle difference between the sensor data and the auxiliary sensor data.

[0189] According to an embodiment, the above invention is also characterized by: receiving an object map captured by one or more sensors of the first vehicle; receiving an auxiliary object map captured by one or more sensors of the second vehicle; generating an overlapping object map by rotating and / or translating the object map or one of the auxiliary object maps to align the auxiliary object map with the object map; and determining whether one or more common objects are identified in the object map and the auxiliary object map based on the overlapping object map.

[0190] According to an embodiment, the above invention is also characterized by: determining whether the one or more common objects satisfy the trust threshold of the application of the first vehicle; if the one or more common objects satisfy the trust threshold of the application, then authenticating the communication between the application of the first vehicle and the second vehicle; and if the one or more common objects do not satisfy the trust threshold of the application, then rejecting the communication between the application of the first vehicle and the second vehicle.

[0191] According to an embodiment, the above invention is also characterized by: determining whether the one or more common objects identified in the object graph and the auxiliary object graph meet the global trust threshold of the first vehicle; if the one or more common objects meet the global trust threshold, then authenticating the communication between the first vehicle and the second vehicle.

[0192] According to an embodiment, the above invention is also characterized in that: determining whether the authentication meets the trust threshold of the first vehicle includes verifying one or more of the following: the identity of the second vehicle; whether the second vehicle is near the first vehicle; the presence of one or more common objects sensed by the sensor data and the auxiliary sensor data; or determining the ground speed based on the equality of one or more common objects of the sensor data and the auxiliary sensor data.

[0193] According to an embodiment, the above invention is further characterized by permitting communication between the first vehicle and the second vehicle if the verification satisfies the trust threshold of the first vehicle.

[0194] According to the present invention, a system is provided, which has: a first vehicle, the first vehicle includes one or more sensors that provide sensor data; and a vehicle controller, the vehicle controller communicates with the one or more sensors and is configured to receive auxiliary sensor data from a second vehicle; wherein the vehicle controller includes a non-transitory computer-readable storage medium storing instructions, and the instructions, when executed by one or more processors, cause the one or more processors to: receive sensor data from the one or more sensors of the first vehicle; receive auxiliary sensor data from the second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication including one or more of the following: the proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data; and determine whether the authentication meets the trust threshold of the first vehicle.

[0195] According to an embodiment, the sensor data includes images received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle;

[0196] The auxiliary sensor data includes auxiliary images received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0197] According to an embodiment, the instructions also cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0198] According to an embodiment, the instructions also cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of the following: the distance of the second vehicle relative to the first vehicle; or the azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0199] According to an embodiment, the instructions further cause the one or more processors to swap the heading angle of the first vehicle or the second vehicle to compensate for an angular difference between the sensor data and the auxiliary sensor data.

[0200] According to the present invention, a non-transitory computer-readable storage medium stores instructions, which, when executed by one or more processors, cause the one or more processors to: receive sensor data from one or more sensors of a first vehicle; receive auxiliary sensor data from a second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication including one or more of the following: the proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data; and determine whether the authentication meets a trust threshold of the first vehicle.

[0201] According to an embodiment, the sensor data includes images received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle;

[0202] The auxiliary sensor data includes auxiliary images received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0203] According to the present invention, a method is provided, which receives sensor data from a first vehicle and auxiliary sensor data from a second vehicle; based on the sensor data and the auxiliary sensor data, extracts authentication including one or more of the following: the proximity of the second vehicle to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data; and determines whether the authentication meets the trust threshold of the first vehicle.

[0204] According to an embodiment, said authentication comprises equal detected ground speeds of a common object identified by said sensor data and said auxiliary sensor data.

[0205] According to an embodiment, the sensor data includes an image received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor or an ultrasonic sensor of the first vehicle; the auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor or an ultrasonic sensor of the second vehicle.

[0206] According to an embodiment, the above invention is also characterized in that: authentication points are extracted from the image received by the camera of the first vehicle, wherein the authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0207] According to an embodiment, the present invention is further characterized by: verifying the authentication point based on the ranging data, wherein verifying the authentication point comprises: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point comprises one or more of the following: the distance of the second vehicle relative to the first vehicle; or

[0208] The azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0209] According to an embodiment, the above invention is further characterized by exchanging the heading angle of the first vehicle or the second vehicle to compensate for the angle difference between the sensor data and the auxiliary sensor data.

[0210] According to an embodiment, the above invention is also characterized by: receiving an object map captured by one or more sensors of the first vehicle; receiving an auxiliary object map captured by one or more sensors of the second vehicle; generating an overlapping object map by rotating and / or translating the object map or one of the auxiliary object maps to align the auxiliary object map with the object map; and determining whether one or more common objects are identified in the object map and the auxiliary object map based on the overlapping object map.

[0211] According to an embodiment, the above invention is also characterized by: determining whether the one or more common objects satisfy the trust threshold of the application of the first vehicle; if the one or more common objects satisfy the trust threshold of the application, then authenticating the communication between the application of the first vehicle and the second vehicle; and if the one or more common objects do not satisfy the trust threshold of the application, then rejecting the communication between the application of the first vehicle and the second vehicle.

[0212] According to an embodiment, the above invention is also characterized by: determining whether the one or more common objects identified in the object graph and the auxiliary object graph meet the global trust threshold of the first vehicle; if the one or more common objects meet the global trust threshold, then authenticating the communication between the first vehicle and the second vehicle.

[0213] According to an embodiment, the above invention is also characterized in that: determining whether the authentication meets the trust threshold of the first vehicle includes verifying one or more of the following: the identity of the second vehicle; whether the second vehicle is near the first vehicle; the presence of one or more common objects sensed by the sensor data and the auxiliary sensor data; or determining the ground speed based on the equality of one or more common objects of the sensor data and the auxiliary sensor data.

[0214] According to an embodiment, the above invention is further characterized by permitting communication between the first vehicle and the second vehicle if the verification satisfies the trust threshold of the first vehicle.

[0215] According to the present invention, a system is provided, which has: a first vehicle, the first vehicle includes one or more sensors that provide sensor data; and a vehicle controller, the vehicle controller communicates with the one or more sensors and is configured to receive auxiliary sensor data from a second vehicle; wherein the vehicle controller includes a non-transitory computer-readable storage medium storing instructions, and the instructions, when executed by one or more processors, cause the one or more processors to: receive sensor data from the one or more sensors of the first vehicle; receive auxiliary sensor data from the second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication including one or more of the following: the proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data; and determine whether the authentication meets the trust threshold of the first vehicle.

[0216] According to an embodiment, the sensor data includes images received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle;

[0217] The auxiliary sensor data includes auxiliary images received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0218] According to an embodiment, the instructions also cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0219] According to an embodiment, the instructions also cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting a ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of the following: the distance of the second vehicle relative to the first vehicle; or the azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0220] According to an embodiment, the instructions further cause the one or more processors to swap the heading angle of the first vehicle or the second vehicle to compensate for an angular difference between the sensor data and the auxiliary sensor data.

[0221] According to the present invention, a non-transitory computer-readable storage medium stores instructions, which, when executed by one or more processors, cause the one or more processors to: receive sensor data from one or more sensors of a first vehicle; receive auxiliary sensor data from a second vehicle; based on the sensor data and the auxiliary sensor data, extract an authentication including one or more of the following: the proximity of the second vehicle relative to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of a common object identified by the sensor data and the auxiliary sensor data; and determine whether the authentication meets a trust threshold of the first vehicle.

[0222] According to an embodiment, the sensor data includes images received from a camera of the first vehicle; the sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle;

[0223] The auxiliary sensor data includes auxiliary images received from a camera of the second vehicle; and the auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle.

[0224] According to an embodiment, the instructions also cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0225] According to an embodiment, the above invention is also characterized in that: the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting the ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of the following:

[0226] The distance of the second vehicle relative to the first vehicle; or the azimuth between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

[0227] According to an embodiment, the instructions also cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: the distance of the second vehicle relative to the first vehicle; the azimuth angle between the second vehicle and the first vehicle; or vehicle identification of the second vehicle.

[0228] According to an embodiment, the above invention is also characterized in that: the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point includes: extracting the ranging authentication point from the ranging data, wherein the ranging authentication point includes one or more of the following:

[0229] The distance of the second vehicle relative to the first vehicle; or the azimuth between the second vehicle and the first vehicle; cross-checking the authentication point extracted from the image with a corresponding ranging authentication point; and verifying that the authentication point is equal to the corresponding ranging authentication point within an error limit.

Claims

1. A vehicle-to-vehicle communication authentication method, comprising: receiving primary sensor data from a first vehicle and auxiliary sensor data from a second vehicle; Extracting authentication based on the primary sensor data and the auxiliary sensor data including one or more of the following: a proximity of the second vehicle to the first vehicle, the proximity being a relative position of the second vehicle with respect to the first vehicle; or a common object identified by the sensor data and the auxiliary sensor data; as well as determining whether the authentication satisfies a trust threshold of the first vehicle; Also includes: receiving ground velocity vectors of surrounding objects detected by the first vehicle; receiving ground velocity vectors of surrounding objects detected by the second vehicle; and Comparing whether ground velocity vectors of surrounding objects detected by the first vehicle and the second vehicle match each other. 2 . The method of claim 1 , wherein the authentication comprises equal detected ground speeds of common objects identified by the sensor data and the auxiliary sensor data.

3. The method of claim 1, wherein: The sensor data includes images received from a camera of the first vehicle; The sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; The auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; The auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle; and The method further includes extracting authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: a distance of the second vehicle relative to the first vehicle; The heading angle between the second vehicle and the first vehicle; or A vehicle identification of the second vehicle.

4. The method of claim 3, further comprising verifying the authentication point based on the ranging data, wherein verifying the authentication point comprises: Extracting ranging authentication points from the ranging data, wherein the ranging authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; or the azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication points extracted from the image with corresponding range-measured authentication points; as well as Verify that the authentication point is equal to the corresponding ranging authentication point within an error limit.

5. The method of claim 1, further comprising one or more of the following: exchanging the heading angle of the first vehicle or the second vehicle to compensate for the angular difference between the sensor data and the auxiliary sensor data; or If the authentication satisfies the trust threshold of the first vehicle, communication between the first vehicle and the second vehicle is permitted.

6. The method of claim 1, further comprising: receiving an object map captured by one or more sensors of the first vehicle; receiving an auxiliary object map captured by one or more sensors of the second vehicle; generating an overlaid object graph by rotating and / or translating one of the object graph or the auxiliary object graph to align the auxiliary object graph with the object graph; as well as A determination is made based on the overlapping object graphs whether one or more common objects are identified in the object graph and the auxiliary object graph.

7. The method of claim 6, further comprising: determining whether the one or more common objects satisfy a trust threshold for an application of the first vehicle; authenticating communications between the application of the first vehicle and the second vehicle if the one or more common objects satisfy the trust threshold of the application; denying communication between the application of the first vehicle and the second vehicle if the one or more common objects do not satisfy the trust threshold of the application; determining whether the one or more common objects identified in the object graph and the auxiliary object graph satisfy a global trust threshold for the one vehicle; as well as If the one or more common objects satisfy the global trust threshold, then authenticating communications between the first vehicle and the second vehicle.

8. The method of claim 1 , wherein determining whether the authentication satisfies the trust threshold of the first vehicle comprises: Verify one or more of the following: the identity of the second vehicle; whether the second vehicle is near the first vehicle; the presence of one or more common objects sensed by the sensor data and the auxiliary sensor data; or Based on equal determined ground speeds of one or more common objects of the sensor data and the auxiliary sensor data.

9. A vehicle-to-vehicle communication authentication system, comprising: a first vehicle including one or more primary sensors providing primary sensor data; as well as a vehicle controller in communication with the one or more sensors and configured to receive auxiliary sensor data from a second vehicle; wherein the vehicle controller includes a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: receiving primary sensor data from the one or more primary sensors of the first vehicle; receiving auxiliary sensor data from the second vehicle; Extracting authentication based on the sensor data and the auxiliary sensor data including one or more of the following: a proximity of the second vehicle to the first vehicle, the proximity being a relative position of the second vehicle to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of common objects identified by the sensor data and the auxiliary sensor data; determining whether the authentication satisfies a trust threshold of the first vehicle; as well as receiving ground velocity vectors of surrounding objects detected by the first vehicle; receiving a ground velocity vector of surrounding objects detected by the second vehicle; as well as Comparing whether ground velocity vectors of surrounding objects detected by the first vehicle and the second vehicle match each other.

10. The system of claim 9, wherein: The sensor data includes images received from a camera of the first vehicle; The sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; The auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; The auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle; and The instructions further cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: a distance of the second vehicle relative to the first vehicle; The heading angle between the second vehicle and the first vehicle; or A vehicle identification of the second vehicle.

11. The system of claim 10, wherein the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point comprises: Extracting ranging authentication points from the ranging data, wherein the ranging authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; or the azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication points extracted from the image with corresponding range-measured authentication points; as well as Verify that the authentication point is equal to the corresponding ranging authentication point within an error limit.

12. The system of claim 9, wherein the instructions further cause the one or more processors to swap a heading angle of the first vehicle or the second vehicle to compensate for an angular difference between the sensor data and the auxiliary sensor data.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: receiving sensor data from one or more sensors of the first vehicle; receiving auxiliary sensor data from a second vehicle; Extracting authentication based on the sensor data and the auxiliary sensor data including one or more of the following: a proximity of the second vehicle to the first vehicle, the proximity being a relative position of the second vehicle to the first vehicle; a common object identified by the sensor data and the auxiliary sensor data; or equal detected ground speeds of common objects identified by the sensor data and the auxiliary sensor data; and A determination is made as to whether the authentication satisfies a trust threshold for the first vehicle.

14. The non-transitory computer-readable storage medium of claim 13, wherein: The sensor data includes images received from a camera of the first vehicle; The sensor data also includes ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the first vehicle; The auxiliary sensor data includes an auxiliary image received from a camera of the second vehicle; The auxiliary sensor data also includes auxiliary ranging data received from one or more of a radar sensor, a lidar sensor, a sonar sensor, or an ultrasonic sensor of the second vehicle; and The instructions further cause the one or more processors to extract authentication points from the image received from the camera of the first vehicle, wherein the authentication points include one or more of: a distance of the second vehicle relative to the first vehicle; The heading angle between the second vehicle and the first vehicle; or A vehicle identification of the second vehicle.

15. The non-transitory computer-readable storage medium of claim 14, wherein the instructions further cause the one or more processors to verify the authentication point based on the ranging data, wherein verifying the authentication point comprises: Extracting ranging authentication points from the ranging data, wherein the ranging authentication points include one or more of the following: the distance of the second vehicle relative to the first vehicle; or the azimuth angle between the second vehicle and the first vehicle; cross-checking the authentication points extracted from the image with corresponding range-measured authentication points; as well as Verify that the authentication point is equal to the corresponding ranging authentication point within an error limit.

Images