Physical and virtual identity association

Abstract

This invention relates to the association of physical and virtual identifiers. A system for associating physical and virtual identifiers of a target vehicle includes: a data processor located within the vehicle, the data processor including a wireless communication module; and a plurality of sensing sensors located within the vehicle and adapted to collect data related to the physical identifier of the target vehicle, and to transmit the data related to the physical identifier of the target vehicle to the data processor via a communication bus; the data processor is adapted to receive data related to the virtual identifier of the target vehicle via a wireless communication channel, and to associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle.

Description

Technical Field

[0001] This disclosure relates to a system for associating a physical identity of a target vehicle detected by a sensing sensor within the vehicle with a virtual identity of the target vehicle received via wireless communication between the vehicle and the target vehicle. Background Technology

[0002] In the current system, an autonomous vehicle using a wireless vehicle-to-vehicle or vehicle-to-infrastructure communication channel receives information transmitted from a target vehicle. This information includes identification information about the target vehicle, allowing the autonomous vehicle to recognize it. This information provides a virtual identifier for the target vehicle. This allows the autonomous vehicle to determine the target vehicle's position relative to itself, thus enabling the autonomous vehicle to take actions such as cooperative maneuvering and localization, as well as infrastructure-assisted coordination.

[0003] Furthermore, the autonomous vehicle will use sensing sensors located within the vehicle, such as lidar, radar, and cameras, to identify objects, such as target vehicles approaching the autonomous vehicle. This provides a physical identifier of the detected target vehicle. Typically, the autonomous vehicle's sensing sensors can detect multiple target vehicles. Current systems generally trust received virtual identifier information without verifying that the received virtual identifier information is associated with the correct physical identifier information. In other words, current systems do not verify that the wirelessly transmitted information corresponds to the correct one of the multiple target vehicles physically identified by the autonomous vehicle. Therefore, while current systems achieve their intended purpose, new and improved systems and methods are needed to associate the physical identifier of a target vehicle detected by the sensing sensors within the autonomous vehicle with the virtual identifier of the target vehicle received via wireless communication between the autonomous vehicle and the target vehicle. Summary of the Invention

[0004] According to several aspects of this disclosure, a method for associating a physical identifier and a virtual identifier of a target vehicle includes: collecting data related to the physical identifier of the target vehicle using multiple sensing sensors, and transmitting the data related to the physical identifier of the target vehicle to a data processor via a communication bus; collecting data related to the virtual identifier of the target vehicle using the data processor via a wireless communication channel; and associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor.

[0005] According to another aspect, associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor further includes: using the data processor to utilize a Bayesian Interference Model, and using the data processor to estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle.

[0006] According to another aspect, associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor further includes: using the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle using the data processor, and using the data processor to estimate the real-time state of the target vehicle.

[0007] According to another aspect, the data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0008] According to another aspect, computer vision features created for each model of all vehicles are stored in a cloud-based vehicle profile database, and the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle. The method includes using the model information received from the target vehicle and utilizing the data processor to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

[0009] According to another aspect, the model information sent by the target vehicle includes brand, model, year, and color.

[0010] According to another aspect, the cloud-based vehicle profile database is a deep neural network.

[0011] According to another aspect, the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle.

[0012] According to another aspect, the data associated with the virtual identifier of the target vehicle includes observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights, and structures.

[0013] According to another aspect, the data associated with the virtual identifier of the target vehicle also includes computer vision features of the target vehicle stored in a cloud-based vehicle profile database.

[0014] According to several aspects of this disclosure, a system for associating physical and virtual identifiers of a target vehicle includes: a data processor located within the vehicle itself, the data processor including a wireless communication module; and a plurality of sensing sensors located within the vehicle itself and adapted to collect data associated with the physical identifier of the target vehicle and to transmit the data associated with the physical identifier of the target vehicle to the data processor via a communication bus; the data processor is adapted to receive data associated with the virtual identifier of the target vehicle via a wireless communication channel and to associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle.

[0015] According to another aspect, when the physical identifier of the target vehicle is associated with the virtual identifier of the target vehicle, the data processor is also adapted to utilize a Bayesian inference model and estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle.

[0016] According to another aspect, when the physical identifier of the target vehicle is associated with the virtual identifier of the target vehicle, the data processor is also adapted to use the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle and estimate the real-time state of the target vehicle.

[0017] According to another aspect, the data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0018] According to another aspect, the system also includes a cloud-based vehicle profile database, which includes computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle. The data processor is also adapted to use the model information received from the target vehicle and to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

[0019] According to another aspect, the model information sent by the target vehicle includes brand, model, year, and color.

[0020] According to another aspect, the cloud-based vehicle profile database is a deep neural network.

[0021] According to another aspect, the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle, including observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights, and structures.

[0022] According to another aspect, the system also includes a cloud-based vehicle profile database, which includes computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle also includes the computer vision features of the target vehicle.

[0023] The present invention also includes the following technical solutions.

[0024] Solution 1. A method for associating the physical and virtual identifiers of a target vehicle, comprising:

[0025] Data related to the physical identifier of the target vehicle is collected using multiple sensing sensors, and the data related to the physical identifier of the target vehicle is transmitted to the data processor via a communication bus.

[0026] The data processor collects data related to the virtual identifier of the target vehicle via a wireless communication channel; and

[0027] The data processor is used to associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle.

[0028] Option 2. According to the method of Option 1, wherein associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor further includes: using the data processor to utilize a Bayesian inference model, and using the data processor to estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle.

[0029] Option 3. According to the method of Option 2, the method of associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor further includes: using the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle using the data processor, and using the data processor to estimate the real-time state of the target vehicle.

[0030] Option 4. The method according to Option 3, wherein the data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0031] Option 5. The method according to Option 2, wherein computer vision features created for each model of all vehicles are stored in a cloud-based vehicle profile database, and the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle, the method comprising using the model information received from the target vehicle, and utilizing the data processor to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

[0032] Option 6. The method described in Option 5, wherein the model information sent by the target vehicle includes brand, model, year, and color.

[0033] Option 7. The method according to Option 6, wherein the cloud-based vehicle configuration file database is a deep neural network.

[0034] Option 8. The method according to Option 2, wherein the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle.

[0035] Option 9. The method according to Option 8, wherein the data associated with the virtual identifier of the target vehicle includes observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights, and structures.

[0036] Option 10. The method according to Option 9, wherein the data associated with the virtual identifier of the target vehicle further includes computer vision features of the target vehicle stored in a cloud-based vehicle profile database.

[0037] Option 11. A system for associating physical and virtual identifiers of a target vehicle, comprising:

[0038] A data processor located within the vehicle, the data processor including a wireless communication module; and

[0039] Multiple sensing sensors, located within the autonomous vehicle and adapted to collect data related to the physical identifier of the target vehicle, and transmit the data related to the physical identifier of the target vehicle to the data processor via a communication bus;

[0040] The data processor is adapted to receive data related to the virtual identifier of the target vehicle via a wireless communication channel, and associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle.

[0041] Solution 12. The system according to Solution 11, wherein when the physical identifier of the target vehicle is associated with the virtual identifier of the target vehicle, the data processor is further adapted to utilize a Bayesian inference model and estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle.

[0042] Solution 13. The system according to Solution 12, wherein when the physical identifier of the target vehicle is associated with the virtual identifier of the target vehicle, the data processor is further adapted to use the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle and estimate the real-time state of the target vehicle.

[0043] Option 14. The system according to Option 13, wherein the data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0044] Option 15. The system according to Option 12 further includes a cloud-based vehicle profile database, the vehicle profile database including computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle including model information sent by the target vehicle, the data processor being further adapted to use the model information received from the target vehicle and to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

[0045] Option 16. The system according to Option 15, wherein the model information sent by the target vehicle includes brand, model, year and color.

[0046] Option 17. The method according to Option 16, wherein the cloud-based vehicle configuration file database is a deep neural network.

[0047] Option 18. The method according to Option 12, wherein the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle, including observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights, and structures.

[0048] Option 19. The system according to Option 18 further includes a cloud-based vehicle profile database, the cloud-based vehicle profile database including computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle also includes the computer vision features of the target vehicle.

[0049] Option 20. A method for associating the physical and virtual identifiers of a target vehicle, comprising:

[0050] Data related to the physical identifier of the target vehicle is collected using multiple sensing sensors, and the data related to the physical identifier of the target vehicle is transmitted to the data processor via a communication bus.

[0051] The data processor collects data related to the virtual identifier of the target vehicle via a wireless communication channel; and

[0052] The data processor associates the physical identifier of the target vehicle with the virtual identifier of the target vehicle by using a Bayesian inference model and by estimating the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle, and one of the following:

[0053] Using the data associated with the physical identifier of the target vehicle, including global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, the relative position of the target vehicle is determined using the data processor, and the real-time state of the target vehicle is estimated using the data processor, wherein the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0054] Using the model information received from the target vehicle, and utilizing the data processor to receive corresponding vehicle profile data from a cloud-based vehicle profile database, the cloud-based vehicle profile database being a deep neural network and including computer vision features created for each model of all vehicles, wherein the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle, including brand, model, year, and color; and

[0055] The data processor associates the physical identifier of the target vehicle with the virtual identifier of the target vehicle, wherein the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle, including observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights and structures, as well as computer vision features of the target vehicle stored in a cloud-based vehicle profile database.

[0056] Other applicable areas will become apparent from the description provided herein. It should be understood that the descriptions and specific examples are intended for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0057] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0058] Figure 1 This is a schematic diagram of a system for associating physical and virtual identifiers of a target vehicle according to exemplary embodiments of the present disclosure;

[0059] Figure 2 This is a schematic diagram of the application of the system disclosed herein, wherein the self-vehicle is associated with physical and virtual identifiers for each of two target vehicles;

[0060] Figure 3 It is a schematic diagram illustrating the relationship between the identified physical identifier, the received virtual identifier, and the actual position of the target vehicle relative to the vehicle itself;

[0061] Figure 4 It is a probability distribution map of the target vehicle's physical identifier, virtual identifier, and actual location;

[0062] Figure 5 It is a probability distribution map of the characteristic distances of the target vehicle;

[0063] Figure 6 This is a schematic diagram of the application of the system disclosed herein, in which the autonomous vehicle utilizes the perception data of the target vehicle;

[0064] Figure 7 This is a schematic diagram illustrating the relationship between the identified physical identifier and the received virtual identifier for each of two target vehicles.

[0065] Figure 8 This is a schematic diagram of the application of the system disclosed herein, in which an autonomous vehicle uses the system for cooperative lane changing;

[0066] Figure 9This is a schematic diagram of the application of the system disclosed herein, wherein the system is used for the manipulation of infrastructure coordination and the precise positioning of infrastructure assistance;

[0067] Figure 10 This is a schematic diagram of the application of the system disclosed herein, wherein the system is used to share physical and virtual identification association information between vehicles;

[0068] Figure 11 This is a schematic diagram illustrating the relationship between the identified physical identifiers and received virtual identifiers for each of two target vehicles, where one of the target vehicles shares its perception information with its own vehicle; and

[0069] Figure 12 This is a schematic flowchart illustrating a method for using a system to associate physical and virtual identifiers of a target vehicle.

[0070] The accompanying drawings are not necessarily drawn to scale, and some features may be enlarged or minimized, for example, to show details of specific components. In some cases, well-known components, systems, materials, or methods have not been described in detail to avoid obscuring this disclosure. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but rather serve only as the basis for the claims and as an illustrative basis for teaching those skilled in the art to apply this disclosure in various ways. Detailed Implementation

[0071] The following description is exemplary in nature only and is not intended to limit this disclosure, its application, or use. Furthermore, it is not intended to be bound by any expressions or implied theories presented in the foregoing technical field, background art, summary of the invention, or the following detailed description. It should be understood that throughout the drawings, corresponding reference numerals indicate the same or corresponding parts and features. As used herein, the term "module" means any hardware, software, firmware, electronic control components, processing logic, and / or processor device, individually or in any combination, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or grouped) and memories executing one or more software or firmware programs, combinational logic circuits, and / or other suitable components providing said functionality. Although the drawings shown herein depict examples with certain element arrangements, additional intermediate elements, devices, features, or components may be present in actual embodiments. It should also be understood that these drawings are illustrative only and may not be drawn to scale.

[0072] As used herein, the term "vehicle" is not limited to automobiles. While this article primarily describes the technology in conjunction with automobiles, the technology is not limited to automobiles. These ideas can be used in a wide range of applications, such as in conjunction with aircraft, ships, other vehicles, and consumer electronics components.

[0073] refer to Figure 1 The system 10 within the self-vehicle 12 for associating the physical and virtual identifiers of the target vehicle 14 includes a data processor 16, which includes a wireless communication module 18 located within the self-vehicle 12.

[0074] The data processor 16 is a non-general-purpose electronic control device that includes a pre-programmed digital computer or processor, memory or non-transitory computer-readable medium for storing data such as control logic, software applications, instructions, computer code, data, lookup tables, etc., and a transceiver or input / output port. Computer-readable medium includes any type of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard disk drive, compact optical disc (CD), digital video optical disc (DVD), or any other type of memory. "Non-transitory" computer-readable medium does not include wired, wireless, optical, or other communication links that transmit transient electrical or other signals. Non-transitory computer-readable medium includes media that permanently store data and media that can store data and subsequently be rewritten, such as rewritable optical discs or erasable memory devices. Computer code includes any type of program code, including source code, object code, and executable code.

[0075] The data processor 16 includes a wireless communication module 18 adapted to allow wireless communication between the vehicle 12 and other vehicles or other external sources. The data processor 16 is adapted to collect information from a database 22 via a wireless data communication network 20 through a wireless communication channel such as WLAN, 4G / LTE, or 5G networks. Such a database 22 may communicate directly via the Internet or may be a cloud-based database. Information that can be collected by the data processor 16 from such external sources includes, but is not limited to, road and highway databases maintained by the Department of Transportation, GPS, the Internet, other vehicles via V2V communication networks, traffic information sources, vehicle-based support systems such as OnStar, etc.

[0076] The wireless communication module 18 enables bidirectional communication between the data processor 16 of the self-vehicle 12 and other vehicles, mobile devices and infrastructure for the purpose of triggering important communications and events.

[0077] System 10 also includes a plurality of perception sensors 24 located within the autonomous vehicle 12. These perception sensors 24 include sensors adapted to collect data related to the physical identification of the target vehicle 14. Such sensors 24 include, but are not limited to, radar, LiDAR, and cameras, which allow the autonomous vehicle to “see” nearby objects. The plurality of perception sensors 24 transmit the data related to the physical identification of the target vehicle 14 to a data processor 16 via a communication bus 26 within the autonomous vehicle 12.

[0078] The data processor 16 is also adapted to receive data related to the virtual identifier of the target vehicle 14 via the wireless communication channel 20 and associate the physical identifier of the target vehicle 14 with the virtual identifier of the target vehicle 14. The target vehicle 14 includes a plurality of sensing sensors 24' located within the target vehicle 14 and a data processor 16' equipped with a wireless communication module 18'. The plurality of sensing sensors 24' communicate with the data processor 16' via a communication bus 26' within the target vehicle 14.

[0079] The wireless communication module 18' within the target vehicle 14 allows the target vehicle 14 to transmit data related to its virtual identifier to its own vehicle 12 via the wireless communication network 20.

[0080] refer to Figure 2 In an exemplary scenario, the plurality of sensing sensors 24 within the autonomous vehicle 12 detect a first target vehicle 14A and a second target vehicle 14B near the autonomous vehicle 12. The autonomous vehicle 12 also wirelessly receives data associated with a virtual identifier of the first target vehicle 14A, as shown in 26. Such virtual identifier data may include, but is not limited to, information such as IP address, VIN, license plate number, and GPS coordinates. However, both the first and second target vehicles 14A and 14B may have the same model and color, making it difficult for the autonomous vehicle 12 to properly associate the virtual identifier information with the correct one of the first and second target vehicles 14A and 14B. It is important that the autonomous vehicle 12 properly associates the virtual identifier with the correct one of the first and second target vehicles 14A and 14B. For the autonomous vehicle 12 to make effective and safe decisions regarding lane changes, speed adjustments, and other such operations, it is crucial that the autonomous vehicle 12 correctly associates the virtual identifier with the correct physical identifier, namely, the correct one of the first and second target vehicles 14A and 14B. In this way, the autonomous vehicle 12 will ensure that it communicates with the correct one of the first and second target vehicles 14A and 14B. Furthermore, the autonomous vehicle 12 can receive virtual identification data from each of the first and second target vehicles 14A, 14B. Proper association between virtual and physical identification will ensure that the autonomous vehicle 12 knows which virtual data is associated with which of the first and second target vehicles 14A, 14B.

[0081] When associating the physical identifier of target vehicle 14 with its virtual identifier, data processor 16 is also adapted to utilize a Bayesian inference model and estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle 14. In other words, data processor 16 uses a Bayesian inference model to match the data received from target vehicle 14 with the physical observations of its own vehicle 12.

[0082] When using a Bayesian inference model, the data processor 16 constructs a two-dimensional discrete probability distribution table, for example:

[0083] ,

[0084] in, .

[0085] There are m virtual identifiers (V1 … V m ) and n physical identifiers (P1 … P n p i,j It is P j With V i The probability of a match. For each physical identifier, such a state model is created, and multiple such state models for all physical identifiers will form a two-dimensional table.

[0086] Bayes' theorem is given by the following formula:

[0087] Where D represents the data and h represents the hypothesis. The calculation is given below:

[0088] ,and

[0089] ,in

[0090] D represents two sets of sensor observations (physical and virtual);

[0091] h j,i This represents the assumption that physical j matches virtual i;

[0092] It is either sensor data given a hypothesis, or the likelihood probability distribution of two sets of observation data given that hypothesis;

[0093] It is a prior hypothesis, or a hypothetical prior probability distribution (the state definition at t-1). Initially, If ten target vehicles are identified, initially each will have a probability of 10%, and this will then be updated.

[0094] P(D) is the evidence probability of the two sets of sensor observations; and

[0095] It is the posterior hypothesis, or the posterior probability distribution of the hypothesis (the state at time t). The state table (hypothesis) is updated using sensor observation data. As new data arrives, the state table is updated to represent a more accurate probability of a physical identifier matching a virtual identifier.

[0096] The Bayesian inference algorithm is as follows:

[0097] Step 1: Collect sensor data from two sources: local sensing sensors (physical) and wireless communication channel 20 (virtual).

[0098] Step 2: Create or update the two-dimensional status table (if a new identifier is detected, create a new row / column, and delete rows / columns for identifiers that no longer exist). If a new row is created, the columns in that new row will be initialized to... .

[0099] Step 3: Use the state table as the prior probability distribution .

[0100] Step 4: Use sensor data to calculate And P(D).

[0101] Step 5: Update the posterior probability distribution .

[0102] Step 6: It was used to update the two-dimensional state table.

[0103] Step 7: In the state table, find the hypothesis (j,i) with the highest probability as the current output of the algorithm, i.e., the hypothesis with probability p. j,i The physical identifier i.

[0104] Step 8: Return to Step 1.

[0105] In one exemplary embodiment, when associating the physical identifier of the target vehicle 14 with its virtual identifier, the data processor is further adapted to use data associated with the physical identifier of the target vehicle 14 to determine the relative position of the target vehicle 14 and estimate the real-time state of the target vehicle 14. The data associated with the physical identifier of the target vehicle 14 includes global positioning coordinates, velocity, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle 14 also includes global positioning coordinates, velocity, acceleration, yaw rate, and direction of travel.

[0106] In this embodiment, target vehicle 14 transmits only basic safety information, including GPS coordinates, speed, acceleration, yaw rate, and direction of travel. Autonomous vehicle 12 uses the multiple sensing sensors 24 to determine the relative position of one or more target vehicles and estimate their real-time state, i.e., GPS coordinates, speed, acceleration, yaw rate, and direction of travel. Autonomous vehicle 12 receives the basic safety information from one or more target vehicles, and the data processor within autonomous vehicle 12 runs a Bayesian inference algorithm and calculates... And P(D).

[0107] refer to Figure 3 An example is shown where the autonomous vehicle 12 uses multiple perception sensors to detect a first target vehicle 14A and a second target vehicle 14B. For the first target vehicle 14A, the vehicle's position (physical identifier), as shown in P1, is observed by the autonomous vehicle's perception sensor 24 (camera). As shown in V4, the GPS position (virtual identifier) ​​of the first target vehicle 14A is reported by the first target vehicle 14A via a wireless communication channel. In one hypothesis, h... 1,4 P1 and V4 are the same identifier, while the true location of the first target vehicle group is indicated at 14A. In other words, P1 and V4 are the same observations of 14A from both sets of sensors. Sensor fusion can then be used to estimate the probability distribution of the ground reality G1. The G1 distribution can be calculated using a second set of Bayesian inference models.

[0108] refer to Figure 4 The graph shows the probability distributions of P1, V4, and G1, where:

[0109] ,and

[0110] .

[0111] In another exemplary embodiment, the system also includes a cloud-based vehicle profile database 22', such as SIFT, SURF, BRIEF, and ORB, which includes computer vision features created for each model of all vehicles. Such a database 22' resides in the cloud 28 and is accessible via wireless communication channel 20. In one example, the target vehicle 14 uses wireless communication channel 20 or a cellular network to transmit its model information (brand, model, year, color, etc.) to other nearby vehicles.

[0112] The data processor 16 is also adapted to use model information received from the target vehicle 14 and to receive corresponding vehicle profile data from the cloud-based vehicle profile database 22'. The self-vehicle 12 receives this model information and uses it to "look up" the corresponding vehicle profile data for the target vehicle 14 from the cloud-based vehicle profile database 22'. Then, the self-vehicle 12, having a set of physical identifiers from its camera-based perception sensors 24 and a set of identifiers and profiles wirelessly received via the communication channel 20, runs a Bayesian inference algorithm and calculates... And P(D).

[0113] In one hypothesis, h 1,4 P1 (physical identifier) ​​and V4 (virtual identifier) ​​are the same identifier. The feature of P1 can be calculated as FEATURE(P1), and the feature of V4 can be represented by FEATURE(V4). The feature distance can be calculated as: F_Distance(P1,V4) = |FEATURE(P1) – FEATURE(V4)|. The feature distance will follow a specific probability distribution G(distance) 30, which can be created through field measurements, such as... Figure 5 As shown, where:

[0114] ,and

[0115] .

[0116] In an exemplary embodiment, the cloud-based vehicle profile database 22' is a deep neural network (DNN). This DNN is adapted to learn a unique feature vector for each captured image of a vehicle. The feature vector should be robust to various lighting / weather conditions, camera characteristics, viewing angles, etc. This is achieved using a well-balanced training dataset and effective data augmentation methods.

[0117] Given a vehicle image x, a DNN defines a feature extractor F:x i – V i This makes the classifier C and the target category label y compatible. i Together, C * F is trained to minimize a series of loss functions, given below:

[0118] ,

[0119] Where Ω(F) is the regularization term, and α is the weight factor of the regularization term.

[0120] In yet another exemplary embodiment, the data associated with the virtual identity of the target vehicle 14 includes data related to the surrounding environment of the target vehicle 14 collected by the perception sensors 24' on the target vehicle 14, including observed lane lines, surrounding vehicles, vulnerable road users (VRUs), road signs, traffic lights, and structures. The data associated with the virtual identity of the target vehicle 14 may also include computer vision features of the target vehicle 14 retrieved from a cloud-based vehicle profile database 22'.

[0121] Target vehicle 14 uses its onboard perception sensors 24' to observe its surrounding environment. Target vehicle 14 shares the observed environmental information with its own vehicle 12 via communication channel 20. For example, referencing... Figure 6 The target vehicle TV2 can share "one dashed lane line on the left, one dashed lane line on the right, one vehicle on the left (TV1), and one vehicle in front (TV3)". (Reference) Figure 7 The autonomous vehicle 12 can use its own onboard perception sensor 24 to observe the physical identifier P1 of the target vehicle TV1, the physical identifier P2 of the target vehicle V2, and the three lane lines (one solid line on the left and two dashed lines on the right). The autonomous vehicle 12 matches its own observations with the shared perception data of the target vehicle TV2 by using the proposed Bayesian inference model.

[0122] The Bayesian inference model is used as follows. (Reference) Figure 7 From the perspective of the voluntary vehicle 12, the voluntary vehicle 12 observes the physical sign P1 of the target vehicle TV1 and receives observation data from the virtual sign V4 of the target vehicle TV1. The voluntary vehicle 12 further observes the physical sign P2 of the target vehicle TV2 and receives observation data from the virtual sign V7 of the target vehicle TV2. Finally, the voluntary vehicle 12 observes the two dashed lane lines on the right and the one solid lane line on the left.

[0123] Virtual identification data V7 from target vehicle TV2 describes its observation of two dashed lane lines on the right, a target vehicle (TV1) on the left, and a target vehicle (TV3) ahead. Target vehicle TV3 is invisible and therefore not physically observed by its own vehicle 12. An assumption is h 2,7 – P2 and V7 are the same identifier, given by the following formula:

[0124] .

[0125] Regarding lane markings:

[0126] G(P2|h 2,7 The probability is that the car (12) observes a target vehicle with one solid line and one dashed line on its left and one dashed line on its right. For example... Figure 7 As shown in the figure, G(P2|h 2,7 ) = 1 / 1 = 100%.

[0127] G(V7|h 2,7 This represents the probability that the target observes a dashed line on both the left and right sides of the target vehicle. For example... Figure 7 As shown, G(V7|h 2,7 ) = ½ = 50%.

[0128] Therefore, P(D|h2,7) = 100% * 50% = 50%. Similarly, P(D|h2,7) can be calculated for other nearby vehicles. 2,7 ).

[0129] refer to Figure 8 One application of System 10 disclosed herein is cooperative lane changing. Autonomous vehicle 12 wants to change to the right lane. Perception sensors on the autonomous vehicle detect two target vehicles TV2 and TV3 in the right lane, and there is insufficient space between them for autonomous vehicle 12 to fit between them. Assuming that target vehicles TV2 and TV3 are each intelligent vehicles, they can broadcast their identification information to adjacent vehicles via a 5G communication channel. After receiving the virtual identification information of target vehicles TV2 and TV3 from the communication channel, autonomous vehicle 12 can use the proposed method to correctly match the virtual identifications of target vehicles TV2 and TV3 with the physical identifications observed by autonomous vehicle 12. Then, autonomous vehicle 12 can issue a "lane change" request to target vehicle TV2, requesting it to accelerate, and to target vehicle TV3, requesting it to decelerate, thereby increasing the space between target vehicles TV2 and TV3 and allowing autonomous vehicle 12 to safely change lanes by moving between them, as shown by arrow 32. Without a correct physical / virtual identifier match, the ego vehicle may issue incorrect requests, such as asking target vehicle V1 to slow down instead of target vehicle V3.

[0130] refer to Figure 9 Another application of the disclosed system 10 is for the manipulation of infrastructure coordination. In this case, the infrastructure behaves much like a "self-driving vehicle." The infrastructure uses sensing sensors, such as... Figure 9The camera 34 shown detects vehicles 36 and uses the algorithm described above to associate physical identifiers with virtual identifiers. This infrastructure communicates with a cloud-based data processor 22'' via a wireless communication network 20'' such as WLAN, 4G / LTE, or 5G. The cloud-based data processor 22'' calculates optimized handling plans for all vehicles 36 and sends consultation instructions to the identified vehicles 36 via the wireless communication network 20''.

[0131] For example, within the cloud-based data processor 22'', a mini-database of two vehicle profiles is created using SIFT features. When these two vehicles 36 approach an intersection, they share their vehicle model information (brand, model, color) with the cloud-based data processor 22'' (infrastructure) via the wireless communication network 20''. The cloud-based data processor 22'' retrieves vehicle features and finds a correct match between the physical identifier, i.e., the image captured by the infrastructure camera, and the virtual identifier, i.e., the information shared via wireless network communication.

[0132] Furthermore, the system 10 disclosed herein can be used for infrastructure-assisted precise positioning. This infrastructure uses sensing sensors, such as… Figure 9 The camera 34 shown detects vehicles 36 and uses the algorithm described above to associate physical identifiers with virtual identifiers. Since the GPS location of the infrastructure camera 34 can be accurately determined in advance, the infrastructure camera 34 can indirectly infer the precise location of each vehicle 36 based on the camera's perception results. The infrastructure sends the inferred vehicle locations to the virtual identifiers via wireless network communication 20''. Vehicles receiving this information can use this location data to assist in navigation or autonomous driving in urban environments where GPS signals are blocked.

[0133] refer to Figure 10 and Figure 11 Another application of the system disclosed herein is for sharing physical / virtual identifier associations. The ego vehicle 12 receives a virtual identifier V4 of a first target vehicle TV1 and a virtual identifier V7 of a second target vehicle TV2 via a wireless communication network. In this example, the virtual identifier V4 of the first target vehicle TV1 indicates that the first target vehicle TV1 is about to change lanes. The virtual identifier V7 of the second target vehicle TV2 indicates that the second target vehicle TV2 has observed two dashed lane lines, a vehicle V7P1 ahead, and indicates that it is about to change lanes to the left. V7P1 is the physical identifier of the first target vehicle TV1 perceived by the second target vehicle TV2 and is virtually shared with the ego vehicle 12. The virtual identifier V7 of the second target vehicle TV2 further confirms using onboard sensors that V7P1 is V4 (the first target vehicle TV1) and that the first target vehicle TV1 is changing lanes.

[0134] The autonomous vehicle 12 also observes the physical identifier P2 of the second target vehicle TV2 and three lane markings (two dashed lines and one solid line) from its own perception sensors. The physical identifiers of the first target vehicle TV1 and the third target vehicle TV3 are hidden from the autonomous vehicle 12. One assumption is that h2,7 --- P2 and V7 are the same identifier (target vehicle TV2).

[0135] .

[0136] Then, the autonomous vehicle 12 uses the virtual identifier V7 of the second target vehicle TV2 to confirm that V7P1 is V4 (the first target vehicle TV1), and determines that the first target vehicle TV1 is about to cut in front of it and decelerate. This can be extended to other scenarios where a direct virtual-physical identifier cannot be associated (i.e., there is no line of sight) but can be accomplished indirectly.

[0137] refer to Figure 12 The method 100 for associating the physical and virtual identifiers of a target vehicle 14 includes, starting at block 102, collecting data associated with the physical identifier of the target vehicle 14 using a plurality of sensing sensors 24, and transmitting the data associated with the physical identifier of the target vehicle 14 to a data processor 16 via a communication bus.

[0138] Moving to block 104, the method further includes using data processor 16 to collect data related to the virtual identifier of target vehicle 14 via wireless communication channel 20, and moving to block 106, using data processor 16 to associate the physical identifier of target vehicle 14 with the virtual identifier of target vehicle 14. In an exemplary embodiment, associating the physical identifier of target vehicle 14 with the virtual identifier of target vehicle 14 using data processor 16 further includes using a Bayesian Interference Model and using data processor 16 to estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle 14.

[0139] In one exemplary embodiment, moving from block 104 to block 108 and associating the physical identifier of the target vehicle 14 with the virtual identifier of the target vehicle 14 using the data processor 16 further includes: using data associated with the physical identifier of the target vehicle 14 to determine the relative position of the target vehicle 14 using the data processor 16, and using the data processor 16 to estimate the real-time state of the target vehicle 14.

[0140] Data associated with the physical identifier of the target vehicle 14 includes global satellite positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and data associated with the virtual identifier of the target vehicle 14 includes global satellite positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

[0141] In another exemplary embodiment, moving from box 104 to box 110, computer vision features created for each model of all vehicles are stored in a cloud-based vehicle profile database 22', and data associated with the virtual identifier of the target vehicle 14 includes model information sent by the target vehicle 14. Method 100 includes using the model information received from the target vehicle 14 and utilizing data processor 16 to receive corresponding vehicle profile data from the cloud-based vehicle profile database 22'. The model information transmitted by the target vehicle 14 includes, but is not limited to, brand, model, year, and color. In another exemplary embodiment, the cloud-based vehicle database 22' is a deep neural network.

[0142] In yet another exemplary embodiment, moving from box 104 to box 112, the data associated with the virtual identity of the target vehicle 14 includes data related to the surrounding environment of the target vehicle 14 collected by the perception sensors 24' on the target vehicle 14. The data associated with the virtual identity of the target vehicle 14 may include, but is not limited to, observed lane lines, surrounding vehicles, vulnerable road users (VRUs), street signs, traffic lights, and structures, and in some exemplary embodiments, the data associated with the virtual identity of the target vehicle 14 also includes computer vision features of the target vehicle 14 stored in a cloud-based vehicle profile database 22'.

[0143] The systems and methods disclosed herein offer several advantages. These advantages include allowing a self-vehicle to correctly associate the physical identifier of a target vehicle detected by sensing sensors within the self-vehicle with the virtual identifier of the target vehicle received via wireless communication between the self-vehicle and the target vehicle. This ensures that the self-vehicle knows which vehicles it may be communicating with and that it knows the correct location of nearby target vehicles. This allows the self-vehicle to operate appropriately and safely on roads and highways, performing tasks such as cooperative lane changing, infrastructure-coordinated maneuvering, infrastructure-assisted precise positioning, and sharing physical / virtual association information with nearby vehicles.

[0144] The description in this disclosure is exemplary in nature only, and variations thereof that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A method for associating the physical identifier and virtual identifier of a target vehicle, comprising: Data related to the physical identifier of the target vehicle is collected using multiple sensing sensors, and the data related to the physical identifier of the target vehicle is transmitted to the data processor via a communication bus. The data processor collects data related to the virtual identifier of the target vehicle via a wireless communication channel; as well as The data processor is used to associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle. This associating the physical identifier of the target vehicle with the virtual identifier further includes: using the data processor to employ a Bayesian inference model, and using the data processor to estimate the probability that data associated with the physical identifier and data associated with the virtual identifier belong to the same target vehicle. The computer vision features created for each model of all vehicles are stored in a cloud-based vehicle profile database, and the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle. The method includes using the model information received from the target vehicle and utilizing the data processor to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

2. The method according to claim 1, wherein, Associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle using the data processor further includes: using the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle using the data processor, and using the data processor to estimate the real-time state of the target vehicle.

3. The method according to claim 2, wherein, The data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

4. The method according to claim 1, wherein, The model information sent by the target vehicle includes the brand, model, year, and color.

5. The method according to claim 4, wherein, The cloud-based vehicle configuration file database is a deep neural network.

6. The method according to claim 1, wherein, The data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle.

7. The method according to claim 6, wherein, The data associated with the virtual identifier of the target vehicle includes observed lane lines, surrounding vehicles, vulnerable road users, road signs, traffic lights, and structures.

8. The method according to claim 7, wherein, The data associated with the virtual identifier of the target vehicle also includes computer vision features of the target vehicle stored in a cloud-based vehicle profile database.

9. A system for associating physical and virtual identifiers of a target vehicle, comprising: A data processor located within the vehicle, the data processor including a wireless communication module; as well as Multiple sensing sensors, located within the autonomous vehicle and adapted to collect data related to the physical identifier of the target vehicle, and transmit the data related to the physical identifier of the target vehicle to the data processor via a communication bus; The data processor is adapted to receive data related to the virtual identifier of the target vehicle via a wireless communication channel, and associate the physical identifier of the target vehicle with the virtual identifier of the target vehicle. Wherein, when associating the physical identifier of the target vehicle with the virtual identifier of the target vehicle, the data processor is further adapted to utilize a Bayesian inference model and estimate the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle, and It also includes a cloud-based vehicle profile database, which includes computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle. The data processor is also adapted to use the model information received from the target vehicle and to receive corresponding vehicle profile data from the cloud-based vehicle profile database.

10. The system according to claim 9, wherein, When the physical identifier of the target vehicle is associated with the virtual identifier of the target vehicle, the data processor is also adapted to use the data associated with the physical identifier of the target vehicle to determine the relative position of the target vehicle and estimate the real-time state of the target vehicle.

11. The system according to claim 10, wherein, The data associated with the physical identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, and the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel.

12. The system according to claim 9, wherein, The model information sent by the target vehicle includes the brand, model, year, and color.

13. The system according to claim 12, wherein, The cloud-based vehicle configuration file database is a deep neural network.

14. The system according to claim 9, wherein, The data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle, including observed lane lines, surrounding vehicles, vulnerable road users, road signs, traffic lights, and structures.

15. The system of claim 14, further comprising a cloud-based vehicle profile database, the cloud-based vehicle profile database including computer vision features created for each model of all vehicles, and the data associated with the virtual identifier of the target vehicle also including the computer vision features of the target vehicle.

16. A method for associating the physical identifier and virtual identifier of a target vehicle, comprising: Data related to the physical identifier of the target vehicle is collected using multiple sensing sensors, and the data related to the physical identifier of the target vehicle is transmitted to the data processor via a communication bus. The data processor collects data related to the virtual identifier of the target vehicle via a wireless communication channel; as well as The data processor associates the physical identifier of the target vehicle with the virtual identifier of the target vehicle by using a Bayesian inference model and by estimating the probability that the data associated with the physical identifier and the data associated with the virtual identifier are for the same target vehicle, and one of the following: Using the data associated with the physical identifier of the target vehicle, including global positioning coordinates, speed, acceleration, yaw rate, and direction of travel, the relative position of the target vehicle is determined using the data processor, and the real-time state of the target vehicle is estimated using the data processor, wherein the data associated with the virtual identifier of the target vehicle includes global positioning coordinates, speed, acceleration, yaw rate, and direction of travel. Using the model information received from the target vehicle, and utilizing the data processor to receive corresponding vehicle profile data from a cloud-based vehicle profile database, the cloud-based vehicle profile database being a deep neural network and including computer vision features created for each model of all vehicles, wherein the data associated with the virtual identifier of the target vehicle includes model information sent by the target vehicle, including brand, model, year, and color; and The data processor associates the physical identifier of the target vehicle with the virtual identifier of the target vehicle, wherein the data associated with the virtual identifier of the target vehicle includes data related to the surrounding environment of the target vehicle collected by perception sensors on the target vehicle, including observed lane lines, surrounding vehicles, vulnerable road users, road signs, traffic lights and structures, as well as computer vision features of the target vehicle stored in a cloud-based vehicle profile database.

Images