Vehicle-to-Everything (V2X) Fraud Detection Using Local Dynamic Map Data Models
Patent Information
- Application Number
- JP2023542759
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-09-23
- Filing Date
- 2021-11-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-11-29
AI Technical Summary
Existing V2X systems face challenges in detecting fraudulent or inaccurate data transmitted by vehicles, which can lead to safety risks and inefficiencies in traffic management.
A fraud management system within V2X equipment uses a local dynamic map (LDM) data model to compare received messages with locally maintained data, identifying inconsistencies to detect fraud conditions and generate reports to a management agency.
Enhances the reliability of V2X communication by accurately identifying and mitigating fraudulent data, improving traffic safety and efficiency by ensuring accurate vehicle positioning and collision avoidance.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] Related Applications This application claims the benefit of priority to U.S. Provisional Application No. 63 / 138,909, filed January 19, 2021, entitled “Vehicle-to-Everything (V2X) Misbehavior Detection Using an LDM Data Model,” the entire contents of which are incorporated herein by reference for all purposes. [Background technology]
[0002] Cellular Vehicle-to-Everything (C-V2X) protocols serve as the foundation for vehicle-based wireless communications and can be used to support intelligent highways, autonomous vehicles, and semi-autonomous vehicles, as well as to improve the overall efficiency and safety of highway transportation systems. C-V2X specifies two transmission modes that together provide 360° non-line-of-sight awareness and higher levels of predictability for enhanced road safety and autonomous driving. The first transmission mode includes Direct C-V2X, which includes vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P), and provides extended communication range and reliability within a dedicated Intelligent Transport System (ITS) 5.9 gigahertz (GHz) spectrum that is independent of cellular networks. The second transmission mode includes vehicle-to-network (V2N) communications in mobile broadband systems and technologies, such as third generation wireless mobile communication technologies (3G) (e.g., Global System for Mobile Communications (GSM) Evolution (EDGE) systems, Code Division Multiple Access (CDMA) 2000 systems, etc.), fourth generation wireless mobile communication technologies (4G) (e.g., Long Term Evolution (LTE) systems, LTE-Advanced systems, Mobile Worldwide Interoperability for Microwave Access (Mobile WiMAX) systems, etc.), fifth generation wireless mobile communication technologies (5G NR systems, etc.). Other V2X wireless technologies are also under consideration in different regions of the world. The techniques described in this patent are applicable to any V2X wireless technology.
[0003] Several regions of the world are developing standards for vehicle-based communication systems and functions, such as the Institute of Electrical and Electronics Engineers (IEEE) standard 1609 and the Society of Automotive Engineers (SAE) standards developed for use in North America, or the European Telecommunications Standards Institute (ETSI) and the European Committee for Standardization (CEN) standards developed for use in Europe. Part of the system is the ability for vehicles to broadcast Basic Safety Messages (BSMs) in North America, or Cooperative Awareness Messages (CAMs) in Europe, that can be received and processed by other vehicles to improve road safety. The processing of such messages in the transmitting and receiving vehicles is performed in the on-board equipment that provides the V2X function (referred to herein as "V2X on-board equipment"). Summary of the Invention [Means for solving the problem]
[0004] Various aspects include a method performed by a misbehavior management system running on a V2X equipment processor for detecting misbehavior conditions in a received V2X message by comparing data in the received V2X message with data contained in a locally maintained or stored local dynamic map (LDM) data model. The LDM aggregates and synthesizes information received by a V2X system participant from all relevant inputs (including but not limited to V2X messages and local sensor inputs) to create a model of the local environment surrounding the V2X system participant. The LDM may be updated based on observed dynamics of objects tracked in the LDM as well as based on new inputs.
[0005] Various aspects may include receiving a V2X message from another V2X system participant, the V2X message including data regarding an environment surrounding the vehicle; comparing the data included in the received V2X message with data in a locally maintained or stored local dynamic map (LDM) data model to detect a misbehavior condition; generating a misbehavior report identifying the misbehavior condition based on the comparison and in response to detecting the misbehavior condition; and transmitting the generated misbehavior report to a misbehavior management authority.
[0006] Some aspects may include monitoring a number of sensors in the vehicle to collect additional data regarding an environment surrounding the vehicle, generating an LDM data model representing the environment surrounding the vehicle based at least in part on an aggregation of the additional data collected from the number of sensors, and maintaining or storing the LDM data model in a local memory.
[0007] Some aspects may further include, in response to determining that a tampering condition is not detected, performing a calculation based on at least one of observed dynamics of the object in the LDM data model or new data input received from the V2X message, modifying the LDM data model to incorporate the calculation and the data included in the received V2X message, and replacing the LDM data model maintained or stored in memory with the modified LDM model.
[0008] In some aspects, transmitting the generated fraud report to a fraud management authority may include transmitting a representation of the LDM data model.
[0009] In some aspects, the representation of the LDM data model may include an incomplete data set for the LDM data model.
[0010] Some aspects may further include receiving feedback from a fraud management agency, the feedback including corrective actions to mitigate the fraud situation.
[0011] In some aspects, the data regarding the environment surrounding the vehicle included in the received V2X message may include traffic information. In some aspects, the data regarding the environment surrounding the vehicle included in the received V2X message includes location information of adjacent vehicles based on Global Navigation Satellite System (GNSS) (e.g., Global Positioning System (GPS)) data. In some aspects, the data regarding the environment surrounding the vehicle included in the received V2X message includes map data specifying road geometry and street furniture.
[0012] In some aspects, comparing the data included in the received V2X message to the locally maintained or stored LDM data model to detect a tampering condition may include determining whether any data included in the received V2X message is inconsistent with information in the locally maintained or stored LDM data model.
[0013] In some aspects, comparing the data included in the received V2X message to the locally maintained or stored LDM data model to detect a misbehavior condition may include selecting a subset of data elements in the locally maintained or stored LDM data model for comparison with the data included in the received V2X message and determining whether any data included in the received V2X message is inconsistent with the selected subset of data elements in the locally maintained or stored LDM data model.
[0014] In some aspects, comparing the data included in the received V2X message to a locally maintained or stored LDM data model to detect a misbehavior condition may include determining whether status or location information of an adjacent vehicle that transmitted the received V2X message is inconsistent with status or location information of the adjacent vehicle in the locally maintained or stored LDM data model.
[0015] Further aspects may include a V2X device having a processor configured to perform one or more operations of the methods summarized above. Further aspects may include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the V2X device to perform operations of the methods summarized above. Further aspects include a V2X device having means for performing the functions of the methods summarized above.
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description given below, serve to explain the features of the claims. [Brief description of the drawings]
[0017] [Figure 1A] FIG. 1 is a component block diagram illustrating a vehicle suitable for implementing various embodiments. [Figure 1B] FIG. 1 is a component block diagram illustrating a vehicle suitable for implementing various embodiments. [Figure 1C] FIG. 1 is a component block diagram illustrating components of a vehicle suitable for implementing various embodiments. [Figure 1D] FIG. 1 is a schematic block diagram illustrating a subset of a V2X communication system suitable for implementing various embodiments. [Figure 2A] FIG. 1 is a component block diagram illustrating components of an exemplary vehicle management system, in accordance with various embodiments. [Figure 2B]1 is a component block diagram illustrating components of another exemplary vehicle management system, in accordance with various embodiments. [Diagram 3] FIG. 1 is a block diagram illustrating components of a system-on-chip for use in a vehicle, in accordance with various embodiments. [Figure 4] FIG. 1 is a component block diagram illustrating a system configured to generate local dynamic map data, according to various embodiments. [Diagram 5] FIG. 1 is a process flow diagram illustrating the operation of a method performed by a processor of a V2X device for detecting a misbehavior condition in a V2X message by comparing data in a received V2X message with data in an LDM data model, according to various embodiments. [Figure 6] FIG. 1 is a process flow diagram illustrating the operation of a method for comparing data in a received V2X message with data in an LDM data model, according to various embodiments. [Figure 7] FIG. 1 is a component block diagram illustrating an exemplary mobile computing device suitable for use with the various embodiments. [Figure 8] FIG. 1 is a component block diagram illustrating an exemplary mobile computing device suitable for use with the various embodiments. [Figure 9] FIG. 1 is a component block diagram illustrating an exemplary server suitable for use with the various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] Various embodiments are described in detail with reference to the accompanying drawings. Whenever possible, the same reference numbers are used throughout the drawings to refer to the same or like parts. References made to specific examples and implementations are for illustrative purposes only and do not limit the scope of the claims.
[0019] In V2X communications, it is important that inaccurate, corrupted, or hacked (i.e., bad) data is detected to prevent such inaccurate data from spreading further. If V2X equipment is sending inaccurate, corrupted, or hacked (i.e., bad) data, the consequences may be merely minor inconvenience and traffic jams, but may also be life-threatening. It is therefore desirable to subject the detection of misbehavior conditions to rigorous analysis against a comprehensive set of information to ensure that any such misbehavior conditions are reliably detected.
[0020] The term "mobile device" is used herein to refer to any one or all of wireless router devices, wireless appliances, cellular telephones, smartphones, portable computing devices, personal or mobile multimedia players, laptop computers, tablet computers, smartbooks, ultrabooks, palmtop computers, wireless email receivers, multimedia Internet-enabled cellular telephones, medical devices and equipment, biometric sensors / devices, wearable devices including smart watches, smart clothing, smart glasses, smart wristbands, and smart jewelry (e.g., smart rings, smart bracelets, etc.), entertainment devices (e.g., wireless game controllers, music and video players, satellite radio, etc.), smart meters / sensors, industrial manufacturing equipment, and large and small machines and appliances for home or business use, wireless communication elements in autonomous and semi-autonomous vehicles, mobile devices fixed or embedded in various mobile platforms, global positioning system devices, and similar electronic devices including memory, wireless communication components, and programmable processors.
[0021] The term "system on chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that contains multiple resources and / or processors integrated on a single substrate. A single SOC may include circuitry for digital, analog, mixed signal, and radio frequency functions. A single SOC may also include any number of general purpose and / or special purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, Flash, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.
[0022] The term "system in package" (SIP) may be used herein to refer to a single module or package containing multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged in a unifying substrate. A SIP may also include multiple independent SOCs packaged in close proximity coupled to each other via high-speed communication circuitry, such as on a single motherboard or within a single mobile device. The proximity of the SOCs facilitates high-speed communication and sharing of memory and resources.
[0023] Terms such as "component," "system," "unit," "module," and the like, as used in this application, include computer-related entities, such as, but not limited to, hardware, firmware, a combination of hardware and software, software, or software in execution, configured to perform certain operations or functions. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of example, both an application running on a communication device and the communication device may be referred to as a component. One or more components may reside within a process and / or thread of execution, and a component may be localized on one processor or core and / or distributed among two or more processors or cores. In addition, these components may execute from various non-transitory computer-readable media having various instructions and / or data structures stored thereon. Components may communicate by way of local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, and other computer, processor, and / or process related communication methods known.
[0024] Generally, various embodiments include methods and mechanisms for detecting a tampering condition by a V2X system participant by comparing a received V2X message to a local dynamic map (LDM) data model of the vehicle and determining whether an inconsistency exists between the data received in the V2X message and a locally maintained or stored LDM data model.
[0025] V2X systems and technologies hold great promise for improving traffic flow and vehicle safety by allowing vehicles to share information about their location, speed, heading, braking, and other factors that may be useful to other vehicles for collision avoidance and other safety functions. Vehicles equipped with V2X / V2V onboard equipment will transmit their vehicle information frequently (e.g., up to 20 times per second) in packets called Basic Safety Messages (BSMs) or CAMs. If all V2X-equipped vehicles transmit such BSM / CAM messages, every receiving vehicle will have the information needed to control its own speed and direction to avoid collisions and efficiently and safely position vehicles relative to each other. It is envisioned that V2X-equipped vehicles may be able to improve traffic flow by safely reducing separation distances, platooning several vehicles together, and avoiding vehicles experiencing malfunctions.
[0026] For ease of reference, some of the embodiments are described in this application using a misconduct management system operating within V2X terminology. However, it should be understood that various embodiments encompass any or all of V2X / V2V or vehicle-based communication standards, messages, or technologies. Thus, nothing in this application should be construed to limit the claims to V2X / V2V systems unless expressly recited as such in the claims. Additionally, the embodiments described herein describe on-board equipment for performing V2X / V2V communications. In a V2X / V2V system, the system participant equipment may include, but is not limited to, vehicle on-board equipment, mobile devices, and roadside units (RSUs). RSUs may include fixed devices such as traffic signals, roadside beacons, traffic cameras, etc. Each of the system participant equipment may broadcast information to other system participant equipment. V2X communications between system participants may enable applications running on each system participant to provide safety applications (e.g., applications that may determine an imminent hazard, such as a vehicle braking suddenly or emerging from a blind crossroad at high speed) or mobility (planning for traffic signal changes) to vehicles, or other useful functions within the vehicular traffic system as a whole.
[0027] A local dynamic map (LDM) is a data model typically constructed by a mobile device to support navigation within its environment. A mobile device obtains information about its environment from one or more sensors and may receive other LDM data from other mobile devices (e.g., via a V2X communication system) or from network elements such as cloud-based servers, and uses such data to construct its LDM. The LDM may be a dynamic data model that evolves over time, even when new data is not received that updates the location of other V2X system participants, via dead reckoning, rather than by messages from those other V2X system participants. This LDM data model may aggregate and synthesize information received by a V2X system participant from all relevant inputs (including, but not limited to, V2X messages and sensor inputs) to create a model of the local environment around the V2X system participant. The LDM may be updated based on observed dynamics of objects tracked within the LDM, as well as updated based on new inputs.
[0028] A misconduct management system running on a V2X participant's equipment may build an LDM by aggregating information obtained from one or more sensors (e.g., camera, radar, LIDAR, etc.) of the host vehicle, from one or more other mobile devices or vehicles received via V2X messages, and / or from remote data sources, such as via roadside units, and network elements, such as cloud-based servers. The vehicle V2X system may process this information to generate and update locally maintained or stored LDM data in a usable or presentable form, such as a digital map. Portions of the LDM map may also be received from external sources, such as computing devices capable of performing intensive processing operations. Such an LDM data model may include many types of information, which may be structured or organized in several layers or data elements. For example, the LDM data model may include a physical map of roads, such as one downloaded from a map database, a data layer of observed road conditions (e.g., rough or smooth, wet, dry, or icy, etc.), a data layer of observed other vehicle positions and speeds, a data layer of network-reported road modifications (e.g., construction, closed lanes), a data layer regarding traffic signals in the vicinity (e.g., the duration of the signal cycle at the traffic light ahead of the vehicle), and other information that is useful for autonomous driving, collision avoidance, and general safety functions (e.g., driver alerts).
[0029] Information used by fraud management systems is typically limited to data that may be maintained or stored in memory (e.g., static maps) and data from on-board sensors. LDM data received from on-board sensors and from other mobile devices may be limited by the sensitivity, field of view, and perceptual limitations of each sensor. LDM data received from distant network elements typically does not include very recent changes in the environment close to the mobile device's vehicle and therefore may not reflect highly dynamic environmental conditions (e.g., road closures, construction, accidents, etc.). By combining all sources of information about the road ahead of the vehicle as well as other vehicles in the vicinity, vehicle systems (e.g., fraud management systems) can generate more comprehensive LDM data models that are useful for complex processes such as autonomous driving and semi-autonomous driver assistance functions.
[0030] LDM data models can be structured in various types that reflect the degree to which such information can change dynamically. For example, LDM data can be classified (e.g., in the relevant ETSI standards): Type 1 for persistent static information such as road locations and geographic features, which may be considered map data; Type 2 for transient static information, which may include signals not included in map data such as speed limits; Type 3 for transient dynamic information such as weather and traffic congestion and other traffic condition information; and Type 4 for highly dynamic information such as automobile sensor data, other vehicles in motion, pedestrians, locations of parked vehicles, traffic signal status, and other highly transient conditions. Examples of LDM implementations have been reported, including PG-LDM by Bosch® and Tele Atlas®, and NAVTEQ-LDM by NAVTEQ®. The PG-LDM implementation employs PostgreSQL as its database engine and provides PostGIS stored procedures and spatial operations. On the other hand, the NAVTEQ-LDM implementation employs SQLite as its database engine.
[0031] In various embodiments, the misconduct management system running on the V2X equipment processor may receive initial LDM data from one or more data sources other than the V2X system participants, which may include neighboring vehicles, mobile devices and RSUs, data sources that may transmit CAM messages or Decentralized Environmental Notification Message (DENM) messages, as well as various internet or cloud-based resources. In some embodiments, the received initial LDM data may be Type 4 information, i.e., “highly dynamic” information reflecting highly transient conditions. In some embodiments, the received LDM data may be obtained from a sensor or another information source within a threshold amount of time, such as 2 seconds, 1 second, 250 milliseconds, or another suitable threshold or time window. In some embodiments, the initial LDM data may include data collected by multiple sensors onboard the vehicle and mobile device. Such sensor data may include data such as speed, temperature, revolutions per minute, GPS location, image data, audio data, or vehicle / device operating condition data. The misbehavior management system may aggregate all of the collected sensor data along with the received V2X message data to generate an LDM data model that represents the environment surrounding the V2X participants. For ease of reference, various embodiments are described with reference to the environment surrounding the V2X participants as the environment surrounding the vehicle, although the V2X participants may be other devices external to the vehicle, such as RSUs and other fixed devices.
[0032] The LDM data model thus generated may be useful for assessing the accuracy or precision of information contained in a received V2X message. The LDM data model may be composed of data received from various information sources and may therefore include one or more data elements related to information provided in the V2X message. However, not all information contained in the LDM data model may be related to information in the V2X message that is to be verified or confirmed.
[0033] Various embodiments may be implemented in a variety of vehicles, an exemplary vehicle 101 of which is shown in FIG. 1A and FIG. 1B. With reference to FIG. 1A and FIG. 1B, the vehicle 101 may include a control unit 140 and a number of sensors 144-170, including a satellite geographic positioning system receiver 142, occupancy sensors 144, 146, 148, 150, 152, tire pressure sensors 154, 156, cameras 158, 160, microphones 162, 164, impact sensors 166, radar 168, and lidar 170. The multiple sensors 144-170 disposed in or on the vehicle may be used for a variety of purposes, such as autonomous and semi-autonomous navigation and control, crash avoidance, position determination, and to provide sensor data regarding objects and people in or on the vehicle 101. The sensors 144-170 may include one or more of a wide variety of sensors capable of detecting a variety of information useful for navigation and collision avoidance. Each of the sensors 144-170 may be in wired or wireless communication with the control unit 140 and with each other. In particular, the sensors may include one or more cameras 158, 160, or other light or photo-optic sensors. The sensors may further include other types of object detection and ranging sensors, such as radar 168, lidar 170, IR sensors, and ultrasonic sensors. The sensors may further include tire pressure sensors 154, 156, humidity sensors, temperature sensors, satellite geopositioning sensors 142, control input sensors 145, accelerometers, vibration sensors, gyroscopes, gravity meters, shock sensors 166, strength meters, stress meters, strain sensors, fluid sensors, chemical sensors, gas content analyzers, pH sensors, radiation sensors, Geiger counters, neutron detectors, biological material sensors, microphones 162, 164, occupancy sensors 144, 146, 148, 150, 152, proximity sensors, and other sensors.
[0034] Vehicle control unit 140 may be configured with processor-executable instructions for performing navigation and collision avoidance operations using information received from various sensors, particularly cameras 158, 160. In some embodiments, control unit 140 may supplement the processing of camera images using distance and relative position (e.g., relative azimuth), which may be obtained from radar 168 and / or lidar 170 sensors. Control unit 140 may be further configured to control steering, braking, and speed of vehicle 101 when operating in autonomous or semi-autonomous modes using information about other vehicles determined using various embodiments.
[0035] FIG. 1C is a component block diagram illustrating a communication system 100 of components and support systems suitable for implementing various embodiments. With reference to FIGS. 1A-1C, the vehicle 101 may include a control unit 140, which may include various circuits and devices used to control the operation of the vehicle 101. In the example shown in FIG. 1C, the control unit 140 includes a processor 140a, a memory 140b, an input module 140c, an output module 140d, and a wireless module 140e. The control unit 140 may be coupled to and configured to control the driving control components 172a, the navigation components 172b, and one or more sensors 172c of the vehicle 101. The processor 140a may be configured with processor-executable instructions for controlling the steering, navigation, and / or other operations of the vehicle 101, including the operation of various embodiments. The processor 140a may be coupled to the memory 140b.
[0036] The radio module 140e may be configured for wireless communication. The radio module 140e may exchange signals (e.g., command signals for controlling steering, signals from navigation mechanisms, etc.) with a network transceiver (e.g., base station 110) via communication link 122 and may provide signals to the processor 140a and / or navigation unit 172b. In some embodiments, the radio module 140e may enable the vehicle 101 to communicate with the wireless communication device 120 through wireless communication link 124. The wireless communication link 124 may be a bidirectional or unidirectional communication link and may use one or more communication protocols as described.
[0037] The input module 140c may receive sensor data from one or more vehicle sensors 172c, as well as electronic signals from other components, including the driving control component 172a and the navigation component 172b. The output module 140d may communicate with or activate various components of the vehicle 101, including the driving control component 172a, the navigation component 172b, and the sensors 172c.
[0038] Control unit 140 may be coupled to driving control components 172a to control physical elements of vehicle 101 related to the steering and navigation of the vehicle, such as the engine, motor, throttle, steering elements, piloting elements, braking or deceleration elements, etc. Driving control components 172a may also include components to control other devices of the vehicle, including environmental controls (e.g., air conditioning and heating), exterior and / or interior lighting, interior and / or exterior information displays (which may include display screens or other devices for displaying information), safety devices (e.g., haptic devices, audible alarms, etc.), and other similar devices.
[0039] The control unit 140 may be coupled to the navigation component 172b, may receive data from the navigation component 172b, and may be configured to use such data to determine the current position and orientation of the vehicle 101 and an appropriate course towards a destination. The navigation component 172b may include or be coupled to a GNSS receiver system (e.g., one or more GPS receivers) that enables the vehicle 101 to determine its current position using GNSS signals. Alternatively, or in addition, the navigation component 172b may include a wireless navigation receiver for receiving navigation beacons or other signals from wireless nodes, such as Wi-Fi access points, cellular network sites, radio stations, remote computing devices, other vehicles, etc. Through control of the driving control element 172a, the processor 140a may control the vehicle 101 to navigate and steer. The processor 140a and / or navigation component 172b may be configured to communicate with network elements, such as servers, within a communications network (e.g., core network 132) via wireless communications links 122, 126 to receive commands for controlling navigation, receive data useful in navigation, provide real-time position reports, and assess other data.
[0040] The control unit 140 may be coupled to one or more sensors 172c, which may include sensors 144-170 as described, and may be configured to provide various data to the processor 140a.
[0041] Although control unit 140 is described as including separate components, in some embodiments some or all of the components (e.g., processor 140a, memory 140b, input module 140c, output module 140d, and wireless module 140e) may be integrated within a single device or module, such as a system-on-chip (SOC) processing device. Such a SOC processing device may be configured for use in a vehicle and may be configured with processor-executable instructions, etc., executing within processor 140a to perform navigation and collision avoidance operations using the LDM data when installed within the vehicle.
[0042] 1D illustrates a portion of a V2X system 103 that includes three vehicles 12, 14, 16. In the illustrated example, each vehicle 12, 14, 16 includes V2X onboard equipment 102, 104, 106 that is configured to periodically broadcast basic safety messages 30, 40, 50 for receipt and processing by the onboard equipment (e.g., 102, 104, 106) of the other vehicles, respectively. By sharing vehicle location, speed, direction, braking, and other information, the vehicles can maintain a safe separation and identify and avoid potential collisions. For example, a trailing vehicle 12 that receives a basic safety message 40 from a leading vehicle 16 can determine the speed and location of the vehicle 16, which in turn enables the vehicle 12 to match speeds and maintain a safe separation distance 20. By being notified through the basic safety message 40 when the leading vehicle 16 brakes, the V2X equipment 102 in the following vehicle 12 can simultaneously brake to maintain a safe separation distance 20 even when the leading vehicle 16 suddenly stops. As another example, the V2X equipment 104 in the truck vehicle 14 can receive the basic safety messages 30, 50 from the two vehicles 12, 16 and thus be notified that the truck vehicle 14 should stop at an intersection to avoid a collision. Each of the vehicle V2X on-board equipment 102, 104, 106 can communicate with each other using any of a variety of close proximity communication protocols. In addition, the vehicles may be capable of transmitting data and information regarding the detected basic safety messages as well as the detected misconduct reports to the original equipment manufacturer (OEM) (70, 72) and / or a remote misconduct management agency 74 via the communication links 60, 62 through the communication network 18 (e.g., cellular, WiFi, etc.). The MBR may be sent directly to the fraud management authority 74 (e.g., over communications links 64, 66). In other embodiments, the MBR may first be sent over communications links 64, 66 to an MBR pre-processing unit, such as an OEM server 70, 72, for pre-processing. The pre-processed MBR may then be sent from the MBR pre-processing server 70, 72 over communications links 64, 66 to the fraud management authority 74.
[0043] FIG. 2A is a component block diagram illustrating components of an exemplary fraud management system 200. The vehicle management system 200 may include various subsystems, communication elements, computational elements, computing devices, or computing units that may be utilized within the vehicle 101. With reference to FIGS. 1A-2A, the various computational elements, computing devices, or computing units within the fraud management system 200 may be implemented within a system (i.e., subsystems) of interconnected computing devices that communicate data and commands with each other (e.g., as indicated by the arrows in FIG. 2A). In some implementations, the various computational elements, computing devices, or computing units within the fraud management system 200 may be implemented within a single computing device, such as separate threads, processes, algorithms, or computational elements. Thus, each subsystem / computational element illustrated in FIG. 2A is also generally referred to herein as a "layer" within the computational "stack" that makes up the fraud management system 200. However, the use of the terms layer and stack in describing various embodiments does not imply or require that the corresponding functions are implemented within a single autonomous (or semi-autonomous) vehicle management system computing device, although this is a possible implementation embodiment. Rather, use of the term "layer" is intended to encompass subsystems having independent processors, computational elements (e.g., threads, algorithms, subroutines, etc.) that execute within one or more computing devices and combinations of subsystems and computational elements.
[0044] The fraud management system stack may include a radar perception layer 202, a camera perception layer 204, a positioning engine layer 206, a map fusion and reconciliation layer 208, a route planning layer 210, a sensor fusion and road world model (RWM) management layer 212, a trip planning and control layer 214, and a behavior planning and prediction layer 216. Layers 202-216 are only examples of some layers in one exemplary configuration of the fraud management system stack 200. In other configurations, other layers may be included, such as additional layers for other perception sensors (e.g., a LIDAR perception layer, etc.), additional layers for planning and / or control, additional layers for modeling, and / or some of the layers 202-216 may be excluded from the fraud management system stack 200. Each of the layers 202-216 may exchange data, computations, and commands, as illustrated by the arrows in FIG. 2A. Additionally, the fraud management system stack 200 may receive and process data from sensors (e.g., radar, lidar, cameras, inertial measurement units (IMUs), etc.), navigation systems (e.g., GPS receivers, IMUs, etc.), vehicle networks (e.g., controller area network (CAN) bus), and databases in memory (e.g., digital map data). The fraud management system stack 200 may output vehicle control commands or signals to a drive by wire (DBW) system / control unit 220, which is a system, subsystem, or computing device that interfaces directly with the vehicle steering, throttle, and braking controls. The configuration of the fraud management system stack 200 and DBW system / control unit 220 shown in FIG. 2A is merely an example configuration and other configurations of the vehicle management system and other vehicle components may be used.As an example, the configuration of fraud management system stack 200 and DBW system / control unit 220 shown in FIG. 2A may be used in a vehicle configured for autonomous or semi-autonomous operation, although a different configuration may be used in a non-autonomous vehicle.
[0045] The radar perception layer 202 may receive data from one or more detection and ranging sensors, such as radar (e.g., 168) and / or lidar (e.g., 170), and may process that data to recognize and determine the location of other vehicles and objects within the vicinity of the vehicle 101. The radar perception layer 202 may include the use of neural network processing and artificial intelligence methods to recognize objects and vehicles, and may convey such information to the sensor fusion and RWM management layer 212.
[0046] The camera perception layer 204 may receive data from one or more cameras, such as cameras (e.g., 158, 160), and may process that data to recognize and determine the location of other vehicles and objects within the vicinity of the vehicle 100. The camera perception layer 204 may include the use of neural network processing and artificial intelligence methods to recognize objects and vehicles, and may convey such information to the sensor fusion and RWM management layer 212.
[0047] The positioning engine layer 206 may receive data from various sensors and process the data to determine the position of the vehicle 100. The various sensors may include, but are not limited to, GPS sensors, IMUs, and / or other sensors connected via a CAN bus. The positioning engine layer 206 may also utilize inputs from one or more cameras, such as cameras (e.g., 158, 160), and / or any other available sensors, such as radar, LIDAR, etc.
[0048] The fraud management system 200 may include or be coupled to a vehicle wireless communications subsystem 230. The wireless communications subsystem 230 may be configured to communicate with other vehicle computing devices and highway communications systems, such as via vehicle-to-vehicle (V2V) communications links, and / or to remote information sources, such as cloud-based resources, via cellular wireless communications systems, such as 5G networks. In various embodiments, the wireless communications subsystem 230 may communicate with other V2X system participants via wireless communications links to receive LDM data.
[0049] The map fusion and arbitration layer 208 may access LDM data received from other V2X system participants, receive output received from the positioning engine layer 206, and process the data to further determine the location of the vehicle 101 in a map, such as a location in a travel lane, a location in a street map, etc. The LDM data may be maintained or stored in a memory (e.g., memory 432) of the vehicle. For example, the map fusion and arbitration layer 208 may convert latitude and longitude information from the GPS to a location in a ground map of the roads in the LDM data. GPS position determinations include errors, so the map fusion and arbitration layer 208 may function to determine a best guess location of the vehicle in the road based on arbitration between the GPS coordinates and the LDM data. For example, the GPS coordinates may identify the location of the vehicle near the center of a two-lane road in the LDM data, but the map fusion and arbitration layer 208 may determine that from the heading direction, the vehicle is most likely aligned in the heading lane that matches the heading direction. The map fusion and reconciliation layer 208 may pass the map-based location information to the sensor fusion and RWM management layer 212.
[0050] The route planning layer 210 may utilize the LDM data, as well as input from an operator or dispatcher, to plan a route to be followed by the vehicle 101 to a particular destination. The route planning layer 210 may pass map-based location information to the sensor fusion and RWM management layer 212. However, the use of traditional maps by other layers, such as the sensor fusion and RWM management layer 212, is not required. For example, as perception data is received, other stacks may build lanes, boundaries, and notions of a local map to operate and / or control the vehicle based solely on the perception data without the provided map.
[0051] The sensor fusion and RWM management layer 212 may receive data and outputs produced by the radar perception layer 202, the camera perception layer 204, the map fusion and arbitration layer 208, and the route planning layer 210, and may use some or all of such inputs to estimate or refine the location and state of the vehicle 101 relative to the road, other vehicles on the road, and other objects in the vicinity of the vehicle 100. For example, the sensor fusion and RWM management layer 212 may combine image data from the camera perception layer 204 with arbitrated map location information from the map fusion and arbitration layer 208 to refine the determined position of the vehicle within a traffic lane. As another example, the sensor fusion and RWM management layer 212 may combine object recognition and image data from the camera perception layer 204 with object detection and ranging data from the radar perception layer 202 to determine and refine the relative positions of other vehicles and objects in the vicinity of the vehicle. As another example, the sensor fusion and RWM management layer 212 may receive information regarding other vehicle positions and headings from vehicle-to-vehicle (V2V) communications (such as via a CAN bus) and combine that information with information from the radar perception layer 202 and camera perception layer 204 to improve the location and operation of other vehicles. The sensor fusion and RWM management layer 212 may output improved location and status information for the vehicle 100, as well as improved location and status information for other vehicles and objects in the vicinity of the vehicle, to the operation planning and control layer 214 and / or behavior planning and prediction layer 216.
[0052] As a further example, the sensor fusion and RWM management layer 212 may use dynamic traffic control commands to instruct the vehicle 101 to change speed, lane, direction of travel, or other navigation elements, and may combine that information with other received information to determine improved location and status information. The sensor fusion and RWM management layer 212 may output the improved location and status information of the vehicle 101, as well as improved location and status information of other vehicles and objects in the vicinity of the vehicle 101, to the trip planning and control layer 214, the behavior planning and prediction layer 216, and / or to devices remote from the vehicle 101, such as data servers, other vehicles, etc., via wireless communications, such as through a C-V2X connection, other wireless connections, etc.
[0053] As yet a further example, the sensor fusion and RWM management layer 212 may monitor perception data from various sensors, such as perception data from the radar perception layer 202, the camera perception layer 204, other perception layers, etc., and / or data from one or more sensors themselves to analyze conditions in the vehicle sensor data. The sensor fusion and RWM management layer 212 may be configured to detect conditions in the sensor data, such as whether a sensor measurement is at, above, or below a threshold, some type of sensor measurement is made, and may output the sensor data to the behavior planning and prediction layer 216 and / or as part of improved location and status information of the vehicle 101 provided to devices remote from the vehicle 100, such as data servers, other vehicles, etc., via wireless communications, such as through a C-V2X connection, other wireless connections, etc.
[0054] Improved location and status information includes vehicle specifications (e.g., size, weight, color, on-board sensor types, etc.); vehicle position, speed, acceleration, heading, attitude, orientation, destination, fuel / power levels, and other status information; vehicle emergency status (e.g., whether the vehicle is an emergency vehicle or a private individual in an emergency); vehicle constraints (e.g., heavy / wide loads, turning constraints, high occupancy vehicle (HOV) The sensor data may include vehicle descriptors associated with the vehicle and the vehicle owner and / or vehicle operator, such as vehicle authorization, vehicle capabilities (e.g., all wheel drive, four wheel drive, snow tires, chains, supported connection types, on-board sensor operational status, on-board sensor resolution level, etc.), equipment issues (e.g., low tire pressure, weak brakes, sensor outage, etc.), owner / operator travel preferences (e.g., preferred lanes, roads, routes, and / or destinations, preferences to avoid tolls or highways, preferences for fastest route, etc.), authorization to provide sensor data to a data authority server (e.g., 184), and / or owner / operator identification information.
[0055] The behavior planning and prediction layer 216 of the autonomous vehicle system stack 200 may use the improved location and state information of the vehicle 101 and the location and state information of other vehicles and objects output from the sensor fusion and RWM management layer 212 to predict future behavior of other vehicles and / or objects. For example, the behavior planning and prediction layer 216 may use such information to predict the future relative positions of other vehicles in the vicinity of the vehicle based on its own vehicle position and speed and the positions and speeds of the other vehicles. Such predictions may take into account information from LDM data and route planning to anticipate changes in relative vehicle positions as the host and other vehicles proceed along the road. The behavior planning and prediction layer 216 may output behavior and location predictions of other vehicles and objects to the operation planning and control layer 214. Additionally, the behavior planning and prediction layer 216 may plan using object behaviors in combination with location predictions and generate control signals to control the operation of the vehicle 101. For example, based on the route planning information, the improved location in the road information, and the relative locations and operations of other vehicles, the behavior planning and prediction layer 216 may determine that the vehicle 101 needs to change lanes and accelerate, such as to maintain or achieve a minimum separation from other vehicles and / or to prepare for a turn or exit. As a result, the behavior planning and prediction layer 216 may calculate or otherwise determine the steering angle for the wheels and the change in throttle setting to be commanded to the trip planning and control layer 214 and the DBW system / control unit 220, along with such various parameters necessary to accomplish such lane change and acceleration. One such parameter may be the calculated steering wheel command angle.
[0056] The operation planning and control layer 214 may receive data and information output from the sensor fusion and RWM management layer 212 and other vehicle and object behavior, as well as location predictions from the behavior planning and prediction layer 216, and may use this information to plan and generate control signals for controlling the operation of the vehicle 101, and may verify that such control signals meet safety requirements for the vehicle 100. For example, based on the route planning information, improved location in road information, and the relative locations and operations of other vehicles, the operation planning and control layer 214 may verify various control commands or instructions, which may be passed to the DBW system / control unit 220.
[0057] The DBW system / control unit 220 may receive commands or instructions from the fleet planning and control layer 214 and may convert such information into mechanical control signals for controlling the wheel angles, brakes, and throttle of the vehicle 100. For example, the DBW system / control unit 220 may respond to a calculated steering wheel command angle by sending a corresponding control signal to a steering wheel controller.
[0058] In various embodiments, the wireless communications subsystem 230 may communicate with other V2X system participants via wireless communications links to transmit sensor data, location data, vehicle data, and data collected by on-board sensors about the environment surrounding the vehicle. Such information may be used by the other V2X system participants to update the LDM data for relay to the other V2X system participants.
[0059] In various embodiments, the fraud management system stack 200 may include functionality to perform safety checking or oversight of various commands, planning or other decisions of various layers that may affect the safety of the vehicle and occupants. Such safety checking or oversight functionality may be implemented in a dedicated layer or may be distributed among the various layers and included as part of the functionality. In some embodiments, various safety parameters may be stored in memory and the safety checking or oversight functionality may compare determined values (e.g., relative spacing to nearby vehicles, distance from road centerline, etc.) with corresponding safety parameters and may issue warnings or commands if a safety parameter is or will be violated. For example, a safety or supervisory function in the behavior planning and prediction layer 216 (or in a separate layer) may determine (e.g., based on a world model improved by the sensor fusion and RWM management layer 212) a current or future separation distance between the vehicle and another vehicle (as defined by the sensor fusion and RWM management layer 212), compare the separation distance to a safe separation distance parameter stored in memory, and issue a command to the trip planning and control layer 214 to accelerate, decelerate, or turn if the current or predicted separation distance violates the safe separation distance parameter. As another example, a safety or supervisory function in the trip planning and control layer 214 (or in a separate layer) may compare a determined or commanded steering wheel command angle to a safe wheel angle limit or parameter, and may issue an override command and / or an alert in response to a commanded angle exceeding the safe wheel angle limit.
[0060] Some safety parameters stored in memory may be static (i.e., unchanged over time), such as maximum vehicle speed. Other safety parameters stored in memory may be dynamic, in that the parameters may be continuously or periodically determined or updated based on vehicle state information and / or environmental conditions. Non-limiting examples of safety parameters include maximum safe speed, maximum braking pressure, maximum acceleration, and safe wheel angle limits, all of which may be a function of road and weather conditions.
[0061] 2B illustrates an example of a subsystem, computational element, computing device, or computing unit in a vehicle management system 250 that may be utilized in a vehicle 101. With reference to FIGS. 1A-2B, in some embodiments, layers 202, 204, 206, 208, 210, 212, and 216 of the misconduct management system stack 250 may be similar to those described with reference to FIG. 2A, and the misconduct management system stack 250 may operate similarly to the misconduct management system stack 200, except that the misconduct management system stack 250 may pass various data or instructions to the vehicle safety and crash avoidance system 252 rather than the DBW system / control unit 220. For example, the configuration of the misconduct management system stack 250 and the vehicle safety and crash avoidance system 252 illustrated in FIG. 2B may be used in a non-autonomous vehicle.
[0062] In various embodiments, the behavior planning and prediction layer 216 and / or the sensor fusion and RWM management layer 212 may output data to the vehicle safety and crash avoidance system 252. For example, the sensor fusion and RWM management layer 212 may output sensor data as part of improved location and state information of the vehicle 101 provided to the vehicle safety and crash avoidance system 252. The vehicle safety and crash avoidance system 252 may use the improved location and state information of the vehicle 101 to make safety decisions regarding the vehicle 101 and / or occupants of the vehicle 100. As another example, the behavior planning and prediction layer 216 may output behavior models and / or predictions related to the operation of other vehicles to the vehicle safety and crash avoidance system 252. The vehicle safety and crash avoidance system 252 may use the behavior models and / or predictions related to the operation of other vehicles to make safety decisions regarding the vehicle 101 and / or occupants of the vehicle 101.
[0063] In various embodiments, vehicle safety and crash avoidance system 252 may include functionality to perform safety inspections or oversight of various commands, various layers of planning or other decisions, and human driver actions that may affect the safety of the vehicle and its occupants. In some embodiments, various safety parameters may be stored in memory, and vehicle safety and crash avoidance system 252 may compare determined values (e.g., relative spacing to nearby vehicles, distance from a road centerline, etc.) with corresponding safety parameters, and may issue warnings or commands if a safety parameter is or will be violated. For example, vehicle safety and crash avoidance system 252 may determine (e.g., based on a world model improved by sensor fusion and RWM management layer 212) a current or future separation distance between its vehicle and another vehicle (as defined by sensor fusion and RWM management layer 212), compare the separation distance to safety separation distance parameters stored in memory, and issue commands to the driver to accelerate, decelerate, or turn if the current or predicted separation distance violates the safety separation distance parameters. As another example, the vehicle safety and crash avoidance system 252 may compare a human driver's changes in steering wheel angle to safe wheel angle limits or parameters and may issue override commands and / or warnings in response to steering wheel angles that exceed the safe wheel angle limits.
[0064] FIG. 3 illustrates an exemplary SOC architecture of a processing device system-on-chip (SOC) 300 suitable for implementing various embodiments in a vehicle. With reference to FIGS. 1A-3, the processing device SOC 300 may include several heterogeneous processors, such as a digital signal processor (DSP) 303, a modem processor 304, an image and object recognition processor 306, a mobile display processor 307, an application processor 308, and a resource and power management (RPM) processor 317. The processing device SOC 300 may also include one or more co-processors 310 (e.g., vector co-processors) connected to one or more of the heterogeneous processors 303, 304, 306, 307, 308, 317. Each of the processors may include one or more cores and an independent / internal clock. Each processor / core may perform operations independent of the other processors / cores. For example, processing device SOC 300 may include a processor that executes a first type of operating system (e.g., FreeBSD, LINUX, OS X, etc.) and a processor that executes a second type of operating system (e.g., Microsoft Windows). In some embodiments, application processor 308 may be a main processor, central processing unit (CPU), microprocessor unit (MPU), arithmetic logic unit (ALU), etc. of SOC 300. Graphics processor 306 may be a graphics processing unit (GPU).
[0065] The processing device SOC 300 may include analog and custom circuitry 314 for managing sensor data, analog-to-digital conversion, wireless data transmission, and for performing other specialized operations such as processing encoded audio and video signals for rendering in a web browser. The processing device SOC 300 may further include system components and resources 316, such as voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components, used to support a processor and a software client (e.g., a web browser) running on the computing device.
[0066] The processing device SOC 300 also includes dedicated circuitry for a camera operation and management processor 305 that includes, provides, controls, and / or manages the operation of one or more cameras 158, 160 (e.g., primary camera, webcam, 3D camera, etc.), video display data from camera firmware, image processing, video pre-processing, video front-end (VFE), in-line JPEG, high definition video codec, etc. The camera operation and management processor 305 may be an independent processing unit and / or may include an independent or internal clock.
[0067] In some embodiments, the image and object recognition processor 306 may be configured with processor-executable instructions and / or dedicated hardware configured to perform the image processing and object recognition analysis involved in various embodiments. For example, the image and object recognition processor 306 may be configured to perform operations to process images received from the cameras (e.g., 158, 160) via the camera operation and management processor 305 to recognize and / or identify other vehicles and otherwise perform the functions of the camera perception layer 204 as described. In some embodiments, the processor 306 may be configured to process radar or lidar data and perform the functions of the radar perception layer 202 as described.
[0068] The system components and resources 316, analog and custom circuitry 314, and / or camera actuation and management processor 305 may include circuitry for interfacing with peripheral devices such as cameras 158, 160, radar 168, lidar 170, electronic displays, wireless communication devices, external memory chips, etc. The processors 303, 304, 306, 307, 308 may be interconnected to one or more memory elements 312, system components and resources 316, analog and custom circuitry 314, camera actuation and management processor 305, and RPM processor 317 via an interconnect / bus module 324 that may include an array of reconfigurable logic gates and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). Communications may be provided by advanced interconnects such as a high-performance network-on-chip (NoC).
[0069] The processing device SOC 300 may further include input / output modules (not shown) for communicating with resources external to the SOC, such as a clock 318 and a voltage regulator 320. Resources external to the SOC (e.g., clock 318, voltage regulator 320) may be shared by two or more of the internal SOC processors / cores (e.g., DSP 303, modem processor 304, graphics processor 306, application processor 308, etc.).
[0070] In some embodiments, the processing device SOC 300 may be included in a control unit (e.g., 140) for use in a vehicle (e.g., 101). The control unit may include communications links for communication with a telephone network (e.g., 180), the Internet, and / or a network server (e.g., 184) as described.
[0071] The processing device SOC 300 may also include additional hardware and / or software components suitable for collecting sensor data from sensors including motion sensors (e.g., accelerometers and gyroscopes of an IMU), user interface elements (e.g., input buttons, touch screen display, etc.), microphone arrays, sensors for monitoring physical conditions (e.g., location, direction, navigation, orientation, vibration, pressure, etc.), cameras, compasses, GPS receivers, communications circuitry (e.g., Bluetooth, WLAN, WiFi, etc.), as well as other well-known components of modern electronic devices.
[0072] FIG. 4 is a component block diagram illustrating a system 400 configured to generate a local dynamic map data model, according to various embodiments. In some embodiments, the system 400 may include one or more computing platforms 402 of a V2X device and / or one or more other V2X system participants 404. With reference to FIGS. 1A-4 and 7-9, the V2X device 402 may include a processor (e.g., 434, 702, 802). The V2X device 402 may be configured by machine-executable instructions 406. The machine-executable instructions 406 may include one or more instruction modules. The instruction modules may include computer program modules. The instruction modules may include one or more of an LDM data receiving module 408, an LDM data integration module 410, an LDM data determining module 412, an LDM data providing module 414, a map generating module 416, a map transmitting module 418, and / or other instruction modules.
[0073] The LDM data reception module 408 may be configured to receive fresh LDM data for the misconduct management system running on the V2X equipment processor. In some embodiments, the LDM data reception module 408 may be configured to receive registration messages from other V2X system participants 404. In some embodiments, the LDM data reception module 408 may be configured to receive planned route information from other V2X system participants 404. In some embodiments, the LDM data reception module 408 may be configured to receive mobile device kinematic information from other V2X system participants 404. In some embodiments, the LDM data reception module 408 may be configured to receive data from other V2X system participants 404, such as, for example, sensor data, image data, audio data, or operating condition data acquired by the other V2X system participants 404.
[0074] The LDM data integration module 410 can be configured to integrate fresh LDM data into the LDM data model.
[0075] The LDM data determination module 412 may be configured to determine LDM data of the LDM data model associated with another particular V2X system participant 404. In some embodiments, the LDM data determination module 412 may be configured to determine LDM data associated with another particular V2X system participant 404 based on information included in the registration message. In some embodiments, the LDM data determination module 412 may be configured to determine LDM data associated with another particular V2X system participant 404 based on planned route information. In some embodiments, the LDM data determination module 412 may be configured to determine LDM data associated with another particular V2X system participant 404 based on kinematics information. In some embodiments, the LDM data determination module 412 may be configured to determine information associated with the LDM data from the received data.
[0076] The LDM data providing module 414 may be configured to provide the determined relevant LDM data to other V2X system participants 404. In some embodiments, the determined relevant LDM data may include highly dynamic LDM information.
[0077] The map generation module 416 may be configured to generate a digital map covering an area within a predetermined distance of the other V2X system participants. In some embodiments, the map transmission module 418 may be configured to transmit the digital map to the other V2X system participants 404. The digital map may be generated and transmitted in a format suitable for use in the autonomous navigation of the other V2X system participants 404.
[0078] In some implementations, the V2X equipment 402, other V2X system participant devices 404, and / or external resources 430 may be operatively linked via one or more electronic communications links. For example, such electronic communications links may be established at least in part via a network, such as the Internet and / or other networks. It will be appreciated that this is not intended to be limiting and that the scope of the present disclosure includes implementations in which the V2X equipment 402, other V2X system participants 404, and / or external resources 430 may be operatively linked via some other communications medium.
[0079] Each of the other V2X system participants 404 may include one or more processors configured to execute computer program modules that may be configured to enable an expert or user associated with a given V2X system participant 404 to interface with the system 400 and / or external resources 430 and / or provide other functionality attributed to the other V2X system participants 404 herein.
[0080] The external resources 430 may include sources of information outside of the V2X system 400, external entities participating in the V2X system 400, and / or other resources. In some implementations, some or all of the functionality attributed to the external resources 430 herein may be provided by resources included within the system 400.
[0081] The V2X device 402 may include electronic storage 432, one or more processors 434, and / or other components. The V2X device 402 may include communication lines or ports to enable exchange of information with a network and / or other computing platforms. The illustration of the V2X device 402 in FIG. 4 is not intended to be limiting. The V2X device 402 may include multiple hardware, software, and / or firmware components that operate together to provide the functionality attributed to the V2X device 402 herein. For example, the V2X device 402 may be implemented by a cloud of computing platforms that operate the V2X device 402 together.
[0082] The electronic storage 432 may comprise a non-transitory storage medium that electronically stores information. The electronic storage medium of the electronic storage 432 may include one or both of system storage that is integral with the V2X device 402 (i.e., substantially non-removable) and / or removable storage that is removably connectable to the V2X device 402, for example, via a port (e.g., a Universal Serial Bus (USB) port, a Firewire port, etc.) or a drive (e.g., a disk drive, etc.). The electronic storage 432 may include one or more of an optically readable storage medium (e.g., an optical disk, etc.), a magnetically readable storage medium (e.g., a magnetic tape, a magnetic hard drive, a floppy drive, etc.), a charge-based storage medium (e.g., an EEPROM, a RAM, etc.), a solid-state storage medium (e.g., a flash drive, etc.), and / or other electronically readable storage medium. The electronic storage 432 may include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and / or other virtual storage resources). The electronic storage 432 may store software algorithms, information determined by the processor 434, information received from the V2X device 402, information received from other V2X system participants 404, and / or other information that enables the V2X device 402 to function as described herein.
[0083] The processor 434 may be configured to provide information processing capabilities within the V2X device 402. Thus, the processor 434 may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although the processor 434 is shown in FIG. 4 as a single entity, this is for illustrative purposes only. In some implementations, the processor 434 may include multiple processing units. These processing units may be physically located within the same device, or the processor 434 may represent the processing functions of multiple devices operating in concert. The processor 434 may be configured to execute the modules 408, 410, 412, 414, 416, 418, and / or other modules. The processor 434 may be configured to execute modules 408, 410, 412, 414, 416, 418, and / or other modules through software, hardware, firmware, or some combination of software, hardware, and / or firmware, and / or other mechanisms for configuring processing power on the processor 434. As used herein, the term "module" may refer to any component or set of components that performs the functions attributed to a module. This may include one or more physical processors in execution of processor-readable instructions, processor-readable instructions, circuitry, hardware, storage media, or any other component.
[0084] 4 as being implemented within a single processing unit, it should be appreciated that in implementations in which the processor 434 includes multiple processing units, one or more of the modules 408-418 may be implemented remotely from the other modules. The description of the functionality provided by the various modules 408-418 described below is for illustrative purposes and is not intended to be limiting, as any of the modules 408-418 may provide more or less functionality than described. For example, one or more of the modules 408-418 may be excluded, with some or all of its functionality being provided by other of the modules 408-418. As another example, the processor 434 may be configured to execute one or more additional modules that may perform some or all of the functionality attributed to one of the following modules 408-418.
[0085] 5 is a process flow diagram illustrating operations of a method 500 performed by a processor of an observing vehicle's V2X equipment to detect a misbehavior condition by comparing a received V2X message to a local dynamic map (LDM) data model of the vehicle and determining whether an inconsistency exists between the data received in the V2X message and a locally maintained or stored LDM data model. With reference to FIGS. 1-5, operations of method 500 may be performed by a processor of the observing vehicle's V2X equipment (e.g., vehicle 12 in FIG. 1D).
[0086] At block 502, the processor may monitor a number of vehicle sensors related to controlling the steering, navigation, and / or other operations of the observing vehicle (e.g., vehicle 12). By monitoring data from the vehicle's own sensors, the misbehavior management system operating on the observing vehicle's V2X equipment 402 may begin to collect data that can be used to generate an LDM data model that represents the environment surrounding the observing vehicle. This may include information about suspicious vehicles sending V2X messages with data indicative of a misbehavior condition. In addition, the LDM data model may include information about neighboring vehicles.
[0087] At block 504, an LDM data model representing the environment surrounding the observed vehicle may be generated by the V2X equipment 402 based at least in part on the aggregation of additional data collected from multiple sensors, as described above with respect to Figures 2A-4. The LDM data model may include Types 1-4 data. At block 506, the generated LDM data model may be maintained or stored locally in memory (e.g., electronic storage 432).
[0088] At block 508, the fraud management system running on the V2X equipment processor may receive a V2X message from another V2X system participant 404. The V2X message may include traffic information, GPS information of the reporting vehicle and other adjacent vehicles calculated from on-board equipment of the reporting vehicle and / or provided by the adjacent vehicles themselves, and / or map data specifying road geometry and street furniture.
[0089] In decision block 510, the misbehavior management system running on the V2X equipment processor may determine whether the received V2X message includes data indicative of a misbehavior condition by comparing the data included in the received V2X message with a locally maintained or stored LDM data model to detect a misbehavior condition. For example, the present vehicle (called the observation vehicle) may receive a V2X message that includes data indicating that another vehicle (called the reporting vehicle) may be passing between two adjacent vehicles. In some embodiments, the reporting vehicle and the two adjacent vehicles may all be V2X system participants. However, in some situations, neither the reporting vehicle nor the two adjacent vehicles are V2X system participants. In other situations, the reporting vehicle and some of the two adjacent vehicles may be V2X system participants, while others are not.
[0090] In this example, the observation vehicle may obtain information about the location / position, heading, speed, motion, etc. of the reporting vehicle and of the two adjacent vehicles through its own sensors, e.g., a camera and a LIDAR installed on board the observation vehicle. In addition, the observation vehicle may receive V2X messages from any and / or all of the reporting vehicle and the two adjacent vehicles. Such received V2X messages may include camera and / or LIDAR information corroborating the observation vehicle's observations about the location / position, heading, speed, motion, etc. of the reporting vehicle and of the two adjacent vehicles. In addition, V2X messages from either the reporting vehicle or the two adjacent vehicles may include GPS location / position information, speedometer data, etc. Yet further camera and LIDAR information about the reporting vehicle or either the two adjacent vehicles may be received from the RSU V2X system participant.
[0091] Any and all of this data received in the various V2X messages may augment a locally maintained LDM data model created by the observing vehicle to represent the environment surrounding the observing vehicle. Thus, the observing vehicle's LDM data model may include data regarding the location / position, heading, and speed of the reporting vehicle and two adjacent vehicles. The observing vehicle may detect a misbehavior condition by comparing the data included in the received V2X message with the data in the observing vehicle's LDM data model. For example, a V2X message received from a reporting vehicle may indicate that the reporting vehicle is passing between two adjacent vehicles. However, based on all of the other information collected by the observing vehicle, the observing vehicle's LDM data model may indicate that there is insufficient space for the reporting vehicle to pass the two adjacent vehicles. Thus, a misbehavior management system running on the observing vehicle's V2X equipment processor may identify a V2X message received from a reporting vehicle as including or evidencing a misbehavior condition. Any of the data received from a reporting vehicle in a V2X message indicating location / position, speed, heading, etc. may be inaccurate, corrupted, or deliberately altered.
[0092] Based on the comparison in block 510, in response to a determination that the V2X message received from the reporting vehicle includes data indicative of a misbehavior condition (i.e., decision 510=Yes), the misbehavior management system operating within the V2X equipment 402 of the observing vehicle may generate a misbehavior report (MBR) in block 512 that identifies the misbehavior condition and the suspect vehicle that sent the V2X message.
[0093] At block 514, the misbehavior management system operating in the V2X equipment 402 of the observing vehicle may send the MBR to a misbehavior managing authority (MA) for further analysis, reporting, and corrective action. For example, the MA may send a message to the suspect vehicle that its sensor is required to be serviced or replaced. To perform an accurate and comprehensive analysis of the misbehavior condition, the MA may utilize the data in the LDM data model such that a holistic analysis of the circumstances giving rise to the misbehavior condition may be determined. Additionally, by analyzing the data in the LDM data model, more appropriate and efficient corrective actions may be issued to mitigate the misbehavior condition. To enable this, some embodiments include a misbehavior management system that provides better quality misbehavior detection (e.g., provides fewer false negative detections) and improves the robustness and resilience of the V2X system 103.
[0094] However, as more and more vehicles are equipped with V2X equipment, the amount of misconduct that may be detected is growing at an exponential rate. In addition, given the bandwidth limitations for transmitting a large number of MBRs, the amount of data that may be included in the MBR may be extremely large. To address this and to reduce the amount of data that may be transmitted with the MBR, in some embodiments, the misconduct management system may transmit only a representation of the LDM data model to the MA. In some embodiments, the misconduct management system may transmit an incomplete data set for the LDM data model when transmitting the MBR to the MA, such as including only data most relevant to the detected misconduct condition and / or excluding data unrelated to the misconduct condition.
[0095] After transmitting the MBR to the MA in block 514, the misbehavior management system operating in the V2X equipment 402 of the observation vehicle may again perform the operations in block 502 to continue monitoring the sensors of the observation vehicle to collect data regarding the environment surrounding the observation vehicle.
[0096] In response to a determination that the data contained in the received V2X message does not indicate a misbehavior condition (i.e., decision 510=No), the misbehavior management system may perform calculations based on at least one of the observed dynamics of the object in the LDM data model or new data inputs received from the V2X message, then modify the LDM data model to incorporate the calculations and the data based on and contained in the received V2X message to augment, improve, or otherwise update the LDM data model of the observed vehicle in block 516, and then store the updated LDM data in block 506. Thus, the misbehavior management system operating in the V2X equipment 402 of the observed vehicle may continually improve the LDM data model. The updated LDM data model may include additional data from the V2X message that enhances and / or improves the LDM data model representing the environment surrounding the observed vehicle. The updated LDM data model may be maintained or stored locally in memory in block 506.
[0097] Following transmission of the MBR to the MA in block 514 or storage of the updated LDM data model in block 506, the misconduct management system operating in the V2X equipment 402 of the observed vehicle may continue to receive other V2X messages in block 508 as described.
[0098] In some embodiments, the misbehavior management system operating within the V2X equipment 402 of the observing vehicle may also periodically perform the operations in block 502 to continue to monitor the sensors of the observing vehicle to gather data regarding the environment surrounding the observing vehicle. In this manner, the universe of data representative of the environment surrounding the observing vehicle may be continually expanded to improve and update the LDM so that misbehavior conditions may be more accurately detected.
[0099] 6 is a process flow diagram illustrating example operations that may be performed as part of block 510 of method 500 to determine whether a misbehavior condition is detected in a received V2X message. With reference to FIGS. 1A-6, the operations of block 510 may be performed by a misbehavior management system operating in the V2X equipment 402 of the observed vehicle.
[0100] After receiving the V2X message at block 508 of method 500, the fraud management system may obtain selected data (e.g., location / position, speed, heading, temperature, etc.) from the received V2X message at block 518. As part of the operations at block 518, an identifier of the originating transmitter of the received V2X message may be obtained.
[0101] At optional block 519, the misconduct management system may select a data element, or data elements in the LDM data model, to use in determining whether the information reported in the V2X message indicates or is the product of misconduct. As described, the LDM data model will include a number of data elements that define the environment surrounding the vehicle and other vehicles in the vicinity, which may be useful for assessing the accuracy or reliability of the information received in the V2X message. Thus, in some embodiments, the misconduct management system may select a number of data elements (e.g., a subset of information) in the LDM data model that will be used to verify or validate the received V2X message. In some embodiments, the misconduct management system may select information or elements in the LDM data model based on or in response to the type of information in the received V2X message. For example, if a V2X message includes the location of another vehicle ahead of the own vehicle, or of a road hazard, the fraud management system may select location-related data elements in the LDM data model for use in validating the V2X message and avoid accessing data elements related to speed, weather conditions, road conditions, or location behind the own vehicle. As another example, if a V2X message includes dynamic information about another vehicle (e.g., turn angle, speed, braking status, etc.), data elements in the LDM data model that are static or outdated will not be helpful in evaluating the V2X message. By selecting and accessing a subset of information in the LDM data model that is relevant to validating or verifying information in a received V2X message, the fraud management system may conserve processing resources and memory utilization and enable faster evaluation of messages than if all data in the LDM data model were used in the message evaluation process.
[0102] In block 520, the fraud management system operating within the V2X equipment of the observation vehicle may compare the parsed data included in the received V2X message against data in an LDM data model maintained or stored in a memory (e.g., storage device 432) of the observation vehicle.
[0103] At decision block 522, the fraud management system may determine whether any of the parsed data included in the received V2X message contradicts or is inconsistent with data in an LDM data model maintained or stored in a memory (e.g., storage device 432) of the observing vehicle.
[0104] For example, in decision block 522, the fraud management system may analyze location information of the vehicle issuing the received V2X message to determine whether the information reported in the V2X message is inconsistent with or contradicts location information in the LDM data model (e.g., indicating a location that corresponds to the location of another vehicle in the LDM).
[0105] As another example of a possible misbehavior state that may be detected using the LDM data in decision block 522, the observation vehicle may be tracking an adjacent vehicle through the observation vehicle's sensors. The adjacent vehicle may not be a V2X system participant. The suspect vehicle may send a V2X message to the observation vehicle that implies that the suspect vehicle is located at a position that overlaps with the position of the tracked adjacent vehicle as observed and determined by the observation vehicle's sensors. In other words, the suspect vehicle may send a V2X message to the observation vehicle that implies that the suspect vehicle location matches the position of the tracked adjacent vehicle as observed and determined by the observation vehicle's sensors (e.g., cameras). In such a case, the V2X message received by the observation vehicle from the suspect vehicle contains data that is inconsistent or inconsistent with the data in the observation vehicle's LDM data model. Thus, the observation vehicle may generate an MBR that indicates the suspect vehicle's misbehavior state.
[0106] Another example of a possible misbehavior condition that may be detected using the LDM data in decision block 522 may be an obstacle in the road. For example, a sofa being transported may fall off a flatbed truck. The observation vehicle may detect the obstacle through the observation vehicle's sensors. The observation vehicle may receive a V2X message from the suspect vehicle that includes data that suggests the suspect vehicle maintained its heading and speed as it drove through the obstacle without slowing down. In such a case, the V2X message received by the observation vehicle from the suspect vehicle includes data that is inconsistent or inconsistent with data in the observation vehicle's LDM data model. Thus, the observation vehicle may generate an MBR that indicates the misbehavior condition of the suspect vehicle.
[0107] As another example of a possible misbehavior condition that may be detected using LDM data in decision block 522, the observing vehicle may be tracking an adjacent vehicle through the observing vehicle's sensors. The adjacent vehicle may not be a V2X system participant. The view of the adjacent vehicle may be temporarily blocked from the observing side (i.e., something gets in the way for a short period of time so that the observing vehicle cannot see the AV directly) and then becomes trackable again. During the time of obstruction, the suspect vehicle may send a V2X message to the observing vehicle about the suspect vehicle's heading (i.e., track). The observing vehicle may reconstruct the route the adjacent vehicle may have taken while it was out of view. Based on this information, the observing vehicle may realize that all possible routes imply a collision between the suspect vehicle and the adjacent vehicle. Because no collision was recorded by the observing vehicle, the V2X message received by the observing vehicle from the suspect vehicle contains data that is inconsistent or inconsistent with the data in the observing vehicle's LDM data model. Thus, the observer vehicle may generate an MBR that indicates the tampering status of the suspect vehicle.
[0108] As another example of a possible misbehavior state that may be detected using the LDM data at decision block 522, the observing vehicle may receive a V2X traffic light message from an RSU indicating that the traffic light is red. The observing vehicle may receive a V2X message from a suspect vehicle indicating that the suspect vehicle includes traveling through an intersection. While the suspect vehicle may actually be running a red light, the V2X message received by the observing vehicle from the suspect vehicle includes data that seemingly contradicts or is inconsistent with data in the observing vehicle's LDM data model. Thus, the observing vehicle may generate an MBR that indicates the suspect vehicle's misbehavior state.
[0109] As another example of a possible misbehavior condition that may be detected using LDM data in decision block 522, an observing vehicle may receive a V2X message from an RSU indicating that roadworks are causing a lane change. The lane change is not indicated in the static map available to the observing vehicle. However, based on the incoming V2X message, the observing vehicle notices that all adjacent vehicles behave as if a lane change is in place, i.e., they all change one lane to the left at a particular location. A suspicious vehicle may send a V2X message that implies that the suspicious vehicle does not observe a lane change, i.e., the suspicious vehicle appears to drive straight ahead, even though it should be visible to the driver that other adjacent vehicles are performing lane changes. The implying is that the suspicious vehicle has its message generated remotely from the actual location by an attacker who does not know about the lane change. In such a case, the V2X message received by the observing vehicle from the suspicious vehicle contains data that is inconsistent or inconsistent with data in the observing vehicle's LDM data model. Thus, the observer vehicle may generate an MBR that indicates the tampering status of the suspect vehicle.
[0110] In response to a determination that the parsed data included in the received V2X message is inconsistent or inconsistent with data in the LDM data model maintained or stored in a memory (e.g., storage device 432) of the observing vehicle (i.e., decision 522=Yes), the misconduct management system operating in the V2X equipment 402 of the observing vehicle may perform the operations at block 512 of method 500 to generate an MBR, as described.
[0111] In response to a determination that the parsed data included in the received V2X message is consistent with or consistent with data in the LDM data model maintained or stored in the memory (e.g., storage device 432) of the observing vehicle (i.e., decision 522=No), the misconduct management system operating in the V2X equipment 402 of the observing vehicle may perform operations at block 516 to modify or update the LDM data model, as described.
[0112] Various embodiments (including but not limited to those described above with reference to FIGS. 1A-6) may be implemented in a wide variety of computing systems including in-vehicle equipment as well as mobile computing devices, one example of a mobile computing device suitable for use with various embodiments is shown in FIG. 7. The mobile computing device 700 may include a processor 702 coupled to a touch screen controller 704 and an internal memory 706. The processor 702 may be one or more multi-core integrated circuits, general purpose or designated for specific processing tasks. The internal memory 706 may be volatile or non-volatile memory, and may be secure and / or encrypted or non-secure and / or non-encrypted memory, or any combination thereof. Examples of memory types that may be utilized include, but are not limited to, DDR, LPDDR, GDDR, WIDEIO, RAM, SRAM, DRAM, P-RAM, R-RAM, M-RAM, STT-RAM, and embedded DRAM. The touchscreen controller 704 and processor 702 may also be coupled to a touchscreen panel 712, such as a resistive-sensing touchscreen, a capacitive-sensing touchscreen, an infrared-sensing touchscreen, etc. Additionally, the display of the mobile computing device 700 need not have touchscreen capabilities.
[0113] The mobile computing device 700 may have one or more wireless signal transceivers 708 (e.g., Peanut, Bluetooth, ZigBee, Wi-Fi, RF radio) for transmitting and receiving communications, and an antenna 710, coupled to each other and / or to the processor 702. The transceivers 708 and antennas 710 may be used with the circuitry described above to implement various wireless transmission protocol stacks and interfaces. The mobile computing device 700 may include a cellular network wireless modem chip 716 that enables communication over a cellular network and is coupled to the processor.
[0114] The mobile computing device 700 may include a peripheral device connection interface 718 coupled to the processor 702. The peripheral device connection interface 718 may be solely configured to accept one type of connection or may be configured to accept various types of common or proprietary physical and communication connections, such as Universal Serial Bus (USB), FireWire, Thunderbolt, or PCIe. The peripheral device connection interface 718 may also be coupled to a similarly configured peripheral device connection port (not shown).
[0115] The mobile computing device 700 may also include a speaker 714 for providing audio output. The mobile computing device 700 may also include a housing 720 constructed from plastic, metal, or a combination of materials for housing all or a portion of the components described herein. Those skilled in the art will recognize that the housing 720 may be a dashboard console of a vehicle in an on-board embodiment. The mobile computing device 700 may include a power source 722 coupled to the processor 702, such as a disposable battery or a rechargeable battery. The rechargeable battery may also be coupled to a peripheral device connection port to receive a charging current from a source external to the mobile computing device 700. The mobile computing device 700 may also include a physical button 724 for receiving user input. The mobile computing device 700 may also include a power button 726 for turning the mobile computing device 700 on and off.
[0116] Various embodiments (including but not limited to those described above with reference to FIGS. 1A-6) may be implemented in a wide variety of computing systems, including a laptop computer 800, an example of which is shown in FIG. 8. Many laptop computers include a touchpad touch surface 817 that serves as the computer's pointing device and may therefore receive drag, scroll, and flick gestures similar to those implemented on the computing devices described above equipped with touchscreen displays. The laptop computer 800 typically includes a processor 802 coupled to a volatile memory 812 and a large capacity non-volatile memory, such as a flash memory disk drive 813. In addition, the computer 800 may have one or more antennas 808 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and / or a cellular telephone transceiver 816 coupled to the processor 802. The computer 800 may also include a floppy disk drive 814 and a compact disk (CD) drive 815 coupled to the processor 802. In a notebook configuration, the computer housing includes a touchpad 817, a keyboard 818, and a display 819, all coupled to the processor 802. Other configurations of computing devices may include a computer mouse or trackball coupled to the processor (e.g., via a USB input), as is well known, and may also be used with the various embodiments.
[0117] Various embodiments (including, but not limited to, those described above with reference to FIGS. 1A-6) may also include a fraud management agency that utilizes a fixed computing system, such as any of a variety of commercially available servers. An exemplary server 900 is shown in FIG. 9. Such a server 900 typically includes one or more multi-core processor assemblies 901 coupled to a volatile memory 902 and a large capacity non-volatile memory, such as a disk drive 904. As shown in FIG. 9, multi-core processor assemblies 901 may be added to the server 900 by inserting them into a rack of assemblies. The server 900 may also include a network access port 907 coupled to the multi-core processor assemblies 901 for establishing a network interface connection with a network 908, such as a local area network, the Internet, a public switched telephone network, and / or a cellular data network (e.g., CDMA, TDMA, GSM, PCS, 3G, 4G, 5G, LTE, or any other type of cellular data network) coupled to other broadcast system computers and servers.
[0118] A number of different cellular and mobile communication services and standards are available or are contemplated in the future, all of which may implement and benefit from the various embodiments. Such services and standards include, but are not limited to, Third Generation Partnership Project (3GPP®), Long Term Evolution (LTE) systems, third generation wireless mobile communication technology (3G), fourth generation wireless mobile communication technology (4G), fifth generation wireless mobile communication technology (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (cdmaOne, CDMA1020®, etc.), Enhanced Data Rates for GSM Evolution (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136 / TDMA), Evolution Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), and the like. Examples of common standards include wireless telecommunications, Worldwide Interoperability for Microwave Access (WiMAX), Wireless Local Area Networks (WLAN), Wi-Fi Protected Access I & II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies involves, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any reference to terminology and / or technical details relating to a particular telecommunications standard or technology is for illustrative purposes only and does not limit the scope of the claims to a particular communications system or communications technology unless specifically recited in the claim language.
[0119] Example implementations are described in the following paragraphs. Some of the example implementations are described with respect to example methods, however, further example implementations may include the example methods described in the following paragraphs implemented by a fraud management system running on a V2X equipment processor, which may be an on-board unit, a mobile device unit, a mobile computing unit, or a fixed roadside unit, including a processor configured with processor-executable instructions for performing operations of the methods of the following implementations, the example methods described in the following paragraphs implemented by a V2X equipment including means for performing the functions of the methods of the following implementations, and the example methods described in the following paragraphs implemented as a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the V2X equipment to perform operations of the methods of the following implementations.
[0120] Example 1. A method for detecting a misbehavior condition in a Vehicle-to-Everything (V2X) system executed by a processor in a vehicle, the method comprising: receiving a V2X message from another V2X system participant, the V2X message including data regarding an environment surrounding the vehicle; comparing the data included in the received V2X message with a locally maintained or stored Local Dynamic Map (LDM) data model to detect the misbehavior condition; generating a misbehavior report identifying the misbehavior condition based on the comparison and in response to detecting the misbehavior condition; and transmitting the generated misbehavior report to a misbehavior management authority.
[0121] Example 2. The method of Example 1, further comprising: monitoring a plurality of sensors within the vehicle to collect additional data regarding an environment surrounding the vehicle; generating an LDM data model representing the environment surrounding the vehicle based at least in part on an aggregation of the additional data collected from the plurality of sensors; and storing the LDM data model in a memory.
[0122] Example 3. The method of any of Examples 1 or 2, further comprising, in response to determining that a tampering condition is not detected, performing a calculation based on at least one of observed dynamics of the object in the LDM data model or new data input received from the V2X message, modifying the LDM data model to incorporate the calculation and the data contained in the received V2X message, and replacing the LDM data model maintained or stored locally in memory with the modified LDM model.
[0123] Example 4. The method of any of Examples 1 to 3, wherein the step of transmitting the generated fraud report to a fraud management authority includes transmitting a representation of the LDM data model.
[0124] Example 5. The method of example 4, wherein the representation of the LDM data model includes an incomplete data set for the LDM data model.
[0125] Example 6. The method of any of Examples 1 to 5, further comprising the step of receiving feedback from a fraud management agency, the feedback including corrective actions to mitigate the fraud situation.
[0126] Example 7. The method of any of Examples 1 to 6, wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes traffic information.
[0127] Example 8. The method of any of Examples 1 to 7, wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes location information of adjacent vehicles based on GNSS (e.g., GPS) data.
[0128] Example 9. The method of any of Examples 1 to 8, wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes map data specifying road geometry and street furniture.
[0129] Example 10. The method of any of Examples 1 to 9, wherein comparing the data included in the received V2X message to a locally maintained or stored LDM data model to detect a fraudulent activity condition includes determining whether any data included in the received V2X message is inconsistent with information in the locally maintained or stored LDM data model.
[0130] Example 11. The method of any of Examples 1 to 10, wherein the step of comparing the data included in the received V2X message with the locally maintained or stored LDM data model to detect a misbehavior condition includes the steps of selecting a subset of data elements in the locally maintained or stored LDM data model for comparison with the data included in the received V2X message, and determining whether any data included in the received V2X message is inconsistent with the selected subset of data elements in the locally maintained or stored LDM data model.
[0131] Example 12. The method of any of Examples 1 to 11, wherein comparing the data included in the received V2X message to the locally maintained or stored LDM data model to detect a misbehavior condition includes determining whether a location of a first adjacent vehicle included in the received V2X message matches a location of a second adjacent vehicle in the locally maintained or stored LDM data model.
[0132] Example 13. The method of any of Examples 1 to 12, wherein comparing data included in the received V2X message to the locally maintained or stored LDM data model to detect a misbehavior condition includes determining whether status information of an adjacent vehicle that transmitted the received V2X message is inconsistent with status information of the adjacent vehicle in the locally maintained or stored LDM data model.
[0133] The various embodiments shown and described are provided merely as examples to illustrate various features of the claims. However, features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment, and may be used with or combined with other embodiments shown and described. Moreover, the claims are not intended to be limited by any single exemplary embodiment.
[0134] The method descriptions and process flow diagrams set forth above are provided only as illustrative examples and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of operations in the above-described embodiments may be performed in any order. Terms such as "then," "then," and "next" do not limit the order of operations, but rather these terms are used to guide the reader through the method descriptions. Additionally, any reference to claim elements in the singular, for example using the articles "a," "an," or "the," should not be construed as limiting the element to the singular.
[0135] Various exemplary logic blocks, modules, components, circuits, and algorithmic operations described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.
[0136] The hardware used to implement the various example logic, logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed using general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of receiver smart objects, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.
[0137] In one or more embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable or processor-readable storage medium. The operations of the methods or algorithms disclosed herein may be embodied in a processor-executable software module or processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that may be accessed by a computer or processor. By way of example and not limitation, such a non-transitory computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above are also included within the scope of non-transitory computer-readable media and non-transitory processor-readable media. Furthermore, operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable storage medium and / or a non-transitory computer-readable storage medium, which may be incorporated into a computer program product.
[0138] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein. [Explanation of symbols]
[0139] 12 vehicles, following vehicles 14 vehicles, trucks 16 vehicles, leading vehicles 18. Communications Networks 20 Safe Separation Distance 30, 40, 50 Basic safety message 60, 62, 64, 66 Communication links 70, 72 Original Equipment Manufacturer (OEM), OEM Server, MBR Pretreatment Server 74 Fraud Control Agency 100 Communication Systems 101 vehicles 102, 104 V2X in-vehicle equipment, V2X equipment, vehicle V2X in-vehicle equipment 103 V2X System 106 V2X in-vehicle equipment, vehicle V2X in-vehicle equipment 110 base station 120 Wireless communication devices 122 Communication Links, Wireless Communication Links 124, 126 Wireless communication links 132 Core Network 140 Control unit, vehicle control unit 140a, 434, 702, 802 processors 140b Memory 140c Input Module 140d Output Module 140e Wireless Module 142 Satellite geographic positioning system receiver, satellite geographic positioning sensor 144, 146, 148, 150, 152 Occupancy sensors 144~170 Sensor 145 Control Input Sensor 154, 156 Tire pressure sensor 158, 160 Camera 162, 164 microphones 166 Impact Sensor 168 Radar 170 Rider 172a Operational control components, operational control elements 172b Navigation components, navigation units 172c Sensor, Vehicle Sensor 200 Fraud Management System, Fleet Management System, Fraud Management System Stack, Autonomous Vehicle System Stack 202 Radar Perception Layer, Layer 204 Camera Perception Layer, Layer 206 Positioning Engine Layer, Layer 208 Map Fusion and Mediation Layers, Layers 210 Route Planning Layer, Layer 212 Sensor Fusion and Road World Model (RWM) Management Layer,Layer,Sensor Fusion and RWM Management Layer 214 Operation Planning and Control Layer, Layer 216 Behavior Planning and Prediction Layer, Layer 220 Drive-by-wire (DBW) system / control unit, DBW system / control unit 230 Vehicle wireless communication subsystem, wireless communication subsystem 250 Fleet Management System, Fraud Management System Stack 252 Vehicle Safety and Crash Avoidance Systems 300 Processing Device System on Chip (SOC), Processing Device SOC, SOC 303 Digital Signal Processor (DSP), Heterogeneous Processor, Processor, DSP 304 Modem processor, heterogeneous processor, processor 305 Camera Operation and Management Processor 306 Image and object recognition processors, heterogeneous processors, graphics processors, processors 307 Mobile display processor, heterogeneous processor, processor 308 Application Processors, Heterogeneous Processors, Processors 310 Coprocessor 312 Memory Elements 314 Analog and Custom Circuit Configurations, Analog and Custom Circuit Configurations 316 System Components and Resources 317 Resource and Power Management (RPM) Processors, Heterogeneous Processors, RPM Processors 318 Clock 320 Voltage Regulator 324 Interconnect / Bus Modules 400 system, V2X system 402 Computing platform, V2X equipment 404 other V2X system participants, other specific V2X system participants, another specific V2X system participant, other V2X system participant devices, a given V2X system participant, another V2X system participant 406 Machine Executable Instructions 408 LDM data receiving module, module 410 LDM Data Integration Module, Module 412 LDM data determination module, module 414 LDM data provision module, module 416 Map Generation Module, Module 418 Map transmission module, module 430 External Resources 432 Memory, Electronic Storage Devices, Storage Devices 700 Mobile Computing Devices 704 Touch Screen Controller 706 Internal Memory 708 Radio signal transceiver, transceiver 710, 808 Antenna 712 Touch Screen Panel 714 Speaker 716 Cellular Network Wireless Modem Chip 718 Peripheral device connection interface 720 Housing 722 Power supply 724 Physical Buttons 726 Power Button 800 Laptop Computers, Computers 812, 902 Volatile Memory 813, 904 Disk Drive 814 Floppy Disk Drive 815 Compact Disc (CD) Drive 816 Wireless Data Link and / or Cellular Telephone Transceivers 817 Touchpad touch surface, touchpad 818 Keyboard 819 Display 900 Server 901 Multi-core Processor Assembly 907 Network Access Port 908 Network
Claims
1. 1. A method for detecting a misbehavior condition in a vehicle-to-everything (V2X) system executed by a processor in a vehicle, the method comprising: receiving a V2X message from another V2X system participant, the V2X message including data regarding an environment surrounding the vehicle; comparing data contained in the received V2X message with a locally maintained or stored Local Dynamic Map (LDM) data model to detect a misbehavior condition; generating a fraud report in response to detecting the fraud condition based on the comparison, the fraud report identifying the fraud condition; sending the generated fraud report to a fraud management authority; The method includes:
2. monitoring a plurality of sensors within the vehicle to collect additional data regarding an environment surrounding the vehicle; generating the LDM data model representing the environment surrounding the vehicle based at least in part on an aggregation of the additional data collected from the plurality of sensors; storing the LDM data model in a memory; The method of claim 1, further comprising:
3. In response to determining that a fraud condition is not detected, performing calculations based on at least one of observed dynamics of objects in the LDM data model or new data inputs received from the V2X messages; modifying the LDM data model to incorporate the calculation and data contained in the received V2X message; replacing the LDM data model maintained or stored in memory with the modified LDM data model; The method of claim 1, further comprising:
4. The method of claim 1 , wherein transmitting the generated fraud report to a fraud management authority comprises transmitting a representation of the LDM data model.
5. The method of claim 4 , wherein the representation of the LDM data model includes an incomplete data set for the LDM data model.
6. The method of claim 1 , further comprising the step of receiving feedback from the fraud management authority, the feedback including corrective actions to mitigate the fraud condition.
7. The method of claim 1 , wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes traffic information.
8. 2. The method of claim 1, wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes location information of adjacent vehicles based on global navigation satellite system data.
9. 2. The method of claim 1, wherein the data regarding the environment surrounding the vehicle included in the received V2X message includes map data specifying road geometry and street furniture.
10. 2. The method of claim 1, wherein comparing data included in the received V2X message to the locally maintained or stored LDM data model to detect a misbehavior condition comprises determining whether any data included in the received V2X message is inconsistent with information in the locally maintained or stored LDM data model.
11. comparing data contained in the received V2X message with the locally maintained or stored LDM data model to detect a misbehavior condition; selecting a subset of data elements within the locally maintained or stored LDM data model for comparison with data included in the received V2X message; determining whether any data included in the received V2X message conflicts with the selected subset of data elements in the locally maintained or stored LDM data model; 2. The method of claim 1, comprising:
12. 2. The method of claim 1, wherein comparing data contained in the received V2X message to the locally maintained or stored LDM data model to detect a misbehavior condition comprises determining whether status or location information of an adjacent vehicle that transmitted the received V2X message is inconsistent with status or location information of the adjacent vehicle in the locally maintained or stored LDM data model.
13. 1. A vehicle-to-everything (V2X) processing device, comprising: a processor, the processor comprising: receiving a V2X message from another V2X system participant, the V2X message including data regarding an environment surrounding a vehicle in which the V2X processing device is installed; comparing data contained in the received V2X message with a locally maintained or stored Local Dynamic Map (LDM) data model to detect a misbehavior condition; generating a fraud report in response to detecting the fraud condition based on the comparison, the fraud report identifying the fraud condition; and Transmitting said generated fraud report to a fraud management agency. a V2X processing device comprising processor-executable instructions for:
14. A V2X processing device as described in claim 13, further configured to execute a method according to any one of claims 2 to 12.
15. A non-transitory computer-readable recording medium storing processor-executable instructions configured to cause a processor of a V2X device to perform a method according to any one of claims 1 to 12.
