System and method for providing data streams for sensor sharing

Abstract

The present invention describes systems and methods for providing data streams for sensor sharing. According to one embodiment, a device configured for use in a host vehicle includes a memory and a processor. The processor includes a receiving module, a metadata module, a grouping module, and a transmission module. The receiving module is configured to receive sensor data from at least one vehicle sensor of the host vehicle. The metadata module is configured to generate metadata for the sensor data. The grouping module is configured to form metadata packets including the metadata and sensor data packets including the sensor data. The transmission module is configured to transmit the metadata packets at a first carrier frequency and transmit the sensor data packets at a second carrier frequency different from the first carrier frequency.

Description

Background Art

[0001] Different services and applications have diverse network requirements and limitations. Generally speaking, 5G is based on 3GPP Long Term Evolution (LTE) Advanced (LTE-Adv) and potentially other new radio access technologies (RATs) to enrich people's lives with better, simpler, and more seamless wireless connectivity solutions. However, while these technologies offer high throughput capabilities, they are subject to high mobility scenarios, such as vehicular communications. Summary of the Invention

[0002] According to one aspect, a device configured for use in a host vehicle includes a memory and a processor. The processor includes a receiving module, a metadata module, a grouping module, and a transmission module. The receiving module is configured to receive sensor data from at least one vehicle sensor of the host vehicle. The metadata module is configured to generate metadata for the sensor data. The grouping module is configured to form metadata packets including the metadata and sensor data packets including the sensor data. The transmission module is configured to transmit the metadata packets at a first carrier frequency and transmit the sensor data packets at a second carrier frequency different from the first carrier frequency.

[0003] According to another aspect, a computer-implemented method for providing a data stream for sensor sharing is provided. The method includes receiving sensor data from at least one vehicle sensor of a host vehicle. The method also includes generating metadata for the sensor data. The method also includes forming a metadata packet including the metadata and a sensor data packet including the sensor data. The method includes transmitting the metadata packet at a first carrier frequency and transmitting the sensor data packet at a second carrier frequency different from the first carrier frequency.

[0004] According to yet another aspect, a device configured to be employed in a host vehicle includes a memory and a processor. The processor includes a receiving module, a metadata module, a grouping module, and a transmission module. The receiving module is configured to receive sensor data from at least one vehicle sensor of the host vehicle. The metadata module is configured to generate metadata for the sensor data. The grouping module is configured to form a metadata packet including the metadata and a sensor data packet including the sensor data. The transmission module is configured to transmit the metadata packet at a first carrier frequency of a spectrum associated with a fifth generation (5G) and transmit the sensor data packet at a second carrier frequency of a spectrum different from the first carrier frequency. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram illustrating an exemplary network system with a vehicle and eNB / gNB in ​​a core network that may be used in conjunction with various aspects described herein.

[0006] Figure 2 is a diagram illustrating example components of a network device, such as a vehicle or eNB / gNB, that may be employed in accordance with various aspects discussed herein.

[0007] Figure 3 is a diagram illustrating an example interface of baseband circuitry that may be employed in accordance with various aspects discussed herein.

[0008] Figure 4 is a block diagram illustrating a system for providing a data stream employable at a vehicle for sensor sharing in accordance with various aspects described herein.

[0009] Figure 5 is a block diagram illustrating a system that may be employed at a base station (BS) / evolved node (eNB) / new radio / next generation node (gNB) that implements beam reporting and beam management procedures in accordance with various aspects described herein.

[0010] Figure 6 is a flow diagram illustrating an example for providing data streams for sensor sharing according to various aspects described herein.

[0011] Figure 7 is a schematic diagram of a host vehicle's sensing area on a road, according to one embodiment.

[0012] Figure 8 is a schematic diagram of an exemplary traffic scene on a road according to one embodiment.

[0013] Figure 9 is a flow diagram illustrating an example for advertising data streams for sensor sharing in accordance with various aspects described herein. DETAILED DESCRIPTION

[0014] To achieve high throughput and long-range communications while utilizing low-frequency communications, the systems and methods described herein utilize directional beamforming technology. For example, a base station can advertise its participation in large-scale sensor sharing. Vehicles can confirm their ability to participate in this sensor sharing and their location and path prediction information, such as navigation, trajectory, and / or orientation data. The base station can then determine the time, frequency, and / or beam training information to share. For example, the base station can indicate the specific frequency to use for communication with a vehicle at a given time.

[0015] Using the communication parameters set by the base station, vehicles close to the base station can communicate with each other. Communication between vehicles (including host vehicles and remote vehicles) can be facilitated by fifth-generation cellular network (5G) technology. For example, raw data from the host vehicle can be processed, analyzed, or used by the remote vehicle. Thus, the host vehicle can transmit raw sensor data as well as metadata required to process the sensor data of the host vehicle. Different types of data are sent using different networks. For example, raw sensor data can be sent using millimeter waves, while metadata can be sent using sub-6GHz. Therefore, the systems and methods herein may use a 5G New Radio (NR) system that operates in different spectrums based on the type of data being sent, such as operating in sub-6GHz spectrum and millimeter waves.

[0016] The remote vehicle can then perform sensor fusion to determine the correspondence between the received sensor data and the metadata. Furthermore, the remote vehicle can use the metadata to authenticate the sensor data. In another embodiment, previously received metadata can be used to authenticate and analyze incoming sensor data from the host vehicle. Thus, metadata can be used to identify and secure communications between the host vehicle and the remote vehicle.

[0017] definition

[0018] The following includes definitions of selected terms used herein. The definitions include various examples and / or forms of components that fall within the scope of a term and that can be used in embodiments. These examples are not intended to be limiting. Furthermore, one of ordinary skill in the art will appreciate that the components discussed herein can be combined, omitted, or organized with other components or into different architectures.

[0019] As used herein, "bus" refers to an interconnection architecture that is operatively connected to other computer components within a computer or between computers. A bus can transfer data between computer components. A bus can be a memory bus, a memory processor bus, a peripheral bus, an external bus, a crossbar switch, and / or a local bus, among others. A bus can also be a vehicle bus that interconnects components within a vehicle using protocols such as Media Oriented Systems Transport (MOST), Controller Area Network (CAN), or Local Interconnect Network (LIN).

[0020] As used herein, "component" refers to a computer-related entity (e.g., hardware, firmware, instructions in execution, and combinations thereof). A computer component may include, for example, a process running on a processor, a processor, an object, an executable, a thread of execution, and a computer. A computer component may reside within a process and / or thread. A computer component may be localized on one computer and / or distributed across multiple computers.

[0021] As used herein, "computer communication" refers to communication between two or more communication devices (e.g., computers, personal digital assistants, cellular phones, network devices, vehicles, vehicle computing devices, infrastructure devices, roadside equipment), and can be, for example, network transmissions, data transmissions, file transmissions, applet transmissions, email, Hypertext Transfer Protocol (HTTP) transmissions, etc. Computer communication can occur over any type of wired or wireless system and / or network having any type of configuration, for example, a local area network (LAN), a personal area network (PAN), a wireless personal area network (WPAN), a wireless network (WAN), a wide area network (WAN), a metropolitan area network (MAN), a virtual private network (VPN), a cellular network, a token ring network, a point-to-point network, an ad hoc network, a mobile ad hoc network, a vehicle ad hoc network (VANET), a vehicle-to-vehicle (V2V) network, a vehicle-to-everything (V2X) network, a vehicle-to-infrastructure (V2I) network, etc. Computer communications may utilize any type of wired, wireless, or network communication protocol, including but not limited to Ethernet (e.g., IEEE 802.3), WiFi (e.g., IEEE 802.11), Communication Access for Land Mobile Devices (CALM), WiMax, Bluetooth, Zigbee, Ultra-Wideband (UWAB), Multiple-Input Multiple-Output (MIMO), telecommunication and / or cellular network communications (e.g., SMS, MMS, 3G, 4G, LTE, 5G, GSM, CDMA, WAVE), satellite, dedicated short range communications (DSRC), etc.

[0022] As used herein, a "communication interface" may include input and / or output devices for receiving input and / or devices for outputting data. The input and / or output may be used to control different vehicle features, including various vehicle components, systems, and subsystems. Specifically, the term "input device" includes, but is not limited to, keyboards, microphones, pointing and selecting devices, cameras, imaging devices, video cards, displays, buttons, knobs, and the like. The term "input device" further includes graphical input controls that occur within a user interface, which may be displayed by various types of mechanisms, such as software and hardware-based controls, interfaces, touch screens, touchpads, or plug-and-play devices. "Output device" includes, but is not limited to, display devices and other devices for outputting information and functionality.

[0023] As used herein, "computer-readable medium" refers to a non-transitory medium that stores instructions and / or data. Computer-readable media can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media can include, for example, optical disks, magnetic disks, etc. Volatile media can include, for example, semiconductor memories, dynamic memories, etc. Common forms of computer-readable media can include, but are not limited to, floppy disks, flexible magnetic disks, hard disks, magnetic tapes, other magnetic media, ASICs, CDs, other optical media, RAM, ROM, memory chips or cards, memory sticks, and other media that can be read from by a computer, processor, or other electronic device.

[0024] As used herein, "database" is used to refer to a table. In other examples, "database" can be used to refer to a set of tables. In other examples, "database" can refer to a set of data repositories and methods for accessing and / or manipulating those data repositories. A database can be stored, for example, on disk, in a data repository, and / or in memory.

[0025] As used herein, a "data repository" may be, for example, a magnetic disk drive, a solid-state hard drive, a floppy disk drive, a tape drive, a Zip drive, a flash memory card, and / or a memory stick. Furthermore, a disk may be a CD-ROM (Compact Disc ROM), a CD-Recordable Drive (CD-R Drive), a CD-Rewritable Drive (CD-RW Drive), and / or a Digital Video ROM Drive (DVDROM). The disk may store an operating system that controls or allocates resources of a computing device.

[0026] As used herein, a "display" may include, but is not limited to, LED display panels, LCD display panels, CRT displays, plasma display panels, touch screen displays, and the like commonly found in vehicles for displaying information about the vehicle. The display may receive input from a user (e.g., touch input, keyboard input, input from various other input devices, etc.). The display may be accessed through various devices, for example, through a remote system. The display may also be physically located on a portable device, mobile device, or vehicle.

[0027] As used herein, "logic circuitry" includes, but is not limited to, hardware, firmware, a non-transitory computer-readable medium storing instructions, instructions executed on a machine, and / or instructions for causing (e.g., executing) actions from another logic circuit, module, method, and / or system. A logic circuit may include and / or be part of a processor controlled by an algorithm, discrete logic (e.g., an ASIC), an analog circuit, a digital circuit, a programmed logic device, a memory device containing instructions, or the like. A logic may include one or more gates, combinations of gates, or other circuit components. Where multiple logics are described, the multiple logics may be merged into one physical logic. Similarly, where a single logic is described, the single logic may be distributed among multiple physical logics.

[0028] As used herein, "memory" may include volatile memory and / or non-volatile memory. Non-volatile memory may include, for example, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programm ...

[0029] As used herein, a "module" includes, but is not limited to, a non-transitory computer-readable medium storing instructions, instructions executed on a machine, hardware, firmware, software executed on a machine, and / or combinations thereof to perform a function or action, and / or cause a function or action from another module, method, and / or system. A module may also include logic, a software-controlled microprocessor, a discrete logic circuit, an analog circuit, a digital circuit, a programmed logic device, a memory device containing execution instructions, logic gates, combinations of gates, and / or other circuit components. Multiple modules may be combined into one module, and a single module may be distributed among multiple modules.

[0030] As used herein, an "obstacle" refers to any object in a roadway and may include pedestrians, other vehicles, animals, debris, potholes, etc. crossing the roadway. Furthermore, an "obstacle" may include most any traffic condition, road condition, weather condition, building, landmark, obstacle in a roadway, road segment, intersection, etc. Thus, obstacles may be identified, detected, or associated with a path, etc., along a route that a vehicle is traveling on or is expected to travel along.

[0031] An "operable connection" or a connection that "operably connects" entities is a connection that can send and / or receive signals, physical communications, and / or logical communications. An operable connection may include a wireless interface, a physical interface, a data interface, and / or an electrical interface.

[0032] As used herein, a "portable device" is a computing device that typically has a display screen with user input (e.g., touch, keyboard) and a processor for computing. Portable devices include, but are not limited to, handheld devices, mobile devices, smartphones, laptops, tablets, e-readers, smart speakers. In some embodiments, a "portable device" may refer to a remote device that includes a processor for computing and / or a communication interface for remotely receiving and transmitting data.

[0033] As used herein, a "processor" processes signals and performs general computational and arithmetic functions. The signals processed by a processor may include digital signals, data signals, computer instructions, processor instructions, messages, bits, bit streams that can be received, transmitted, and / or detected. Generally speaking, a processor can be a variety of different processors, including multiple single-core and multi-core processors and coprocessors and other multiple single-core and multi-core processor and coprocessor architectures. A processor may include logic circuits that execute actions and / or algorithms.

[0034] As used herein, "value" and "level" may include, but are not limited to, numerical values ​​or other types of values ​​or levels, such as percentages, non-numerical values, discrete states, discrete values, continuous values, and the like. As used throughout this detailed description and in the claims, the term "value of X" or "level of X" refers to any numerical value or other type of value used to distinguish between two or more states of X. For example, in some cases, the value or level of X may be given as a percentage between 0% and 100%. In other cases, the value or level of X may be a value within a range between 1 and 10. In still other cases, the value or level of X may not be a numerical value, but may be associated with a given discrete state, such as "not X," "slightly x," "x," "very x," and "extremely x."

[0035] As used herein, "vehicle" refers to any mobile vehicle capable of carrying one or more users and driven by any form of energy. The term "vehicle" includes, but is not limited to, cars, trucks, vans, minivans, SUVs, motorcycles, scooters, boats, go-karts, recreational vehicles, rail transport, personal watercraft, and airplanes. In some cases, a motor vehicle includes one or more engines. In addition, the term "vehicle" may refer to an electric vehicle (EV) capable of carrying one or more users and powered entirely or partially by one or more motors powered by batteries. EV may include battery-powered electric vehicles (BEVs) and plug-in hybrid electric vehicles (PHEVs). The term "vehicle" may also refer to an autonomous vehicle and / or self-driving vehicle powered by any form of energy. An autonomous vehicle may carry one or more users. In addition, the term "vehicle" may include an automatic or non-automatic vehicle that drives along a predetermined path or drives freely.

[0036] As used herein, a "vehicle occupant" may include, but is not limited to, one or more living beings located in a vehicle. A vehicle occupant may be a driver or a passenger of the vehicle. A vehicle occupant may be a person (e.g., an adult, a child, an infant) or an animal (e.g., a pet, a dog, a cat).

[0037] As used herein, a "vehicle system" may include, but is not limited to, any automatic or manual system that can be used to enhance vehicle, driving, and / or safety. Exemplary vehicle systems include, but are not limited to, electronic stability control systems, anti-lock braking systems, brake assist systems, automatic brake pre-fill systems, low-speed follow systems, cruise control systems, collision warning systems, collision mitigation braking systems, automatic cruise control systems, lane departure warning systems, blind spot indicator systems, lane keeping assist systems, navigation systems, steering systems, transmission systems, brake pedal systems, electronic power steering systems, vision devices (e.g., camera systems, proximity sensor systems), climate control systems, electronic preload systems, monitoring systems, passenger detection systems, vehicle suspension systems, vehicle seat configuration systems, vehicle cabin lighting systems, audio systems, sensory systems, interior or exterior camera systems, and the like.

[0038] I. System Overview

[0039] Reference is now made to the drawings, which are for purposes of illustrating one or more exemplary embodiments and not for purposes of limiting these embodiments. Figure 1 is an exemplary component diagram of an operating environment according to some embodiments, Figure 1A network architecture 100 of a network is shown. The network architecture 100 is shown as including a first vehicle 101 and a second vehicle 102. Vehicles 101 and 102 are shown as vehicles, but may also include other forms of user equipment (UE), such as a smartphone (e.g., a handheld touchscreen mobile computing device that can connect to one or more cellular networks), or any mobile or non-mobile computing device, such as a personal data assistant (PDA), a pager, a laptop computer, a desktop computer, a wireless handheld device, or any computing device that includes a wireless communication interface.

[0040] In some embodiments, either vehicle 101 or 102 may include an Internet of Things (IoT) UE, which may include a network access layer designed for low-power IoT applications that utilize ephemeral UE connections. The IoT UE may utilize technologies such as machine-to-machine (M2M) or machine-type communication (MTG) to exchange data with an MTG server or device via a public land mobile network (PLMN), proximity services (ProSe), or device-to-device communication, a sensor network, or an IoT network. M2M or MTG data exchanges may be machine-initiated data exchanges. The IoT network describes IoT vehicles interconnected using ephemeral connections, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure). The IoT vehicles may execute background applications (e.g., keep-alive messages, status updates, etc.) to facilitate connectivity to the IoT network.

[0041] Vehicles 101 and 102 may be configured to connect, e.g., be communicatively coupled, to a radio access network (RAN) 110, which may be, for example, an evolved universal mobile telecommunications system (UMTS), an terrestrial radio access network (E-UTRAN), a next generation RAN (NG RAN), or some other type of RAN. Vehicles 101 and 102 utilize connections 103 and 104, respectively, each of which includes a physical communication interface or layer (discussed in further detail below); in this example, connections 103 and 104 are shown as air interfaces to achieve communicative coupling and may be consistent with a cellular communication protocol, such as a global system for mobile communications (GSM) protocol, a code division multiple access (CDMA) network protocol, a push-to-talk (PTT) protocol, a PTT over cellular (POC) protocol, a universal mobile telecommunications system (UMTS) protocol, a 3GPP long term evolution (LTE) protocol, a fifth generation (5G) protocol, a new radio (NR) protocol, or the like.

[0042] In this embodiment, the vehicles 101 and 102 may also directly exchange communication data via the Prose interface 105. The Prose interface 105 may alternatively be referred to as a sidelink interface, which includes one or more logical channels, including but not limited to a physical sidelink control channel (PSCCH), a physical sidelink shared channel (PSSCH), a physical sidelink discovery channel (PSDCH), and a physical sidelink broadcast channel (PSBCH).

[0043] The second vehicle 102 is shown as being configured to access an access point (AP) 106 via a connection 107. The connection 107 may comprise a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein the AP 106 would include Wi-Fi. In this example, AP 106 is shown connected to the Internet without being connected to the core network of the wireless system (described in further detail below).

[0044] The RAN 110 may include one or more access nodes that implement connections 103 and 104. These access nodes (ANs) may be referred to as base stations (BSs), Node Bs, evolved Node Bs (eNBs), next-generation Node Bs (gNBs), RAN nodes, etc., and may include ground stations (e.g., terrestrial access points) or satellite stations that provide coverage within a geographic area (e.g., a cell). The RAN 110 may include one or more RAN nodes for providing macro cells, such as the macro RAN node 111, and one or more RAN nodes for providing femto cells or pico cells (e.g., cells with smaller coverage areas, smaller user capacity, or higher bandwidth than macro cells), such as the low power (LP) RAN node 112.

[0045] Any of the RAN nodes 111 and 112 may terminate the air interface protocol and may be the first point of contact for the vehicles 101 and 102. In some embodiments, any of the multiple RAN nodes 111 and 112 may perform various logical functions of the RAN 110, including but not limited to functions of a radio network controller (RNC), such as radio bearer management, uplink and downlink dynamic radio resource management and data packet scheduling, and mobility management.

[0046] According to some embodiments, vehicles 101 and 102 may be configured to communicate with each other or with any of RAN nodes 111 and 112 using orthogonal frequency division multiplexing (OFDM) communication signals over a multi-carrier communication channel according to various communication techniques, such as, but not limited to, orthogonal frequency division multiple access (OFDMA) communication techniques (e.g., for downlink communications) or single-carrier frequency division multiple access (SC-FDMA) communication techniques (e.g., for uplink and ProSe or sidelink communications), although the scope of the embodiments is not limited in this respect. The OFDM signal may include multiple orthogonal subcarriers.

[0047] In some embodiments, a downlink resource grid can be used for downlink transmissions from any RAN node 111 and 112 to vehicles 101 and 102, while uplink transmissions can utilize similar techniques. The grid can be a time-frequency grid, called a resource grid or time-frequency resource grid, which represents the physical resources in the downlink in each time slot. This time-frequency plane representation is a convention in OFDM systems, making radio resource allocation intuitive and easy to understand. Each column and row of the resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. The duration of the resource grid in the time domain corresponds to a time slot in a radio frame. The smallest time-frequency unit in the resource grid is represented as a resource element. Each resource grid includes multiple resource blocks, which describe the mapping of specific physical channels to resource elements. Each resource block includes a collection of resource elements; in the frequency domain, this can represent the minimum number of resources currently allocable. There are several different physical downlink channels that are transported using such resource blocks.

[0048] The physical downlink shared channel (POSCH) can carry user data and higher-layer signaling to vehicles 101 and 102. The physical downlink control channel (PDCCH) can carry information about, among other things, the transport format and resource allocation associated with the POSCH channel. It can also inform vehicles 101 and 102 about the transport format, resource allocation, and H-ARQ (Hybrid Automatic Repeat Request) information associated with the uplink shared channel. Typically, downlink scheduling (allocation of control and shared channel resource blocks to the second vehicle 102 within the cell) can be performed at either RAN node 111 or 112 based on channel quality information fed back from either vehicle 101 or 102. Downlink resource allocation information can be sent on the POCCH for (e.g., allocated to) each of vehicles 101 and 102.

[0049] POCCH can use control channel elements (CCE) to convey control information. Before being mapped to resource elements, POCCH complex-valued symbols can first be organized into quadruples, which can then be arranged using a sub-block interleaver for rate matching. One or more of these CCEs can be used to transmit each POCCH, where each CCE corresponds to a set of nine four physical resource elements called resource element groups (REGs). Four quadrature phase shift keying (QPSK) symbols can be mapped to each REG. Depending on the size of the downlink control information (OCI) and the channel conditions, one or more CCEs can be used to transmit POCCH. Four or more different POCCH formats with different numbers of CCEs (e.g., aggregation levels, L=1, 2, 4, or 8) are defined in LTE.

[0050] Some embodiments may use the concept of resource allocation for control channel information, which is an extension of the above concept. For example, some embodiments may utilize an enhanced physical downlink control channel (EPOCCH) that uses POSCH resources for control information transmission. EPOCCH can be transmitted using one or more enhanced control channel elements (ECCEs). Similar to the above, each EGGE may correspond to a set of nine four physical resource elements called an enhanced resource element group (EREG). In some cases, an EGGE may have other numbers of EREGs.

[0051] RAN 110 is shown as being communicatively coupled to a core network (CN) 120 via an S1 interface 113. In an embodiment, CN 120 may be an evolved packet core (EPC) network, a next generation packet core (NPC) network, or some other type of CN. In this embodiment, S1 interface 113 may be divided into two parts: an S1-U interface 114, which carries traffic data between RAN nodes 111 and 112 and a serving gateway (S-GW) 122; and an S1 mobility management entity (MME) interface 115, which is a signaling interface between RAN nodes 111 and 112 and MME 121.

[0052] In this embodiment, CN 120 includes MME 121, S-GW 122, Packet Data Network (PON) Gateway (P-GW) 123, and Home Subscriber Server (HSS) 124. MME 121 can be functionally similar to the control plane of a conventional Serving General Packet Radio Service (GPRS) Support Node (SGSN). MME 121 can manage mobility aspects of access, such as gateway selection and tracking area list management. HSS 124 can include a database for network users, which includes subscription-related information used to support network entities handling communication sessions. CN 120 can include one or several HSSs 124, depending on the number of mobile subscribers, equipment capacity, network organization, etc. For example, HSS 124 can provide support for routing / roaming, authentication, authorization, naming and / or addressing solutions, location dependencies, etc.

[0053] The S-GW 122 may terminate the S1 interface 113 towards the RAN 110 and route data packets between the RAN 110 and the CN 120. Additionally, the S-GW 122 may be the local mobility anchor for handovers between RAN nodes and may also provide an anchor for inter-3GPP mobility. Other responsibilities may include lawful interception, charging, and some policy enforcement.

[0054] The P-GW 123 may terminate the SGi interface toward the PON. The P-GW 123 may route data packets between the EPC network and an external network, such as a network including an application server 130 (alternatively referred to as an application function (AF), via an Internet Protocol (IP) interface 125. Typically, the application server 130 may be an element that provides applications using IP bearer resources utilizing a core network (e.g., a UMTS packet service (PS) domain, an LTE PS data service, etc.). In this embodiment, the P-GW 123 is shown as being communicatively coupled to the application server 130 via the IP communication interface 125. The application server 130 may also be configured to support one or more communication services (e.g., voice over internet protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, etc.) for the vehicles 101 and 102 via the CN 120.

[0055] P-GW 123 may also be a node for policy enforcement and charging data collection. Policy and Charging Enforcement Function (PCRF) 126 is the policy and charging control element of CN 120. In a non-roaming scenario, there may be a single PCRF in the Home Public Land Mobile Network (HPLMN) associated with the UE's Internet Protocol Connectivity Access Network (IP-CAN) session. In a roaming scenario with local traffic breakout, there may be two PCRFs associated with the UE's IP-CAN session: a Home PCRF (H-PCRF) in the HPLMN and a Visited PCRF (V-PCRF) in the Visited Public Land Mobile Network (VPLMN). PCRF 126 may be communicatively coupled to application server 130 via P-GW 123. Application server 130 may signal PCRF 126 to indicate a new service flow and select appropriate quality of service (QoS) and charging parameters. PCRF 126 may provision the rules in a Policy and Charging Enforcement Function (PCEF) (not shown) with the appropriate Traffic Flow Template (TFT) and QoS Class Identifier (QCI), which initiates QoS and charging as specified by application server 130 .

[0056] Figure 2 1 shows exemplary components of a device 200 according to some embodiments. In some embodiments, the device 200 may include at least application circuitry 202, baseband circuitry 204, radio frequency (RF) circuitry 206, front-end module (FEM) circuitry 208, one or more antennas 210, and power management circuitry (PMC) 212 coupled together as shown. The components of the device 200 shown may be included in a UE or a RAN node. In some embodiments, the device 200 may include fewer components (e.g., a RAN node may not utilize application circuitry 202 but instead include a processor / controller to process IP data received from an EPC). In some embodiments, the device 200 may include additional components such as memory / storage, a display, a camera, sensors, or input / output (I / O) interfaces. In other embodiments, the components described below may be included in more than one device (e.g., for a cloud-RAN (C-RAN) embodiment, circuitry may be individually included in more than one device).

[0057] The application circuitry 202 may include one or more application processors. For example, the application circuitry 202 may include circuitry such as, but not limited to, one or more single-core or multi-core processors. The processors may include any combination of general-purpose processors and specialized processors (e.g., graphics processors, application processors, etc.). The processors may be coupled to or include a memory / storage device and may be configured to execute instructions stored in the memory / storage device to enable various applications or operating systems to run on the device 200. In some embodiments, the processors of the application circuitry 202 may process IP data packets received from the EPC.

[0058] The baseband circuitry 204 may include circuitry such as, but not limited to, one or more single-core or multi-core processors. The baseband circuitry 204 may include one or more baseband processors or control logic to process baseband signals received from the receive signal path of the RF circuitry 206 and generate baseband signals for the transmit signal path of the RF circuitry 206. The baseband circuitry 204 may interface with the application circuitry 202 for baseband signal generation and processing and for controlling operations of the RF circuitry 206. For example, in some embodiments, the baseband circuitry 204 may include a third-generation (3G) baseband processor 204A, a fourth-generation (4G) baseband processor 204B, a fifth-generation (5G) baseband processor 204C, or other baseband processors 204D of other existing generations, generations under development, or generations to be developed in the future (e.g., second generation (2G), sixth generation (6G), etc.). The baseband circuitry 204 (e.g., one or more of the baseband processors 204A-204D) may handle various radio control functions that enable communication with one or more radio networks via the RF circuitry 206. In other embodiments, some or all of the functionality of baseband processors 204A-204D may be included in modules stored in memory 204G and executed via central processing unit (CPU) 204E. Radio control functions may include, but are not limited to, signal modulation / demodulation, encoding / decoding, radio frequency shifting, etc. In some embodiments, the modulation and / or demodulation circuitry of baseband circuitry 204 may include fast Fourier transform (FFT), precoding or constellation mapping and / or demapping functionality. In some embodiments, the encoding / decoding circuitry of baseband circuitry 204 may include convolution, tail-biting, turbo, Viterbi, or low-density parity check (LDPC) encoder / decoder functionality. The implementation of modulation and / or demodulation and encoder and / or decoder functionality is not limited to these examples and may include other suitable functionality in other embodiments.

[0059] In some embodiments, the baseband circuitry 204 may include one or more audio digital signal processors (DSPs) 204F. The audio DSPs 204F may include components for compression and / or decompression and echo cancellation, and in other embodiments may include other suitable processing elements. In some embodiments, the components of the baseband circuitry may be appropriately combined in a single chip, a single chipset, or disposed on the same circuit board. In some embodiments, some or all of the components of the baseband circuitry 204 and the application circuitry 202 may be implemented together, for example, on a system on a chip (SOC).

[0060] In some embodiments, baseband circuitry 204 can provide communications compatible with one or more radio technologies. For example, in some embodiments, baseband circuitry 204 can support communications with an Evolved Universal Terrestrial Radio Access Network (EUTRAN) or other wireless metropolitan area network (WMAN), wireless local area network (WLAN), or wireless personal area network (WPAN). Embodiments in which baseband circuitry 204 is configured to support radio communications using more than one wireless protocol may be referred to as multi-mode baseband circuitry.

[0061] RF circuitry 206 can use modulated electromagnetic radiation passing through a non-solid medium to facilitate communication with a wireless network. In various embodiments, RF circuitry 206 can include switches, filters, amplifiers, etc. to facilitate communication with the wireless network. RF circuitry 206 can include a receive signal path that can include circuitry for down-converting RF signals received from FEM circuitry 208 and providing baseband signals to baseband circuitry 204. RF circuitry 206 can also include a transmit signal path that can include circuitry for up-converting baseband signals provided by baseband circuitry 204 and providing an RF output signal to FEM circuitry 208 for transmission.

[0062] In some embodiments, the receive signal path of RF circuitry 206 may include mixer circuitry 206a, amplifier circuitry 206b, and filter circuitry 206c. In some embodiments, the transmit signal path of RF circuitry 206 may include filter circuitry 206c and mixer circuitry 206a. RF circuitry 206 may also include synthesizer circuitry 206d, which is used to synthesize frequencies used by mixer circuitry 206a in the receive and transmit signal paths. In some embodiments, mixer circuitry 206a in the receive signal path may be configured to downconvert the RF signal received from FEM circuitry 208 based on the synthesized frequency provided by synthesizer circuitry 206d. Amplifier circuitry 206b may be configured to amplify the downconverted signal, and filter circuitry 206c may be a low-pass filter (LPF) or a band-pass filter (BPF) configured to remove unwanted signals from the downconverted signal to generate an output baseband signal. The output baseband signal may be provided to baseband circuitry 204 for further processing. In some embodiments, the output baseband signal may be a zero-frequency baseband signal, although this is not required.In some embodiments, the mixer circuit 206a of the receive signal path may include a passive mixer, although the scope of the embodiments is not limited in this respect.

[0063] In some embodiments, mixer circuit 206a of the transmit signal path can be configured to upconvert an input baseband signal based on a synthesized frequency provided by synthesizer circuit 206d to generate an RF output signal for FEM circuit 208. The baseband signal can be provided by baseband circuit 204 and can be filtered by filter circuit 206c.

[0064] In some embodiments, the mixer circuit 206a of the receive signal path and the mixer circuit 206a of the transmit signal path may include two or more mixers and may be arranged for quadrature down-conversion and up-conversion, respectively. In some embodiments, the mixer circuit 206a of the receive signal path and the mixer circuit 206a of the transmit signal path may include two or more mixers and may be arranged for image rejection (e.g., Hartley image rejection). In some embodiments, the mixer circuit 206a of the receive signal path and the mixer circuit 206a of the transmit signal path may be arranged for direct down-conversion and direct up-conversion, respectively. In some embodiments, the mixer circuit 206a of the receive signal path and the mixer circuit 206a of the transmit signal path may be configured for superheterodyne operation.

[0065] In some embodiments, the output baseband signal and the input baseband signal may be analog baseband signals, although the scope of the embodiments is not limited in this respect. In some alternative embodiments, the output baseband signal and the input baseband signal may be digital baseband signals. In these alternative embodiments, RF circuitry 206 may include analog-to-digital converter (ADC) and digital-to-analog converter (DAG) circuitry, and baseband circuitry 204 may include a digital baseband interface to communicate with RF circuitry 206.

[0066] In some dual-mode embodiments, separate radio IC circuits may be provided to process signals for each spectrum, although the scope of the embodiments is not limited in this respect.

[0067] In some embodiments, synthesizer circuit 206 d can be a fractional-N synthesizer or a fractional-N / N+1 synthesizer, although the scope of the embodiments is not limited in this respect, as other types of frequency synthesizers may also be suitable. For example, synthesizer circuit 206 d can be a delta-sigma synthesizer, a frequency multiplier, or a synthesizer including a phase-locked loop with a frequency divider. Synthesizer circuit 206 d can be configured to synthesize an output frequency for use by mixer circuit 206 a of RF circuit 206 based on a frequency input and a frequency divider control input. In some embodiments, synthesizer circuit 206 d can be a fractional-N / N+1 synthesizer.

[0068] In some embodiments, the frequency input may be provided by a voltage controlled oscillator (VCO), although this is not required. The divider control input may be provided by baseband circuitry 204 or application circuitry 202 depending on the desired output frequency. In some embodiments, the divider control input (e.g., N) may be determined from a lookup table based on the channel indicated by application circuitry 202.

[0069] The synthesizer circuit 206d of the RF circuit 206 may include a frequency divider, a delay-locked loop (DLL), a multiplexer, and a phase accumulator. In some embodiments, the frequency divider may be a dual-mode frequency divider (DMD), and the phase accumulator may be a digital phase accumulator (DPA). In some embodiments, the DMD may be configured to divide the input signal by N or N+1 (e.g., based on the output) to provide a fractional frequency division ratio. In some exemplary embodiments, the DLL may include a set of cascaded tunable delay elements, a phase detector, a charge pump, and a type-0 flip-flop. In these embodiments, the delay elements may be configured to divide the VCO cycle into Nd equal phase groups, where Nd is the number of delay elements in the delay line. In this way, the DLL provides negative feedback to help ensure that the total delay through the delay line is one VCO cycle.

[0070] In some embodiments, the synthesizer circuit 206d can be configured to generate a carrier frequency as the output frequency, while in other embodiments, the output frequency can be a multiple of the carrier frequency (e.g., twice the carrier frequency, four times the carrier frequency) and used in conjunction with a quadrature generator and divider circuit to generate multiple signals at the carrier frequency with multiple different phases relative to each other. In some embodiments, the output frequency can be the LO frequency (fLO). In some embodiments, the RF circuit 206 can include an IQ / polarity converter.

[0071] The FEM circuitry 208 may include a receive signal path that may include circuitry configured to operate on RF signals received from one or more antennas 210, amplify the received signals, and provide an amplified version of the received signals to the RF circuitry 206 for further processing. The FEM circuitry 208 may also include a transmit signal path that may include circuitry configured to amplify signals provided by the RF circuitry 206 for transmission via one or more of the one or more antennas 210. In various embodiments, amplification by the transmit or receive signal path may be performed only in the RF circuitry 206, only in the FEM circuitry 208, or in both the RF circuitry 206 and the FEM circuitry 208.

[0072] In some embodiments, the FEM circuitry 208 may include a TX / RX switch to switch between transmit and receive modes of operation. The FEM circuitry may include a receive signal path and a transmit signal path. The receive signal path of the FEM circuitry may include an LNA to amplify a received RF signal and provide the amplified received RF signal as an output (e.g., to the RF circuitry 206). The transmit signal path of the FEM circuitry 208 may include a power amplifier (PA) to amplify an input RF signal (e.g., provided by the RF circuitry 206) and one or more filters to generate an RF signal for subsequent transmission (e.g., via one or more of the one or more antennas 210).

[0073] In some embodiments, PMC 212 can manage the power provided to baseband circuitry 204. Specifically, PMC 212 can control power source selection, voltage scaling, battery charging, or DC-DC conversion. PMC 212 is typically included when device 200 is capable of being powered by a battery, such as when the device is included in a UE. PMC 212 can increase power conversion efficiency while providing desired implementation size and heat dissipation characteristics.

[0074] Although Figure 2The PMC 212 is shown coupled only to the baseband circuitry 204. However, in other embodiments, the PMC 212 may additionally or alternatively be coupled to other components (such as, but not limited to, the application circuitry 202, the RF circuitry 206, or the FEM circuitry 208) and perform similar power management operations.

[0075] In some embodiments, the PMC 212 may control or otherwise be part of various power saving mechanisms of the device 200. For example, if the device 200 is in the RRC_Connected state, in which it remains connected to the RAN node for a short period of time because it expects to receive traffic, then after a period of inactivity, the device may enter a state known as discontinuous reception mode (DRX). During this state, the device 200 may be powered down for short intervals and thereby save power.

[0076] If there is no data traffic activity for an extended period of time, the device 200 may transition to the RRC_Idle state, in which the device is disconnected from the network and does not perform operations such as channel quality feedback, handover, etc. The device 200 enters a very low power state and performs paging, in which the device periodically wakes up again to listen to the network and then powers down again. In this state, the device 200 cannot receive data and, in order to receive data, must transition back to the RRC connected state.

[0077] An additional power saving mode can prevent the device from using the network for a period exceeding the paging interval (ranging from a few seconds to several hours). During this period, the device is completely unable to connect to the network and can be completely powered down. Any data sent during this period will incur significant delays, assuming that the delay is acceptable.

[0078] The processors of application circuitry 202 and baseband circuitry 204 may be used to execute elements of one or more instances of a protocol stack. For example, the processors of baseband circuitry 204 (alone or in combination) may be used to execute layer 3, layer 2, or layer 1 functions, while the processors of baseband circuitry 204 may utilize data received from these layers (e.g., packet data) and further execute layer 4 functions (e.g., a transport communication protocol (TCP) layer and a user datagram protocol (UDP) layer). As mentioned herein, layer 3 may include a radio resource control (RRC) layer, described in further detail below. As mentioned herein, layer 2 may include a medium access control (MAC) layer, a radio link control (RLC) layer, and a packet data convergence protocol (PDCP) layer, described in further detail below. As mentioned herein, layer 1 may include a physical (PHY) layer of a UE / RAN node, described in further detail below.

[0079] Figure 31 shows an exemplary interface of a baseband circuit according to some embodiments. As described above, Figure 2 The baseband circuit 204 may include processors 204A-204E and a memory 204G utilized by the processors. Each of the processors 204A-204E may include a memory interface 304A-304E, respectively, to send / receive data to / from the memory 204G.

[0080] The baseband circuit 204 may also include one or more interfaces, such as a memory interface 312 (e.g., an interface for sending / receiving data to / from a memory external to the baseband circuit 204), an application circuit interface 314 (e.g., an interface for sending / receiving data to / from a memory external to the baseband circuit 204), and an application circuit interface 315 (e.g., an interface for sending / receiving data to / from a memory external to the baseband circuit 204). Figure 2 an interface for transmitting / receiving data to / from the application circuit 202), an RF circuit interface 316 (for example, an interface for transmitting / receiving data to / from the application circuit 202), and an RF circuit interface 316 (for example, an interface for transmitting / receiving data to / from the application circuit 202). Figure 2 an interface for sending / receiving data to / from the RF circuit 206), a wireless hardware connectivity interface 318 (e.g., for sending / receiving data to / from a near field communication (NFC) component, part( Low power consumption), components and other communication components to send / receive data) and a power management interface 320 (eg, an interface for sending / receiving power or control signals to / from the PMC 212) to communicatively couple to other circuits / devices.

[0081] As discussed above, user equipment may include a vehicle, such as Figure 1 The first vehicle 101 and the second vehicle 102 are shown. The vehicles have an operating environment that allows them to provide data streams to each other and other vehicles for sensor sharing. As used herein, the host vehicle 400 refers to a vehicle that uses at least a portion of the operating environment. Therefore, the first vehicle 101 or the second vehicle 102 can be relative to Figure 4 The operating environment shown in FIG serves as a host vehicle 400 .

[0082] exist Figure 4 In the embodiment of the present invention, the host vehicle 400 includes a vehicle computing device (VCD) 402, a vehicle system 404, and a vehicle sensor 406. Generally speaking, the VCD 402 includes a processor 408, a memory 410, a disk 412, and an input / output (I / O) device 414, each of which is operatively connected to communicate with the host vehicle via a bus 416 and / or other wired and wireless technologies defined herein. The VCD 402 includes provisions for processing, communicating, and interacting with various components of the host vehicle and other components of the operating environment, including other vehicles.

[0083] In one embodiment, the VCD 402 may be implemented with the host vehicle, for example, as part of a telematics unit, a head unit, an infotainment unit, an electronic control unit, an onboard unit, or as part of a particular vehicle control system, etc. In other embodiments, the VCD 402 may be implemented remotely from the host vehicle, for example, with a portable device (not shown), a remote device (not shown), or a remote server 418 connected via the network architecture 100 .

[0084] The processor 408 may include processing / logic circuitry having a hardware, firmware, and software architecture framework for facilitating shared autonomy through collaborative autonomy and control of the host vehicle 400 and / or remote vehicles. For example, the first vehicle 101 may be the host vehicle 400 and the second vehicle 102 may be a remote vehicle. In other embodiments, the second vehicle 102 may be multiple remote vehicles.

[0085] The processor 408 may store application frameworks, core programs, libraries, drivers, application program interfaces, etc. to execute and control the hardware and functions discussed herein. For example, the processor 408 may include a receiving module 420, a metadata module 422, a packetization module 424, and a transmission module 426, although the processor 408 may be configured as other architectures. In addition, in some embodiments, the memory 410 and / or disk 412 may store similar components to those stored by the processor 408 for execution by the processor 408.

[0086] I / O devices 414 may include software and hardware to facilitate the input and output of data between components of VCD 402 and other components of the operating environment. Specifically, I / O devices 414 may include a network interface controller (not shown) and other hardware and software that manages and / or monitors connections between I / O devices 414 and other components of the operating environment using, for example, network architecture 100 and controls the bidirectional transfer of data therebetween.

[0087] As described above, in some embodiments, data transmission can be performed at and / or using other infrastructure and servers as described above. For example, VCD 402 can use the network architecture to directly or indirectly transmit and receive information such as road data or vehicle data to and from remote server 418. Remote server 418 may include a remote processor 428, a remote memory 430, remote data 432, and a communication interface 434 configured to communicate with each other. For example, VCD 402 can receive and transmit information from remote server 418 and other servers, processors, and information providers via network architecture 100. VCD 402 can receive and transmit information to and from remote server 418, including but not limited to vehicle data, traffic data, road data, curb data, vehicle location and heading data, high-traffic event schedules, weather data, or other transportation-related data. In some embodiments, remote server 418 can be linked to one or more vehicles, other entities, transportation infrastructure, and / or devices via network architecture 100. In this way, vehicles equipped for sensor sharing can communicate.

[0088] Referring again to host vehicle 400, vehicle systems 404 may include any type of vehicle control system and / or vehicle described herein to improve the host vehicle and / or the driving of the host vehicle. For example, vehicle systems 404 may include an autonomous driving system, a driver assistance system, an adaptive cruise control system, a lane departure warning system, a lane assist system, a highway merge, exit and lane change system, a collision warning system, an integrated onboard safety system and an automatically navigated vehicle system or any other advanced driver assistance system (ADAS). As will be described, one or more systems in vehicle systems 404 may be controlled according to the systems and methods discussed herein. Here, vehicle systems 404 include a navigation system 436 and an infotainment system 438. Navigation system 436 stores, calculates and provides route and destination information, and facilitates features such as turn-by-turn navigation. Infotainment system 438 provides visual information and / or entertainment to vehicle occupants and may include display 440.

[0089] Vehicle sensors 406 that may be implemented using vehicle system 404 may include various types of sensors for use with the host vehicle and / or vehicle system 404 to detect and / or sense parameters of the host vehicle, vehicle system 404, and / or the environment surrounding the host vehicle. For example, vehicle sensors 406 may include data regarding the host vehicle and / or downstream objects in proximity to the host vehicle. For example, vehicle sensors 406 may include, but are not limited to, accelerometers, speed sensors, brake sensors, proximity sensors, vision sensors, range sensors, seat sensors, seatbelt sensors, door sensors, environmental sensors, yaw rate sensors, steering sensors, GPS sensors, and the like. Vehicle sensors 406 may include any type of sensor, such as acoustic, electrical, environmental, optical, imaging, light, pressure, force, thermal, temperature, proximity, and the like.

[0090] Using the system and network configurations discussed above, onboard communication using sensor data can provide cooperative autonomy and vehicle control based on real-time information from the vehicles. Detailed embodiments describing exemplary methods using the above-described system and network configurations will now be discussed in detail.

[0091] refer to Figure 5 , shows a block diagram of a system 500 that can be employed at a BS (base station) according to various aspects described herein to facilitate mapping of one or more codewords to one or more MIMO layers of 5G NR. System 500 may include one or more processors 510 (e.g., one or more baseband processors, such as in conjunction with Figure 2 and / or Figure 3 One or more baseband processors discussed herein), the one or more processors including processing circuitry and associated memory interfaces (e.g., in conjunction with Figure 3); communication circuitry 520 (e.g., which may include circuitry for one or more wired (e.g., X2, etc.) connections and / or may include transceiver circuitry that includes transmitter circuitry (e.g., associated with one or more transmit chains) or receiver circuitry (e.g., associated with one or more receive chains), wherein the transmitter circuitry and the receiver circuitry may employ common circuit elements, different circuit elements, or a combination thereof); and memory 530 (which may include any of a variety of storage media and may store instructions and / or data associated with one or more processors in processor 510 or communication circuitry 520). In various aspects, system 500 may be included in an Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Node B (evolved Node B, eNodeB, or eNB), a next generation Node B (gNodeB or gNB), or other base stations in a wireless communication network. In some aspects, processor 510, communication circuitry 520, and memory 530 may be included in a single device, while in other aspects, they may be included in different devices, such as as part of a distributed architecture.

[0092] II. Method Overview

[0093] Now refer to Figure 6 , a method 600 for providing a data stream for sensor sharing will now be described according to an exemplary embodiment. Reference will also be made to Figures 1 to 5 、 Figure 7 and Figure 8 To describe Figure 6 .like Figure 6 As shown, the method for distance prediction can be described as three stages, namely (A) data management stage, (B) packaging stage, and (C) transmission stage. For simplicity, method 600 will be described as these stages, but it should be understood that the elements of method 600 can be organized into different architectures, blocks, stages and / or processes. For example, the reconstruction and prediction stages can be performed in parallel.

[0094] A. Data management stage

[0095] At block 602, the receiving module 420 receives sensor data from the vehicle sensors 406. The vehicle sensors 406 include at least one sensor for sensing objects and the surrounding environment surrounding the host vehicle 400. In an exemplary embodiment, the surrounding environment of the host vehicle 400 can be defined as a predetermined area around the host vehicle 400 (in front, to the sides, behind, above, below) and including the road environment in front of, to the sides, and / or behind the host vehicle 400 that may be within the vehicle's path. For example, the vehicle sensors 406 may include a light sensor 710 for capturing sensor data in a light sensing area 711; and one or more host image sensors 712a, 712b, 712c, 712d, 712e, and 712f for capturing sensor data in corresponding image sensing areas 713a, 713b, 713c, 713d, 713e, and 713f.

[0096] Light sensor 710 can be used to capture light data in a light sensing area 711. The size of light sensing area 711 can be defined by the position, range, sensitivity, and / or actuation of light sensor 710. For example, light sensor 710 can rotate 360 ​​degrees around host vehicle 400 and sweep across light sensing area 711 to collect sensor data. Conversely, light sensor 710 can be omnidirectional and simultaneously collect sensor data from all directions of light sensing area 711. For example, light sensor 710 can emit one or more laser beams of ultraviolet light, visible light, or near-infrared light in light sensing area 711 to collect sensor data.

[0097] The light sensor 710 may be configured to receive one or more reflected laser waves (e.g., signals) reflected from one or more objects in the light sensing area 711. In other words, when one or more laser beams are emitted through the light sensing area 711, the one or more laser beams may be reflected as laser waves by one or more traffic-related objects (e.g., motor vehicles, pedestrians, trees, guardrails, etc.) within the light sensing area 711 and returned to be received at the light sensor 710.

[0098] One or more host image sensors 712a, 712b, 712c, 712d, 712e, and 712f may also be positioned around the host vehicle 400 to capture additional sensor data from corresponding image sensing areas 713a, 713b, 713c, 713d, 713e, and 713f. The size of the image sensing areas 713a-713f may be defined by the location, range, sensitivity, and / or actuation of the one or more host image sensors 712a-712f.

[0099] One or more host image sensors 712a-712f may be disposed at exterior front and / or side portions of the host vehicle 400, including but not limited to the vehicle bumper, the vehicle headlight unit, the vehicle fenders, and various portions of the windshield. One or more host image sensors 712a-712f may be positioned on a flat-panel scan gantry (not shown) that allows the one or more host image sensors 712a-712f to swing so as to capture images of the exterior environment of the host vehicle 400 at various angles. Additionally, one or more host image sensors 712a-712f may be positioned at interior portions of the host vehicle 400, including the vehicle dashboard (e.g., a dashboard-mounted camera), the rear side of a vehicle rearview mirror, and the like.

[0100] The sensor data includes sensor data captured from the at least one sensor of the host vehicle 400. In this example, the sensor data is captured from the light sensing area 711 and the image sensing areas 713a-713f. Therefore, the sensor data is from the sensor area defined by the light sensing area 711 and the image sensing areas 713a-713f.

[0101] In addition to sensing objects and the surrounding environment of the host vehicle 400, the sensor data may also include vehicle data about the host vehicle 400. The vehicle data may include speed, acceleration, velocity, yaw rate, steering and restrictor angles, range or distance data, navigation data, etc. The vehicle data may also include course direction data, course history data, predicted course data, motion data, current vehicle position data, and any other vehicle information about the remote vehicle and the environment surrounding the remote vehicle.

[0102] At block 604, metadata module 422 generates metadata for the sensor data. Metadata describes the sensor data and may include descriptive metadata, structural metadata, administrative metadata, reference metadata, and statistical metadata. To generate the metadata, metadata module 422 may identify the source of the metadata. For example, assume a first set of sensor data is received from light sensor 710, and a second set of sensor data is received from host image sensor 712a. Metadata module 422 generates metadata that distinguishes between light sensor 710 and host image sensor 712a. The metadata may also include timing information, such as the time at which the sensor data was captured, the length of the sensor data, and the amount of time it took to receive the sensor data. For example, assume host image sensor 712a captures sensor data for image sensing area 713a during a journey. The metadata corresponding to the sensor data may include, for example, via a timestamp, the time at which the sensor data was captured by host image sensor 712a. The length of time to which the sensor data corresponds may be, for example, ten minutes of a twenty-minute journey. Furthermore, in this example, the amount of time it took for the sensor data to be received at receiving module 420 may be two milliseconds (ms) after the data was captured by host image sensor 712a. In this manner, metadata may be generated based on information from vehicle sensors 406 and / or receiving module 420 .

[0103] Metadata may also be generated based on information from the vehicle system 404. Returning to the above example, assume that the host image sensor 712a captures sensor data for the image sensing area 713a of the road (eg, the road environment 800), e.g. Figure 8 As shown. The navigation system 436 can provide positioning and location information corresponding to the image sensing area 713a. For example, it can also be based on, for example, Metadata, such as positioning and location data, is generated by connecting information received from a portable device (not shown) associated with the host vehicle. In another embodiment, metadata, such as positioning and location data, may also be generated based on information received from roadside equipment, road infrastructure, etc.

[0104] The metadata may also include information about the source. For example, the metadata may include the location, orientation, calibration parameters, and capture parameters associated with at least one of the vehicle sensors 406 of the host vehicle 400. For example, the metadata may include information about the location and orientation of the host image sensor 712a relative to the host image sensors 712b-712f. The capture parameters may include image center, focal length, focus distance, distortion model, global or rolling shutter, etc. The metadata may also include information about the size of the received sensor data, such as the size relative to the image from the host image sensor 712a. For another example, metadata about the light sensor 710 may include, for example, the radar pose in the main camera, the type of tracking mode, the detection threshold, the scan angle, the scan angle uncertainty, and the points per scan.

[0105] Metadata may also be generated based on analysis of sensor data by vehicle system 404. For example, light sensor 710 may be configured to receive one or more reflected laser waves (e.g., signals) reflected from one or more objects in light sensing area 711. In other words, when one or more laser beams are transmitted through light sensing area 711, the one or more laser beams may be reflected as laser waves by one or more traffic-related objects (e.g., motor vehicles, pedestrians, trees, guardrails, etc.) within light sensing area 711 and returned to be received at light sensor 710. Vehicle system 404 may analyze the sensor data from light sensor 710 to identify the type of objects sensed in the sensor data, the number of objects, and the like. Metadata may also include information regarding the distance, luminescence, Doppler analysis, orientation, return amplitude, distance uncertainty, Doppler uncertainty, luminescence uncertainty, and angular resolution of the sensed objects. Thus, metadata may be generated based on information from multiple sources, including VCD 402 of host vehicle 400, vehicle system 404, and vehicle sensors 406. Metadata may also be generated based on, for example, proximity based on other devices associated with the host vehicle 400 via a network, such as portable devices or infrastructure devices.

[0106] B. Packaging stage

[0107] At block 606, the grouping module 424 forms metadata packets including metadata and sensor data packets including sensor data. The grouping module 424 distinguishes between data types and packages each data type separately. For example, the grouping module 424 receives metadata retrieved and collected by the metadata module 422 and packages the metadata into metadata packets. Sensor data may be received from the receiving module 420 and / or the vehicle sensors 406, and the grouping module 424 packages the sensor data into sensor data packets separate from the metadata packets.

[0108] Although metadata packets and sensor data packets are encapsulated separately, packetization module 424 can similarly encapsulate packets. For example, packetization module 424 can formulate packets based on a communication protocol and deliver these packets to a transport layer. The communication protocol can be a cellular communication protocol or a sidelink. Assuming the cellular communication protocol is a 5G protocol, packets can be formed based on the layers of the 5G protocol. Furthermore, encapsulation can be performed based on a portion of the radio spectrum. For example, packetization module 424 can formulate packets with compression, data rate, etc. based on a specific portion of the radio spectrum to be used by the cellular communication protocol.

[0109] The packets may be formulated based on the type of data being sent. In one embodiment, the metadata packets and / or sensor data packets may be formed to utilize one or more of the multiple-input multiple-output (MIMO) layers of the network architecture 100. In another embodiment, formulating the packets may include formatting the metadata and / or sensor data according to frames. Assume that the packet module 424 is forming metadata packets having one or more frames. The packet module 424 may arrange the metadata into specific frames in the one or more frames based on the layer of the communication protocol being used by the vehicles 101 and 102. For example, spatial multiplexing may be used below 6 GHz, while MIMO may be used above 6 GHz.

[0110] In one embodiment, a frame of the one or more frames includes a preamble having an identification value that identifies the raw data corresponding to the metadata of the metadata packet. The remaining portion of the frame of the metadata packet after the preamble may be composed of multiple subframes having metadata generated and collected by metadata module 422. The identification value may also indicate the number of frames of the metadata packet, the number of metadata packets, the size of the frames and / or metadata packets, etc., when the next frame begins. In another embodiment, the identification value may be appended to the one or more frames of the metadata packet. In addition, the sensor data packet may also include an identification value that corresponds to and is appended in a similar manner to the identification value of the metadata packet.

[0111] C.Transmission phase

[0112] At box 608, the transmission module 426 transmits the metadata packets at the first carrier frequency and the sensor data packets at the second carrier frequency. The transmission module 426 can access the baseband circuit 204 associated with the cellular communication protocol. Different parts of the radio spectrum can be used to transmit the carrier frequencies. For example, the first carrier frequency can be sent in the low frequency band or the mid-frequency band of 5G. In one embodiment, the first carrier frequency is in the lower 6 GHz spectrum (e.g., 600 MHz, 3100-3550 MHz, 3700-4200 MHz, etc.). Conversely, the second carrier frequency can be sent in the high frequency band of 5G, such as millimeter waves between 24 GHz and 100 GHz (e.g., 27.5-28.35 GHz, 37-40 GHz, 64-71 GHz, etc.).

[0113] In some embodiments, the transmission module 426 may select the first and second carrier frequencies based on communication parameters, such as spectrum authorization, infrastructure development, and frequency bandwidth support. For example, millimeter wave (mmWave) is an unlicensed range of spectrum that offers greater bandwidth. Therefore, due to the size of sensor data packets, mmWave may be selected as the second carrier frequency. Conversely, spectrum below 6 GHz is available over a wide area and is licensed for vehicles 101 and 102, and thus may be used for direct communication.

[0114] Because the transmission module 426 uses different carrier frequencies for different types of data, the carrier frequencies can be sent via different beam paths. Figure 8 800, assuming that metadata packets and sensor data packets are being transmitted from the host vehicle 400 to the remote vehicle 802. The receiving module 420 of the host vehicle 400 can receive sensor data and corresponding metadata from the light sensor 710 and / or one or more host image sensors 712a, 712b, 712c, 712d, 712e and 712f. The remote vehicle 802 can have similarly positioned sensors. For example, the remote vehicle 802 can have a remote sensor 804 that operates in a similar manner to the light sensor 710 described above. The receiving module 420 can also receive sensor data and corresponding metadata from road infrastructure such as street lights 806. In this way, the host vehicle 400 can receive sensor data and / or metadata directly from the road environment 800.

[0115] Continuing with the above example, assume that the first carrier frequency is in the sub-6 GHz spectrum and the second carrier frequency is in the mmWave spectrum. Although both frequencies utilize a 5G network, the first carrier frequency may be routed through the base station 808 in the first branch 810 and the second branch 812, unlike the second carrier frequency, which may communicate using a direct path 814. Because beams in high-mobility situations may be complex, the base station 808 may provide beam training information to vehicles participating in providing data streams for sensor sharing, as will be described in the example given below.

[0116] Figure 9 is a flow chart illustrating an example of an advertisement data stream for sensor sharing according to various aspects described herein. At block 902, the base station 808 may advertise 816 providing a data stream for sensor sharing that is available to vehicles in the road environment 800, such as the host vehicle 400 and the remote vehicle 802. The processor 510 of the base station 808 may be configured to generate and transmit the advertisement 816. The advertisement 816 may include the minimum communication standards required for participation. For example, the minimum communication may include network capabilities, such as 5G capabilities, hardware requirements, minimum sensing capabilities, etc. In another embodiment, the minimum communication standards may be based on the manner in which the vehicle is driven, the manufacturer of the vehicle, the members of the group, etc.

[0117] One or more vehicles of the road environment 800 receive the advertisement 816 at the receiving module 420. While any number of vehicles can receive the advertisement 816, the examples described herein will be discussed with respect to the host vehicle 400. Thus, at block 904, the host vehicle 400 receives the advertisement 816 broadcast by the base station 808. In response to receiving the advertisement 816, the transmitting module 426 of the host vehicle 400 (if it is capable of participating in sensor sharing) transmits a positive response to the base station 808 at block 906.

[0118] The positive response may include confirmation of the minimum communication standards and may include detailed information about how the host vehicle 400 meets the minimum communication standards. For example, assume that the minimum communication standards require the host vehicle to have a threshold amount of sensor coverage. The confirmation may include a sensor map detailing the light sensing area 711 and the image sensing areas 713a, 713b, 713c, 713d, 713e, and 713f. The confirmation may also include path planning information from, for example, the navigation system 436, such as route and destination information, directions, turn-by-turn navigation, global positioning system (GPS) information, and the like.

[0119] At box 908, the base station 808 provides host vehicle beam training information to the host vehicle 400 based on the path planning information. The beam training information includes directional information that one or more vehicles in the vehicle (such as the host vehicle 400) can use for communication. In particular, the directional information indicates where the carrier frequency should be directed based on where the vehicle will be. Therefore, even if the vehicle presents a high mobility scenario, the carrier frequency can be transmitted based on the predicted location of the vehicle. The beam training information can also include network information about the network architecture 100, such as which baseband circuit, carrier frequency, channel, etc. to use to provide data flow for sensor sharing. Therefore, the base station 808 can set and transmit communication parameters to the participating vehicles to achieve communication.

[0120] The vehicle can then be operated according to the present invention, for example Figure 6 The various aspects described above provide data streams for sensor sharing. In response to receiving the metadata packets and the sensor data packets, at block 910, each of the metadata packets and the sensor data packets may be authenticated using corresponding identification values. For example, the grouping module 424 may compare the identification values ​​of the metadata packets and the sensor data packets to determine whether the identification values ​​match or reference each other, the metadata packets, and / or the sensor data packets. In this way, the metadata packets and / or the sensor data packets may be authenticated to protect the vehicle from spoofing, hacking, and the like.

[0121] At block 912, in response to the metadata packet and / or the sensor data packet being authenticated, sensor fusion of the metadata packet and the sensor data packet is performed by the grouping module 424. In sensor fusion, the metadata packet and the sensor data packet are correlated such that the metadata of the metadata packet corresponds to the raw sensor data of the sensor data packet. Furthermore, assuming that the remote vehicle 802 receives and authenticates the metadata packet and the sensor data packet, the grouping module 424 of the remote vehicle 802 may fuse sensor data from the vehicle sensors 406 of the remote vehicle 802 with information from the sensor data of the sensor data packet. In this way, vehicles such as the host vehicle 400 and the remote vehicle 802 can utilize different frequency spectrums to provide data streams for sensor sharing.

[0122] Aspects discussed herein can be described and implemented in the context of non-transitory computer-readable storage media that store computer-executable instructions. Non-transitory computer-readable storage media include computer storage media and communication media. For example, flash memory drives, digital versatile disks (DVDs), compact disks (CDs), floppy disks, and magnetic tape cassettes. Non-transitory computer-readable storage media can include volatile or non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, modules, or other data.

[0123] It will be appreciated that various of the features and functions disclosed above and other features and functions, or alternatives or variations thereof, may be ideally combined into many other different systems or applications. Furthermore, various currently unforeseen or unanticipated substitutions, modifications, variations, or improvements thereto may be made by those skilled in the art, which are also intended to be encompassed by the appended claims.

Claims

1. A device configured to be employed in a host vehicle, comprising: Memory; and A processor, the processor comprising: a receiving module configured to receive sensor data from at least one vehicle sensor of the host vehicle; a metadata module configured to generate metadata for the sensor data; a grouping module configured to form a metadata group including the metadata and a sensor data group including the sensor data; and A transmission module configured to transmit the metadata packets at a first carrier frequency and transmit the sensor data packets at a second carrier frequency different from the first carrier frequency, wherein the first carrier frequency is in a sub-6 GHz spectrum and the second carrier frequency is in a millimeter wave spectrum.

2. The device according to claim 1, wherein The transmission module is configured to transmit the metadata packet and the sensor data packet to another vehicle.

3. The apparatus of claim 1 , wherein the metadata comprises one or more of a location of the at least one vehicle sensor, an orientation of the at least one vehicle sensor, a calibration parameter of the at least one vehicle sensor, and a capture parameter of the at least one vehicle sensor. 4 . The apparatus of claim 1 , wherein frequencies of the first carrier frequency and the second carrier frequency are set by a base station in communication with the host vehicle.

5. The apparatus of claim 1, wherein the metadata packets and the sensor data packets include beam training information.

6. The apparatus of claim 1 , wherein the sensor data packets are formed to utilize one or more layers of a multiple-input multiple-output (MIMO) layer of a fifth generation (5G) New Radio (NR) network. 7 . The apparatus of claim 1 , wherein the metadata packet and the sensor data packet include an identification value for authentication.

8. A computer-implemented method for providing a data stream for sensor sharing using a host vehicle, the method comprising: receiving sensor data from at least one vehicle sensor of the host vehicle; Generate metadata for sensor data; forming a metadata packet including the metadata and a sensor data packet including the sensor data; as well as The metadata packets are transmitted at a first carrier frequency and the sensor data packets are transmitted at a second carrier frequency different from the first carrier frequency, wherein the first carrier frequency is in a sub-6 GHz spectrum and the second carrier frequency is in a millimeter wave spectrum.

9. The computer-implemented method of claim 8, wherein: The metadata packet and the sensor data packet are transmitted to another vehicle.

10. The computer-implemented method of claim 8, wherein the metadata comprises one or more of a location of the at least one vehicle sensor, an orientation of the at least one vehicle sensor, a calibration parameter of the at least one vehicle sensor, and a capture parameter of the at least one vehicle sensor. 11 . The computer-implemented method of claim 8 , wherein frequencies of the first carrier frequency and the second carrier frequency are set by a base station in communication with the host vehicle.

12. The computer-implemented method of claim 8, wherein the metadata packets and the sensor data packets include beam training information.

13. The computer-implemented method of claim 8, wherein the sensor data packets are formed to utilize one or more multiple-input multiple-output (MIMO) layers of a fifth generation (5G) New Radio (NR) network.

14. The computer-implemented method of claim 8, wherein the metadata packet and the sensor data packet include an identification value for authentication.

15. A device configured to be employed in a host vehicle, comprising: Memory; and A processor, the processor comprising: a receiving module configured to receive sensor data from at least one vehicle sensor of the host vehicle; a metadata module configured to generate metadata for the sensor data; a grouping module configured to form a metadata group including the metadata and a sensor data group including the sensor data; and A transmission module configured to transmit the metadata packets at a first carrier frequency of a spectrum associated with a fifth generation (5G) and to transmit the sensor data packets at a second carrier frequency of a spectrum different from the first carrier frequency, wherein the first carrier frequency is in a spectrum below 6 GHz and the second carrier frequency is in a millimeter wave spectrum.

16. The apparatus according to claim 15, wherein The transmission module is configured to transmit the metadata packet and the sensor data packet to another vehicle.

17. The apparatus of claim 15, wherein the metadata comprises one or more of a location of the at least one vehicle sensor, an orientation of the at least one vehicle sensor, a calibration parameter of the at least one vehicle sensor, and a capture parameter of the at least one vehicle sensor.

18. The apparatus of claim 15, wherein frequencies of the first carrier frequency and the second carrier frequency are set by a base station in communication with the host vehicle.

19. The apparatus of claim 15, wherein the metadata packets and the sensor data packets include beam training information.

20. The apparatus of claim 15, wherein the sensor data packets are formed to utilize one or more multiple-input multiple-output (MIMO) layers of the 5G-associated spectrum.

Images