Estimating and transmitting object captured by camera

Abstract

A wireless device may obtain an image having at least one captured object from a camera, the at least one captured object may include a plurality of dimensions (802). The image may include a 2D projection of the at least one captured object. The wireless device may calculate at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object (804). The wireless device may estimate an inverse depth of the plurality of dimensions of the at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object (806). The wireless device may transmit an indication of the plurality of dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth (808).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to U.S. Provisional Application Serial No. 63 / 364,185, filed on May 4, 2022, entitled “METHODS FOR ESTIMATING AND TRANSMITTING OBJECTS CAPTURED BY A CAMERA,” and U.S. Non-Provisional Patent Application Serial No. 18 / 311,187, filed on May 2, 2023, entitled “ESTIMATING AND TRANSMITTING OBJECTS CAPTURED BY ACAMERA,” the entire contents of which are expressly incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to communication systems and, more particularly, to the transmission of data captured by an image capture device. Background Art

[0004] Wireless communication systems are widely deployed to provide a variety of telecommunication services, such as telephony, video, data, messaging, and broadcast. Typical wireless communication systems may employ multiple access technologies capable of supporting communication with multiple users by sharing available system resources. Examples of such multiple access technologies include code division multiple access (CDMA) systems, time division multiple access (TDMA) systems, frequency division multiple access (FDMA) systems, orthogonal frequency division multiple access (OFDMA) systems, single carrier frequency division multiple access (SC-FDMA) systems, and time division synchronous code division multiple access (TD-SCDMA) systems.

[0005] These multiple access technologies have been adopted in various telecommunication standards to provide a common protocol that enables different wireless devices to communicate at a city, country, region, or even global level. An example telecommunication standard is 5G New Radio (NR). 5G NR is part of the continued mobile broadband evolution promulgated by the Third Generation Partnership Project (3GPP) to meet new requirements associated with latency, reliability, security, scalability (e.g., with the Internet of Things (IoT)) and other requirements. 5G NR includes services associated with enhanced mobile broadband (eMBB), massive machine type communications (mMTC), and ultra-reliable low latency communications (URLLC). Certain aspects of 5G NR may be based on the 4G Long Term Evolution (LTE) standard. Further improvements to 5G NR technology are needed. In addition, these improvements may also be applicable to other multiple access technologies and telecommunication standards that employ these technologies. Summary of the invention

[0006] A simplified summary of one or more aspects is presented below to provide a basic understanding of these aspects. This summary is not an extensive review of all contemplated aspects. This summary is neither intended to identify key or important elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.

[0007] In one aspect of the present disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may have a memory and at least one processor coupled to the memory. Based at least in part on information stored in the memory, the at least one processor may be configured to obtain an image having at least one captured object from a camera, the at least one captured object may include multiple dimensions. The image may include a two-dimensional (2D) projection of the at least one captured object. The at least one processor may be further configured to calculate at least one of the multiple dimensions of the at least one captured object based on the 2D projection of the at least one captured object. The at least one processor may be further configured to estimate the inverse depth of the multiple dimensions of the at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. The at least one processor may be further configured to send an indication of the multiple dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth.

[0008] To achieve the aforementioned and related purposes, one or more aspects include the features fully described below and particularly pointed out in the claims. The following description and the accompanying drawings set forth in detail some exemplary features of one or more aspects. However, these features indicate only some of the various ways in which the principles of the various aspects can be employed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a diagram illustrating an example of a wireless communication system and an access network.

[0010] Figure 2A is a diagram illustrating an example of a first frame according to various aspects of the present disclosure.

[0011] Figure 2B is a diagram illustrating an example of downlink (DL) channels within a subframe according to aspects of the present disclosure.

[0012] Figure 2C is a diagram illustrating an example of a second frame according to various aspects of the present disclosure.

[0013] Figure 2Dis a diagram illustrating an example of an uplink (UL) channel within a subframe according to various aspects of the present disclosure

[0014] Figure 3 is a diagram illustrating an example of a base station and a user equipment (UE) in an access network.

[0015] Figure 4 is a diagram illustrating an example of UE positioning based on reference signal measurement.

[0016] Figure 5 is a diagram illustrating an example of a side link slot structure.

[0017] Figure 6 is a diagram illustrating a UE and a camera configured to capture at least one image and transmit multiple calculated dimensions of at least one three-dimensional (3D) object captured by the camera.

[0018] Figure 7 Shown by Figure 6 Example images captured by a camera

[0019] Figure 8 is a flow chart of a method of wireless communication.

[0020] Fig. 9 is a flow chart of a method of wireless communication.

[0021] Fig.10 is a flow chart of a method of wireless communication.

[0022] Fig.11 are diagrams illustrating examples of hardware implementations for example apparatuses and / or network entities. DETAILED DESCRIPTION

[0023] Generating accurate street maps can help enable autonomous driving and advanced driver assistance system (ADAS) applications on vehicles. Such street maps can be generated by collecting image data from mobile wireless devices (such as vehicles with cameras configured to collect image data). When combined with the position data of the vehicle, such image data can be used to estimate the road environment around the vehicle. However, sending image data from a moving vehicle may utilize a large amount of wireless resources. Sending the attributes of objects in the image (such as the dimensions of the object) can be a more efficient use of wireless resources. Such attributes may include, for example, the location of the vehicle, the direction the camera is facing, and the dimensions of three-dimensional (3D) objects (e.g., traffic signs, traffic signals, billboards) in the image.

[0024] Multiple dimensions may be used to describe each object in an image. A nonlinear least squares solver may be used to calculate one or more dimensions of an object captured by an image. However, it may be impractical to describe such objects using dimensions such as size, location, and orientation because many of these dimensions cannot be calculated without capturing multiple images and comparing these images to each other. Capturing multiple images about an object may delay the initialization of a nonlinear least squares solver. In addition, describing such objects using a location vector capable of conveying negative depths may allow for impossible negative depths in calculations using a nonlinear least squares solver. Describing such objects using dimensions based on the inverse depth of the object may allow the device to calculate more dimensions of a nonlinear least squares solver based on a single captured image. The device analyzing the object in the image may calculate dimensions other than the inverse depth, and may estimate the inverse depth using a reasonable estimate of the inverse depth. The nonlinear least squares solver may be initialized with the calculated and estimated dimensions, and the estimate of the inverse depth may be improved and refined using data from subsequent images. When using the inverse depth, the nonlinear least squares solver may not consider impossible values ​​such as negative depth values ​​or zero depth values.

[0025] A wireless device may obtain an image having at least one captured object from a camera, the at least one captured object may include multiple dimensions. The image may include a two-dimensional (2D) projection of the at least one captured object. The wireless device may calculate at least one dimension of the multiple dimensions of the at least one captured object based on the 2D projection of the at least one captured object. The wireless device may estimate an inverse depth of the multiple dimensions of the at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. The wireless device may send an indication of the multiple dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth.

[0026] The specific embodiments described below in conjunction with the accompanying drawings are descriptions of various configurations and do not represent the only configurations that can practice the concepts described herein. In order to provide a thorough understanding of various concepts, the specific embodiments include specific details. However, these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid blurring these concepts.

[0027] Several aspects of the telecommunication system are also presented with reference to various apparatuses and methods. These apparatuses and methods are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0028] As an example, an element or any part of an element or any combination of elements can be implemented as a "processing system" including one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on chip (SoCs), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gate logic, discrete hardware circuits, and other suitable hardware configured to perform various functionalities described throughout the present disclosure. One or more processors in a processing system can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description language, or other terms, software should be broadly interpreted as meaning instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, processes, functions, or any combination thereof.

[0029] Therefore, in one or more example aspects, specific implementations and / or use cases, the described functions may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored or encoded as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media. Storage media may be any available media that can be accessed by a computer. By way of example, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of these types of computer-readable media, or any other medium that can be used to store computer-executable code in the form of instructions or data structures that can be accessed by a computer.

[0030] Although various aspects, specific implementations and / or use cases are described in this application by the illustration of some examples, additional or different aspects, specific implementations and / or use cases may be generated in many different arrangements and scenarios. The various aspects, specific implementations and / or use cases described herein can be implemented across many different platform types, devices, systems, shapes, sizes and packaging arrangements. For example, various aspects, specific implementations and / or use cases can be generated via integrated chip specific implementations and other devices based on non-module components (e.g., end-user devices, vehicles, communication equipment, computing equipment, industrial equipment, retail / purchase equipment, medical equipment, devices that enable artificial intelligence (AI), etc.). Although some examples may or may not be specifically for use cases or applications, the described examples may have a wide range of applicability. Various aspects, specific implementations and / or use cases can be in the range from chip-level or modular components to non-modular, non-chip-level specific implementations, and further to the range of aggregated, distributed or original equipment manufacturer (OEM) equipment or systems in conjunction with one or more technologies herein. In some actual settings, the equipment incorporating the various aspects and features described may also include additional components and features for implementing and practicing the claimed and described aspects. For example, the transmission and reception of wireless signals necessarily include multiple components for analog and digital purposes (e.g., hardware components including antennas, RF chains, power amplifiers, modulators, buffers, processors, interleavers, adders / summers, etc.) The various techniques described herein can be practiced in a wide variety of devices, chip-level components, systems, distributed arrangements, aggregated or disaggregated components, end-user devices, etc., of various sizes, shapes, and configurations.

[0031] The deployment of a communication system (such as a 5G NR system) can be arranged with various components or constituent parts in a variety of ways. In a 5G NR system or network, a network node, a network entity, a mobility element of a network, a radio access network (RAN) node, a core network node, a network element or a network equipment (such as a base station (BS) or one or more units (or one or more components) that perform base station functionality) can be implemented in an aggregated or decomposed architecture. For example, a BS (such as a Node B (NB), an evolved NB (eNB), an NR BS, a 5G NB, an access point (AP), a transmit receive point (TRP) or a cell, etc.) can be implemented as an aggregated base station (also referred to as an independent BS or a monolithic BS) or a decomposed base station.

[0032] The aggregated base station can be configured to utilize a radio protocol stack physically or logically integrated within a single RAN node. The decomposed base station can be configured to utilize a protocol stack physically or logically distributed between two or more units, such as one or more central or centralized units (CU), one or more distributed units (DU), or one or more radio units (RU). In some aspects, the CU can be implemented within a RAN node, and one or more DUs can be co-located with the CU, or alternatively, can be geographically or virtually distributed in one or more other RAN nodes. The DU can be implemented to communicate with one or more RUs. Each of the CU, DU, and RU can be implemented as a virtual unit, i.e., a virtual central unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).

[0033] Base station operation or network design may take into account the aggregated nature of base station functionality. For example, a disaggregated base station may be used in an integrated access backhaul (IAB) network, an open radio access network (O-RAN (network configuration such as that initiated by the O-RAN Alliance)), or a virtualized radio access network (vRAN, also known as a cloud radio access network (C-RAN)). Decomposition may include distributing functionality across two or more units at various physical locations, as well as virtually distributing functionality of at least one unit, which may enable flexibility in network design. Various units of a disaggregated base station or disaggregated RAN architecture may be configured for wired or wireless communication with at least one other unit.

[0034] Figure 1 1 is a diagram 100 illustrating an example of a wireless communication system and an access network. The illustrated wireless communication system includes a disaggregated base station architecture. The disaggregated base station architecture may include one or more CUs 110, which may communicate directly with a core network 120 via a backhaul link, or indirectly with the core network 120 through one or more disaggregated base station units (such as a near real-time (near RT) RAN intelligent controller (RIC) 125 via an E2 link, or a non-real-time (non-RT) RIC 115 associated with a service management and orchestration (SMO) framework 105, or both). The CU 110 may communicate with one or more DUs 130 via corresponding midhaul links (such as an F1 interface). The DU 130 may communicate with one or more RUs 140 via corresponding fronthaul links. The RU 140 may communicate with corresponding UEs 104 via one or more radio frequency (RF) access links. In some implementations, a UE 104 may be served simultaneously by multiple RUs 140.

[0035] Each of the units (i.e., CU 110, DU 130, RU 140, and near-RT RIC 125, non-RT RIC 115, and SMO framework 105) may include or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via a wired or wireless transmission medium. Each of the units or an associated processor or controller that provides instructions to the communication interfaces of these units may be configured to communicate with one or more of the other units via a transmission medium. For example, these units may include a wired interface configured to receive signals on a wired transmission medium or to transmit signals to one or more of the other units. In addition, these units may include a wireless interface that may include a receiver, transmitter, or transceiver (such as an RF transceiver) configured to receive signals on a wireless transmission medium or to transmit signals to one or more of the other units, or both.

[0036] In some aspects, CU 110 may host one or more higher layer control functions. Such control functions may include radio resource control (RRC), packet data convergence protocol (PDCP), service data adaptation protocol (SDAP), etc. Each control function may be implemented using an interface configured to communicate signals with other control functions hosted by CU 110. CU110 may be configured to handle user plane functionality (i.e., central unit-user plane (CU-UP)), control plane functionality (i.e., central unit-control plane (CU-CP)), or a combination thereof. In some specific implementations, CU 110 may be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP unit may communicate bidirectionally with the CU-CP unit via an interface (such as an E1 interface). As needed, CU 110 may be implemented to communicate with DU 130 for network control and signaling.

[0037] DU 130 may correspond to a logical unit that includes one or more base station functions for controlling the operation of one or more RUs 140. In some aspects, DU 130 may host one or more of a radio link control (RLC) layer, a medium access control (MAC) layer, and one or more high physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) depending at least in part on a functional split such as those defined by 3GPP. In some aspects, DU 130 may further host one or more low PHY layers. Each layer (or module) may be implemented using an interface that is configured to communicate signals with other layers (and modules) hosted by DU 130 or with control functions hosted by CU 110.

[0038] The lower layer functionality may be implemented by one or more RUs 140. In some deployments, the RU 140 controlled by the DU 130 may correspond to a logical node that hosts RF processing functions or low PHY layer functions (such as performing fast Fourier transform (FFT), inverse FFT (iFFT), digital beamforming, physical random access channel (PRACH) extraction and filtering, etc.), or both, based at least in part on functional splitting (such as lower layer functional splitting). In such an architecture, the RU 140 may be implemented to handle over-the-air (OTA) communications with one or more UEs 104. In some implementations, real-time and non-real-time aspects of control plane and user plane communications with the RU 140 may be controlled by the corresponding DU 130. In some scenarios, this configuration may enable the DU 130 and CU 110 to be implemented in a cloud-based RAN architecture (such as a vRAN architecture).

[0039] The SMO framework 105 may be configured to support RAN deployment and configuration of non-virtualized network elements and virtualized network elements. For non-virtualized network elements, the SMO framework 105 may be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which may be managed via an operation and maintenance interface (such as an O1 interface). For virtualized network elements, the SMO framework 105 may be configured to interact with a cloud computing platform (such as an open cloud (O-Cloud) 190) to perform network element lifecycle management (such as instantiating virtualized network elements) via a cloud computing platform interface (such as an O2 interface). Such virtualized network elements may include, but are not limited to, CU 110, DU 130, RU 140, and near-RT RIC 125. In some specific implementations, the SMO framework 105 may communicate with hardware aspects of the 4G RAN (such as an open eNB (O-eNB) 111) via the O1 interface. Additionally, in some specific implementations, the SMO framework 105 may communicate directly with one or more RUs 140 via the O1 interface. The SMO framework 105 may also include a non-RT RIC 115 configured to support the functionality of the SMO framework 105 .

[0040] The non-RT RIC 115 may be configured to include logic functions that enable non-real-time control and optimization of RAN elements and resources, artificial intelligence (AI) / machine learning (ML) (AI / ML) workflows including model training and updating, or policy-based guidance of applications / features in the near-RT RIC 125. The non-RT RIC 115 may be coupled to or in communication with the near-RT RIC 125 (such as via an A1 interface). The near-RT RIC 125 may be configured to include logic functions that enable near-real-time control and optimization of RAN elements and resources via data collection and actions through an interface (such as via an E2 interface) that connects one or more CUs 110, one or more DUs 130, or both, and O-eNBs with the near-RT RIC 125.

[0041] In some implementations, in order to generate an AI / ML model to be deployed in the near-RT RIC 125, the non-RT RIC 115 may receive parameters or external enrichment information from an external server. Such information may be utilized by the near-RT RIC 125 and may be received from a non-network data source or from a network function at the SMO framework 105 or the non-RT RIC 115. In some examples, the non-RT RIC 115 or the near-RT RIC 125 may be configured to adjust RAN behavior or performance. For example, the non-RT RIC 115 may monitor long-term trends and patterns of performance and employ AI / ML models to perform corrective actions through the SMO framework 105 (such as via reconfiguration of O1) or via the creation of RAN management policies (such as A1 policies).

[0042] At least one of the CU 110, the DU 130, and the RU 140 may be referred to as a base station 102. Therefore, the base station 102 may include one or more of the CU 110, the DU 130, and the RU 140 (each component is indicated by a dotted line to indicate that each component may be included in the base station 102 or may not be included in the base station). The base station 102 provides an access point to the core network 120 for the UE 104. The base station 102 may include a macro cell (a high-power cellular base station) and / or a small cell (a low-power cellular base station). Small cells include femto cells, pico cells, and micro cells. A network including both small cells and macro cells may be referred to as a heterogeneous network. A heterogeneous network may also include a home evolved Node B (eNB) (HeNB), which may provide services to a restricted group referred to as a closed subscriber group (CSG). The communication link between RU 140 and UE 104 may include uplink (UL) (also referred to as reverse link) transmission from UE 104 to RU 140 and / or downlink (DL) (also referred to as forward link) transmission from RU 140 to UE 104. The communication link may use multiple-input multiple-output (MIMO) antenna technology, including spatial multiplexing, beamforming and / or transmission diversity. The communication link may be through one or more carriers. For each carrier allocated in a carrier aggregation of up to Yx MHz (x component carriers) for transmission in each direction, the base station 102 / UE 104 may use a spectrum of up to Y MHz (e.g., 5 MHz, 10 MHz, 15 MHz, 20 MHz, 100 MHz, 400 MHz, etc.) bandwidth. The carriers may or may not be adjacent to each other. The allocation of carriers may be asymmetric with respect to DL and UL (e.g., more or fewer carriers may be allocated for DL ​​compared to UL). The component carrier may include a primary component carrier and one or more secondary component carriers. The primary component carrier may be referred to as a primary cell (PCell) and the secondary component carrier may be referred to as a secondary cell (SCell).

[0043] Some UEs 104 may communicate with each other using a device-to-device (D2D) communication link 158. Such UEs 104 may also communicate with a roadside unit (RSU) 152 using a D2D communication link 158. The D2D communication link 158 may use DL / UL wireless wide area network (WWAN) spectrum. The D2D communication link 158 may use one or more sidelink channels, such as a physical sidelink broadcast channel (PSBCH), a physical sidelink discovery channel (PSDCH), a physical sidelink shared channel (PSSCH), and a physical sidelink control channel (PSCCH). The D2D communication may be through a variety of wireless D2D communication systems, such as, for example, Bluetooth, Wi-Fi, LTE, or NR based on the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard.

[0044] The wireless communication system may also include a Wi-Fi AP 150 that communicates with a UE 104 (also referred to as a Wi-Fi station (STA)) via a communication link 154, e.g., in a 5 GHz unlicensed spectrum, etc. When communicating in an unlicensed spectrum, the UE 104 / AP 150 may perform a clear channel assessment (CCA) to determine whether a channel is available prior to communication.

[0045] The electromagnetic spectrum is typically subdivided into various categories, bands, channels, etc. based on frequency / wavelength. In 5G NR, two initial operating bands have been identified as frequency range designations FR1 (410 MHz–7.125 GHz) and FR2 (24.25 GHz–52.6 GHz). Although a portion of FR1 is greater than 6 GHz, FR1 is often referred to (interchangeably) as the “sub-6 GHz” band in various documents and articles. A similar naming issue sometimes occurs with respect to FR2, which is often (interchangeably) referred to as the “millimeter wave” band in documents and articles, although distinct from the extremely high frequency (EHF) band (30 GHz–300 GHz) identified as the “millimeter wave” band by the International Telecommunication Union (ITU).

[0046] Frequencies between FR1 and FR2 are generally referred to as mid-band frequencies. Recent 5G NR research has identified the operating bands for these mid-band frequencies as frequency range designation FR3 (7.125GHz–24.25GHz). The bands falling within FR3 can inherit FR1 characteristics and / or FR2 characteristics, and therefore the features of FR1 and / or FR2 can be effectively extended to mid-band frequencies. In addition, higher frequency bands are currently being explored to extend 5G NR operations to more than 52.6GHz. For example, three higher operating bands have been identified as frequency range designations FR2-2 (52.6GHz–71GHz), FR4 (71GHz–114.25GHz), and FR5 (114.25GHz–300GHz). Each of these higher frequency bands falls within the EHF band.

[0047] In view of the above, unless otherwise specified, if the term "below 6 GHz" or the like is used in this document, it can broadly refer to frequencies that can be less than 6 GHz, can be within FR1, or can include mid-band frequencies. In addition, unless otherwise specified, if the term "millimeter wave" or the like is used in this document, it can broadly refer to frequencies that can include mid-band frequencies, can be within FR2, FR4, FR2-2, and / or FR5, or can be within the EHF band.

[0048] The base station 102 and the UE 104 may each include multiple antennas (such as antenna elements, antenna panels, and / or antenna arrays) to facilitate beamforming. The base station 102 may send a beamformed signal 182 to the UE 104 in one or more transmit directions. The UE 104 may receive the beamformed signal from the base station 102 in one or more receive directions. The UE 104 may also send a beamformed signal 184 to the base station 102 in one or more transmit directions. The base station 102 may receive the beamformed signal from the UE 104 in one or more receive directions. The base station 102 / UE 104 may perform beam training to determine the best receive direction and transmit direction for each of the base station 102 / UE 104. The transmit direction and receive direction of the base station 102 may be the same or may not be the same. The transmit direction and receive direction of the UE 104 may be the same or may not be the same.

[0049] The base station 102 may include and / or be referred to as a gNB, a Node B, an eNB, an access point, a base transceiver station, a radio base station, a radio transceiver, a transceiver function, a basic service set (BSS), an extended service set (ESS), a transmit receive point (TRP), a network node, a network entity, a network equipment, or some other suitable term. The base station 102 may be implemented as an integrated access and backhaul (IAB) node, a relay node, a sidelink node, an aggregated (monolithic) base station with a baseband unit (BBU) (including a CU and a DU) and a RU, or as a decomposed base station including one or more of a CU, a DU, and / or a RU. A set of base stations that may include decomposed base stations and / or aggregated base stations may be referred to as a next generation (NG) RAN (NG-RAN).

[0050] The core network 120 may include an access and mobility management function (AMF) 161, a session management function (SMF) 162, a user plane function (UPF) 163, a unified data management (UDM) 164, one or more location servers 168, and other functional entities. AMF 161 is a control node that handles signaling between UE 104 and the core network 120. AMF 161 supports registration management, connection management, mobility management, and other functions. SMF 162 supports session management and other functions. UPF 163 supports packet routing, packet forwarding, and other functions. UDM 164 supports the generation of authentication and key agreement (AKA) credentials, user identity handling, access authorization, and subscription management. One or more location servers 168 are illustrated as including a gateway mobile location center (GMLC) 165 and a location management function (LMF) 166. However, in general, the one or more location servers 168 may include one or more location / positioning servers, which may include one or more of the GMLC 165, LMF 166, position determination entity (PDE), serving mobile location center (SMLC), mobile positioning center (MPC), etc. The GMLC 165 and LMF 166 support UE location services. The GMLC 165 provides an interface for clients / applications (e.g., emergency services) to access UE positioning information. The LMF 166 receives measurements and assistance information from the NG-RAN and UE 104 via the AMF 161 to calculate the location of the UE 104. The NG-RAN may utilize one or more positioning methods in order to determine the location of the UE 104. Positioning the UE 104 may involve signal measurements, position estimation, and optional speed calculations based on these measurements. Signal measurements may be performed by the UE 104 and / or the serving base station 102. The measured signal can be based on a satellite positioning system (SPS) 170 (e.g., one or more of a global navigation satellite system (GNSS), a global positioning system (GPS), a non-terrestrial network (NTN), or other satellite positioning / positioning systems), LTE signals, wireless local area network (WLAN) signals, Bluetooth signals, a terrestrial beacon system (TBS), sensor-based information (e.g., an atmospheric pressure sensor, a motion sensor), an NR enhanced cell ID (NR E-CID) method, an NR signal (e.g., multi-round trip time (multi-RTT), DL angle of departure (DL-AoD), DL time difference of arrival (DL-TDOA), UL time difference of arrival (UL-TDOA), and UL angle of arrival (UL-AoA) positioning) and / or one or more of other systems / signals / sensors.

[0051] Examples of UE 104 include cellular phones, smart phones, session initiation protocol (SIP) phones, laptops, personal digital assistants (PDAs), satellite radios, global positioning systems, multimedia devices, video devices, digital audio players (e.g., MP3 players), cameras, game consoles, tablet devices, smart devices, wearable devices, vehicles, electric meters, gas pumps, large or small kitchen appliances, healthcare equipment, implants, sensors / actuators, displays, or any other similarly functional devices. Some of the UEs 104 may be referred to as IoT devices (e.g., parking meters, gas pumps, toasters, vehicles, heart monitors, etc.). UE 104 may also be referred to as stations, mobile stations, subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, mobile terminals, wireless terminals, remote terminals, mobile phones, user agents, mobile clients, clients, or some other suitable terminology. In some scenarios, the term UE may also be applied to one or more supporting devices, such as in a device cluster arrangement. One or more of these devices may access the network collectively and / or individually.

[0052] Reference again Figure 1 In certain aspects, the UE 104 may include a dimension generator component 198 that may be configured to obtain an image having at least one captured object from a camera, wherein the at least one captured object includes multiple dimensions, and wherein the image includes a two-dimensional (2D) projection of the at least one captured object. The dimension generator component 198 may also be configured to calculate at least one of the multiple dimensions of the at least one captured object based on the 2D projection of the at least one captured object. The dimension generator component 198 may also be configured to estimate the inverse depth of the multiple dimensions of the at least one captured object based on information associated with one or more attributes of the camera or one or more attributes of at least one reference object associated with the at least one captured object. The dimension generator component 198 may also be configured to send an indication of the multiple dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. The multiple dimensions may include at least one calculated dimension and the estimated inverse depth of the captured object. Although the following description may focus on traffic signs, the concepts described herein may be applicable to any three-dimensional (3D) object captured by a camera that captures a two-dimensional (2D) image, such as a billboard or a traffic light.

[0053] Figure 2A FIG200 is a diagram illustrating an example of a first subframe within a 5G NR frame structure. Figure 2B is a diagram 230 illustrating an example of DL channels within a 5G NR subframe. Figure 2C 250 is a diagram illustrating an example of a second subframe within a 5G NR frame structure. Figure 2D FIG280 is a diagram illustrating an example of UL channels within a 5G NR subframe. The 5G NR frame structure may be frequency division duplex (FDD) (wherein for a particular set of subcarriers (carrier system bandwidth), subframes within that subcarrier set are dedicated to either DL or UL), or may be time division duplex (TDD) (wherein for a particular set of subcarriers (carrier system bandwidth), subframes within that subcarrier set are dedicated to both DL and UL). Figure 2A , Figure 2C In the example provided, the 5G NR frame structure is assumed to be TDD, where subframe 4 is configured with slot format 28 (most of which are DL), where D is DL, U is UL, and F can be used flexibly between DL / UL, and subframe 3 is configured with slot format 1 (all of which are UL). Although subframes 3 and 4 are shown with slot formats 1 and 28, respectively, any particular subframe can be configured with any of the various available slot formats 0 to 61. Slot formats 0 and 1 are DL and UL, respectively. Other slot formats 2 to 61 include a mix of DL, UL and flexible symbols. The UE is configured with the slot format through the received slot format indicator (SFI) (dynamically configured through DL control information (DCI) or semi-statically / statically configured through radio resource control (RRC) signaling). Note that the following description also applies to the 5G NR frame structure as TDD.

[0054] FIG. 2A to FIG. 2D The frame structure is illustrated, and various aspects of the present disclosure may be applicable to other wireless communication technologies that may have different frame structures and / or different channels. A frame (10ms) may be divided into 10 subframes (1ms) of the same size. Each subframe may include one or more time slots. A subframe may also include a microslot, which may include 7, 4, or 2 symbols. Each time slot may include 14 or 12 symbols, depending on whether the cyclic prefix (CP) is normal or extended. For a normal CP, each time slot may include 14 symbols, and for an extended CP, each time slot may include 12 symbols. The symbol on the DL may be a CP orthogonal frequency division multiplexing (OFDM) (CP-OFDM) symbol. The symbol on the UL may be a CP-OFDM symbol (for high throughput scenarios) or a discrete Fourier transform (DFT) extended OFDM (DFT-s-OFDM) symbol (also known as a single carrier frequency division multiple access (SC-FDMA) symbol) (for power-limited scenarios; limited to single-stream transmission). The number of time slots within a subframe is based on the CP and parameter set. The parameter set defines the subcarrier spacing (SCS) and effectively defines the symbol length / duration, which is equal to 1 / SCS.

[0055]

[0056] For normal CP (14 symbols / slot), different parameter sets μ0 to 4 allow 1, 2, 4, 8, and 16 slots per subframe, respectively. For extended CP, parameter set 2 allows 4 slots per subframe. Thus, for normal CP and parameter set μ, there are 14 symbols / slot and 2 μ timeslots / subframe. The subcarrier spacing can be equal to 2 μ *15kHz, where μ is parameter set 0 to 4. Therefore, the subcarrier spacing for parameter set μ=0 is 15kHz, and the subcarrier spacing for parameter set μ=4 is 240kHz. The symbol length / duration is inversely related to the subcarrier spacing. FIG. 2A to FIG. 2D An example is provided for a normal CP with 14 symbols per slot and a parameter set μ=2 with 4 slots per subframe. The slot duration is 0.25 ms, the subcarrier spacing is 60 kHz, and the symbol duration is about 16.67 μs. Within a frame set, there may be one or more different bandwidth parts (BWPs) that are frequency-division multiplexed (see Figure 2B ). Each BWP can have a specific parameter set and CP (normal or extended).

[0057] A resource grid may be used to represent a frame structure. Each slot includes a resource block (RB) (also referred to as a physical RB (PRB)) extending over 12 consecutive subcarriers. The resource grid is divided into a plurality of resource elements (REs). The number of bits carried by each RE depends on the modulation scheme.

[0058] like Figure 2A As illustrated, some of the REs carry reference (pilot) signals (RS) for the UE. The RS may include a demodulation RS (DM-RS) (indicated as R for a particular configuration, but other DM-RS configurations are possible) and a channel state information reference signal (CSI-RS) for channel estimation at the UE. The RS may also include a beam measurement RS (BRS), a beam refinement RS (BRRS), and a phase tracking RS (PT-RS).

[0059] Figure 2BExamples of various DL channels within a subframe of a frame are illustrated. A physical downlink control channel (PDCCH) carries DCI within one or more control channel elements (CCEs) (e.g., 1, 2, 4, 8, or 16 CCEs), each CCE including six RE groups (REGs), each RE group (REG) including 12 consecutive REs in an OFDM symbol of an RB. A PDCCH within a BWP may be referred to as a control resource set (CORESET). The UE is configured to monitor PDCCH candidates in a PDCCH search space (e.g., a common search space, a UE-specific search space) during a PDCCH monitoring opportunity on a CORESET, wherein the PDCCH candidates have different DCI formats and different aggregation levels. Additional BWPs may be located at higher and / or lower frequencies on the channel bandwidth. A primary synchronization signal (PSS) may be within symbol 2 of a particular subframe of a frame. The PSS is used by UE 104 to determine subframe / symbol timing and physical layer identity. A secondary synchronization signal (SSS) may be within symbol 4 of a particular subframe of a frame. The SSS is used by the UE to determine the physical layer cell identity group number and the radio frame timing. Based on the physical layer identity and the physical layer cell identity group number, the UE can determine the physical cell identifier (PCI). Based on the PCI, the UE can determine the location of the DM-RS. The physical broadcast channel (PBCH) carrying the master information block (MIB) can be logically grouped with the PSS and SSS to form a synchronization signal (SS) / PBCH block (also referred to as an SS block (SSB)). The MIB provides the number of RBs in the system bandwidth and the system frame number (SFN). The physical downlink shared channel (PDSCH) carries user data, broadcast system information not sent via the PBCH (such as a system information block (SIB)), and paging messages.

[0060] like Figure 2C As illustrated, some of the REs carry DM-RS (indicated as R for a particular configuration, but other DM-RS configurations are possible) for channel estimation at the base station. The UE may send a DM-RS for a physical uplink control channel (PUCCH) and a DM-RS for a physical uplink shared channel (PUSCH). The PUSCH DM-RS may be sent in the first or first two symbols of the PUSCH. Depending on whether a short PUCCH or a long PUCCH is sent and depending on the specific PUCCH format used, the PUCCH DM-RS may be sent in different configurations. The UE may send a sounding reference signal (SRS). The SRS may be sent in the last symbol of a subframe. The SRS may have a comb structure, and the UE may send the SRS on one of the teeth of the comb. The SRS may be used by the base station for channel quality estimation to achieve frequency-dependent scheduling of the UL.

[0061] Figure 2DExamples of various UL channels within a subframe of a frame are illustrated. The PUCCH may be located at a position as indicated in one configuration. The PUCCH carries uplink control information (UCI), such as scheduling requests, channel quality indicators (CQI), precoding matrix indicators (PMI), rank indicators (RI), and hybrid automatic repeat request (HARQ) acknowledgement (ACK) (HARQ-ACK) feedback (i.e., one or more HARQ ACK bits indicating one or more ACKs and / or negative ACKs (NACKs)). The PUSCH carries data and may additionally be used to carry a buffer status report (BSR), a power headroom report (PHR), and / or UCI.

[0062] Figure 5 Diagrams 500 and 510 are included that illustrate example aspects of a slot structure that may be used for SL communication (e.g., between UEs 104, etc.). In some examples, the slot structure may be within a 5G / NR frame structure. In other examples, the slot structure may be within an LTE frame structure. Although the following description may focus on 5G NR, the concepts described herein may be applicable to other similar areas, such as LTE, LTE-A, CDMA, GSM, and other wireless technologies. Figure 5 The example time slot structure in is only an example, and other SL communications may have different frame structures and / or different channels for SL communications. A frame (10ms) can be divided into 10 subframes of equal size (1ms). Each subframe may include one or more time slots. A subframe may also include a microslot, which may include 7, 4 or 2 symbols. Each time slot may contain 7 or 14 symbols, depending on the time slot configuration. For time slot configuration 0, each time slot may include 14 symbols, and for time slot configuration 1, each time slot may include 7 symbols. Figure 500 illustrates a single resource block that may correspond to a single time slot transmission of, for example, a 0.5ms transmission time interval (TTI). The physical SL control channel may be configured to occupy multiple physical resource blocks (PRBs), for example, 10, 12, 15, 20 or 25 PRBs. The PSCCH may be limited to a single subchannel. For example, the PSCCH duration may be configured to be 2 symbols or 3 symbols. For example, a subchannel may include 10, 15, 20, 25, 50, 75, or 100 PRBs. Resources for SL transmission may be selected from a resource pool including one or more subchannels. As a non-limiting example, a resource pool may include between 1 and 27 subchannels. A PSCCH size may be established for a resource pool, for example, between 10%-100% of a subchannel within a duration of 2 symbols or 3 symbols. Figure 5Diagram 510 in illustrates an example where the PSCCH occupies approximately 50% of a subchannel as an example to illustrate the concept of the PSCCH occupying a portion of a subchannel. The physical SL shared channel (PSSCH) occupies at least one subchannel. In some examples, the PSCCH may include a first portion of SL control information (SCI), and the PSSCH may include a second portion of the SCI.

[0063] A resource grid may be used to represent the frame structure. Each time slot may include a resource block (RB) (also referred to as a physical RB (PRB)) extending 12 consecutive subcarriers. The resource grid is divided into a plurality of resource elements (REs). The number of bits carried by each RE depends on the modulation scheme. Figure 5 As illustrated, some of the REs may include control information in the PSCCH and some of the REs may include a demodulation RS (DMRS).At least one symbol may be used for feedback. Figure 5 An example with two symbols for a physical SL feedback channel (PSFCH) with adjacent gap symbols is illustrated. Symbols before and / or after the feedback may be used for a transition between reception of data and transmission of feedback. The gap enables the device to switch from operating as a transmitting device to preparing to operate as a receiving device, for example, in a subsequent time slot. As illustrated, data may be sent in the remaining REs. The data may include data messages as described herein. The positioning of any of the data, DMRS, SCI, feedback, gap symbols, and / or LBT symbols may differ from Figure 5 Illustrated example. In some aspects, multiple time slots can be aggregated together.

[0064] Figure 3370 is a block diagram of a base station 310 in an access network communicating with a UE 350. In the DL, Internet Protocol (IP) packets may be provided to a controller / processor 375. The controller / processor 375 implements layer 3 and layer 2 functionality. Layer 3 includes a radio resource control (RRC) layer, and layer 2 includes a service data adaptation protocol (SDAP) layer, a packet data convergence protocol (PDCP) layer, a radio link control (RLC) layer, and a medium access control (MAC) layer. The controller / processor 375 provides RRC layer functionality associated with broadcasting of system information (e.g., MIB, SIB), RRC connection control (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), inter-radio access technology (RAT) mobility, and measurement configuration for UE measurement reporting; PDCP layer functionality associated with header compression / decompression, security (encryption, decryption, integrity protection, integrity verification), and handover support functions; RLC layer functionality associated with delivery of upper layer packet data units (PDUs), error correction through ARQ, concatenation, segmentation and reassembly of RLC service data units (SDUs), re-segmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto transport blocks (TBs), demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction through HARQ, priority handling, and logical channel prioritization.

[0065] The transmit (Tx) processor 316 and the receive (Rx) processor 370 implement layer 1 functionality associated with various signal processing functions. Layer 1 (which includes the physical (PHY) layer) may include error detection on the transmission channel, forward error correction (FEC) decoding / decoding of the transmission channel, interleaving, rate matching, mapping to the physical channel, modulation / demodulation of the physical channel, and MIMO antenna processing. The Tx processor 316 handles the mapping to the signal constellation based on various modulation schemes (e.g., binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), M-phase shift keying (M-PSK), M-quadrature amplitude modulation (M-QAM)). The decoded and modulated symbols can then be divided into parallel streams. Subsequently, each stream can be mapped to an OFDM subcarrier, multiplexed with a reference signal (e.g., a pilot) in the time domain and / or frequency domain, and then combined using an inverse fast Fourier transform (IFFT) to generate a physical channel carrying a time domain OFDM symbol stream. The OFDM stream is spatially pre-coded to generate multiple spatial streams. Channel estimates from a channel estimator 374 may be used to determine the coding and modulation schemes, as well as for spatial processing. Channel estimates may be derived based on reference signals and / or channel condition feedback sent by the UE 350. Each spatial stream may then be provided to a different antenna 320 via a separate transmitter 318Tx. Each transmitter 318Tx may modulate a radio frequency (RF) carrier with a corresponding spatial stream for transmission.

[0066] At the UE 350, each receiver 354Rx receives a signal through its corresponding antenna 352. Each receiver 354Rx recovers the information modulated onto the RF carrier and provides the information to a receive (Rx) processor 356. The Tx processor 368 and the Rx processor 356 implement layer 1 functionality associated with various signal processing functions. The Rx processor 356 can perform spatial processing on the information to recover any spatial stream destined for the UE 350. If multiple spatial streams are destined for the UE 350, they can be combined into a single OFDM symbol stream by the Rx processor 356. The Rx processor 356 then converts the OFDM symbol stream from the time domain to the frequency domain using a fast Fourier transform (FFT). The frequency domain signal includes a separate OFDM symbol stream for each subcarrier of the OFDM signal. The symbols on each subcarrier and the reference signal are recovered and demodulated by determining the signal constellation point most likely to be sent by the base station 310. These soft decisions can be based on the channel estimate calculated by the channel estimator 358. The soft decisions are then decoded and deinterleaved to recover the data and control signals originally sent on the physical channel by the base station 310. The data and control signals are then provided to the controller / processor 359, which implements layer 3 and layer 2 functionality.

[0067] The controller / processor 359 may be associated with a memory 360 that stores program codes and data. The memory 360 may be referred to as a computer readable medium. In the UL, the controller / processor 359 provides demultiplexing between transport channels and logical channels, packet reassembly, decryption, header decompression, and control signal processing to recover IP packets. The controller / processor 359 is also responsible for error detection using ACK and / or NACK protocols to support HARQ operations.

[0068] Similar to the functions described in conjunction with DL transmissions performed by the base station 310, the controller / processor 359 provides RRC layer functionality associated with system information (e.g., MIB, SIB) acquisition, RRC connection, and measurement reporting; PDCP layer functionality associated with header compression / decompression and security (encryption, decryption, integrity protection, integrity verification); RLC layer functionality associated with delivery of upper layer PDUs, error correction through ARQ, concatenation, segmentation and reassembly of RLC SDUs, resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto TBs, demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction through HARQ, priority handling, and logical channel prioritization.

[0069] Channel estimates derived by the channel estimator 358 from a reference signal or feedback sent by the base station 310 may be used by the Tx processor 368 to select appropriate coding and modulation schemes and to facilitate spatial processing. The spatial streams generated by the Tx processor 368 may be provided to different antennas 352 via separate transmitters 354Tx. Each transmitter 354Tx may modulate an RF carrier with a corresponding spatial stream for transmission.

[0070] UL transmissions are processed at the base station 310 in a manner similar to that described in conjunction with the receiver functionality at the UE 350. Each receiver 318Rx receives a signal through its corresponding antenna 320. Each receiver 318Rx recovers information modulated onto an RF carrier and provides the information to an Rx processor 370.

[0071] The controller / processor 375 may be associated with a memory 376 that stores program codes and data. The memory 376 may be referred to as a computer readable medium. In the UL, the controller / processor 375 provides demultiplexing between transport channels and logical channels, packet reassembly, decryption, header decompression, control signal processing to recover IP packets. The controller / processor 375 is also responsible for error detection using ACK and / or NACK protocols to support HARQ operations.

[0072] At least one of the Tx processor 368, the Rx processor 356, and the controller / processor 359 may be configured to perform a combined Figure 1 Aspects of the dimension generator component 198.

[0073] Figure 4 4 is a diagram illustrating an example of UE positioning based on reference signal measurement. UE 404 may be at time T SRS_Tx UL-SRS 412 is sent and at time T PRS_Rx A DL positioning reference signal (PRS) (DL-PRS) 410 is received. TRP 406 may be at time T SRS_Rx Receive UL-SRS 412 and at time T PRS_Tx 410. UE 404 may receive DL-PRS 410 before transmitting UL-SRS 412, or may transmit UL-SRS 412 before receiving DL-PRS 410. In both cases, a positioning server (e.g., location server 168) or UE 404 may determine the UL-SRS 412 based on ||T SRS_Rx –T PRS_Tx |–|T SRS_Tx –T PRS_Rx || to determine RTT 414. Thus, multi-RTT positioning may utilize UE ERx-Tx time difference measurements (ie, |T SRS_Tx –T PRS_Rx |) and DL-PRS reference signal received power (RSRP) (DL-PRS-RSRP), and the measured TRP Rx-Tx time difference measurement (ie, |T SRS_Rx –T PRS_Tx |) and UL-SRS-RSRP. The UE 404 measures the UE Rx-Tx time difference measurement (and optionally the DL-PRS-RSRP of the received signal) using the assistance data received from the positioning server, and the TRP 402, 406 measures the gNB Rx-Tx time difference measurement (and optionally the UL-SRS-RSRP of the received signal) using the assistance data received from the positioning server. These measurements may be used at the positioning server or the UE 404 to determine the RTT, which is used to estimate the position of the UE 404. Other methods for determining the RTT are possible, such as, for example, using DL-TDOA and / or UL-TDOA measurements.

[0074] DL-AoD positioning may utilize the measured DL-PRS-RSRP of downlink signals received from multiple TRPs 402, 406 at the UE 404. The UE 404 measures the DL-PRS-RSRP of the received signals using assistance data received from a positioning server, and the resulting measurements, along with the departure azimuth angle (A-AoD), departure zenith angle (Z-AoD), and other configuration information, are used to position the UE 404 relative to neighboring TRPs 402, 406.

[0075] DL-TDOA positioning may utilize DL Reference Signal Time Difference (RSTD) (and optionally DL-PRS-RSRP) of downlink signals received from multiple TRPs 402, 406 at the UE 404. The UE 404 measures the DL RSTD (and optionally DL-PRS-RSRP) of the received signals using assistance data received from a positioning server, and the resulting measurements are used along with other configuration information to position the UE 404 relative to neighboring TRPs 402, 406.

[0076] UL-TDOA positioning may utilize the UL relative time of arrival (RTOA) (and optionally UL-SRS-RSRP) of uplink signals sent from the UE 404 at multiple TRPs 402, 406. The TRPs 402, 406 measure the UL-RTOA (and optionally UL-SRS-RSRP) of the received signals using assistance data received from a positioning server, and the resulting measurements are used along with other configuration information to estimate the position of the UE 404.

[0077] UL-AoA positioning may utilize the measured azimuth angle of arrival (A-AoA) and zenith angle of arrival (Z-AoA) of uplink signals sent from the UE 404 at multiple TRPs 402, 406. The TRPs 402, 406 measure the A-AoA and Z-AoA of the received signals using assistance data received from a positioning server, and the resulting measurements are used along with other configuration information to estimate the position of the UE 404.

[0078] Additional positioning methods may be used to estimate the position of the UE 404, such as, for example, UE-side UL-AoD and / or DL-AoA. Note that data / measurements from various techniques may be combined in various ways to increase accuracy, determine and / or enhance certainty, supplement / complete measurements, and / or replace / provide missing information.

[0079] Generating accurate street maps can help enable autonomous driving and advanced driver assistance system (ADAS) applications on vehicles. Such street maps can be generated by collecting image data from mobile devices (such as vehicles with cameras configured to collect image data). When combined with the location data of the vehicle, such image data can be used to estimate the road environment around the vehicle that collected the image. However, sending image data from a moving vehicle may utilize a large amount of wireless resources. Sending the road estimate of the image to a central location (such as a server or data center) can be a more efficient use of wireless resources. Such road estimates may include, for example, the location of the vehicle, the direction the camera is facing, and the attributes of 3D objects (e.g., traffic signs, traffic signals, billboards) captured for each image. The central location can fuse the estimates together to generate a coherent map estimate.

[0080] Figure 6 Diagram 600 of a UE 602 and a camera 601 configured to capture at least one image is shown. The camera 601 may be configured to capture one or more images about the UE 602, such as a first traffic sign 606, a road 608, a second traffic sign 610, and a tree 612. The camera 601 may be configured to capture a set of images within a certain period of time, such as a set of images taken every 0.1 seconds within a full second. The camera 601 may be configured to capture a set of images periodically, such as a set of images taken every 0.1 seconds within a full second every five seconds. The camera 601 may provide images to the UE 602 directly or indirectly. For example, the camera 601 may be integrated into the UE 602 as a device of the UE 602 and may provide images via a system bus, or the camera 601 may be configured to send one or more images to a storage device on a drive (such as a hard drive of a vehicle), and the UE 602 may be configured to analyze the images stored on the drive. UE 602 may be configured to analyze one or more images captured by camera 601 and send a transmission 603 with attributes of the image, such as the position of UE 602 when the image was captured, the orientation or direction of camera 601, one or more calculated dimensions of an object captured in the image, or a covariance matrix based on one or more calculated dimensions of an object captured in the image. The diagonal of the covariance matrix may include covariances for each of the calculated dimensions, and other values ​​in the covariance matrix may indicate cross-correlations between states. UE 602 may be configured to send diagonal elements of the covariance matrix (e.g., eight scalar diagonal elements of an 8×8 extended matrix estimated using an extended Kalman filter).

[0081] In some aspects, sending 603 may include a set of calculated dimensions, such as a tuple excluding inverse depth and an estimate of inverse depth (τ cf ,R cf,α f ). The UE 602 may associate the inverse depth value with the inverse depth covariance value. The inverse depth covariance value may have an inverse relationship with the accuracy of the estimate, wherein a larger inverse depth covariance value indicates a lower accuracy of the estimate, and a smaller inverse depth covariance value indicates a greater accuracy of the estimate. The UE 602 may have a mapping that associates the inverse depth value with the inverse depth covariance value. For example, an inverse depth value between 1 / 1 and 1 / 20 may be associated with an inverse depth covariance value of 0.1, and an inverse depth value between 1 / 20 and 1 / 50 may be associated with an inverse depth covariance value of 0.2. The mapping that associates the inverse depth value with the inverse depth covariance value may be generated by repeatedly estimating the inverse depth at a known distance from an object captured by the camera to calculate the accuracy of the estimate. The UE 602 may send 603 in response to the inverse depth covariance associated with the estimated inverse depth being less than or equal to a threshold.

[0082] In some aspects, the UE 602 may associate the depth value with a depth covariance value. Similar to the inverse depth covariance value, the depth covariance value may have an inverse relationship with the accuracy of the estimate of the depth, where a larger depth covariance value indicates a lower accuracy of the estimate and a smaller depth covariance value indicates a greater accuracy of the estimate. The UE 602 may have a mapping that associates the depth value with the depth covariance value. For example, a depth value between 1 meter (m) and 20m may be associated with a depth covariance value of 1m, while an inverse depth value between 20m and 50m may be associated with an inverse depth covariance value of 5m. The mapping that associates the depth value with the depth covariance value may be generated by repeatedly estimating the depth at a known distance from an object captured by the camera to calculate the accuracy of the estimate. The UE 602 may calculate the estimated depth value based on the estimated inverse depth. The UE 602 may send a transmission 603 in response to the depth covariance associated with the estimated depth (based on the estimated inverse depth) being less than or equal to a threshold. In some aspects, transmission 603 may include depth covariance, thereby allowing a device receiving transmission 603 to estimate the accuracy of the estimated depth or the estimated inverse depth.

[0083] In one aspect, the UE 602 may be configured to send a transmission 603 with attributes of an image in response to a triggering event. In one aspect, the UE 602 may send the transmission 603 in response to the camera 601 capturing an image with a recognized traffic sign (e.g., a 2D projection associated with a historical reference traffic sign saved to memory). In another aspect, the UE 602 may send the transmission 603 in response to the camera 601 capturing an image that no longer has a 2D projection of a 3D object in a previously captured image. (i.e., the 3D object is no longer in the field of view of the UE 602). In another aspect, the UE 602 may send the transmission 603 in response to one or more calculated or estimated dimensions converging (i.e., the diagonal of the covariance matrix falls within an error margin, and the continuously calculated values ​​fall within an error margin). The UE 602 may publish the calculated / estimated traffic sign state / dimension to other ADAS components of the UE 602, or publish the calculated / estimated traffic sign state / dimension on a V2X or V2V link via the transmission 603. The states / dimensions may be published along with the diagonal elements of the corresponding covariance matrix.

[0084] UE 602 may send transmission 603 to a roadside unit (RSU) 622 via sidelink resources and / or to a base station 624 via cellular (e.g., UE-UTRAN (Uu)) resources (such as uplink or downlink resources). Base station 624 may be configured to send Uu transmissions to UE 602. Uu communications may be based on a combination of FIG. 2A to FIG. 2D The RSU 622 may be configured to send a sidelink transmission 623 to the UE 602. The sidelink communication may be based on a combination of Figure 5 The time slot structure of the various aspects described. For example, the sidelink transmission from the UE 602 or RSU 622 may include a control channel (e.g., PSCCH) and / or a corresponding data channel (e.g., PSSCH). The control channel may include information for decoding the data channel (e.g., sidelink control information (SCI)), which includes reservation information, such as information about time and / or frequency resources reserved for data channel transmission. For example, the SCI may indicate the TTI and the number of RBs that will be occupied by the data transmission. The SCI can also be used by the receiving device to avoid interference by avoiding transmission on reserved resources. In addition to sidelink reception, the UE 602 and the RSU 622 may each be capable of sidelink transmission. The sidelink transmission may be unicast, broadcast, or multicast to nearby devices. The sidelink transmission may include transmissions between V2V or V2X type applications (including map crowdsourcing). The UE 602 may include, for example, in combination with Figure 1 The dimension generator component 198 is described.

[0085] The sidelink communication may be based on different types or modes of resource allocation mechanisms. In a first resource allocation mode (which may be referred to herein as "mode 1"), centralized resource allocation may be provided by a network entity. For example, a base station 624 may determine resources for sidelink communication and may allocate resources to a UE 602 and an RSU 622 intended for sidelink transmissions. The base station 624 may provide the allocated sidelink resources as one or more Uu transmissions 625 (e.g., downlink transmissions).

[0086] In a second resource allocation mode (which may be referred to herein as "Mode 2"), a distributed resource allocation may be provided. In Mode 2, UE 602 may autonomously determine the resources intended for sidelink transmissions. To coordinate the selection of sidelink resources by UE 602, UE 602 may use listening techniques to monitor the resources transmitted by other sidelink UEs (such as Figure 1 The device communicating based on the sidelink may determine one or more radio resources used by other devices in the time domain and the frequency domain in order to select transmission resources that avoid conflicts with other devices.

[0087] Sidelink transmissions and / or resource reservations may be periodic or aperiodic, where the UE may reserve resources for transmission in the current slot and up to two future slots (as discussed below).

[0088] Thus, in a second mode (e.g., Mode 2), each UE may autonomously select resources for sidelink transmission, for example, without a central entity (such as a base station indicating resources for the device). A first UE may reserve the selected resources in order to inform other UEs about the resources that the first UE intends to use for sidelink transmission.

[0089] In some examples, resource selection for sidelink communication can be based on a listening-based mechanism. For example, before selecting resources for data transmission, the UE can first determine whether the resources have been reserved by other UEs.

[0090] For example, as part of the listening mechanism for resource allocation mode 2, the UE may determine (e.g., listen) whether the selected side link resource has been reserved by other UEs before selecting the side link resource for data transmission. If the UE determines that the side link resource has not been reserved by other UEs, the UE may use the selected side link resource for sending data, for example, in PSSCH transmission. The UE may estimate or determine which radio resources (e.g., side link resources) may be in use and / or reserved by other UEs by detecting and decoding the side link control information (SCI) sent by other UEs. The UE may estimate or determine which radio resources are in use and / or reserved by other UEs using a listening-based resource selection algorithm. The UE may receive an SCI from another UE, the SCI including reservation information based on a resource reservation field included in the SCI. The UE may continuously monitor (e.g., listen) and decode the SCI from a peer UE. The SCI may include reservation information, for example, indicating that a particular UE has selected a time slot and RB for future transmission. The UE may exclude resources used and / or reserved by other UEs from the candidate resource set used by the UE for sidelink transmission, and the UE may select / reserve resources for sidelink transmission from resources that are not used and thus form the candidate resource set. The UE may continuously perform listening on the SCI with resource reservations in order to maintain a candidate resource set, from which the UE may select one or more resources for sidelink transmission. Once the UE selects a candidate resource, the UE may send an SCI indicating its own reservation of resources for sidelink transmission. The number of resources reserved by the UE (e.g., subchannels per subframe) may depend on the size of the data to be sent by the UE. Although this example is described for a UE receiving reservation information from another UE, the reservation information may also be received from an RSU or other device communicating based on a sidelink.

[0091] The images captured by the camera 601 may be analyzed by the UE 602 and the properties of the 3D objects captured by the camera 601 may be transmitted, for example, to generate an accurate street map. In one aspect, the wireless device may estimate and transmit the properties of the 3D objects about the vehicle by analyzing at least one image obtained from the camera. A nonlinear least squares solver may be used to estimate the dimensions of the objects in the image. The nonlinear least squares solver may include, for example, a Levenberg Marquardt solver, a Gauss Newton solver, an extended Kalman filter, an unscented Kalman filter, or a particle filter. Each object in the image may be described using multiple dimensions estimated by such a nonlinear least squares solver.

[0092] Figure 72D image 700 obtained from a camera is shown. 2D image 700 may have portions including 2D projections of 3D objects, such as 2D projection 702 of a first traffic sign, 2D projection 704 of a road, 2D projection 706 of a second traffic sign, and 2D projection 708 of a tree. 2D image 700 may be a 3D image obtained using Figure 6 6, wherein 2D projection 702 may be a 2D projection of a first traffic sign 606, 2D projection 704 may be a 2D projection of a road 608, 2D projection 706 may be a 2D projection of a second traffic sign 610, and 2D projection 708 may be a 2D projection of a tree 612. 2D image 700 may be associated with a location (e.g., the location of a UE with GNSS positioning performed when the image was captured). 2D image 700 may also be associated with an orientation of a camera used to capture 2D image 700. The orientation may be described using three dimensions such as pitch, roll, and yaw. Each of the 2D projections (2D projection 702 and 2D projection 706) of the sign may be described using eight dimensions of a nonlinear least squares solver.

[0093] The 2D projection of the 3D object can be analyzed by locating the corner points of the 2D projection of the 3D object in the image. The device analyzing the image can identify the position of the corner points of the 2D projection in any suitable manner, such as by identifying a rectangular object in the image, by identifying an object with a color similar to that used for a known traffic sign, or by using a deep neural network to train an image analysis algorithm to identify the 2D projection of the object of interest. For example, the device analyzing the 2D image 700 can determine that the 2D projection 702 of the first traffic sign has four corner points 702a, 702b, 702c, and 702d. Similarly, the device analyzing the 2D image 700 can also determine that the 2D projection 702 of the second traffic sign has four corner points 706a, 706b, 706c, and 706d. The four corner points 706a, 706b, 706c, and 706d of the 2D projection 706 can be used to describe the 2D projection 706 of the first traffic sign. The 2D projection 706 can be described as a rotated rectangle in the 2D image 700. Such a device may use a perception engine to detect 2D pixels on the 2D image 700 that correspond to corners of the traffic sign.

[0094] Each corner point may be described in any suitable manner to identify the location along the surface of the 2D image at which the corner point is located. In one aspect, a device analyzing the 2D image 700 may determine each corner point as a pixel of the 2D image 700 having an xy-axis plot with an x-axis and a y-axis defined by the resolution of the 2D image 700, where the top left corner point (corner point 702a of the 2D projection 702) may be designated as (0,0). In another aspect, a device analyzing the 2D image 700 may calibrate a camera to determine a camera calibration matrix and camera distortion parameters, thereby allowing such points to be processed to be expressed in terms of ideal camera coordinates rather than in terms of image pixels or device pixels.

[0095] A rectangular 3D object such as a traffic sign can be modeled as a general rectangle with an arbitrary orientation and eight degrees of freedom (i.e., eight dimensions). Two of the eight dimensions can represent the size of the rectangular 3D object, such as height and width. Three of the eight dimensions can represent the positioning of the rectangular 3D object, such as the distance forward / backward, the distance to the left / right, and the distance up / down relative to the direction of the camera (such as the direction facing the vehicle). The positioning can be reported relative to a position (such as the position of the vehicle capturing the image). The positioning of the object described in the image can be described as having a positive forward / backward distance because the camera cannot capture the image of an object behind the lens of the camera. Three of the eight dimensions can represent the orientation of the rectangular 3D object, such as pitch, roll, and yaw. Eight dimensions can be used in a nonlinear least squares solver to represent a rectangular traffic sign in an image.

[0096] The variables p, q, and r can be used as reference frames. It can be expressed as the translation from q frame to r frame expressed in the p frame coordinate system, where make represents the rotation of the 3D object from frame q to frame r. Therefore, Can be used to relate translations in different reference frames.

[0097] In one aspect, a rectangular 3D object can be described using a reference frame f fixed to the upper left corner of the object, oriented so that the rectangle lies in the xy plane of the frame, with the x direction to the right, the y direction down, and the z direction being the rectangle normal. Using such a frame f, the size a of the rectangular 3D object relative to the frame f The width a measured in the x direction of f f (0) and the height a measured in the y direction of f f (1) Definition. In other words, the stack of a f (0) and a f (1) Can form a vector Rectangular 3D objects can be represented by a tuple (t cf ,R cf ,af ) is described relative to the camera frame c. Let Represents a point in three dimensions. For example, t(0) may represent the distance of the 3D object from the camera in the x direction (e.g., to the right), t(1) may represent the distance of the 3D object from the camera in the y direction (e.g., up), and t(2) may represent the distance of the 3D object from the camera in the z direction (e.g., forward).

[0098] Estimate R for a rectangular 3D object facing the camera cf The value of may be simpler for rectangular 2D projections, since such 3D objects may be assumed to have a neutral orientation. Figure 7 t of a 3D object in a 2D image 700) cf and a f The value of may be difficult because such values ​​may not be directly observable using a single 2D image from a camera. For example, we may not know the t of a 3D object. cf The z value of the depth is unknown because the distance of the 3D object from the camera may not be known based on a single 2D projection of the 3D object. f The values ​​of , because measuring the length and height of a 2D projection may not represent the length and height of a 3D object without knowing the appropriate scale multiplier for the object being measured.

[0099] In some aspects, the eight dimensions of size, location and orientation of the 2D projection of a rectangular traffic sign in a 2D image may be estimated by tracking the traffic sign from many locations of the image captured by the camera. However, tracking the 3D object from many locations of the captured image may delay the initialization of the nonlinear least squares solver. From an implementation point of view, the delayed initialization may be troublesome. In some aspects, tracking the 3D object from many locations may use a separate tracking mechanism, which may further complicate and delay the initialization of the nonlinear least squares solver. If the traffic sign is observed by the vehicle at a short distance for the first time (e.g., due to obstructed visibility), and the vehicle must perform one or more control actions that move the vehicle from a specified route to fully track the traffic sign to describe the eight dimensions of size, location and orientation, then this delay may be problematic.

[0100] In addition, using a nonlinear least squares solver with eight dimensions to send a description of a 3D object may mean sending the estimated mean and covariance information of the nonlinear least squares solver. The covariance matrix of the nonlinear least squares solver with eight dimensions may be an 8×8 matrix, which may have 64 scalar elements). Sending all 64 scalar elements may utilize a large amount of wireless resources. Although the symmetry of the covariance matrix of the extended Kalman filter can be used to reduce the transmission of 64 scalar elements to 36 elements, sending all 36 unique scalar covariance values ​​may also use a large amount of bandwidth specifications.

[0101] In one aspect, the depth-normalized position (τ), depth-normalized size (α), and orientation (R cf ) to describe the 2D projection of a 3D object on a 2D image.

[0102] The depth normalized localization τ can be a point in three dimensions The inverse depth parameterization of .

[0103] For example,

[0104] In other words, t to τ can be understood as a one-to-one mapping, except for the case t(2) = 0 (where the inverse depth parameterization is undefined). However, such cases may be rare, since the 3D objects captured by the camera should be at least a certain distance in front of the camera's lens. The mapping from t to τ may be an involution (i.e., the inverse mapping may be equal to the mapping itself)

[0105] Likewise, the depth normalized size α can be a point in two dimensions that can represent the size of a rectangular object The inverse depth parameterization of .

[0106] For example, α=a / α(2).

[0107] Likewise, a to α can be understood as a one-to-one mapping, except for the case t(2) = 0 (where the inverse depth parameterization is undefined).

[0108] Therefore, the tuple (t cf ,R cf ,a f ) can be reparameterized as (τ cf ,R cf ,α f ). Although the tuple (t cf ,R cf ,a f ) may have at least three of the eight dimensions that are not observable when analyzing a 2D projection of a 3D image (e.g., a f (0), a f (1) and t cf(2)), but the tuple (τ cf ,R cf ,α f ) may have one of eight dimensions that is not observable when analyzing a 2D projection of a 3D image by using estimation calculations with high accuracy.

[0109] Available To define the position of the corner points of the rectangular object, i,j∈{0,1} and

[0110]

[0111] Because by definition, f 00 =f.

[0112] The distance measurement between such corner points can be denoted as The measurement can be modeled as

[0113] here, It can be point f ij Compared to The depth normalized translation of frame c is given.

[0114] here, Can be additive measurement noise. ij can have a mean of zero and a covariance matrix of The noise term z is a Gaussian distribution. ij may be independent across i,j∈{0,1}. In some aspects, Σ may not be diagonal, since the measurements may be expressed in ideal camera coordinates and may include relevant effects that undistort the original image coordinates.

[0115] Measurements of a single rectangular 2D projection (such as 2D projection 702 or 2D projection 706) may be used To calculate the tuple (τ cf ,R cf ,α f ) of the eight dimensions. You can use y 00 Directly calculate τ with high accuracy cf (0:1) depth normalized positioning value. In other words,

[0116] Given the measurements from the initial camera frame, the maximum likelihood estimate for the traffic sign state can be Although the above optimization problem may not solve the unobservable quantity τ cf (2) However, can be calculated as.

[0117] here, It can be calculated as follows To express:

[0118]

[0119] Divide the numerator and denominator by t cf (2) We can get:

[0120]

[0121] therefore, can be calculated with high accuracy as

[0122]

[0123] therefore,

[0124]

[0125] Will Applying to optimization problems This optimization problem may be solved numerically using a nonlinear least squares solver such as a Levenberg Marquardt solver or a Gauss Newton solver, an extended Kalman filter, an unscented Kalman filter, or a particle filter. Such solvers may be iterative in nature and may require reasonable values ​​such as one of the reasonable estimates provided below to begin with.

[0126] orientation Based on the use of measurement y 00 ,y 01 ,y 10 and 11 The vanishing points of the 2D projections are calculated with high accuracy. The homogeneous version of Constructible vanishing point v x and v y .

[0127] This corresponds to the x-axis of the f-frame.

[0128] This corresponds to the y-axis of the f frame.

[0129] These noise vanishing points can be used to directly estimate R cf For example,

[0130] Due to the influence of noise, the matrix May not be a rotation matrix. A rotation matrix with high accuracy may be chosen. Calculate, as the closest to

[0123] in the Frobenius norm The rotation matrix of . For example, in Can be The singular value decomposition of .

[0131] Depth normalization size α cf The difference y can be used 10 -y 00 and 01 -y 00 to calculate.

[0132] For example, define Here, e0 may be the first column of the 3×3 identity matrix, and e1 may be the second column of the 3×3 identity matrix.

[0133]

[0134] Due to the relationship between the depth normalization business size α cf It can be a 2D vector equation, so premultiply both sides by a fixed vector It can be concluded that:

[0135]

[0136] Solving for α cf (0) It is concluded that:

[0137]

[0138] therefore, It can be a reasonable calculation.

[0139] Similarly, solving for α cf (1) It is concluded that:

[0140]

[0141] therefore, It can be a reasonable calculation.

[0142] therefore, in

[0143] and

[0144]

[0145] Similarly, in

[0146]

[0147] Using the above calculations, for A reasonable initial estimate of can be calculated and used as the least squares optimization problem The solution to this optimization problem provides Refined estimate of .

[0148] Therefore, it is possible to use a single 2D image such as Figure 7 2D image 700) to calculate the tuple (τ cf ,R cf ,α f )(Except τ cf (2)). The calculated values ​​of the seven dimensions can be used to initialize a nonlinear least squares solver such as a Levenberg Marquardt solver, a Gauss Newton solver, an extended Kalman filter, an unscented Kalman filter, or a particle filter.

[0149] The inverse depth τ can be calculated with high accuracy without using a single rectangular 2D projection cf (2). This parameter can be calculated with high accuracy from the motion. For example, the inverse depth τ can be calculated with high accuracy by analyzing a series of images. cf (2) where at least two of the images contain 2D projections of the same 3D object from two different locations, where the location at which each image was taken is known. However, a reasonable estimate (such as 1 / 20 m -1 )(the 20m estimate from the camera) makes the inverse depth parameter τ cf (2) Initialization. Therefore, use 1 / 20m -1 A reasonable estimate of the initialization of a nonlinear least squares solver (e.g., an extended Kalman filter) may have a coverage from 0m -1 Up to 1 / 20m -1 The prior of the interval .

[0150] In the case of non-inverse depth expression, this prior interval can be considered as 20m to ∞m. In other words, the initial uncertainty can be large enough to model the diffusion prior so that all depths up to infinity can be estimated, while also excluding negative depths from the interval.

[0151] The device can be based on a camera (such as Figure 6 One or more properties of the camera 601 in or associated with at least one captured object (such as Figure 6 The inverse depth τ is provided by associating information with one or more properties of a reference object (the first traffic sign 606) in FIG. cf(2) A reasonable estimate of the distance. For example, the resolution of the camera may provide a number of pixels high and wide for the 2D image 700. When related to other information (such as the location of the UE, or historical measurements of objects similar to the 2D projection of the captured object), the number of pixels for the resolution of the camera may allow a device that counts the number of pixels between the corners of the traffic sign to better estimate the distance. Historical measurements of objects similar to the 2D projection of the captured object may be used to provide additional context. In one example, a first image captured by a first camera with a first resolution may capture a 2D representation of a first sign with a known height dimension and width dimension from a known distance. The 2D representation may have a first height dimension and a first width dimension. A second image captured by a second camera with the same resolution as the first camera may have a 2D representation of a second sign, the second sign having the same height dimension and the same width dimension as the sign in the first image. The second sign may have the same height dimension and width dimension as the first sign. The UE may estimate the distance to the second sign as the same distance from the first camera to the first sign when the first image was captured. In another example, a traffic sign may be identified by the text on the sign or the color of the sign, as some signs may be authorized to have certain background colors. The color of the logo can be used to determine the actual size of the logo, which may not provide the intended use for alpha f values, but can also provide context about other distance measurements.

[0152] In one aspect, a traffic sign may have historically been captured by a camera with a resolution of 1920×1180 and have a size of 400 pixels by 200 pixels at a distance of 40 m. A 2D projection of the traffic sign with the same background color may also have a size of 400 pixels by 200 pixels in an image with a resolution of 1920×1180. Thus, a device analyzing the 2D projection may provide τ cf (2) = 1 / 40m -1 On the other hand, the traffic sign may have historically been captured by one type of camera with a 2D projection of the traffic sign and have dimensions of 250 pixels by 250 pixels at a distance of 15 m. A device analyzing a 2D projection of a traffic sign with the same dimensions captured by the same type of camera may provide τ by determining that the object in the 2D projection cf (2) = 1 / 20m -1 The one or more properties of the reference object and / or one or more properties of the camera may be stored in a memory accessible to the device. The device may be configured to reference such properties to provide an inverse depth estimate τ cf A reasonable estimate of (2) is obtained to initialize the nonlinear least squares solver.

[0153] In another aspect, reference objects such as traffic signs may have size properties, shape properties, and position in the image properties associated with each reference object. For example, where the reference objects include traffic signs, the reference objects may include diamond roadside signs (classified by diamond signs captured on one side of the road), overhead signs (classified by rectangular signs captured above the road), and rectangular speed limit signs (classified by rectangular signs captured on one side of the road). Each sign may be identified by properties shared with the 2D projection and may be associated with a size. For example, a diamond road sign may be 3ft×3ft, and an overhead sign may be 10ft×5ft. The perception engine may be used to associate the 2D projection with a type of sign (e.g., by using corner points to detect the shape and position of the 2D projection) that may be referenced as a reference object to derive the size of the 3D object (i.e., a f ). The reference object may be associated with the region. For example, when determining that the location of the captured image is within the first region, the device may compare the 2D projection with the reference object associated with the first region, and when determining that the location of the captured image is within the second region, the device may compare the 2D projection with the reference object associated with the second region. Once the size of the 3D object is known, the analysis device may determine the depth normalized size (i.e., α) by comparing the pixel distance of the corner points with the known size of the 3D object based on the properties of the reference object. f ). f =a f / t cf (2) The equation applied to these values ​​of the device can be solved for t cf (2) to determine the depth of the 3D object, which can then be used to derive the inverse depth t cf (2) A reasonable estimate.

[0154] This reasonable estimate Can be used with The refined estimate is used together with the inverse depth The diffusion initialization is used to initialize the Kalman filter state and provide Reasonable initialization value for .

[0155] In one aspect, the association module of the device can be configured to first attempt to associate the measurement (e.g., the measurement between the corner points) with the existing traffic sign state of the extended Kalman filter stored in the memory. If the association module fails, the module can declare that a new traffic sign state is to be added, or can declare that the association failed.

[0156] A device that computes the dimensions of a 2D projection of a 3D object captured in a 2D image may use the tuple (τ cf ,R cf ,α f) to calculate seven of the eight dimensions with high accuracy, and the eighth dimension can be estimated with lossy, diffused lower accuracy. A more accurate inverse depth τ can be calculated by analyzing at least one additional 2D image with a 2D projection of the same 3D object based on the distance between the locations where the images were captured and the distance between the corners of the 2D projection. cf (2) to improve the lower accuracy.

[0157] Compute the tuple (τ) with high accuracy cf ,R cf ,α f ) and estimating the eighth dimension allows the analysis device to initialize the nonlinear least squares solver without delay. This calculation also separates the seven instantaneously observable dimensions from the non-instantaneously observable depth dimension.

[0158] Compute the tuple representing the 3D object (τ cf ,R cf ,α f ) may be configured to send the entire covariance matrix of the nonlinear least squares solver (e.g., the joint covariance matrix ), or may be configured to send the diagonal elements of the covariance matrix, or a function of the diagonal elements of the covariance matrix (e.g., the square root of the diagonal elements used to calculate the standard deviation rather than the variance). For example, a device may be configured to send the eight scalar diagonal elements of the covariance matrix of the extended Kalman filter. The receiving device may then assume that the off-diagonal elements are zero. This reduces the number of scalar values ​​to be transmitted from 36 to 8. Since the tuple (τ cf ,R cf ,α f ) is confined to the inverse depth τ cf (2), so the error of the extended Kalman filter will likely be in a single dimension (i.e., the inverse depth dimension, or δτ cf (2)). Therefore, the diagonal approximation to the covariance matrix can be a faithful representation of the covariance matrix. This error covariance can be used by downstream components (such as map crowdsourcing applications) to better fuse information from multiple such devices.

[0159] Figure 8 800 is a flow chart of a method of wireless communication. The method may be performed by a wireless device, a component of a wireless device, a UE, or a component of a UE (e.g., UE 104, UE 350, UE 404, UE 602; apparatus 1104). The methods described herein may provide several benefits, such as improved resource utilization and / or power conservation.

[0160] At 802, the UE may obtain an image having at least one captured object from a camera. The at least one captured object may include multiple dimensions, and wherein the image includes a 2D projection of the at least one captured object. For example, 802 may be performed by Figure 6 The UE 602 in the embodiment may obtain an image from the camera 601, such as Figure 7 700 in the 2D image. At least one captured object, such as the first traffic sign 606 or the second traffic sign 610, may have multiple dimensions, such as depth-normalized position, depth-normalized size, and orientation. Figure 7 The 2D image 700 may include Figure 6 702 of the 3D object of the first traffic sign 606 in FIG. Fig.11 Component 198 in is executed.

[0161] At 804, the UE may calculate at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object. For example, 804 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 7 2D projection 702 in τ to calculate the tuple (τ) representing the first traffic sign 606 cf ,R cf ,α f ). These seven dimensions may include τ cf Two of the three dimensions, such as the two dimensions τ cf (0:1) and may include an R cf (e.g., Rodrigues angle or Euler angle) and may include α f In addition, 804 can be represented by Fig.11 Component 198 in is executed.

[0162] At 806, the UE may estimate inverse depths of multiple dimensions of at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. For example, 806 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 7 2D projection 702 in to estimate the tuple (τ cf ,R cf ,α f ) of the inverse depth τ cf (2), the tuple may represent the first traffic sign 606. The estimate may be based on the resolution of the camera and / or may be based on the dimensions of the traffic sign associated with the first traffic sign 606. In addition, 806 may be composed of Fig.11Component 198 in is executed.

[0163] Finally, at 808, the UE may send an indication of multiple dimensions of at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. For example, 808 may be performed by Figure 6 602 in the UE, the UE may send a pair of tuples (τ cf ,R cf ,α f ) of the calculated dimensions (including such as τ cf (0:1) two dimensions, R cf The three dimensions and α f ) and the tuple (τ cf ,R cf ,α f ) is the estimated inverse depth τ cf (2) In addition, 808 can be Fig.11 Component 198 in is executed.

[0164] Fig. 9 Flowchart 900 is a method of wireless communication. The method may be performed by a wireless device, a component of a wireless device, a UE, or a component of a UE (e.g., UE 104, UE 350, UE 404, UE 602; apparatus 1104). The methods described herein may provide several benefits, such as improved resource utilization and / or power conservation.

[0165] At 902, the UE may obtain an image having at least one captured object from a camera. The at least one captured object may include multiple dimensions, and wherein the image includes a 2D projection of the at least one captured object. For example, 902 may be performed by Figure 6 The UE 602 in the embodiment may obtain an image from the camera 601, such as Figure 7 700 in the 2D image. At least one captured object, such as the first traffic sign 606 or the second traffic sign 610, may have multiple dimensions, such as depth-normalized position, depth-normalized size, and orientation. Figure 7 The 2D image 700 may include Figure 6 702 of the 3D object of the first traffic sign 606 in FIG. Fig.11 Component 198 in is executed.

[0166] At 904, the UE may calculate at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object. For example, 904 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 72D projection 702 in τ to calculate the tuple (τ) representing the first traffic sign 606 cf ,R cf ,α f ). These seven dimensions may include τ cf Two of the three dimensions, such as the two dimensions τ cf (0:1) and may include an R cf (e.g., Rodrigues angle or Euler angle) and may include α f In addition, 904 can be Fig.11 Component 198 in is executed.

[0167] At 906, the UE may estimate inverse depths of multiple dimensions of at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. For example, 906 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 7 2D projection 702 in to estimate the tuple (τ cf ,R cf ,α f ) of the inverse depth τ cf (2), the tuple may represent the first traffic sign 606. The estimate may be based on the resolution of the camera and / or may be based on the dimensions of the traffic sign associated with the first traffic sign 606. In addition, 906 may be composed of Fig.11 Component 198 in is executed.

[0168] At 908, the UE may send an indication of multiple dimensions of at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. For example, 908 may be Figure 6 602 in the UE, the UE may send a pair of tuples (τ cf ,R cf ,α f ) of the calculated dimensions (including such as τ cf (0:1) two dimensions, R cf The three dimensions and α f ) and the tuple (τ cf ,R cf ,α f ) is the estimated inverse depth τ cf (2) Instructions. In addition, 908 can be Fig.11 Component 198 in is executed.

[0169] At 910, the UE may associate the plurality of corner points with a nonlinear least squares solver. For example, 910 may be performed by Figure 6 The UE 602 in the embodiment may associate multiple corner points (such as four corner points 702a, 702b, 702c and 702d) with a nonlinear least squares solver (such as a Levenberg Marquardt solver, a Gauss Newton solver, an extended Kalman filter, an unscented Kalman filter or a particle filter). In addition, 910 may be performed by Fig.11 Component 198 in is executed.

[0170] At 912, the UE may estimate at least one dimension based on a nonlinear least squares solver. For example, 912 may be performed by Figure 6 Executed by UE 602 in the UE, the UE may use a nonlinear least squares solver to estimate at least one dimension, such as the height or width of the traffic sign. In addition, 912 may be performed by Fig.11 Component 198 in is executed.

[0171] At 914, the UE may calculate at least one dimension of the plurality of dimensions based on the 2D projection of the at least one captured object. For example, 914 may be performed by Figure 6 Executed by UE 602 in the UE, the UE may be based on a 2D projection (such as a first traffic sign 606) of at least one captured object (such as a first traffic sign 606). Figure 7 2D projection 702 in the image) to calculate at least one dimension in multiple dimensions (such as the height or width of the traffic sign). In addition, 914 can be calculated by Fig.11 Component 198 in is executed.

[0172] At 916, the UE may calculate at least two of the three dimensions of the depth normalized position based on the 2D projection of the at least one captured object. The plurality of dimensions may include the depth normalized position. The depth normalized position may include three of the plurality of dimensions. One of the three dimensions may include an inverse depth. For example, 916 may be composed of Figure 6 The UE 602 in the embodiment may calculate at least two of the three dimensions of the depth normalized positioning based on the 2D projection of at least one captured object, such as two dimensions τ cf (0:1). Multiple dimensions may include depth normalized positioning, such as τ cf The depth normalized localization may include three dimensions among multiple dimensions, denoted as τ cf (0:1:2). One of the three dimensions may include inverse depth, such as τ cf (2). In addition, 916 can be Fig.11 Component 198 in is executed.

[0173] At 918, the UE may calculate three dimensions of the orientation based on the 2D projection of the at least one captured object. The multiple dimensions may include the orientation. The orientation may include three dimensions of the multiple dimensions. For example, 918 may be composed of Figure 6 The UE 602 in the embodiment may calculate the orientation (such as R) based on the 2D projection of at least one captured object. cf ). The plurality of dimensions may include orientation. The orientation may include three dimensions of the plurality of dimensions, denoted as R cf (0:1:2). In addition, 918 can be Fig.11 Component 198 in is executed.

[0174] At 920, the UE may calculate two dimensions of the depth-normalized size based on the 2D projection of the at least one captured object. The multiple dimensions may include the depth-normalized size. The depth-normalized size may include two dimensions of the multiple dimensions. For example, 920 may be composed of Figure 6 The UE 602 in the embodiment may calculate a depth normalized size (such as α) based on a 2D projection of at least one captured object. f ). The multiple dimensions may include a depth normalized size. The depth normalized size may include two dimensions of the multiple dimensions, denoted as α f (0:1). In addition, 920 can be Fig.11 Component 198 in is executed.

[0175] Fig.10 1000 is a flow chart of a method of wireless communication. The method may be performed by a wireless device, a component of a wireless device, a UE, or a component of a UE (e.g., UE 104, UE 350, UE 404, UE 602; apparatus 1104). The methods described herein may provide several benefits, such as improved resource utilization and / or power conservation.

[0176] At 1002, the UE may obtain an image having at least one captured object from a camera. The at least one captured object may include multiple dimensions, and wherein the image includes a 2D projection of the at least one captured object. For example, 1002 may be performed by Figure 6 The UE 602 in the embodiment may obtain an image from the camera 601, such as Figure 7 700 in the 2D image. At least one captured object, such as the first traffic sign 606 or the second traffic sign 610, may have multiple dimensions, such as depth-normalized position, depth-normalized size, and orientation. Figure 7 The 2D image 700 may include Figure 6 702 of the 3D object of the first traffic sign 606 in FIG. Fig.11 Component 198 in is executed.

[0177] At 1004, the UE may calculate at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object. For example, 1004 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 7 2D projection 702 in τ to calculate the tuple (τ) representing the first traffic sign 606 cf ,R cf ,α f ). These seven dimensions may include τ cf Two of the three dimensions, such as the two dimensions τ cf (0:1) and may include an R cf (e.g., Rodrigues angle or Euler angle) and may include α f In addition, 1004 can be represented by Fig.11 Component 198 in is executed.

[0178] At 1006, the UE may estimate inverse depths of multiple dimensions of at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. For example, 1006 may be performed by Figure 6 UE 602 in the embodiment may perform the operation based on Figure 7 2D projection 702 in to estimate the tuple (τ cf ,R cf ,α f ) of the inverse depth τ cf (2), the tuple may represent the first traffic sign 606. The estimate may be based on the resolution of the camera and / or may be based on the dimensions of the traffic sign associated with the first traffic sign 606. In addition, 1006 may be composed of Fig.11 Component 198 in is executed.

[0179] At 1008, the UE may send an indication of multiple dimensions of at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. For example, 1008 may be performed by Figure 6 602 in the UE, the UE may send a pair of tuples (τ cf ,R cf ,α f ) of the calculated dimensions (including such as τ cf (0:1) two dimensions, R cf The three dimensions and α f ) and the tuple (τ cf ,R cf,α f ) is the estimated inverse depth τ cf (2) Instructions. In addition, 1008 may be Fig.11 Component 198 in is executed.

[0180] At 1010, the UE may calculate the depth covariance based on the inverse depth covariance associated with the inverse depth. For example, 1010 may be calculated by Figure 6 The UE 602 in the embodiment may calculate the depth covariance based on the inverse depth covariance associated with the inverse depth. The inverse depth covariance may be calculated based on historical data. The inverse depth covariance may be associated with a corresponding depth covariance using a standard measurement (such as meters). In addition, 1010 may be performed by Fig.11 Component 198 in is executed.

[0181] At 1012, the UE may send a depth covariance associated with the estimated inverse depth. For example, 1012 may be Figure 6 The UE 602 in the embodiment may send a depth covariance associated with the estimated inverse depth. The receiving device (such as RSU 622) may then use the depth covariance to determine whether the estimate is accurate enough to rely on further estimation / measurement / triggering. In addition, 1012 may be performed by Fig.11 Component 198 in is executed.

[0182] At 1014, the UE may detect at least one triggering event associated with the camera. For example, 1014 may be performed by Figure 6 The UE 602 in the embodiment may detect at least one triggering event associated with the camera. The triggering event may be, for example, the fact that the image is captured by the camera, or may be an indication that the UE 602 has crossed a threshold distance since the previous transmission, or may be the recognition of an object in the image captured by the camera (e.g., the recognition of a sign or street identifier in the image). In addition, 1014 may be performed by Fig.11 Component 198 in is executed.

[0183] At 1016, the UE may send an indication of the multiple dimensions in response to the detected at least one triggering event. For example, 1016 may be Figure 6 1016 may be performed by UE 602 in the embodiment, the UE may send an indication of multiple dimensions in response to detecting at least one triggering event. Fig.11 Component 198 in is executed.

[0184] At 1018, the UE may obtain a set of images from the camera within a certain period of time, wherein at least one image in the set of images may not include at least one captured object. Figure 61018 may be performed by UE 602 in the image collection, which may obtain an image set from a camera within a time period of, for example, one second or half a second, and the image set may include images such as 2D image 700. At least one image in the image set may not include at least one captured object. For example, one image may not include 2D projection 702, while another image may include 2D projection 702. When recognizing that 2D projection 702 is a sign, UE 602 may calculate the dimension of the object associated with 2D projection 702. In addition, 1018 may be performed by Fig.11 Component 198 in is executed.

[0185] At 1020, the UE may identify at least one triggering event based on the image set. For example, 1020 may be performed by Figure 6 The UE 602 in the embodiment may identify at least one triggering event based on the image set, such as the presence of an object or the movement of the UE 602 from a zone not associated with the identified object to another zone associated with the identified object. Fig.11 Component 198 in is executed.

[0186] At 1022, the UE may send an indication of the plurality of dimensions in response to the depth covariance associated with the estimated inverse depth being less than or equal to a threshold. Figure 6 The UE 602 in the embodiment may be executed by the UE 602, which may send an indication of the multiple dimensions in response to the depth covariance associated with the estimated inverse depth being less than or equal to the threshold. The UE 602 may calculate the depth covariance based on the estimated inverse depth, and may be configured to trigger the sending based on a comparison between the depth covariance and the threshold. The threshold may be configurable via a user interface of the UE 602. In addition, 1022 may be configured by Fig.11 Component 198 in is executed.

[0187] Fig.111100 is a diagram illustrating an example of a hardware implementation for an apparatus 1104. The apparatus 1104 may be a UE, a component of a UE, or may implement UE functionality. In some aspects, the apparatus 1104 may include a cellular baseband processor 1124 (also referred to as a modem) coupled to one or more transceivers 1122 (e.g., a cellular RF transceiver). The cellular baseband processor 1124 may include on-chip memory 1124'. In some aspects, the apparatus 1104 may also include one or more subscriber identity module (SIM) cards 1120 and an application processor 1106 coupled to a secure digital (SD) card 1108 and a screen 1110. The application processor 1106 may include on-chip memory 1106'. In some aspects, the device 1104 may also include a Bluetooth module 1112, a WLAN module 1114, an SPS module 1116 (e.g., a GNSS module), one or more sensor modules 1118 (e.g., an atmospheric pressure sensor / altimeter; a motion sensor such as an inertial management unit (IMU), a gyroscope, and / or an accelerometer; light detection and ranging (LIDAR), radio-aided detection and ranging (RADAR), sound navigation and ranging (SONAR), a magnetometer, audio, and / or other technologies for positioning), an additional memory module 1126, a power source 1130, and / or a camera 1132. The Bluetooth module 1112, the WLAN module 1114, and the SPS module 1116 may include an on-chip transceiver (TRx) (or in some cases, only a receiver (Rx)). The Bluetooth module 1112, the WLAN module 1114, and the SPS module 1116 may include their own dedicated antennas and / or communicate using an antenna 1180. The cellular baseband processor 1124 communicates with the UE 104 and / or with the RU associated with the network entity 1102 through the transceiver 1122 via one or more antennas 1180. The cellular baseband processor 1124 and the application processor 1106 can each include a computer-readable medium / memory 1124', 1106', respectively. The additional memory module 1126 can also be considered as a computer-readable medium / memory. Each computer-readable medium / memory 1124', 1106', 1126 can be non-transitory. The cellular baseband processor 1124 and the application processor 1106 are each responsible for general processing, including executing software stored on the computer-readable medium / memory. The software enables the cellular baseband processor 1124 / application processor 1106 to perform the various functions described above when executed by the cellular baseband processor 1124 / application processor 1106. The computer-readable medium / memory can also be used to store data manipulated by the cellular baseband processor 1124 / application processor 1106 when executing the software.The cellular baseband processor 1124 / application processor 1106 may be a component of the UE 350 and may include the memory 360 and / or at least one of the Tx processor 368, the Rx processor 356, and the controller / processor 359. In one configuration, the device 1104 may be a processor chip (modem and / or application) and include only the cellular baseband processor 1124 and / or the application processor 1106, while in another configuration, the device 1104 may be the entire UE (e.g., see. Figure 3 UE 350) and includes additional modules of device 1104.

[0188] As discussed above, component 198 may be configured to obtain an image having at least one captured object from a camera, the at least one captured object may include multiple dimensions. The image may include a 2D projection of the at least one captured object. Component 198 may be configured to calculate at least one of the multiple dimensions of the at least one captured object based on the 2D projection of the at least one captured object. Component 198 may be configured to estimate the inverse depth of the multiple dimensions of the at least one captured object based on information associated with one or more attributes of the camera or one or more attributes of at least one reference object associated with the at least one captured object. Component 198 may be configured to send an indication of the multiple dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. Component 198 may be within the cellular baseband processor 1124, the application processor 1106, or both the cellular baseband processor 1124 and the application processor 1106. Component 198 may be one or more hardware components specifically configured to perform the stated process / algorithm, implemented by one or more processors configured to perform the stated process / algorithm, stored in a computer-readable medium for implementation by one or more processors, or some combination of the foregoing. As shown, the device 1104 may include a variety of components configured for various functions. In one configuration, the device 1104, and specifically the cellular baseband processor 1124 and / or the application processor 1106, may include a component for obtaining an image having at least one captured object from a camera. The device 1104 may also include a component for calculating at least one of a plurality of dimensions of the at least one captured object based on a 2D projection of the at least one captured object. The device 1104 may also include a component for estimating an inverse depth of a plurality of dimensions of the at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. The device 1104 may also include a component for sending an indication of a plurality of dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth. The device 1104 may also include means for associating the plurality of corner points with a nonlinear least squares solver. The device 1104 may also include means for estimating at least one dimension based on the nonlinear least squares solver. The device 1104 may also include means for calculating at least one of the plurality of dimensions based on a 2D projection of at least one captured object. The device 1104 may also include means for calculating at least two of the three dimensions of the depth normalized positioning based on the 2D projection of at least one captured object. The device 1104 may also include means for calculating at least three of the three dimensions of the orientation based on the 2D projection of at least one captured object.The device 1104 may also include a component for calculating two dimensions of the depth normalized size based on a 2D projection of at least one captured object. The device 1104 may also include a component for detecting at least one trigger event associated with the camera. The device 1104 may also include a component for sending an indication of multiple dimensions in response to detecting at least one trigger event. The device 1104 may also include a component for obtaining a set of images from the camera within a certain period of time. The device 1104 may also include sending an indication of multiple dimensions in response to the depth covariance associated with the estimated inverse depth being less than or equal to a threshold. The device 1104 may also include sending a depth covariance associated with the estimated inverse depth. The device 1104 may also include calculating the depth covariance based on the inverse depth covariance associated with the inverse depth. The component may be a component 198 of the device 1104 configured to perform the functions recorded by the component. As described above, the device 1104 may include a Tx processor 368, an Rx processor 356, and a controller / processor 359. Thus, in one configuration, the means may be the Tx processor 368, the Rx processor 356, and the controller / processor 359 configured to perform the functions recited by the means.

[0189] It should be understood that the specific order or hierarchy of the blocks in the disclosed process / flowchart is only an illustration of the exemplary method. It should be understood that the specific order or hierarchy of the blocks in the process / flowchart can be rearranged based on design preferences. Further, some blocks can be combined or omitted. The attached method claims provide the elements of various blocks in a sample order, but are not limited to the specific order or hierarchy provided.

[0190] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects. Therefore, the claims are not limited to the various aspects described herein, but should be given the full scope consistent with the language claims. Unless specifically stated otherwise, reference to an element in the singular form does not mean "one and only one", but "one or more". Terms such as "if", "when" and "while" do not mean a direct temporal relationship or reaction. That is, these phrases, such as "when ......", do not mean an immediate action in response to the occurrence of an action or during the occurrence of an action, but simply imply that if the conditions are met, then the action will occur, but no specific or immediate time limit is required for the occurrence of the action. The word "exemplary" is used herein to mean "used as an example, instance or illustration". Any aspect described as "exemplary" herein is not necessarily interpreted as being preferred or having advantages over other aspects. Unless otherwise specifically stated, the term "some" refers to one or more. Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, where any such combination may contain one or more members of A, B, or C. A set should be interpreted as a set of elements, where the number of elements is one or more. Thus, for a set of X, X will include one or more elements. If the first device receives data from the second device or sends data to the second device, data can be directly received / sent between the first device and the second device, or data can be indirectly received / sent between the first device and the second device through a collection of devices. All structural and functional equivalents that are known or will be known later to those of ordinary skill in the art throughout the various aspects of the elements described in this disclosure are expressly incorporated herein by reference and are covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recorded in the claims. Words such as "module", "mechanism", "element", "device" cannot replace the word "component". Therefore, no claim element will be understood as a functional component unless the element is explicitly recorded using the phrase "component for...".

[0191] As used herein, the phrase "based on" should not be interpreted as referring to a closed set of information, one or more conditions, one or more factors, etc. In other words, the phrase "based on A" (where "A" can be information, a condition, a factor, etc.) should be interpreted as "based at least on A" unless specifically stated differently.

[0192] A device configured to "output" data (such as, transmit, signal or message) may, for example, transmit the data using a transceiver, or may transmit the data to a device that transmits the data. A device configured to "obtain" data (such as, transmit, signal or message) may, for example, receive the data using a transceiver, or may obtain the data from a device that receives the data.

[0193] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.

[0194] Aspect 1 is a method of wirelessly communicating at a wireless device (e.g., a user equipment (UE)), comprising obtaining an image having at least one captured object from a camera. The at least one captured object may include multiple dimensions. The image may include a two-dimensional (2D) projection of the at least one captured object. The method may also include calculating at least one of the multiple dimensions of the at least one captured object based on the 2D projection of the at least one captured object. The method may also include estimating an inverse depth of the multiple dimensions of the at least one captured object based on information associated with one or more properties of the camera or one or more properties of at least one reference object associated with the at least one captured object. The method may also include sending an indication of the multiple dimensions of the at least one captured object, the indication including the calculated at least one dimension and the estimated inverse depth.

[0195] Aspect 2 is a method according to aspect 1, wherein the image may include a 2D image and the at least one captured object may include a three-dimensional (3D) object. A portion of the 2D image may include the 2D projection of the at least one captured object.

[0196] Aspect 3 is a method according to aspect 2, wherein the at least one reference object may include a traffic sign.

[0197] Aspect 4 is a method according to any one of aspects 1 to 3, wherein the 2D projection of the at least one captured object may include a plurality of corner points of the at least one captured object.

[0198] Aspect 5 is a method according to aspect 4, wherein calculating the at least one dimension of the multiple dimensions may include associating the multiple corner points with a nonlinear least squares solver. Calculating the at least one dimension of the multiple dimensions may also include estimating the at least one dimension based on the nonlinear least squares solver.

[0199] Aspect 6 is a method according to any one of Aspects 1 to 5, wherein calculating the at least one dimension among the multiple dimensions may include calculating the at least one dimension among the multiple dimensions based on the 2D projection of the at least one captured object.

[0200] Aspect 7 is a method according to any one of aspects 1 to 6, wherein the multiple dimensions may include at least one of (a) depth-normalized position, (b) orientation, or (c) depth-normalized size.

[0201] Aspect 8 is a method according to aspect 7, wherein the depth normalized positioning may include three dimensions among the multiple dimensions. One dimension among the three dimensions may include the inverse depth.

[0202] Aspect 9 is a method according to aspect 8, wherein calculating the at least one dimension of the multiple dimensions may include calculating at least two dimensions of the three dimensions of the depth normalized positioning based on the 2D projection of the at least one captured object.

[0203] Aspect 10 is a method according to any one of aspects 7 to 9, wherein the orientation may include three dimensions of the plurality of dimensions.

[0204] Aspect 11 is a method according to aspect 10, wherein calculating the at least one dimension of the multiple dimensions may include calculating at least three dimensions of the three dimensions of the orientation based on the 2D projection of the at least one captured object.

[0205] Aspect 12 is a method according to any one of aspects 7 to 11, wherein the depth normalized size may include two dimensions of the multiple dimensions.

[0206] Aspect 13 is a method according to aspect 12, wherein calculating the at least one dimension of the plurality of dimensions may include calculating the two dimensions of the depth-normalized size based on the 2D projection of the at least one captured object.

[0207] Aspect 14 is a method according to any one of aspects 1 to 13, wherein the one or more properties of the camera may include at least one of a camera type or a resolution of the camera.

[0208] Aspect 15 is a method according to any one of aspects 1 to 14, wherein the indication of the multiple dimensions may include at least one of the diagonal elements of a covariance matrix or a function of the diagonal elements of the covariance matrix.

[0209] Aspect 16 is a method according to any one of aspects 1 to 15, further comprising detecting at least one triggering event associated with the camera, wherein sending the indication of the plurality of dimensions is in response to detecting the at least one triggering event.

[0210] Aspect 17 is a method according to aspect 16, further comprising detecting the at least one triggering event associated with the camera by obtaining a set of images from the camera within a certain period of time. At least one image in the set of images may not include the at least one captured object.

[0211] Aspect 18 is a method according to aspect 1, wherein sending the indication of the multiple dimensions may include: sending the indication of the multiple dimensions in response to a depth covariance associated with the estimated inverse depth being less than or equal to a threshold.

[0212] Aspect 19 is a method according to aspect 1, wherein the method may include sending a depth covariance associated with the estimated inverse depth.

[0213] Aspect 20 is a method according to aspect 19, wherein the method may include calculating the depth covariance based on an inverse depth covariance associated with the inverse depth.

[0214] Aspect 21 is an apparatus for performing wireless communications at a wireless device, comprising: at least one memory; and at least one processor, the at least one processor being coupled to the at least one memory, and the at least one processor being configured to implement any one of Aspects 1 to 20 based at least in part on information stored in the at least one memory.

[0215] Aspect 22 is the apparatus of aspect 21, further comprising at least one of an antenna or a transceiver coupled to the at least one processor.

[0216] Aspect 23 is an apparatus for wireless communication, comprising means for implementing any one of aspects 1 to 20.

[0217] Aspect 24 is a computer-readable medium (eg, non-transitory computer-readable medium) storing computer-executable code, wherein the code, when executed by a processor, causes the processor to implement any one of aspects 1 to 20.

Claims

1. An apparatus for wireless communication at a wireless device, comprising: at least one memory; and at least one processor coupled to the at least one memory and based at least in part on information stored in the at least one memory, the at least one processor configured to: obtaining, from a camera, an image having at least one captured object, wherein the at least one captured object comprises a plurality of dimensions, and wherein the image comprises a two-dimensional (2D) projection of the at least one captured object; calculating at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object; estimating an inverse depth of the plurality of dimensions of the at least one captured object based on first information associated with a first set of properties of the camera or a second set of properties of at least one reference object associated with the at least one captured object; as well as An indication of the plurality of dimensions of the at least one captured object is sent, the indication comprising the calculated at least one dimension and the estimated inverse depth.

2. The apparatus of claim 1, wherein the image is a 2D image and the at least one captured object is a three-dimensional (3D) object, wherein a portion of the 2D image includes the 2D projection of the at least one captured object. The apparatus of claim 2 , wherein the at least one reference object comprises a traffic sign. 4 . The apparatus of claim 1 , wherein the 2D projection of the at least one captured object comprises a plurality of corner points of the at least one captured object.

5. The apparatus of claim 4, wherein to calculate the at least one dimension of the plurality of dimensions, the at least one processor is configured to: associating the plurality of corner points with a nonlinear least squares solver; and The at least one dimension is estimated based on the nonlinear least squares solver.

6. The apparatus of claim 1 , wherein to calculate the at least one dimension of the plurality of dimensions, the at least one processor is configured to: The at least one dimension of the plurality of dimensions is calculated based on the 2D projection of the at least one captured object. 7 . The apparatus of claim 1 , wherein the plurality of dimensions comprises at least one of: depth-normalized position, orientation, or depth-normalized size. 8 . The apparatus of claim 7 , wherein the depth-normalized positioning comprises three dimensions of the plurality of dimensions, wherein one dimension of the three dimensions comprises the inverse depth.

9. The apparatus of claim 8, wherein to calculate the at least one dimension of the plurality of dimensions, the at least one processor is configured to: At least two of the three dimensions of the depth-normalized position fix are calculated based on the 2D projection of the at least one captured object.

10. The apparatus of claim 7, wherein the orientation comprises three dimensions of the plurality of dimensions.

11. The apparatus of claim 10, wherein to calculate the at least one dimension of the plurality of dimensions, the at least one processor is configured to: The three dimensions of the orientation are calculated based on the 2D projection of the at least one captured object.

12. The apparatus of claim 7, wherein the depth-normalized size comprises two dimensions of the plurality of dimensions.

13. The apparatus of claim 12, wherein to calculate the at least one dimension of the plurality of dimensions, the at least one processor is configured to: The two dimensions of the depth-normalized size are calculated based on the 2D projection of the at least one captured object.

14. The device of claim 1, wherein the first set of properties of the camera comprises at least one of a camera type or a resolution of the camera.

15. The apparatus of claim 1, wherein the indication of the plurality of dimensions comprises at least one of diagonal elements of a covariance matrix or a function of the diagonal elements of the covariance matrix.

16. The apparatus of claim 1, wherein the at least one processor is further configured to: At least one triggering event associated with the camera is detected, and wherein to send the indication of the plurality of dimensions, the at least one processor is configured to send the indication of the plurality of dimensions in response to the detected at least one triggering event.

17. The apparatus of claim 16, wherein to detect the at least one triggering event associated with the camera, the at least one processor is configured to: obtaining a set of images from the camera over a period of time, wherein at least one image in the set of images does not include the at least one captured object; and The at least one triggering event is identified based on the set of images.

18. The apparatus of claim 1, wherein to send the indication of the plurality of dimensions, the at least one processor is configured to: The indication of the plurality of dimensions is sent in response to a depth covariance associated with the estimated inverse depth being less than or equal to a threshold.

19. The apparatus of claim 1, wherein the at least one processor is further configured to: A depth covariance associated with the estimated inverse depth is sent.

20. The apparatus of claim 19, wherein the at least one processor is further configured to: The depth covariance is calculated based on an inverse depth covariance associated with the inverse depth.

21. The apparatus of claim 1 , further comprising a transceiver coupled to the at least one processor, wherein to transmit the indication of the plurality of dimensions, the at least one processor is configured to: The indication of the plurality of dimensions is transmitted via the transceiver.

22. A method of wireless communication at a wireless device, comprising: obtaining, from a camera, an image having at least one captured object, wherein the at least one captured object comprises a plurality of dimensions, and wherein the image comprises a two-dimensional (2D) projection of the at least one captured object; calculating at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object; estimating an inverse depth of the plurality of dimensions of the at least one captured object based on information associated with a first set of properties of the camera or a second set of properties of at least one reference object associated with the at least one captured object; as well as An indication of the plurality of dimensions of the at least one captured object is sent, the indication comprising the calculated at least one dimension and the estimated inverse depth.

23. The method of claim 22, wherein the 2D projection of the at least one captured object comprises a plurality of corner points of the at least one captured object, wherein calculating the at least one dimension of the plurality of dimensions comprises: associating the plurality of corner points with a nonlinear least squares solver; as well as The at least one dimension is estimated based on the nonlinear least squares solver.

24. The method of claim 22, wherein calculating the at least one of the plurality of dimensions comprises calculating the at least one of the plurality of dimensions based on the 2D projection of the at least one captured object.

25. The method of claim 22, wherein the plurality of dimensions comprises at least one of: depth-normalized position, orientation, or depth-normalized size.

26. A method according to claim 25, wherein the depth normalized positioning includes three dimensions of the multiple dimensions, wherein one dimension of the three dimensions includes the inverse depth, and wherein calculating the at least one dimension of the multiple dimensions includes calculating at least two of the three dimensions of the depth normalized positioning based on the 2D projection of the at least one captured object.

27. A method according to claim 25, wherein the orientation includes three dimensions of the multiple dimensions, and wherein calculating at least one of the multiple dimensions includes calculating at least three of the three dimensions of the orientation based on the 2D projection of the at least one captured object.

28. The method of claim 25, wherein the depth-normalized size comprises two of the plurality of dimensions, wherein calculating the at least one of the plurality of dimensions comprises calculating the two dimensions of the depth-normalized size based on the 2D projection of the at least one captured object.

29. An apparatus for wireless communication at a wireless device, comprising: means for obtaining, from a camera, an image having at least one captured object, wherein the at least one captured object comprises a plurality of dimensions, and wherein the image comprises a two-dimensional (2D) projection of the at least one captured object; means for calculating at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object; means for estimating an inverse depth of the plurality of dimensions of the at least one captured object based on information associated with a first set of properties of the camera or a second set of properties of at least one reference object associated with the at least one captured object; and Means for transmitting an indication of the plurality of dimensions of the at least one captured object, the indication comprising the calculated at least one dimension and the estimated inverse depth.

30. A computer readable medium storing computer executable code at a wireless device, the code, when executed by a processor, causes the processor to: obtaining, from a camera, an image having at least one captured object, wherein the at least one captured object comprises a plurality of dimensions, and wherein the image comprises a two-dimensional (2D) projection of the at least one captured object; calculating at least one dimension of the plurality of dimensions of the at least one captured object based on the 2D projection of the at least one captured object; estimating an inverse depth of the plurality of dimensions of the at least one captured object based on information associated with a first set of properties of the camera or a second set of properties of at least one reference object associated with the at least one captured object; as well as An indication of the plurality of dimensions of the at least one captured object is sent, the indication comprising the calculated at least one dimension and the estimated inverse depth.

Images