Electronic device, method for electronic device, and computer-readable storage medium

Abstract

Provided are an electronic device, a method for an electronic device, and a computer-readable storage medium. An electronic device of one embodiment may comprise at least one processor and at least one memory, wherein the at the least one memory comprises a computer program code The computer program code comprised in the at least one memory, when executed by the at least one processor, causes the electronic device to: determine a communication uncertainty and / or a perception accuracy of vehicle state data acquired by means of wireless communication; and use the vehicle state data to estimate a vehicle state based on the determined communication uncertainty and / or perception accuracy.

Description

Electronic devices, methods for using electronic devices, and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202410927195.8, filed on July 10, 2024, entitled "Electronic Device, Method for Electronic Device and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of wireless communication technology, and more specifically, to an electronic device capable of performing vehicle state estimation, such as in intelligent transportation networks, a method for the electronic device, and a computer-readable storage medium. Background Technology

[0003] In intelligent transportation systems, roadside perception systems (such as roadside-mounted sensors) are used to monitor the status of autonomous and non-autonomous vehicles. For example, in a vehicle-road-cloud system, a digital twin system in the cloud estimates the status of road vehicles based on vehicle status data (perception data that indicates vehicle status) sensed by the roadside perception system and transmitted to the cloud via wireless communication, and updates (maps) the status of the corresponding virtual vehicles accordingly.

[0004] In order to achieve accurate estimation and / or updating of vehicle status in the cloud, it is desirable to use the above vehicle status data in an appropriate manner. Summary of the Invention

[0005] A brief overview of this disclosure is given below to provide a basic understanding of certain aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0006] At least one aspect of this disclosure aims to provide an electronic device, a method for using an electronic device, and a computer-readable storage medium that can estimate the vehicle state using vehicle state data based on the uncertainty inherent in the vehicle state data itself.

[0007] According to one aspect of this disclosure, an electronic device is provided, comprising at least one processor and at least one memory, wherein the at least one memory includes computer program code. The at least one memory and the computer program code are configured, via the at least one processor, to cause the electronic device to perform: compensation for communication uncertainty in vehicle state data based on determined communication uncertainty; and estimation of vehicle state using the compensated vehicle state data.

[0008] According to another aspect of this disclosure, a method for an electronic device is also provided, the method comprising: compensating for communication uncertainty in vehicle state data based on determined communication uncertainty; and estimating the vehicle state using the compensated vehicle state data.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, which stores computer program code that causes an electronic device to perform the method provided according to the above aspects via a processor included in the electronic device.

[0010] In accordance with other aspects of this disclosure, computer program code and computer program products for implementing the methods described above according to this disclosure are also provided.

[0011] According to at least one aspect of the embodiments of this disclosure, it is possible to take into account the uncertainty of the vehicle state data itself during the process of estimating the vehicle state using vehicle state data, thereby using the vehicle state data in an appropriate manner to estimate the vehicle state based on the communication uncertainty in the wireless transmission of the vehicle state data and the uncertainty (perception accuracy) in the perception process of the data.

[0012] Other aspects of embodiments of this disclosure are set forth in the following description section, wherein preferred embodiments of the present disclosure are described in detail without limiting them. Attached Figure Description

[0013] The accompanying drawings described herein are for illustrative purposes only and not for all possible implementations, and are not intended to limit the scope of this disclosure. In the drawings:

[0014] Figure 1 is a schematic diagram illustrating an example scenario of acquiring vehicle status data through roadside sensing;

[0015] Figure 2 is a block diagram illustrating a first configuration example of an electronic device according to an embodiment of the present disclosure;

[0016] Figure 3 is a schematic diagram illustrating an example algorithm for state estimation;

[0017] Figure 4 is a schematic diagram illustrating an example of a neural network model used for time delay prediction;

[0018] Figure 5 is a schematic diagram illustrating an example of the time delay-prediction accuracy relationship between kinematic prediction and neural network prediction;

[0019] Figure 6 is a block diagram illustrating a second configuration example of an electronic device according to an embodiment of the present disclosure;

[0020] Figure 7 is a flowchart illustrating an example signaling interaction according to an embodiment of the present disclosure;

[0021] Figure 8 is a flowchart illustrating a process example of a method for an electronic device according to an embodiment of the present disclosure;

[0022] Figure 9 is a block diagram illustrating an example of a schematic configuration of a server to which the technologies of this disclosure can be applied;

[0023] Figure 10 is a block diagram illustrating a first example of a schematic configuration of an eNB to which the technologies of this disclosure can be applied;

[0024] Figure 11 is a block diagram illustrating a second example of a schematic configuration of an eNB to which the technologies of this disclosure can be applied;

[0025] Figure 12 is a block diagram illustrating an example of a schematic configuration of a smartphone to which the technologies of this disclosure can be applied;

[0026] Figure 13 is a block diagram illustrating an example of a schematic configuration of a car navigation device to which the technology of this disclosure can be applied.

[0027] While this disclosure is readily subject to various modifications and substitutions, specific embodiments thereof have been shown by way of example in the accompanying drawings and are described in detail herein. However, it should be understood that the description of specific embodiments herein is not intended to limit this disclosure to the specific forms disclosed, but rather, this disclosure is intended to cover all modifications, equivalents, and substitutions falling within the spirit and scope of this disclosure. It should be noted that throughout the drawings, corresponding reference numerals indicate corresponding parts. Detailed Implementation

[0028] Examples of this disclosure will now be described more fully with reference to the accompanying drawings. The following description is merely exemplary and is not intended to limit the disclosure, its application, or its uses.

[0029] Example embodiments are provided so that this disclosure will become exhaustive and will fully convey its scope to those skilled in the art. Numerous specific details, such as examples of particular components, apparatus, and methods, are set forth to provide a detailed understanding of embodiments of this disclosure. It will be apparent to those skilled in the art that the specific details are not required, and that the example embodiments may be implemented in many different forms, none of which should be construed as limiting the scope of this disclosure. In some example embodiments, well-known processes, well-known structures, and well-known techniques are not described in detail.

[0030] The description will proceed in the following order:

[0031] 1. Overview

[0032] 2. Configuration examples of electronic devices

[0033] 2.1 First Configuration Example

[0034] 2.2 Second Configuration Example

[0035] 2.3 Example Signaling Interaction

[0036] 4. Method Examples

[0037] 5. Application Examples

[0038] <1. Overview>

[0039] In intelligent transportation networks, roadside units (RSUs) may be equipped with one or more devices for acquiring vehicle state data, such as lidar, millimeter-wave radar, optical cameras, and other sensors, to form a roadside perception system. The acquired vehicle state data is then transmitted wirelessly to the cloud (network side, cloud server side, or edge server side). Similarly, vehicles can also acquire vehicle state data using their onboard sensor devices and transmit it to the cloud. The cloud (e.g., a cloud-based V2X server, such as a server in a digital twin system) estimates the state of road vehicles based on the received vehicle state data and updates (maps) the state of the corresponding virtual vehicles accordingly. Figure 1 schematically illustrates an example scenario of perception using roadside sensors in an intelligent transportation network.

[0040] However, due to factors such as the sensing errors of the sensors used by the roadside unit and the uncertainties in the wireless communication data transmission between the roadside unit and the cloud, the cloud's direct use of the received vehicle status data to estimate the vehicle status may have problems in terms of accuracy and / or real-time performance.

[0041] To this end, the inventors proposed the basic concept of the present invention, which is to take into account the uncertainty of the vehicle state data itself during the process of using vehicle state data for vehicle state estimation, and then use the vehicle state data in an appropriate manner to estimate the vehicle state based on the communication uncertainty of the data in wireless communication transmission and / or the uncertainty (perception accuracy) in the data perception process.

[0042] Next, embodiments of the apparatus / method based on the above-described inventive concept, as well as various preferred examples and processes, will be described with reference to the roadside perception example scenario shown in Figure 1. Note that although the detailed description mainly uses the acquisition of vehicle status data from roadside perception in the cloud as an example scenario, those skilled in the art will understand based on this disclosure that the embodiments of this disclosure can be similarly applied to scenarios where vehicle status data is acquired using sensors mounted on the vehicle, and will not be elaborated further here.

[0043] <2. Configuration Example of the Electronic Device in the First Embodiment>

[0044] [2.1 First Configuration Example]

[0045] (Basic configuration example)

[0046] Figure 2 is a block diagram illustrating a first configuration example of an electronic device according to an embodiment of the present disclosure.

[0047] As shown in Figure 2, the electronic device 200 includes: a determining unit 220 configured to determine the communication uncertainty and / or perception accuracy of vehicle state data acquired via wireless communication; and an estimating unit 230 configured to estimate the vehicle state using the vehicle state data based on the determined communication uncertainty and / or perception accuracy. Furthermore, as shown in Figure 2, the electronic device 200 also includes an optional communication unit 210 configured to receive vehicle state data (e.g., but not limited to, vehicle state data from roadside units) via wireless communication.

[0048] Each unit in the electronic device 200 shown in Figure 2 can be implemented by one or more processing circuits and at least one memory. The processing circuit can be, for example, a chip, a processor, etc., and the at least one memory can be any form of storage device such as RAM, ROM, or flash memory. The at least one memory is used, for example, to store computer program code and data required for the processing circuits to perform processing. Furthermore, it should be understood that the functional units in the electronic device shown in Figure 2 are merely logical modules divided according to their specific functions, and are not intended to limit the specific implementation method.

[0049] When applied to an intelligent transportation system, the electronic device 200 is preferably located in the cloud. The cloud here can be, for example, a network side, a cloud server side, or an edge server side, and the network side can be, for example, a core network or a base station side. Alternatively, if the roadside unit (RSU) or vehicle-side device has corresponding processing capabilities, the electronic device 200 can also be located in the roadside unit (RSU) or vehicle-side device. The vehicle-side device, more generally, can refer to various user devices located on the vehicle and capable of accessing various sensors.

[0050] It should also be noted that the electronic device 200 can be implemented at the chip level or at the device level. For example, the electronic device 200 can function as a server, base station, vehicle, or user equipment itself. In this case, the processing circuitry and memory used to implement the various units in the electronic device 200 can additionally implement the general functions of the server, base station, vehicle, or user equipment. For example, the memory can be used to store programs and related data information that need to be executed to implement various functions of the server, base station, vehicle, or user equipment; alternatively or additionally, the electronic device 200 can additionally have other components or external devices for implementing these functions. The implementation details of the aforementioned other components or external devices are not the focus of this invention and will not be described further.

[0051] Here, the vehicle state data received by the electronic device 200, such as through the communication unit 210, is raw or processed sensing data that can indicate the vehicle state, and may indicate, for example, vehicle position, speed, and / or heading angle. For simplicity, the following detailed description uses processed sensing data as an example of vehicle state data.

[0052] The communication uncertainty determined by the determining unit 220 can indicate uncertainties that affect the real-time performance and accuracy of vehicle state estimation by acquiring vehicle state data via wireless communication, such as the real-time performance and reliability (e.g., integrity) of communication transmission, thereby enabling the estimating unit 230 to perform corresponding communication compensation for the vehicle state data in response to these uncertainties.

[0053] As an example, the communication uncertainty determined by the determining unit 220 may include the latency and / or packet loss rate of wireless communication, preferably including real-time latency and packet loss rate. The determining unit 220 may use various existing methods for determining latency and / or packet loss rate in wireless communication transmission to perform the above determination, such as, but not limited to, real-time measurement. Optionally, the determining unit 220 may use a pre-trained timing prediction model to predict the current latency and / or packet loss rate (described in detail later).

[0054] The perception accuracy determined by the determining unit 220 can indicate the accuracy of the vehicle state data itself obtained by sensing using sensors.

[0055] As an example, the determining unit 220 can be configured to determine the perception accuracy (or perception error) of the vehicle state data based on the error of the mounting posture of the sensor used to acquire the vehicle state data and / or the sensing accuracy of the sensor itself. The determining unit 220 can determine that the perception accuracy (or perception error) of the vehicle state data is higher when the error of the sensor's mounting posture is smaller and the sensing accuracy of the sensor itself is higher. The error of the sensor's mounting posture may, for example, include the error range between the actual posture and the calibrated posture or ideal posture (e.g., the error range of the posture angle). Furthermore, taking an image sensor or camera sensor as an example, the sensor's sensing accuracy may include the resolution of the acquired image.

[0056] Preferably, the estimation unit 230 can be configured to compensate for the uncertainty of vehicle state data based on the determined communication uncertainty, and to estimate the vehicle state using the compensated vehicle state data. The estimation unit 230 can compensate for or correct the corresponding uncertainty in the vehicle state data caused by the communication uncertainty. For example, if the communication uncertainty determined by the determining unit 220 includes the delay and / or packet loss rate of wireless communication, the compensation performed by the estimation unit 230 can include delay compensation to improve the real-time performance of the vehicle state data and / or packet loss compensation to improve the reliability or integrity of the vehicle state data (described in detail later). In this way, the real-time performance and accuracy of subsequent vehicle state estimation can be improved.

[0057] Furthermore, preferably, the estimation unit 230 can be configured to use vehicle state data to correct the current vehicle state predicted based on the previously estimated vehicle state, based on the perception accuracy determined by the determination unit 230, in order to estimate the current vehicle state. Accordingly, the estimation unit 230 can determine the proportion of the vehicle state data in the state estimation according to the perception accuracy, thereby improving the accuracy of the state estimation.

[0058] Figure 3 schematically illustrates an example algorithm for state estimation performed by estimation unit 230. As shown in Figure 3, in a simplified example, Here, x k This represents the estimated vehicle state at the current time k. This represents the vehicle state x estimated based on the previous time step (k-1). k-1 The predicted current vehicle state, α represents the perception accuracy, and z k This represents the vehicle status data at the current time k.

[0059] Preferably, the estimation unit 230 can apply both communication compensation and perception accuracy-weighted correction to the vehicle state data during its state estimation processing. For example, the estimation unit can use the communication-compensated vehicle state data as the current vehicle state data in an algorithm such as that shown in Figure 3.

[0060] The basic configuration of the electronic device 200 and the basic processing of its various units have been described above. Using the above processing, the electronic device of this embodiment can take into account the uncertainty of the vehicle state data itself during the estimation of the vehicle state using vehicle state data, and use the vehicle state data to estimate the vehicle state in an appropriate manner to compensate for or correct this uncertainty, thereby improving the estimation performance. Next, example processing of each unit of the electronic device 200 will be further described.

[0061] (Example processing for determining the unit)

[0062] As mentioned above, preferably, the determining unit 220 of the electronic device 200 can use a pre-obtained timing prediction model to predict real-time latency and packet loss rate.

[0063] Example of determining delay

[0064] In one example, the determining unit 220 can be configured to determine the current delay based on one or more previous delays using a pre-obtained neural network model. For example, a back propagation (BP) neural network or a radial basis function (RBP) neural network can be used to determine the current delay based on a time sequence of previous delays.

[0065] Figure 4 shows an example of an RBF neural network model that can be used for delay prediction, which can be stored, for example, in the memory of an electronic device 200. As shown in Figure 4, the RBF neural network model includes an input layer, hidden layers, and an output layer. The input layer receives a time sequence {τ} of length p with a previous delay. k-1 ,τ k-2 ,….,τ k-p As input data (p is a natural number greater than or equal to 1), it is passed to a set of n radial basis functions. to The hidden layers consist of n (natural numbers greater than or equal to 1), where each basis function performs a nonlinear transformation on the input data. The output layer is assigned corresponding weights ω1 to ω... n The outputs of the hidden layers are weighted and summed to obtain the final output result. The delay of the current time k is used as the prediction. The RBF neural network can take a sequence of delays of any past length as input and the delay of the next time step as output, so there is no restriction on the value of p here.

[0066] The aforementioned RBF model can be obtained by pre-training (and preferably verifying and evaluating) various existing training algorithms using pre-collected and labeled latency data to obtain optimized parameters, which will not be elaborated here. Optionally, the latency here can cover not only the transmission latency of wireless communication, but also the computation latency of acquiring vehicle state data based on raw sensing data.

[0067] Example of determining packet loss rate

[0068] In one example, the determining unit 220 can be configured to use a pre-obtained hidden Markov model to determine the current packet loss rate based on one or more current state parameters related to the packet loss rate in wireless communication.

[0069] Hidden Markov Models (HMMs) are probabilistic models of time series. They describe the process by which a hidden Markov chain randomly generates a sequence of states (a sequence of unobservable hidden states), and then generates an observation (an observed state) from each state, thus producing a random sequence of observations. Each position in the sequence can be considered a time point. The Markov model λ can be described by its parameters (A, B, π), where the state transition probability matrix A and the initial state probability vector π determine the hidden Markov chain, and the observation probability matrix B determines how to generate an observed state from each hidden state. These parameters can be obtained through a pre-defined optimization algorithm.

[0070] In this example, a Hidden Markov Model (HMM) can be used, for example, stored in the memory of electronic device 200. The model's hidden states are M network states E described by L network state parameters related to the packet loss rate, and the observed states are N packet loss states O corresponding to the L packet loss rate values ​​or intervals, respectively. As an example, the network state parameters can be two or more parameters that are highly correlated with the packet loss rate but low in correlation with each other, such as signal-to-noise ratio (SNR) and channel occupancy. For example, when dividing each parameter into two intervals to characterize the merits of each parameter value, M = 2 can be defined based on the L parameters. L There are several network states. L and N can be appropriately determined, for example, L=2, N=3, etc. Using this model, the current packet loss state can be determined based on the current network state, that is, the probability distribution of the specific value or interval of the packet loss rate, and the packet loss rate value or interval with the highest probability can be used as the packet loss rate at the current moment.

[0071] The aforementioned hidden Markov model can be obtained by using the collected hidden state data and observed state data to estimate the optimal parameters of the model using algorithms such as maximum likelihood estimation, which will not be elaborated here.

[0072] Examples of determining perceptual accuracy

[0073] In one example, the determining unit 220 can be configured to determine a perception error that includes vehicle state data. For example, the perception error of the vehicle state data can be determined based on the error of the mounting orientation of the sensor used to acquire the vehicle state data and the perception accuracy. Preferably, the perception error can be determined to have a Gaussian distribution.

[0074] Here, using an image sensor as an example, we describe an example of the process by which the determining unit determines the sensing error.

[0075] Geometric model of image sensor

[0076] As a prelude, we first describe the geometric model of a roadside camera as an example of an image sensor. Taking a monocular camera as an example, it can be represented by a pinhole imaging model. In the pinhole imaging model, we consider the world coordinate system, the camera coordinate system, and the pixel coordinate system. We assume the coordinates of the vehicle being photographed (or the target vehicle) in the world coordinate system (also called world coordinates) are P. w (X w Y w Z w If the coordinates of the two coordinate systems (also known as pixel coordinates) are (u, v), then the transformation relationship between the two coordinate systems is shown in formula (I-1).

[0077] Among them, Z c These are the vehicle's coordinates in the camera's coordinate system. Here, the Z-axis of the camera coordinate system... c The axis is perpendicular to the lens plane of the camera. In the above formula, R, K, and T are all obtained by pre-calibrating the camera through various methods, and are respectively the rotation matrix, intrinsic parameter matrix, and translation matrix, where the rotation matrix R and translation matrix T are extrinsic parameter matrices.

[0078] Since a monocular camera cannot estimate depth, it only focuses on vehicles on the ground, i.e., in Z... w When = 0, the vehicle's Z coordinates in the camera coordinate system are determined using the following formulas (I-2), (I-3), and (I-4). c . Z c =(Z w +Z2) / Z1=Z2 / Z1 (I-4)

[0079] Accordingly, we can obtain the coordinates (u, v) from the pixel coordinate system to the coordinates P in the world coordinate system. w (X w Y w Z w The conversion relationship is shown in formula (I-5).

[0080] As shown in Equation (I-5), the coordinates P in the world coordinate system are obtained by transforming the vehicle's coordinates (u, v) in the pixel coordinate system based on the rotation matrix R and translation matrix T (which are external parameters) and the intrinsic parameter matrix K. w Therefore, the camera's inherent sensing accuracy (i.e., the accuracy or resolution of coordinates (u, v) in the pixel coordinate system) and the calibration errors of each parameter matrix will affect the coordinates P in the world coordinate system obtained through transformation. w The accuracy of the coordinates is affected. Furthermore, the pixel coordinates (u, v) of the target vehicle also influence the error range of the world coordinates. All of these factors can lead to discrepancies between the vehicle coordinates obtained using the camera and the actual coordinates.

[0081] Image sensor mounting / calibration orientation error

[0082] First, consider the calibration error of the extrinsic parameter matrix. When calibrating the extrinsic parameter matrix of a camera, multiple images are often used. Since the extrinsic parameters obtained from each image will not be exactly the same, fitting algorithms such as least squares fitting and RANSAC are used during parameter calibration. This leads to a certain deviation between the calibrated extrinsic parameters and the actual extrinsic parameters, resulting in a discrepancy between the vehicle coordinates perceived by the camera and the actual coordinates.

[0083] To this end, the error between the calibrated extrinsic parameters of the camera and the actual extrinsic parameters can be obtained in advance, for example, in the form of the error of the camera's mounting attitude (attitude angles). More specifically, the camera can be calibrated using an appropriate calibration algorithm to obtain the theoretical extrinsic parameter matrices R and T; any existing calibration algorithm can be used. Furthermore, for example, during the above calibration process, or through an additional calibration process, n rotation matrices R1, R2...Rn can be obtained from n sets of actual calibrations, and the rotation angle difference between these rotation matrices and the theoretical extrinsic parameter matrix R obtained by the calibration algorithm can be calculated, thereby obtaining the errors ψ, γ, and θ of the three attitude angles, namely pitch angle, yaw angle, and roll angle, for n sets. The aforementioned pre-obtained errors of the camera's attitude angles can, for example, be stored in the memory of the electronic device 200.

[0084] In this example, the determining unit 220 can jointly obtain the world coordinate error of the target vehicle based on the error (ψ, γ, θ) of the camera's installation / calibration attitude (attitude angle) obtained in advance and the pixel position (u, v) of the target vehicle perceived by the camera in real time as an example of vehicle state data, and preferably represents the error in a Gaussian distribution.

[0085] As an example, a specific algorithm is described to determine the world coordinate error based on the error (ψ, γ, θ) of the installation attitude (attitude angle) and the pixel position (u, v) of the target vehicle in real time, which can be implemented by the determination unit 220.

[0086] As mentioned earlier, to quantify the impact of extrinsic parameter errors on the sensing results, the deviation can be characterized by the rotation angle of the rotation matrix. Assuming there are deviations ψ, γ, and θ between the calibrated extrinsic parameter attitude and the true attitude, we can rotate from the calibration coordinate system to the actual coordinate system accordingly, and then re-estimate R, T, and Z. c Therefore, P is re-estimated. w To determine the perceived uncertainty caused by the calibration.

[0087] Here, we re-estimate R′ and T′ using formulas (I-6) and (I-7): T′=R′R -1 T (I-7)

[0088] Z is re-estimated using formulas (I-8), (I-9), and (I-10). c ′: Z c ′=(Z w ′+Z2′) / Z1′=Z2′ / Z1′ (I-10)

[0089] Accordingly, the vehicle world coordinates estimated by the camera under the true attitude parameters are P′. w (X′ w ,Y′ w Z′ w This allows us to calculate the error in the estimated vehicle world coordinates under two sets of extrinsic parameters, i.e., the observation error caused by inaccurate extrinsic parameter estimation, as shown in formula (I-11).

[0090] Based on the pre-obtained ranges of ψ, γ, and θ, ΔP can be calculated. w The range, through ΔP w The range can be obtained by representing the range of Δx and Δy by their respective minimum and maximum values. min ~Δx max and Δy min ~Δy maxFurthermore, the error range can be transformed into the form of a Gaussian distribution represented by its respective mean and variance.

[0091] Image sensor resolution error

[0092] Next, consider the impact of resolution on perception accuracy. In an ideal optical system, the ideal image of a point is a point image, but a pixel on the image plane is a block. Four rays of light passing through the four vertices of the corresponding pixel from the center of the camera will form a quadrilateral region on the ground. Changes in points within this quadrilateral region will not cause changes in pixel coordinates, so it is impossible to distinguish points within this region. Therefore, it is necessary to estimate the size of the quadrilateral region projected by this pixel.

[0093] In this example, the determining unit 220 can map the pixel position (u,v) of the target vehicle, which is an example of vehicle state data, as perceived in real time by the camera, to the world coordinate system in a manner with a deviation determined based on the pre-acquired perception accuracy (resolution) of the camera, according to a predetermined mapping relationship (such as the mapping relationship in formula (I-5) above), and obtain the error range in the world coordinate system based on the mapping range in the world coordinate system, preferably represented by a Gaussian distribution.

[0094] Here, as an example of the processing that can be implemented by the determination unit 220, the following specific algorithm is given.

[0095] In this example, assuming the pixel coordinate u has a deviation Δu determined based on resolution, the vehicle's actual world coordinates are as shown in the following formula. P w =R -1 K -1 Z c (u+Δu)-R -1 (I-12)

[0096] For the quadrilateral region obtained by projecting a pixel onto the ground, the deviation of the world coordinates can be estimated by the deviation of one pixel. Let the coordinate errors of the three pixels be denoted as Δu1=[1 0 0]T, Δu2=[0 1 0]T, and Δu3=[1 1 0]T.

[0097] From P1 = R -1 K -1 Z c (u+Δu1)-R -1 P2 = R -1 K -1 Z c (u+Δu2)-R -1 P3 = R -1 K -1 Z c(u+Δu3)-R -1 By obtaining the coordinate differences between one vertex and the other three points of the quadrilateral, the quadrilateral region projected onto the ground by the pixel block can be obtained. Based on this, the ranges of Δx and Δy can be calculated. min ~Δx max and Δy min ~Δy max Since the actual vehicle position could be any point within the quadrilateral, Δx and Δy could be at points within Δx. min ~Δx max and Δy min ~Δy max For any value within the range, the Gaussian distribution of Δx and Δy can be obtained by assuming that Δx and Δy follow a Gaussian distribution within this range.

[0098] Note that the determining unit 220 of the electronic device can simultaneously determine the image sensor's mounting orientation and the perception error caused by its resolution. In this case, the two errors can be added together to obtain the overall error. For example, two Gaussian distributions can be added together to obtain the overall Gaussian distribution.

[0099] (Example processing of the estimation unit)

[0100] As previously described, the estimation unit 230 of the electronic device 200 can be configured to perform communication uncertainty compensation on the vehicle state data based on the determined communication uncertainty, so as to compensate or correct the impact of communication uncertainty (e.g., but not limited to delay compensation and packet loss rate) on the vehicle state data accordingly.

[0101] Example of delay compensation

[0102] In one example, estimation unit 230 can be configured to predict the delayed vehicle state data based on the current vehicle state data to obtain delayed vehicle state data.

[0103] Here, the estimation unit predicts time delays based on kinematic models and / or pre-trained neural network models. The following description uses vehicle state data indicating vehicle position as an example to illustrate the details of the prediction process. However, based on the content of this disclosure, those skilled in the art will understand that the predictions are not limited to vehicle position data, but can be appropriately applied to other types of vehicle state data, which will not be elaborated upon here.

[0104] As a first example, the estimation unit 230 can estimate the vehicle position ξ based on the acquired vehicle state data at the current time t. o (t), vehicle speed v and acceleration a, and the current time delay τ determined by the determining unit 220. dThe time delay τ is estimated based on the following kinematic formula. d The vehicle's position ξ1(t) afterward:

[0105] The vehicle velocity *v* and acceleration *a* in the above formula can be determined in an appropriate manner. For example, it can be assumed that the vehicle moves at a constant speed within a short time window, and thus the acceleration and velocity can be calculated based on the vehicle's position within that time window. Alternatively, if the vehicle state data includes sensor data that can directly indicate the vehicle's velocity and acceleration, the vehicle velocity and acceleration can also be obtained directly based on such sensor data, which will not be elaborated further here.

[0106] As a second example, the estimation unit 230 can, based on a pre-trained neural network model, extract the vehicle position ξ from the acquired vehicle state data at the current time t. o (t) (and optional vehicle speed v and acceleration a, etc.) and current time delay τ d The time delay τ is used as input data to obtain the output of the neural network model. d The vehicle position ξ1(t) is then determined. Here, the aforementioned neural network model can be trained using pre-labeled training data to obtain a model with good predictive performance.

[0107] Taking the prediction of vehicle state data indicating vehicle position as an example, the prediction accuracy can be represented by the Euclidean distance between the actual vehicle position and the predicted vehicle position, and by the Final Displacement Error (FDE) between the actual final position and the predicted final position. The inventors have found that in some applications, simple kinematic models are suitable for short-term prediction, while neural network models are suitable for long-term prediction. For example, Figure 5 shows the relationship between FDE and time delay in kinematic prediction and neural network prediction. As shown in Figure 5, when the time delay τ... d When the time delay is less than the threshold τ0, the kinematic prediction accuracy is higher; conversely, the neural network prediction accuracy is higher. The threshold can be obtained through prior experiments or simulation, which will not be elaborated here. Accordingly, as a third example, the estimation unit 230 can select an appropriate prediction method based on a comparison between the current time delay and the threshold.

[0108] Furthermore, in other applications, computational speed, processing load, and / or prediction accuracy may be considered comprehensively. In such cases, a kinematic model with advantages in computational speed and processing load, or a neural network model with advantages in prediction accuracy (especially in long-term prediction), can be selected based on system design or overall requirements. These will not be elaborated further here.

[0109] As a fourth example, estimation unit 230 may, when the perception accuracy determined by determination unit 220 is low (e.g., below a predetermined threshold), base its estimation on a time-delay compensated previous time step (tT). S The vehicle position ξ1(tT) in the vehicle status data s The vehicle's speed v and acceleration a, as well as the time interval T between the previous and current moments, are also considered. S The time delay τ is estimated based on the following kinematic formula. d The vehicle's position ξ2(t) afterward:

[0110] As an example, T S This can represent the time interval between two frames. That is, in the above formula, the compensated data from the previous frame is used to predict the data for the current frame. In this way, the impact of vehicle state data with low perception accuracy on state estimation can be reduced.

[0111] Example of packet loss compensation (reliability compensation)

[0112] In one example, estimation unit 230 can be configured to correct the current vehicle state data using previous vehicle state data based on the determined current packet loss rate to obtain packet loss compensated vehicle state data. Preferably, estimation unit 230 can perform the above correction if the determined perception accuracy is lower than a predetermined threshold and / or the determined current packet loss rate is higher than a predetermined threshold. Here, packet loss rate is used as an example of communication reliability; however, other methods can be used to characterize communication reliability, such as bit error rate, signal-to-noise ratio, etc., and the examples of this disclosure can be similarly applied to situations where reliability is measured in other ways, which will not be elaborated here.

[0113] Preferably, the "previous vehicle state data" used here is delay-compensated vehicle state data. The estimation unit 230 can use the packet loss rate, an indicator of communication reliability, as a weight θ, and weight the two predicted values ​​ξ1(t) and ξ2(t) using the following formula to obtain the vehicle position ξ3(t) after packet loss compensation: ξ3(t)=(1-θ)ξ1(t)+θξ2(t) (II-3)

[0114] In this way, when the packet loss rate is high, the predicted result ξ2(t) corresponding to the correction value of the more trusted vehicle state data from previous moments accounts for a higher proportion; while when the packet loss rate is low, the predicted result ξ1(t) corresponding to the more trusted vehicle state data from the current moment accounts for a higher proportion. Advantageously, this can prevent large errors in the results due to unreliable received data under high packet loss conditions.

[0115] Preferably, the estimation unit 230 can perform the time delay compensation and packet loss correction described above in an iterative manner, that is, it can use the vehicle state data ξ3(tT) from the previous time moment that has been corrected for both time delay and packet loss. s This is used to predict the vehicle status data at the current moment.

[0116] In this case, the following variation of the above formula (II-2) can be used for prediction:

[0117] Example of state estimation

[0118] As previously described, the estimation unit 230 of the electronic device 200 can be configured to perform vehicle state estimation using vehicle state data that has been compensated for communication uncertainties (e.g., delay compensation and / or reliability compensation).

[0119] Preferably, the estimation unit 230 can use vehicle state data to correct the current vehicle state predicted based on the previously estimated vehicle state, based on the determined perception accuracy, to estimate the current vehicle state. For example, an example algorithm for the above estimation based on the determined perception accuracy can be shown in Figure 3.

[0120] In one example, the perception accuracy of the vehicle state data determined by the determining unit 220 may include a perception error. Preferably, the perception error may have a Gaussian distribution.

[0121] Accordingly, the estimation unit 230 can be configured to use the extended Kalman filter algorithm to estimate the vehicle state as a state variable using vehicle state data as observation data, wherein the perception error of the vehicle state data is used as the measurement noise of the observation data.

[0122] In summary, the Extended Kalman Filter (EPF) algorithm is an algorithm that corrects predicted values ​​based on measured values ​​(observations) according to measurement noise. Its core algorithm can be expressed using the following set of formulas (II-4).

[0123] In the above formula (II-4), the subscripts k and k-1 are used to indicate time. x represents the estimated vehicle state, which can be a vector of a specified length. In one example, it can be a 5-dimensional vector representing the vehicle's x-axis and y-axis positions, velocity, acceleration, and heading angle in the world coordinate system. J F This represents the state transition matrix, used to transform the estimated state x at time k-1. k-1 Move to the next time step k to obtain the predicted value. P is the covariance matrix representing the correlation between observations, Q is an intermediate variable representing prediction noise, z is (e.g., compensated) vehicle state data, and J H This represents the observation matrix, used for transformation from the state domain to the observation domain.

[0124] Furthermore, in the above formula (II-4), K is the Kalman gain, which is determined based on the measurement noise (sensing error) R of the vehicle state data. For example, when z is the vehicle position data previously calculated via formula (II-3), the measurement noise R can have the form of a Gaussian distribution of the position error.

[0125] In the above formula (II-4), x k The Kalman gain is used to measure the observed (measured) value z. k and predicted value The currently estimated vehicle state obtained through fusion.

[0126] [2.2 Second Configuration Example]

[0127] Figure 6 is a block diagram illustrating a second configuration example of an electronic device according to an embodiment of the present disclosure. As shown in Figure 6, the electronic device 600 of the second configuration example may include a communication unit 610, a determination unit 620, and an estimation unit 630 corresponding to the respective units of the electronic device 200 of the first configuration example, and differs from the electronic device 200 in that it additionally includes an optional update unit 630. The update unit 630 may be configured to update the state of the virtual vehicle based on the vehicle state (estimated vehicle state) estimated by the estimation unit 630. In this configuration example, the electronic device 600 can be applied in a digital twin system that represents a real vehicle with a virtual vehicle.

[0128] As an example, the updating unit 630 of the electronic device 600 can update the state of the virtual vehicle corresponding to the real vehicle based on the estimated vehicle state using various appropriate methods, so as to achieve synchronization between the state of the virtual vehicle and the state of the real vehicle. Preferably, the communication unit 210 of the electronic device 600 can provide the updated state of the virtual vehicle to various parties in the intelligent transportation network, such as, but not limited to, vehicles in the intelligent transportation network and optional roadside units (RSUs).

[0129] In one example, the update unit 630 can directly use the estimated vehicle state for the current moment as the virtual vehicle state for the current moment to achieve the above-mentioned synchronous update.

[0130] Optionally, the update unit 630 can update the virtual vehicle's estimated state (x) from a previous time step. k-1 ) and the estimated state at the current time (x)k The difference between the values ​​of the virtual vehicle and the target speed at the current moment can be used to calculate the control inputs of the virtual vehicle using various appropriate methods. These control inputs may include, for example, acceleration and front wheel steering angle. As a simple implementation, a proportional-integral (PID) controller can be used to adjust the acceleration based on the difference between the speed at a previous moment and the target speed at the current moment, where the target speed at the current moment can be given by the velocity component in the estimated state. Alternatively, a pre-built kinematic model of the virtual vehicle can be used to calculate the control inputs of the virtual vehicle based on the estimated state (x) of the virtual vehicle at a previous moment. k-1 ) and the estimated state at the current time (x) k The difference between the calculated control variables (acceleration and front wheel angle) is used to calculate the front wheel steering angle of the virtual vehicle. This disclosure does not limit the specific model used. For example, existing pure tracking algorithms can be used to calculate the front wheel steering angle. Subsequently, the update unit 630 can update the state of the virtual vehicle (e.g., coordinates, heading angle, etc.) based on the calculated control variables such as acceleration and front wheel steering angle, using a pre-built kinematic model of the virtual vehicle. In this way, compared to directly using the estimated state (x) at the current moment... k Updating the state of the virtual vehicle can yield update results that are more consistent with kinematic laws.

[0131] [2.3 Example Signaling Interaction]

[0132] To facilitate understanding, the following example scenario of an intelligent transportation system is used to describe the example signaling interactions between the cloud-based electronic device 200 or 600 and the roadside unit (RSU), and optionally with the vehicle (V).

[0133] Figure 7 illustrates an example signaling interaction between a cloud-based electronic device 200 or 600 and a roadside unit (RSU) and, optionally, a vehicle V. Here, the RSU is a roadside-mounted hardware unit capable of V2X communication and supporting V2X applications, which can acquire vehicle status data using roadside sensors. The vehicle V is the target vehicle whose status is sensed, and optionally has an onboard unit installed and runs applications within the intelligent transportation network. Note that although only one RSU and one vehicle V are shown in the figure, their number is merely illustrative, and this disclosure is not limited thereto.

[0134] As shown in Figure 7, the RSU reports the attributes of the sensors it utilizes (e.g., errors indicating sensor mounting orientation, sensing accuracy, etc.) to the cloud so that the cloud can determine sensing accuracy. In addition, the RSU reports the network status of wireless communication (and / or wireless channel quality) to the cloud so that the cloud can determine communication uncertainties (latency, reliability, etc.). Furthermore, the RSU transmits the acquired vehicle status data to the cloud via wireless communication. The cloud's electronic equipment can determine the communication uncertainties and sensing accuracy of the acquired vehicle status data and use this data to estimate the vehicle's status.

[0135] Optionally, if the electronic device in the cloud is an electronic device 600 as shown in Figure 6, the cloud can also use the estimated vehicle state to update the state of the virtual vehicle. Optionally, the cloud can send the updated virtual vehicle state to vehicle V. Additionally, although not shown in the figure, the cloud can send the updated virtual vehicle state to the RSU.

[0136] <4. Method Examples>

[0137] Corresponding to the above-described apparatus embodiments, this disclosure provides the following method embodiments.

[0138] Figure 8 is a flowchart illustrating a process example for a method for an electronic device, such as an intelligent transportation network, according to an embodiment of the present disclosure.

[0139] As shown in Figure 8, in step S11, the communication uncertainty and / or perception accuracy of the vehicle state data acquired via wireless communication can be determined. In step S12, the vehicle state can be estimated using the vehicle state data based on the determined communication uncertainty and / or perception accuracy.

[0140] As an example, the method shown in Figure 8 can be implemented by an electronic device in the cloud. Furthermore, as shown in Figure 8, in optional step S10, vehicle status data (e.g., vehicle status data from a roadside unit) can be received wirelessly. Additionally, in optional step S13, the virtual vehicle's status can be updated based on the estimated vehicle status.

[0141] Furthermore, in one example, in step S11, communication uncertainty compensation can be applied to the vehicle state data based on the determined communication uncertainty. Accordingly, in step S12, the compensated vehicle state data can be used to estimate the vehicle state.

[0142] For example, in step S11, the determined communication uncertainty may include the delay and / or packet loss rate of wireless communication. Optionally, the current delay can be determined based on one or more previous delays using a pre-obtained neural network model. Optionally, the current packet loss rate can be determined based on one or more current state parameters related to the packet loss rate using a pre-obtained hidden Markov model.

[0143] For example, in step S11, the determined perception accuracy may include the perception error of the vehicle state data. Optionally, the perception error of the vehicle state data may be determined based on the error of the mounting posture of the sensor used to acquire the vehicle state data and the perception accuracy.

[0144] Furthermore, in one example, in step S11, optionally, the determined delayed vehicle state data can be predicted based on the current vehicle state data to obtain delayed vehicle state data. For example, the prediction can be made based on a kinematic model and / or a pre-trained neural network model.

[0145] Furthermore, in one example, in step S11, optionally, the current vehicle state data can be corrected using previous vehicle state data based on the determined current packet loss rate to obtain packet loss compensated vehicle state data. For example, this correction can be performed if the determined perception accuracy is lower than a predetermined threshold and / or the determined current packet loss rate is higher than a predetermined threshold.

[0146] Alternatively, in one example, in step S12, the current vehicle state, predicted based on a previously estimated vehicle state, may be corrected using vehicle state data based on the determined perception accuracy to estimate the current vehicle state.

[0147] According to the embodiments of this disclosure, the subject performing the above method may be the electronic device 300 according to the first embodiment of this disclosure, and therefore all embodiments of the electronic device 300 mentioned above are applicable here.

[0148] According to embodiments of this disclosure, the subject performing the above method may be an electronic device 200 or 600 according to embodiments of this disclosure; therefore, all embodiments of electronic device 200 or 600 described above are applicable here.

[0149] <5. Application Examples>

[0150] The technology disclosed herein can be applied to a variety of products.

[0151] The electronic device 200 or 600 according to the embodiment is preferably implemented in the cloud of the intelligent transportation network, and therefore is preferably implemented as a server or other network-side device, such as a base station device or RSU. However, if the device mounted in the vehicle, such as a general user equipment, has sufficient computing or processing power, it can also be implemented as the aforementioned device. Therefore, it can be various types of devices, including but not limited to servers, base station devices, user equipment, etc.

[0152] For example, electronic device 200 or 600 can be implemented as any type of server, such as a tower server, rack server, or blade server. Electronic device 300 can be a control module installed on the server (such as an integrated circuit module comprising a single chip, or a card or blade inserted into a slot in a blade server).

[0153] Furthermore, electronic devices 200 or 600 can also be implemented as various base stations. Base stations can be implemented as any type of evolved NodeB (eNB) or gNB (5G base station). eNBs include, for example, macro eNBs and small eNBs. Small eNBs can be eNBs covering cells smaller than macro cells, such as pico eNBs, micro eNBs, and femtocell eNBs. A similar situation can occur with gNBs. Alternatively, base stations can be implemented as any other type of base station, such as NodeBs and base transceiver stations (BTSs). A base station can include: a body configured to control wireless communication (also called base station equipment); and one or more remote radio heads (RRHs) located in a different location from the body. Additionally, various types of user equipment can operate as base stations by temporarily or semi-persistently performing base station functions.

[0154] Furthermore, electronic devices 200 or 600 can also be implemented as any type of TRP. This TRP can have sending and receiving functions, for example, it can receive information from terminal devices and base station devices, and it can also send information to terminal devices and base station devices. In a typical example, the TRP can provide services to terminal devices and is controlled by the base station device. Further, the TRP can have a structure similar to that of the base station device, or it can only have the structures related to sending and receiving information found in the base station device.

[0155] Furthermore, electronic device 200 or 600 can be implemented as various user devices. User devices can be implemented as mobile terminals (such as smartphones, tablet PCs, laptop PCs, portable gaming terminals, portable / dongle-type mobile routers, and digital camera devices) or in-vehicle terminals (such as car navigation devices). User devices can also be implemented as terminals performing machine-to-machine (M2M) communication (also known as machine-type communication (MTC) terminals). Additionally, user devices can be wireless communication modules (such as integrated circuit modules comprising a single chip) installed on each of the aforementioned terminals.

[0156] [Application examples of servers]

[0157] Figure 9 is a block diagram illustrating an example of a schematic configuration of a server 1700 to which the technologies of this disclosure can be applied. The server 1700 includes a processor 1701, a memory 1702, a storage device 1703, a network interface 1704, and a bus 1706.

[0158] Processor 1701 may be, for example, a central processing unit (CPU) or a digital signal processor (DSP), and controls the functions of server 1700. Memory 1702 includes random access memory (RAM) and read-only memory (ROM), and stores data and programs executed by processor 1701. Storage device 1703 may include storage media such as semiconductor memory and hard disk.

[0159] Network interface 1704 is a wired communication interface used to connect server 1700 to wired communication network 1705. Wired communication network 1705 can be a core network such as an evolved packet core network (EPC) or a packet data network (PDN) such as the Internet.

[0160] Bus 1706 connects processor 1701, memory 1702, storage device 1703, and network interface 1704 to each other. Bus 1706 may include two or more buses (such as a high-speed bus and a low-speed bus) each with different speeds.

[0161] In the server 1700 shown in FIG9, the communication unit in the electronic device previously described with reference to FIG2 or FIG6 can be implemented through network interface 1704. At least some of the functions of the determination unit, estimation unit, and update unit of the processor in the electronic device can be implemented by processor 1701. For example, processor 1701 can implement at least some of the functions of the determination unit, estimation unit, and update unit by executing instructions stored in memory 1702 or storage device 1703.

[0162] [Application examples of base stations]

[0163] (First application example)

[0164] Figure 10 is a block diagram illustrating a first example of a schematic configuration of an eNB to which the technologies of this disclosure can be applied. The eNB 1800 includes one or more antennas 1810 and a base station device 1820. The base station device 1820 and each antenna 1810 can be connected to each other via RF cables.

[0165] Each of the antennas 1810 includes one or more antenna elements (such as multiple antenna elements included in a multiple-input multiple-output (MIMO) antenna) and is used by the base station device 1820 to transmit and receive wireless signals. As shown in Figure 10, the eNB 1800 may include multiple antennas 1810. For example, multiple antennas 1810 may be compatible with multiple frequency bands used by the eNB 1800. Although Figure 10 shows an example in which the eNB 1800 includes multiple antennas 1810, the eNB 1800 may also include a single antenna 1810.

[0166] The base station equipment 1820 includes a controller 1821, a memory 1822, a network interface 1823, and a wireless communication interface 1825.

[0167] The controller 1821 can be, for example, a CPU or a DSP, and operates various higher-level functions of the base station equipment 1820. For example, the controller 1821 generates data packets based on data in signals processed by the wireless communication interface 1825, and transmits the generated packets via the network interface 1823. The controller 1821 can bundle data from multiple baseband processors to generate bundled packets and transmit the generated bundled packets. The controller 1821 may have logical functions that perform controls such as radio resource control, radio bearer control, mobility management, admission control, and scheduling. This control can be performed in conjunction with nearby eNBs or core network nodes. The memory 1822 includes RAM and ROM, and stores programs executed by the controller 1821 and various types of control data (such as terminal lists, transmission power data, and scheduling data).

[0168] Network interface 1823 is a communication interface used to connect base station equipment 1820 to core network 1824. Controller 1821 can communicate with core network nodes or other eNBs via network interface 1823. In this case, eNB 1800 and core network nodes or other eNBs can be connected to each other via logical interfaces (such as S1 and X2 interfaces). Network interface 1823 can also be a wired communication interface or a wireless communication interface for wireless backhaul. If network interface 1823 is a wireless communication interface, it can use a higher frequency band for wireless communication compared to the frequency band used by wireless communication interface 1825.

[0169] The wireless communication interface 1825 supports any cellular communication scheme (such as LTE and LTE-Advanced) and provides wireless connectivity to terminals located in the cell of eNB 1800 via antenna 1810. The wireless communication interface 1825 typically includes, for example, a baseband (BB) processor 1826 and RF circuitry 1827. The BB processor 1826 can perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and performs various types of signal processing at layers such as L1, Media Access Control (MAC), Radio Link Control (RLC), and Packet Data Convergence Protocol (PDCP). Instead of controller 1821, the BB processor 1826 may have some or all of the above-described logical functions. The BB processor 1826 may be a memory storing communication control programs, or a module including a processor and associated circuitry configured to execute programs. Updates can change the functionality of the BB processor 1826. The module may be a card or blade inserted into a slot in base station equipment 1820. Alternatively, the module may be a chip mounted on a card or blade. Meanwhile, the RF circuit 1827 may include, for example, a mixer, a filter, and an amplifier, and transmits and receives wireless signals via the antenna 1810.

[0170] As shown in Figure 10, the wireless communication interface 1825 may include multiple BB processors 1826. For example, the multiple BB processors 1826 may be compatible with multiple frequency bands used by the eNB 1800. As shown in Figure 10, the wireless communication interface 1825 may include multiple RF circuits 1827. For example, the multiple RF circuits 1827 may be compatible with multiple antenna elements. Although Figure 10 shows an example in which the wireless communication interface 1825 includes multiple BB processors 1826 and multiple RF circuits 1827, the wireless communication interface 1825 may also include a single BB processor 1826 or a single RF circuit 1827.

[0171] In the eNB 1800 shown in Figure 10, the communication unit in the electronic device previously described with reference to Figures 2 or 6 can be implemented via a wireless communication interface 1825 and an optional antenna 1810. At least some of the functions of the determination unit, estimation unit, and update unit in the electronic device can be implemented via a controller 1821. For example, the controller 1821 can implement at least some of the functions of the determination unit, estimation unit, and update unit by executing instructions stored in the memory 1822.

[0172] (Second application example)

[0173] Figure 11 is a block diagram illustrating a second example of a schematic configuration of an eNB to which the technologies of this disclosure can be applied. The eNB 1930 includes one or more antennas 1940, a base station device 1950, and an RRH 1960. The RRH 1960 and each antenna 1940 can be connected to each other via RF cables. The base station device 1950 and the RRH 1960 can be connected to each other via high-speed lines such as fiber optic cables.

[0174] Each of the antennas 1940 includes one or more antenna elements (such as multiple antenna elements included in a MIMO antenna) and is used for transmitting and receiving wireless signals by the RRH 1960. As shown in Figure 11, the eNB 1930 may include multiple antennas 1940. For example, multiple antennas 1940 may be compatible with multiple frequency bands used by the eNB 1930. Although Figure 11 shows an example in which the eNB 1930 includes multiple antennas 1940, the eNB 1930 may also include a single antenna 1940.

[0175] The base station equipment 1950 includes a controller 1951, a memory 1952, a network interface 1953, a wireless communication interface 1955, and a connection interface 1957. The controller 1951, memory 1952, and network interface 1953 are the same as the controller 1821, memory 1822, and network interface 1823 described with reference to FIG10.

[0176] Wireless communication interface 1955 supports any cellular communication scheme (such as LTE and LTE-Advanced) and provides wireless communication to terminals located in the sector corresponding to RRH 1960 via RRH 1960 and antenna 1940. Wireless communication interface 1955 may typically include, for example, a BB processor 1956. The BB processor 1956 is identical to the BB processor 1826 described with reference to FIG10, except that it is connected to the RF circuitry 1964 of RRH 1960 via connection interface 1957. As shown in FIG11, wireless communication interface 1955 may include multiple BB processors 1956. For example, multiple BB processors 1956 may be compatible with multiple frequency bands used by eNB 1930. Although FIG11 shows an example in which wireless communication interface 1955 includes multiple BB processors 1956, wireless communication interface 1955 may also include a single BB processor 1956.

[0177] Connection interface 1957 is an interface for connecting base station equipment 1950 (wireless communication interface 1955) to RRH 1960. Connection interface 1957 can also be a communication module for communication in the aforementioned high-speed line connecting base station equipment 1950 (wireless communication interface 1955) to RRH 1960.

[0178] RRH 1960 includes a connection interface 1961 and a wireless communication interface 1963.

[0179] Connection interface 1961 is an interface for connecting RRH 1960 (wireless communication interface 1963) to base station equipment 1950. Connection interface 1961 can also be a communication module for communication in the aforementioned high-speed line.

[0180] Wireless communication interface 1963 transmits and receives wireless signals via antenna 1940. Wireless communication interface 1963 typically includes, for example, RF circuitry 1964. RF circuitry 1964 may include, for example, mixers, filters, and amplifiers, and transmits and receives wireless signals via antenna 1940. As shown in FIG11, wireless communication interface 1963 may include multiple RF circuits 1964. For example, multiple RF circuits 1964 may support multiple antenna elements. Although FIG11 shows an example in which wireless communication interface 1963 includes multiple RF circuits 1964, wireless communication interface 1963 may also include a single RF circuit 1964.

[0181] In the eNB 1930 shown in Figure 11, the communication unit in the electronic device previously described with reference to Figures 2 or 3 can be implemented, for example, via a wireless communication interface 1963 and an optional antenna 1940. At least some of the functions of the determination unit, estimation unit, and update unit in the electronic device can be implemented by a controller 1951. For example, the controller 1951 can implement at least some of the functions of the determination unit, estimation unit, and update unit by executing instructions stored in the memory 1952.

[0182] [Application examples of terminal devices]

[0183] (First application example)

[0184] Figure 12 is a block diagram illustrating an example of a schematic configuration of a smartphone 2000 to which the technology of this disclosure can be applied. The smartphone 2000 includes a processor 2001, a memory 2002, a storage device 2003, an external connection interface 2004, a camera device 2006, a sensor 2007, a microphone 2008, an input device 2009, a display device 2010, a speaker 2011, a wireless communication interface 2012, one or more antenna switches 2015, one or more antennas 2016, a bus 2017, a battery 2018, and an auxiliary controller 2019.

[0185] The processor 2001 can be, for example, a CPU or a system-on-a-chip (SoC), and controls the application layer and other functions of the smartphone 2000. The memory 2002 includes RAM and ROM, and stores data and programs executed by the processor 2001. The storage device 2003 can include storage media such as semiconductor memory and hard disks. The external connectivity interface 2004 is an interface for connecting external devices (such as memory cards and Universal Serial Bus (USB) devices) to the smartphone 2000.

[0186] The camera device 2006 includes an image sensor (such as a charge-coupled device (CCD) and complementary metal-oxide-semiconductor (CMOS)) and generates captured images. The sensor 2007 may include a set of sensors, such as a measurement sensor, a gyroscope sensor, a geomagnetic sensor, and an accelerometer. The microphone 2008 converts sound input to the smartphone 2000 into an audio signal. The input device 2009 includes, for example, a touch sensor, keypad, keyboard, buttons, or switches configured to detect touches on the screen of the display device 2010 and receives operations or information input from the user. The display device 2010 includes a screen (such as a liquid crystal display (LCD) and an organic light-emitting diode (OLED) display) and displays the output image from the smartphone 2000. The speaker 2011 converts the audio signal output from the smartphone 2000 into sound.

[0187] The wireless communication interface 2012 supports any cellular communication scheme (such as LTE and LTE-Advanced) and performs wireless communication. The wireless communication interface 2012 typically includes, for example, a BB processor 2013 and RF circuitry 2014. The BB processor 2013 can perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and performs various types of signal processing for wireless communication. Meanwhile, the RF circuitry 2014 can include, for example, mixers, filters, and amplifiers, and transmits and receives wireless signals via an antenna 2016. The wireless communication interface 2012 can be a single chip module on which the BB processor 2013 and RF circuitry 2014 are integrated. As shown in Figure 12, the wireless communication interface 2012 can include multiple BB processors 2013 and multiple RF circuitry 2014. Although Figure 12 shows an example where the wireless communication interface 2012 includes multiple BB processors 2013 and multiple RF circuitry 2014, the wireless communication interface 2012 can also include a single BB processor 2013 or a single RF circuitry 2014.

[0188] In addition to cellular communication schemes, the wireless communication interface 2012 can support other types of wireless communication schemes, such as short-range wireless communication schemes, near-field communication schemes, and wireless local area network (LAN) schemes. In this case, the wireless communication interface 2012 may include a BB processor 2013 and RF circuitry 2014 for each wireless communication scheme.

[0189] Each of the antenna switches 2015 switches the connection destination of antenna 916 among multiple circuits (e.g., circuits for different wireless communication schemes) included in the wireless communication interface 2012.

[0190] Each of the antennas 2016 includes one or more antenna elements (such as multiple antenna elements included in a MIMO antenna) and is used by the wireless communication interface 2012 to transmit and receive wireless signals. As shown in Figure 12, the smartphone 2000 may include multiple antennas 2016. Although Figure 12 shows an example in which the smartphone 2000 includes multiple antennas 2016, the smartphone 2000 may also include a single antenna 2016.

[0191] Furthermore, the smartphone 2000 may include an antenna 2016 for each wireless communication scheme. In this case, the antenna switch 2015 can be omitted from the configuration of the smartphone 2000.

[0192] Bus 2017 connects processor 2001, memory 2002, storage device 2003, external connection interface 2004, camera device 2006, sensor 2007, microphone 2008, input device 2009, display device 2010, speaker 2011, wireless communication interface 2012, and auxiliary controller 2019 to each other. Battery 2018 supplies power to the various blocks of smartphone 2000 shown in Figure 12 via feeders, which are partially shown as dashed lines in the figure. Auxiliary controller 2019 operates the minimum necessary functions of smartphone 2000, for example, in sleep mode.

[0193] In the smartphone 2000 shown in Figure 12, the communication unit in the electronic device previously described with reference to Figures 2 or 6 can be implemented via a wireless communication interface 2012 and an optional antenna 2016. At least some of the functions of the determining unit, estimating unit, and updating unit in the electronic device can be implemented by a processor 2001 or an auxiliary controller 2019. For example, the processor 2001 or the auxiliary controller 2019 can implement at least some of the functions of the determining unit, estimating unit, and updating unit by executing instructions stored in the memory 2002 or storage device 2003.

[0194] (Second application example)

[0195] Figure 13 is a block diagram illustrating an example of a schematic configuration of a car navigation device 2120 to which the technology of this disclosure can be applied. The car navigation device 2120 includes a processor 2121, a memory 2122, a Global Positioning System (GPS) module 2124, a sensor 2125, a data interface 2126, a content player 2127, a storage medium interface 2128, an input device 2129, a display device 2130, a speaker 2131, a wireless communication interface 2133, one or more antenna switches 2136, one or more antennas 2137, and a battery 2138.

[0196] The processor 2121 can be, for example, a CPU or a SoC, and controls the navigation function and other functions of the car navigation device 2120. The memory 2122 includes RAM and ROM, and stores data and programs executed by the processor 2121.

[0197] GPS module 2124 uses GPS signals received from GPS satellites to measure the location (such as latitude, longitude, and altitude) of car navigation device 2120. Sensor 2125 may include a set of sensors, such as a gyroscope sensor, a geomagnetic sensor, and an air pressure sensor. Data interface 2126 is connected to, for example, an in-vehicle network 2141 via a terminal not shown, and acquires data generated by the vehicle (such as vehicle speed data).

[0198] Content player 2127 reproduces content stored on storage media (such as CDs and DVDs), which is inserted into storage media interface 2128. Input device 2129 includes, for example, a touch sensor, button, or switch configured to detect touch on the screen of display device 2130, and receives operations or information input from the user. Display device 2130 includes a screen such as an LCD or OLED display and displays images or reproduced content for navigation functions. Speaker 2131 outputs sound for navigation functions or reproduced content.

[0199] The wireless communication interface 2133 supports any cellular communication scheme (such as LTE and LTE-Advanced) and performs wireless communication. The wireless communication interface 2133 typically includes, for example, a BB processor 2134 and RF circuitry 2135. The BB processor 2134 can perform, for example, encoding / decoding, modulation / demodulation, and multiplexing / demultiplexing, and performs various types of signal processing for wireless communication. Meanwhile, the RF circuitry 2135 can include, for example, mixers, filters, and amplifiers, and transmits and receives wireless signals via antenna 2137. The wireless communication interface 2133 can also be a chip module on which the BB processor 2134 and RF circuitry 2135 are integrated. As shown in Figure 13, the wireless communication interface 2133 can include multiple BB processors 2134 and multiple RF circuits 2135. Although Figure 13 shows an example where the wireless communication interface 2133 includes multiple BB processors 2134 and multiple RF circuits 2135, the wireless communication interface 2133 can also include a single BB processor 2134 or a single RF circuitry 2135.

[0200] In addition to cellular communication schemes, wireless communication interface 2133 can support other types of wireless communication schemes, such as short-range wireless communication schemes, near-field communication schemes, and wireless LAN schemes. In this case, for each wireless communication scheme, wireless communication interface 2133 may include BB processor 2134 and RF circuitry 2135.

[0201] Each of the antenna switches 2136 switches the connection destination of the antenna 2137 among multiple circuits (such as circuits for different wireless communication schemes) included in the wireless communication interface 2133.

[0202] Each of the antennas 2137 includes one or more antenna elements (such as multiple antenna elements included in a MIMO antenna) and is used for transmitting and receiving wireless signals through the wireless communication interface 2133. As shown in Figure 13, the car navigation device 2120 may include multiple antennas 2137. Although Figure 13 shows an example in which the car navigation device 2120 includes multiple antennas 2137, the car navigation device 2120 may also include a single antenna 2137.

[0203] Furthermore, the car navigation device 2120 may include an antenna 2137 for each wireless communication scheme. In this case, the antenna switch 2136 can be omitted from the configuration of the car navigation device 2120.

[0204] Battery 2138 supplies power to the various blocks of the car navigation device 2120 shown in Figure 13 via feeders, which are partially shown as dashed lines in the figure. Battery 2138 accumulates the power supplied from the vehicle.

[0205] In the car navigation device 2120 shown in FIG13, the communication unit in the electronic device previously described with reference to FIG2 or FIG3 can be implemented via a wireless communication interface 2133 and an optional antenna 2137. At least some of the functions of the determining unit, estimating unit, and updating unit in the electronic device can be implemented by the processor 2121. For example, the processor 2121 can implement at least some of the functions of the determining unit, estimating unit, and updating unit by executing instructions stored in the memory 2122.

[0206] The technology disclosed herein can also be implemented as an in-vehicle system (or vehicle) 2140 including one or more of the following blocks: a car navigation device 2120, an in-vehicle network 2141, and a vehicle module 2142. The vehicle module 2142 generates vehicle data (such as vehicle speed, engine speed, and fault information) and outputs the generated data to the in-vehicle network 2141.

[0207] Preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, but the present disclosure is by no means limited to the examples described above. Various changes and modifications can be made by those skilled in the art within the scope of the appended claims, and it should be understood that such changes and modifications naturally fall within the technical scope of the present disclosure.

[0208] For example, the units shown in the dashed boxes in the functional block diagrams shown in the attached figures represent that the functional unit is optional in the corresponding device, and the optional functional units can be combined in an appropriate manner to achieve the desired function.

[0209] For example, the multiple functions included in one unit in the above embodiments can be implemented by separate devices. Alternatively, the multiple functions implemented by multiple units in the above embodiments can be implemented by separate devices respectively. In addition, one of the above functions can be implemented by multiple units. Needless to say, such a configuration is included within the scope of the present disclosure.

[0210] In this specification, the steps described in the flowchart include not only processes executed sequentially in the stated order, but also processes executed in parallel or individually, rather than necessarily sequentially. Furthermore, even within the steps of sequential processing, needless to say, the order can be appropriately altered.

[0211] In addition, this disclosure may have the configuration described below.

[0212] 1. An electronic device, comprising:

[0213] At least one processor; and

[0214] At least one memory, including computer program code, wherein the at least one memory and the computer program code are configured to cause the electronic device to execute, via the at least one processor:

[0215] Determine the communication uncertainty and / or perception accuracy of vehicle status data acquired via wireless communication; and

[0216] Vehicle state is estimated using vehicle state data based on the determined communication uncertainty and / or perception accuracy.

[0217] 2. The electronic device according to configuration 1, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0218] Based on the determined communication uncertainty, compensation is performed on the vehicle state data for communication uncertainty; and

[0219] Vehicle status is estimated using compensated vehicle status data.

[0220] 3. The electronic device according to configuration 2, wherein the determined communication uncertainty includes the delay and / or packet loss rate of wireless communication.

[0221] 4. The electronic device according to configuration 3, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0222] The current delay is determined based on one or more previous delays using a pre-obtained neural network model.

[0223] 5. The electronic device according to configuration 3, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0224] Using a pre-obtained Hidden Markov Model, the current packet loss rate is determined based on one or more current state parameters related to the packet loss rate in wireless communication.

[0225] 6. The electronic device according to configuration 3, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0226] Based on the current vehicle status data, predict the delayed vehicle status data to obtain the delayed vehicle status data.

[0227] 7. The electronic device according to configuration 6, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0228] The prediction is made based on a kinematic model and / or a pre-trained neural network model.

[0229] 8. The electronic device according to configuration 3, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0230] Based on the determined current packet loss rate, the current vehicle status data is corrected using previous vehicle status data to obtain packet loss compensated vehicle status data.

[0231] 9. The electronic device according to configuration 8, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0232] The correction is performed when the determined perception accuracy is below a predetermined threshold and / or the determined current packet loss rate is above a predetermined threshold.

[0233] 10. The electronic device according to configuration 1, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0234] Based on the determined perception accuracy, the current vehicle state is estimated by using vehicle state data to correct the current vehicle state prediction based on the previously estimated vehicle state.

[0235] 11. The electronic device according to configuration 10, wherein the determined perception accuracy includes the perception error of vehicle state data.

[0236] 12. The electronic device according to configuration 11, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0237] The perception error of the vehicle status data is determined based on the installation posture error of the sensors used to acquire vehicle status data and the perception accuracy.

[0238] 13. The electronic device according to configuration 11, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0239] The extended Kalman filter algorithm is used to estimate the vehicle state as a state variable by using vehicle state data as observation data, where the perception error of the vehicle state data is used as the measurement noise of the observation data.

[0240] 14. The electronic device according to configuration 1, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0241] Update the virtual vehicle's state based on the estimated vehicle state.

[0242] 15. The electronic device according to configuration 1, wherein the at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor:

[0243] Vehicle status data is received via wireless communication.

[0244] 16. The electronic device according to configuration 1, wherein the electronic device is located in the cloud.

[0245] 17. A method for use in an electronic device, comprising:

[0246] Determine the communication uncertainty and / or perception accuracy of vehicle status data acquired via wireless communication; and

[0247] Vehicle state is estimated using vehicle state data based on the determined communication uncertainty and / or perception accuracy.

[0248] 18. The method according to configuration 17 further includes:

[0249] Update the virtual vehicle's state based on the estimated vehicle state.

[0250] 19. The method according to configuration 17 further includes:

[0251] Vehicle status data is received via wireless communication.

[0252] 20. A non-transitory computer-readable storage medium storing computer program code that causes an electronic device to perform the method as described in any one of configurations 17 to 19 via a processor included in the electronic device.

[0253] While embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, it should be understood that the embodiments described above are merely illustrative and do not constitute a limitation thereof. Those skilled in the art can make various modifications and alterations to the above embodiments without departing from the spirit and scope of the present disclosure. Therefore, the scope of the present disclosure is defined only by the appended claims and their equivalents.

Claims

1. An electronic device, comprising: At least one processor; as well as At least one memory, including computer program code, wherein the at least one memory and the computer program code are claimed to cause the electronic device to execute via the at least one processor: Determine the communication uncertainty and / or perception accuracy of vehicle status data acquired via wireless communication; and Vehicle state is estimated using vehicle state data based on the determined communication uncertainty and / or perception accuracy.

2. The electronic device according to claim 1, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Based on the determined communication uncertainty, compensation is performed on the vehicle status data for communication uncertainty. as well as Vehicle status is estimated using compensated vehicle status data.

3. The electronic device according to claim 2, wherein, The identified communication uncertainties include the latency and / or packet loss rate of wireless communication.

4. The electronic device according to claim 3, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: The current delay is determined based on one or more previous delays using a pre-obtained neural network model.

5. The electronic device according to claim 3, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Using a pre-obtained Hidden Markov Model, the current packet loss rate is determined based on one or more current state parameters related to the packet loss rate in wireless communication.

6. The electronic device according to claim 3, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Based on the current vehicle status data, predict the delayed vehicle status data to obtain the delayed vehicle status data.

7. The electronic device according to claim 6, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: The prediction is made based on a kinematic model and / or a pre-trained neural network model.

8. The electronic device according to claim 3, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Based on the determined current packet loss rate, the current vehicle status data is corrected using previous vehicle status data to obtain packet loss compensated vehicle status data.

9. The electronic device according to claim 8, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: The correction is performed when the determined perception accuracy is below a predetermined threshold and / or the determined current packet loss rate is above a predetermined threshold.

10. The electronic device according to claim 1, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Based on the determined perception accuracy, the current vehicle state is estimated by using vehicle state data to correct the current vehicle state prediction based on the previously estimated vehicle state.

11. The electronic device according to claim 10, wherein, The determined perception accuracy includes the perception error of vehicle status data.

12. The electronic device according to claim 11, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: The perception error of the vehicle status data is determined based on the installation posture error of the sensors used to acquire vehicle status data and the perception accuracy.

13. The electronic device according to claim 11, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: The extended Kalman filter algorithm is used to estimate the vehicle state as a state variable by using vehicle state data as observation data, where the perception error of the vehicle state data is used as the measurement noise of the observation data.

14. The electronic device according to claim 1, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Update the virtual vehicle's state based on the estimated vehicle state.

15. The electronic device according to claim 1, wherein, The at least one memory and the computer program code are further configured to cause the electronic device to execute, via the at least one processor: Vehicle status data is received via wireless communication.

16. The electronic device according to claim 1, wherein, The electronic devices are located in the cloud.

17. A method for use in an electronic device, comprising: Determine the communication uncertainty and / or perception accuracy of vehicle status data acquired via wireless communication; as well as Vehicle state is estimated using vehicle state data based on the determined communication uncertainty and / or perception accuracy.

18. The method of claim 17, further comprising: Update the virtual vehicle's state based on the estimated vehicle state.

19. The method of claim 17, further comprising: Vehicle status data is received via wireless communication.

20. A non-transitory computer-readable storage medium storing computer program code that causes an electronic device to perform the method as described in any one of claims 17 to 19 via a processor included in the electronic device.

Images