An environment perception method in digital twin internet of vehicles based on inter-sensory integration

Abstract

The application relates to an environment sensing method based on integrated sensing in a digital twin Internet of Vehicles, and belongs to the technical field of communication, and proposes an environment collaborative sensing method based on integrated sensing of digital twin, wherein a digital twin system makes an edge collaborative decision according to task requirements and available resources, and selects an integrated sensing mode or an environment sensing instruction mode; the mechanism has two collaborative modes, if the integrated sensing mode ISAC is selected, an RSU uses an integrated sensing beam to sense environment data, and conversely, if the environment sensing instruction mode InsT is selected, after the RSU receives a task instruction from a virtual control unit, since environment data has been locally cached, only perspective transformation needs to be performed on the environment data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of communication and relates to an environment perception method in digital twin vehicle networking based on integrated communication and sensing. BACKGROUND

[0002] With the development of automatic driving based on AR or tactile internet technology, future vehicles will be equipped with many sensors to collect environmental data, such as a sufficient number of cameras and LIDAR. This computationally intensive service requirement often requires 3D reconstruction techniques such as depth map fusion. In fact, neighboring sensor-equipped RSUs can obtain similar environmental data. In addition, within the coverage of the RSU, similar environmental data can also be obtained from various sensor-equipped SeVs. Due to the different perspectives of the RSUs, it is necessary to perform coordinate transformation preprocessing using the coordinates of the TaVs contained in the computing instructions to eliminate the differences between the perceived environmental data. The size of the computing instructions is much smaller than that of the original sensory input data, and the corresponding transmission delay will be greatly reduced. Therefore, due to the cooperation of original sensory data transmission and instruction transmission, it provides new possibilities for reducing overall delay. In the traditional cooperation process, a large amount of input data needs to be passed between edge servers, which not only occupies a large amount of communication resources, but also exceeds the tolerance range of the delay, and has certain limitations. SUMMARY

[0003] Therefore, the purpose of the present application is to provide an environment perception method in digital twin vehicle networking based on integrated communication and sensing. An environment perception method suitable for vehicle networking is proposed in combination with the characteristics of the integrated communication and sensing system, which saves a large amount of spectrum resources and energy consumption and reduces task cooperation delay while ensuring perception accuracy.

[0004] To achieve the above purpose, the present application provides the following technical solutions:

[0005] An environment perception method in digital twin vehicle networking based on integrated communication and sensing, comprising the following steps:

[0006] S1: The vehicle initiates an environment perception task, forwards the task instructions to the road test unit RSU with the most abundant communication resources under the condition of meeting the channel threshold coefficient, and calculates the task transmission time length;

[0007] S2: According to the received task instructions, the digital twin system DT combines some information delivered by the RSU regularly and makes a cooperation decision under the premise of meeting the QoE requirement; the cooperation decision includes the task ratio, the wireless bandwidth ratio and the adaptive transmission strategy;

[0008] S3: The digital twin system checks the real-time status of edge servers (RSUs), selects RSUs with good service status as collaborative nodes, splits complex tasks into subtasks and assigns them to collaborative nodes. Collaborative nodes use edge selection strategies to select nearby service vehicles (SeVs) for vehicle-road collaborative perception.

[0009] S4: If the data required for the task is not cached locally, RSU j uses the ISAC mode to perceive and obtain the resources required to process the task. Otherwise, it uses the environmental perception instruction mode InsT. For the ISAC mode, multi-target sensing and multi-stream communication are performed simultaneously to obtain the environmental data required for processing the task. For the InsT mode, only the locally cached data needs to be preprocessed, that is, the perspective is transformed. The time for data processing depends on the size of the instruction and the coordinate transformation.

[0010] Furthermore, step S1 specifically includes the following steps:

[0011] S11: Using binary numbers v,m To indicate whether the vehicle and RSU m can establish a reliable communication connection, define h v,m is the channel threshold coefficient, then:

[0012]

[0013] h v,m represents the channel threshold coefficient;

[0014] S12: Let the set of RSUs that meet the channel threshold coefficient be Φ={φ1,φ2,...,φ c}, where c represents the number of RSUs that meet the channel threshold coefficient; when the channel threshold coefficient is met, the digital twin system DT allocates the RSU with the richest communication resources to the vehicle access, and this RSU is represented by k:

[0015]

[0016] in They respectively represent the real-time communication resources of RSU that meet the channel threshold coefficient.

[0017] S13: The mission instruction only contains the input data size, computational intensity, and QoE requirements; the channel rate from the mission vehicle to the RSU is as follows:

[0018]

[0019] where zk1(t) is the bandwidth allocated to vehicle by RSU k, pv,k represents the transmission power of the vehicle, and σ 2 represents the noise power, represents the complex channel fading coefficient, is the path loss, a is the path loss exponent, d v,k (t) represents the time-varying distance between the vehicle and RSU k during the task initiation;

[0020] S14: The time required for the task instruction to be unloaded from the vehicle to RSU k is represented as:

[0021]

[0022] D v represents the data size of the task instruction, R v,k represents.

[0023] Further, the step S3 specifically comprises the following steps:

[0024] S31: The DT generates a corresponding decision according to the requirements of the task vehicle, adaptively divides the task into subtasks, and distributes the subtasks to the service vehicles SeV and RSUs for collaborative processing, the SeVs participating in the collaboration form a set I = {1, 2,..., i,..., I}; the RSUs participating in the collaboration form a set J = {1, 2,..., j,..., J};

[0025] S32: The process of assigning tasks is first delivered to the relatively idle RSU node, and then delivered to the nearby computing vehicle SeV i by the RSU; a binary number represents the edge selection strategy of the nearby SeV, which is specifically represented as represents that the SeV i is currently busy and is not selected to participate in the collaborative computing task; represents that the SeV i selects to participate in the collaborative computing task and receives the related task parameters and the data required for processing the task at the nearby RSU; it is assumed that each RSU can deliver tasks to at most M max vehicles SeV, thus obtaining Each SeV selects at most one RSU to receive the task, that is,

[0026] Further, the step S4 specifically comprises the following steps:

[0027] S41: Assuming that the task D task is divided into n parts by the RSU j, and the proportion coefficient of each subtask to the total task is a j , satisfying If the data required for the task is not cached locally, the RSU j acquires the resources required for processing the task by sensing through the ISAC mode, otherwise the InsT mode is used; a binary number x j is defined to indicate whether the indispensable data is cached in the corresponding edge server of the RSU j.

[0028] S42: sequentially calculate the task uploading time delay of ISAC and InsT two parallel cooperation modes;

[0029] S43: if ISAC cooperation mode is selected, which is composed of RSU equipped with N antennas and i task vehicles, each vehicle has M antennas, where IM≤N; the range of interest ROI includes the target sensing area and the interference area, which is divided into cubes of equal size, each cube represents a pixel point; there are O clutters in the interference range around the target as interference; there are two basic tasks in the vehicle-road cooperative wireless network, namely target sensing and information communication, which are completed by multiple task vehicles and multi-antenna RSU cooperation;

[0030] S44: when the RSU obtains similar environment data and has been cached into the corresponding edge server, the InsT mode is selected, the matrix operation containing the task vehicle coordinates in the calculation instruction is used to perform coordinate transformation on the environment data perceived by the RSU, and then the RSU performs the subtask using its own perceived environment data.

[0031] Further, in step S43, the following steps are specifically included:

[0032] The superposition coding transmission signal of the ith vehicle is established, assuming that the ith vehicle constructs a sensing signal To sense p targets in the target area and generate data As a communication signal to transmit information in each time slot, which is specifically described as follows:

[0033]

[0034]

[0035] wherein represents sensing, represents communication, and respectively are the transmit beamforming vectors for the corresponding sensing signal and communication signal;

[0036] For the received signal of the RSU, it is specifically as follows:

[0037]

[0038] wherein, represents sensing, represents clutter interference, represents communication, n is an additive white Gaussian noise AWGN vector, and respectively are the reflection coefficients of the pth target and the Oth clutter; represents the cascade channel, where g i,p and g′ i,p are the channel gains from the i-th service vehicle to the p-th target and from the p-th target to the RSU, respectively; represents the cascade channel, where f i,o and f i ' ,o are the channel gains from the i-th service vehicle to the O-th clutter and from the O-th clutter to the RSU, respectively; is the channel gain from the i-th service vehicle to the RSU; assuming G i,p ,F i,o ,H i Remains constant within a time slot, but fades independently across time slots;

[0039] A linear unbiased estimator is deployed at the RSU to estimate the reflection coefficient of the target; in order to obtain accurate reflection coefficients for target sensing The RSU performs receive beamforming to enhance the desired signal and suppress co-channel interference;

[0040] The mean square error (MSE) is used as the performance indicator of target perception to minimize the difference between the estimated reflection coefficient and the actual reflection coefficient to suppress interference from other signals, where the estimated reflection coefficient of the pth target is Expressed as:

[0041]

[0042] in is the sensing receive beamforming vector adopted at the RSU for the pth target; after the relevant parameters are estimated, the mean square error MSE is used as the performance indicator of target perception; the estimated reflection coefficient is minimized and the actual reflection coefficient z j1 to suppress interference from other signals:

[0043]

[0044] in, is the noise power, R m is the reflection coefficient The root mean square value RMS of the prior probability of occurrence of each type;

[0045] The RSU performs responsive receive beamforming to mitigate interference. The received signal is given by:

[0046]

[0047] in is the communication receive beamforming vector of the lth communication signal from the ith service vehicle, and The relevant signal-to-interference-plus-noise ratio (SINR) is expressed as:

[0048]

[0049] where

[0050]

[0051] The communication latency is thus obtained as:

[0052]

[0053] z i1 (t) represents the allocated channel bandwidth of the SeV i, and α i represents the task data transmission coefficient of each service vehicle, and The upload latency of this stage is thus obtained as i.e., the deadline for data upload required for processing the task is set as t max When , the RSU chooses not to wait for the SeV any longer and proceeds with the task processing directly.

[0054] Further, in step S44, the RSU caches the offloading decision to perceive and preprocess the corresponding environmental data; in the case of InsT, the upload time of the service vehicle depends on the coordinate conversion time, assuming that the service vehicle is always perceiving the environmental data, the coordinate transformation is performed immediately after receiving the computing instruction, the conversion time depends on the amount of data sensed, and the time spent in execution depends on the computing capability of the RSUj's corresponding edge server and the CPU cycles required by the subtask:

[0055]

[0056] where M inst represents the computing intensity of the coordinate transformation, z j2 (t) represents the computing rate of the RSU;

[0057] The time required in step S44 is thus expressed as:

[0058]

[0059] In performing the task of vehicle-initiated environmental perception, the total time required according to the established task processing procedure is expressed as:

[0060]

[0061] The beneficial effects of the present application are that in the traditional collaboration process, a large amount of input data needs to be transmitted between edge servers, not only occupying a large amount of communication resources, but also exceeding the tolerance range of delay, and has certain limitations. Therefore, the RSU and the service vehicle in this paper can be regarded as ISAC nodes. The RSU and the service vehicle equipped with sensors both have the ability to obtain environmental information. Adjacent RSUs can collect similar sensor environmental data. In this case, adjacent edge nodes do not need to transmit a large amount of input data, but upload the computing instructions, which will greatly reduce the transmission time.

[0062] Other advantages, objects, and features of the present application will be understood by those skilled in the art from the following specification in conjunction with the appended drawings in which: BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to make the purposes, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be combined with the drawings as follows, in which:

[0064] Fig. 1 The figure is a schematic diagram of the method of the present application for digital twin assisted vehicle networking edge collaboration scenario;

[0065] Fig. 2 The figure is a schematic diagram of the method of the present application for edge collaboration process;

[0066] Fig. 3 The figure is a schematic diagram of the method of the present application for integrated environmental perception mode. DETAILED DESCRIPTION

[0067] The embodiments of the present application are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied in other different specific embodiments, and the details in the specification can be modified or changed in various ways based on different views and applications without departing from the spirit of the present application. It should be noted that the figures provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and the figures in the following examples and features can be combined with each other without conflict.

[0068] It should be noted that the figures provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and only the components related to the present application are shown in the figures, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change, and the component layout pattern may be more complex.

[0069] In the following description, numerous specific details are discussed in order to provide a thorough explanation of embodiments of the application. It will be apparent, however, to one skilled in the art, that embodiments of the application can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring embodiments of the application.

[0070] As Figs. 1-3 shown, the application provides an environment perception method in a digital twin Internet of Vehicles based on the integration of general perception, including the following steps:

[0071] Step one: the vehicle initiates a computing task and forwards its instructions to the RSU with the most abundant communication resources, including the size of the input data size, the computing intensity (i.e. the total number of CPU cycles of the computing task) and the QoE requirements, etc.; the task transmission time in the cooperative perception process is solved, including the following steps:

[0072] Step 1: the task vehicle uploads the task instructions to any RSU within the connectable range, in order to reduce transmission delay, the vehicle tends to select the RSU with the most ideal communication state with the help of digital twin (DT). In order to ensure this, the binary number con v,m is used to indicate whether the vehicle and the RSU m can establish a reliable communication connection; define h v,m as the channel threshold coefficient, then

[0073]

[0074] Step 2: assuming that the number of RSUs in the system that meet the above requirements is c, which together constitute a new set Φ = {φ1, φ2,..., φ c}. In addition, the bandwidth resources allocated to the vehicle by the RSU is another important factor affecting the transmission time. Therefore, in order to further reduce the transmission time, under the condition of meeting the channel threshold coefficient, the DT tends to allocate the RSU with the most abundant communication resources to the vehicle after querying the internal data, and the expression of RSU k can be obtained:

[0075]

[0076] Step 3: Unlike the previous offloading manner, this cooperative process only needs to transmit the task instruction. In fact, the task instruction only contains the input data size, the computing intensity (i.e., the total number of CPU cycles of the computing task), and the QoE requirement, etc. But in a specific environment, such as a system with extremely high real-time requirements or a scene extremely sensitive to network transmission speed, it means that the time required for transmitting the task must be included in the overall corresponding delay. The channel rate from the task vehicle to the RSU is as follows:

[0077]

[0078] where z k1 (t) is the bandwidth allocated by the RSU k to the vehicle, pv,k represents the transmit power of the vehicle, σ 2 represents the noise power. represents the complex channel fading coefficient, is the path loss, and a is the path loss exponent, d v,k (t) represents the time-varying distance between the vehicle and the RSU k during the task initiation. The time required for offloading the task instruction from the vehicle to the RSU k is represented as:

[0079]

[0080] D v represents the data size of the task instruction.

[0081] Step 2: According to the received task instruction, the digital twin system combines some information (i.e., moving speed, location, computing capability, wireless transmission capability, etc.) delivered by the RSU periodically and makes an offloading decision under the premise of meeting the QoE requirement. The offloading decision includes the task ratio, the wireless bandwidth ratio, and the adaptive transmission strategy (i.e., ISAC or InsT), etc.

[0082] Step 3: The digital twin system checks the real-time edge server status, selects nodes with good service status to split complex tasks into sub-tasks and distributes them to cooperative nodes, and the cooperative nodes select nearby SeVs for cooperative perception through edge selection strategies, including the following steps:

[0083] Step 1: The DT generates the corresponding decision according to the requirements of the task vehicle, adaptively divides the task into sub-tasks, and distributes the sub-tasks to SeVs (service vehicles) and RSUs for cooperative processing. The SeVs participating in the cooperation form a set Ι = {1, 2,..., i,..., I}. The RSUs participating in the cooperation form a set J = {1, 2,..., j,..., J}.

[0084] Step 2: A binary number is used to represent the edge selection strategy of the nearby SeV, which is specifically represented as SeVi indicates that it is currently busy and does not choose to participate in the collaborative computing task; otherwise, SeVi indicates that it chooses to participate in the collaborative computing task and receives relevant task parameters and data required for processing the task at the nearby RSU. It is assumed that each RSU can deliver tasks for at most M max vehicles, so that Similarly, each SeVi chooses at most one RSU to receive the task, that is,

[0085] Step 4: If the data required for the task is not cached locally, RSUj can perceive through the ISAC mode to obtain the required resources for processing the task, otherwise, the InsT mode is used. For the ISAC mode, multi-target sensing and multi-flow communication are simultaneously performed to obtain the environmental data required for processing the task. For the InsT mode, only the cached data needs to be preprocessed, that is, perspective transformation. The data processing time depends on the size of the instruction and the coordinate transformation. Specifically, the following steps are included:

[0086] Step 1: Only with the support of the corresponding data, the task can be successfully completed. It is assumed that the task D task is divided into n parts by RSUj. Each sub-task occupies a proportion of the total task, and the proportion coefficient is a j , satisfying If the data required for the task is not cached locally, RSUj can perceive through the ISAC mode to obtain the required resources for processing the task, otherwise, the InsT mode is used. Define a binary number x j to indicate whether the indispensable data is cached in the corresponding edge server of RSUj;

[0087] Step 2: The task upload delay of the two parallel collaborative modes of ISAC and InsT is calculated in turn;

[0088] Step 3: The integrated environment perception mode is shown in Fig. 3 .

[0089] If the ISAC collaborative mode is selected, it is composed of an RSU equipped with N antennas and i task vehicles, each vehicle has M antennas, and IM≤N. The range of interest (ROI) includes the target sensing area and the interference area, which is divided into cubes of equal size, each cube represents a pixel point. Without loss of generality, there are O clutters as interference in the interference range around the target. There are two basic tasks in the vehicle-road cooperative wireless network, namely target perception and information communication, which are completed by multiple task vehicles and multi-antenna RSUs in collaboration;

[0090] The i-th vehicle constructs a superposition coded transmit signal, assuming the i-th vehicle constructs a sensing signal To sense p targets in the target region and generate data As a communication signal to convey information in each time slot, described as follows:

[0091]

[0092]

[0093] where, are transmit beamforming vectors for the corresponding sensing and communication signals, respectively.

[0094] For the RSU's received signal, described as follows:

[0095]

[0096] where n is an additive white Gaussian noise (AWGN) vector, and are reflection coefficients of the p-th target and the O-th clutter, respectively. denotes a cascaded channel, where g i,p and g′ i,p are channel gains from the i-th service vehicle to the p-th target and from the p-th target to the RSU, respectively. denotes a cascaded channel, where f i,o and f i ′ ,o are channel gains from the i-th service vehicle to the O-th clutter and from the O-th clutter to the RSU, respectively. Furthermore, is a channel gain from the i-th service vehicle to the RSU. Assume that G i,p , F i,o , H i remain constant within a time slot but independently fade across time slots.

[0097] The mean square error of target sensing and the interference plus noise ratio of communication can be further effectively obtained by the above procedure, which can guarantee the time delay of the corresponding mode under the conditions of sensing accuracy and communication quality.

[0098] To improve the accuracy of target sensing, all sensors cooperate to sense the target. Therefore, a linear unbiased estimator is deployed at the RSU to estimate the reflection coefficients of the target. To obtain accurate reflection coefficients for target sensing The RSU performs receive beamforming to enhance the desired signal and suppress co-channel interference.

[0099] After the relevant parameter estimation, the mean square error (MSE) is used as the performance indicator of target perception. The difference between the estimated reflection coefficient and the actual reflection coefficient can be minimized to suppress interference from other signals and achieve high-precision target sensing. The estimated reflection coefficient of the pth target can be expressed as

[0100]

[0101] where is the sensing receive beamforming vector adopted at the RSU for the pth target. After the relevant parameter estimation, the mean square error (MSE) is used as the performance indicator of target perception. The above is not only a function of but also a function of other sensing signals and communication signals for different targets from different service vehicles. In this case, the difference between the estimated reflection coefficient and the actual reflection coefficient z j1 can be minimized to suppress interference from other signals and achieve high-precision target sensing.

[0102]

[0103] where, is the noise power, R m is the reflection coefficient According to the root mean square value (RMS) of the prior probability of each type,

[0104] Secondly, in order to improve the quality of the communication signal, the RSU performs response receive beamforming to mitigate interference. Therefore, the received signal of the communication signal at the RSU is given by

[0105]

[0106] where is the communication receive beamforming vector of the lth communication signal from the ith service vehicle. Since the received SINR determines the quality of the communication signal, it is usually considered as the performance measure of communication. The signal-to-interference-plus-noise ratio (SINR) related to can be expressed as

[0107]

[0108] where

[0109]

[0110] so as to obtain the communication delay

[0111]

[0112] z i1 (t) denotes the allocated SeV i channel bandwidth, and α i denotes the task data transmission coefficient of each service vehicle, and Thus, the upload delay of this stage is That is, the deadline of data upload required for processing the task is set as t max When , the RSU chooses not to wait for the SeV any longer and directly processes the task.

[0113] Step 4: When the environment-aware instruction cooperation mode is selected, the RSU obtains similar environment data and has already cached into the corresponding edge server, although the perspectives are different. In order to eliminate the perspective difference and keep consistent with the input data obtained by the task vehicle, it is necessary to first pre-process the environment data perceived by the RSU, i.e., coordinate transformation. This can be achieved by using matrix operation containing the coordinates of the task vehicle in the computing instruction, and then the RSU performs the offloading sub-task using its own perceived environment data. Not only that, the RSU can cache the offloading decision to perceive and pre-process the corresponding environment data. In the case of InsT, the upload time of the service vehicle mainly depends on the coordinate conversion time. Assuming that the service vehicle has been perceiving the environment data, the coordinate transformation can be performed immediately after receiving the computing instruction. The conversion time depends on the amount of data sensed, which will be much smaller than the time required for uploading data, and the time spent depends on the computing power of the edge server coupled with the RSU j and the CPU cycles required by the sub-task.

[0114]

[0115] where M inst denotes the computing intensity of coordinate transformation, z j2 (t) denotes the computing rate of the RSU.

[0116] In summary, the time required for this step can be listed as follows:

[0117] In the execution of the task of vehicle-initiated environment perception, according to the established task processing procedure, the total time required can be accurately expressed as:

[0118]

[0119] In the above embodiments, reference to "the embodiment" or "this embodiment" in the specification indicates that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments, of the application. The various appearances of "the embodiment" or "this embodiment" are not necessarily referring to the same embodiment.

[0120] In the above embodiments, although the application has been described in conjunction with specific embodiments thereof, numerous alternatives, modifications, and variations will be readily apparent to those of ordinary skill in the art in light of the foregoing descriptions. For example, other storage structures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed. The embodiments of this application are intended to cover all such alternatives, modifications, and variations as come within the scope of the appended claims.

[0121] The embodiments also provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements any of the methods in the embodiments.

[0122] The embodiments also provide an electronic terminal, comprising: a processor and a memory.

[0123] The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the terminal executes any of the methods in the embodiments.

[0124] The computer readable storage medium in the embodiments can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by a hardware provided by a computer program. The foregoing computer programs can be stored in a computer readable storage medium. When the program is executed, the steps of the foregoing method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and various media that can store program codes.

[0125] The electronic terminal provided by the embodiments includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected with the processor and the transceiver and complete communication between each other. The memory is configured to store a computer program, the communication interface is configured to communicate, and the processor and the transceiver are configured to run the computer program, so that the electronic terminal executes each step of the method.

[0126] In the embodiments, the memory can include a random access memory (RAM) and can also include a non-volatile memory, such as at least one disk memory.

[0127] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0128] The present application can be applied in numerous general or special computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, etc.

[0129] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media including memory storage devices.

[0130] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. An environment perception method in a digital twin Internet of Vehicles based on inter-sensory integration, characterized in that: Comprising the following steps: S1: The vehicle initiates an environment perception task, and forwards the task instruction to the RSU, denoted as k: wherein respectively represent the communication resources of the RSU in real time that meet the channel threshold coefficient S2: According to the received task instruction, the digital twin system DT makes a cooperative decision under the premise of meeting the QoE requirement; The cooperative decision includes task ratio, wireless bandwidth ratio and adaptive transmission strategy; S3: The digital twin system checks the real-time RSU state, splits the complex task into subtasks and distributes them to the cooperative nodes, and the cooperative nodes select nearby service vehicles SeV for vehicle-road cooperative perception through edge selection strategy; S41: Assuming a task is divided into n parts by RSU j, and the proportion coefficient of each subtask to the total task is , satisfying ; If the data required by the subtask is not cached locally, RSU j is aware of this through the ISAC mode to acquire the resources required to process the task, otherwise the InsT mode is used; define a binary number to indicate whether the indispensable data is cached in the corresponding edge server of RSU j; S42: The task uploading time delay of ISAC and InsT two parallel cooperation modes is calculated in turn; S43: If ISAC cooperation mode is selected, wherein the RSU equipped with N antennas and I task vehicles are composed, each vehicle has M antennas, and IM ≤ N; The range of interest ROI includes the target sensing area and the interference area, which is divided into cubes of equal size, and each cube represents a pixel point; S44: When the RSU obtains the environment data and has deleted the cache to the corresponding edge server, the InsT mode is selected, the matrix operation is performed using the task vehicle coordinates contained in the calculation instruction, the coordinate transformation of the environment data perceived by the RSU is performed, and then the RSU executes the subtask using the environment data perceived by itself; In step S44, the RSU caches the offloading decision to perceive and preprocess the corresponding environment data; The uploading time of the service vehicle in the InsT case depends on the coordinate conversion time, assuming that the service vehicle is always perceiving environment data, performing coordinate transformation immediately after receiving the calculation instruction, the conversion time depends on the amount of data sensed, and the time spent in execution depends on the computing power of the edge server corresponding to the RSU j and the CPU cycle required by the subtask: wherein represents the computational intensity of the coordinate transformation, represents the computational rate of the RSU; In summary, the time required in step S44 is represented as: When performing the task of vehicle-initiated environment perception, the total time required is expressed as: 。 2. The environment perception method based on the integration of common sense and digital twin in the Internet of Vehicles according to claim 1, characterized in that: Step S1 specifically comprises the following steps: S11: Using binary numbers to indicate whether the vehicle and the RSU m can establish a reliable communication connection: is a channel threshold coefficient; S12: Let the set of RSUs satisfying the channel threshold coefficient be where c represents the number of RSUs satisfying the channel threshold coefficient; S13: The task instruction only contains input data size, calculation intensity and QoE requirement; The channel rate from the task vehicle to the RSU is as follows: wherein is the bandwidth allocated to the vehicle by the RSU k, denotes the transmit power of the vehicle, denotes the noise power, denotes the complex channel fading coefficient, is the path loss between the vehicle and the RSU k during the mission initiation, is the path loss exponent; S14: The time required for the task instruction to be offloaded from the vehicle to the RSU k is represented as: Represents the data size of the task instruction.

3. The environment perception method based on the integration of common sense and digital twin in the Internet of Vehicles according to claim 1, characterized in that: The step S3 specifically comprises the following steps: S31: DT generates a corresponding decision according to the requirements of the task vehicle, adaptively divides the task into subtasks, and distributes the subtasks to the service vehicles SeV and RSUs for collaborative processing, and the SeVs participating in collaboration constitute a set ; the RSUs participating in collaboration constitute a set ; S32: the flow of assigning tasks is first delivered to the idle RSU node, and then delivered to the nearby service vehicle SeVi by the RSU; the binary number represents the edge selection strategy of the nearby SeV, which is specifically represented as ; represents that SeVi is currently busy and is not selected to participate in the cooperative computing task; represents that SeVi selects to participate in the cooperative computing task, and receives the related task parameters and the data required for processing the task from the nearby RSU; it is assumed that each RSU can deliver tasks to at most vehicles SeV, and thus ; each SeV selects at most one RSU to receive the task, that is, .

4. The environment perception method based on the integration of common sense and digital twin in the Internet of Vehicles according to claim 1, characterized in that: In step S43, the following steps are specifically included: Establishing the i-th vehicle superposition encoding transmit signal, assuming the i-th vehicle constructs a sensing signal to sense p targets in a target region and generate data as a communication signal to transmit information in each time slot, described as follows: wherein represents sensing, represents communication, , are transmit beamforming vectors for the corresponding sensing and communication signals, respectively; For the received signal of the RSU, the following is specifically: wherein represents a perception, represents a clutter interference, represents a communication, is an additive white Gaussian noise, AWGN, vector, and are reflection coefficients of the pth target and the Oth clutter, respectively; represents a cascaded channel, wherein and are channel gains from the ith service vehicle to the pth target and from the pth target to the RSU, respectively; represents a cascaded channel, wherein and are channel gains from the ith service vehicle to the Oth clutter and from the Oth clutter to the RSU, respectively; is a channel gain from the ith service vehicle to the RSU; deploying a linear unbiased estimator at the RSU to estimate the reflection coefficient of the target; to obtain accurate reflection coefficient for target sensing the RSU performs receive beamforming to enhance the desired signal and suppress co-channel interference; The mean square error (MSE) is used as a performance indicator for the target perception, which minimizes the difference between the estimated reflection coefficient and the actual reflection coefficient of the pth target to suppress the interference from other signals, where the estimated reflection coefficient of the pth target is represented as: wherein is the sensing receive beamforming vector employed at the RSU for the pth target; after performing the relevant parameter estimation, the mean square error (MSE) is employed as the performance indicator for the target sensing; the difference between the estimated reflection coefficient and the actual reflection coefficient is minimized to suppress the interference from other signals: wherein is the noise power, is the reflection coefficient the root mean square square value of the prior occurrence probability of each type, RMS; The RSU performs response receive beamforming to mitigate interference, at the RSU for a received signal of a communication signal is given by the following equation: wherein is a communication receive beamforming vector for the / th communication signal from the / th serving vehicle, and The signal-to-interference-plus-noise ratio (SINR) associated with the / th communication signal from the / th serving vehicle is expressed as Thus, the communication delay is obtained: ​ denotes the allocated SeV i channel bandwidth, and ; denotes the mission data transmission coefficient of each service vehicle, and ; the upload delay of this stage is obtained as , that is, the deadline of data upload required for processing the task is set as , when , the RSU selects not to wait for the SeV any more and directly processes the task.

Images