Communication and environment perception integrated method and device based on deep learning, and storage medium

Abstract

The invention discloses a communication and environmental perception integrated method based on deep learning, and the method comprises the steps: obtaining signal receiving strengths on different wave beam pairs, and converting the signal receiving strengths into RSRP images; feature patterns generated by the multiple paths and the non-perpendicular incidence paths in the RSRP image are detected, and the arrival angle and the departure angle of the detected multiple paths are estimated; multi-path angle parameters in the user moving process are tracked and associated, and the AOA and AOD changing processes of the same multi-path at different positions in the user moving process are obtained; performing LOS and NLOS path identification on each multipath by using the multipath AOA and AOD change process obtained by tracking; the method comprises the following steps: fitting AOA and AOD motion track points of each path in a user motion process, representing different scatterers generating multiple paths in an environment by using parameters obtained by fitting, extracting environment and user position information contained in an AOA-AOD space, and estimating the orientation of the scatterers and the user position in the environment in combination with sensor information of a user side.

Description

Technical Field

[0001] The present invention proposes a method, device and storage medium for integrating communication and environmental perception based on deep learning, which belongs to the intersection of millimeter wave communication, deep learning and communication perception integration. Background Art

[0002] Millimeter wave communication technology is one of the key technologies for the future sixth generation (6G) wireless communications. Compared with sub-6GHz communication systems, millimeter wave communication systems have larger bandwidths and higher data rates. Array antennas and beamforming technologies are also used to combat the high path loss in the millimeter wave band. These characteristics bring higher latency and angular resolution to millimeter wave communication systems, making it possible for them to have both communication and perception capabilities. In addition, wireless perception systems and millimeter wave communication systems are more similar in frequency bands, channel characteristics, and hardware architecture, making it possible to achieve integrated communication and perception.

[0003] There are many problems and challenges in achieving integrated communication and perception. First, the millimeter wave channel is highly correlated with the environmental geometry, and environmental perception relies on accurate estimation of the channel multipath parameters. Second, in mobile scenarios, the channel multipath parameters change over time, and tracking and correlating the angular parameters of the multipath is a major challenge for beam management and environmental perception. Finally, the LOS path and NLOS path in the millimeter wave channel contain the geometric relationships between the user, base station, and environment, respectively. Identifying the LOS and NLOS paths can more finely use the millimeter wave channel multipath parameters for environmental perception. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for estimating, tracking, identifying LOS / NLOS paths and environmental perception of channel multipath angle parameters with high reliability and real-time performance based on information obtained during beam management.

[0005] Technical solution: In order to achieve the above-mentioned purpose, the present invention proposes a method for integrating communication and environmental perception based on deep learning, which specifically includes the following steps:

[0006] Step 1: During the beam scanning process, the base station and the user obtain the signal receiving strength on different beam pairs and convert it into a signal receiving strength RSRP image;

[0007] Step 2, using a target detection method based on deep learning, detects the characteristic pattern generated by the multipath in the RSRP image, and estimates the arrival angle AOA and departure angle AOD of the detected multipath; the multipath includes a direct path LOS and an indirect path NLOS;

[0008] Step 3: Using a multi-target tracking algorithm, based on the estimated values ​​of multipaths at different positions during the user's movement, the multipath angle parameters during the user's movement are tracked and associated to obtain the change process of AOA and AOD of the same multipath at different positions during the user's movement.

[0009] Step 4: Using the tracked multipath AOA and AOD trajectories and their changing processes, LOS and NLOS are identified for each multipath according to the different changing rules of AOA and AOD of the LOS path and the NLOS path.

[0010] Step 5: Fit the AOA and AOD motion trajectory points of each multipath in the user's motion process, use the fitted parameters to represent the different scatterers that generate multipath in the environment, extract the environment and user location information contained in the RSRP image, and combine the sensor information on the user side to complete the estimation of the azimuth angle between the virtual base station and the base station in the environment and the user's position.

[0011] Furthermore, in step 1, the beam scanning is any one of an exhaustive beam scanning scheme and a layered codebook beam scanning scheme; and the method of converting into an RSRP image is: using RSRP values ​​of different beam pairs to generate a color heat map or a grayscale image.

[0012] Furthermore, in step 2, the algorithm for estimating the angle of arrival AOA and the angle of departure AOD of the detected multipath is any one of the YOLO algorithm, MUISC, ESPRIT, and OMP algorithms based on deep learning.

[0013] Furthermore, in step 3, during the user's movement, the specific method of the multipath angle parameter tracking association algorithm based on the multi-target tracking algorithm ByteTrack is as follows:

[0014] (3.1) Using the tracking algorithm ByteTrack, based on the estimation of AOA and AOD of RSRP image multipath, the position change process of multipath in continuous RSRP image during user movement is tracked as follows:

[0015] (1) The RSRP image I at a certain position in the user's motion is regarded as a frame. In the user's motion trajectory, the RSRP images at different positions arranged in chronological order constitute a continuous RSRP image sequence

[0016] (2) For The kth frame I k , the detection frame detected by YOLO is divided into a set of detection frames above the confidence threshold τ And the set of detection boxes below the threshold

[0017] (3) The set of trajectories tracked in the kth frame is defined as

[0018] (4) In the first frame I 1 In the middle, high confidence detection boxes are initialized as trajectories

[0019] (5) In subsequent frames, each multipath trajectory is k+1 New location First, the Kalman filter is based on Make predictions;

[0020] (6) For the k+1th frame The first step of trajectory association is performed by calculating the intersection over union (IoU). Detection box of trajectory prediction and trajectory of lost tracking These two and The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and and lost track of the tracks Conduct trajectory association;

[0021] (7) After the first association, the successfully associated trajectory is added In the above example, the unassociated predicted trajectory and detection box are denoted as as well as

[0022] (8) The second step is to associate as well as The intersection over union (IoU) is used to calculate The detection box and trajectory prediction in The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and Conduct trajectory association;

[0023] (9) After the second association, the successfully associated trajectories are in the set middle, The unassociated detection frames will no longer be retained. Unassociated tracks will be stored in middle;

[0024] (10) will be associated in subsequent frames. Tracks that cannot be re-tracked within a certain number of frames will be deleted;

[0025] (11) High confidence detection box that is not matched will be initialized as a new trajectory and stored in middle;

[0026] (12) For each multipath in the k+1th frame, based on the results of the current frame tracking Assign an identification ID to each path;

[0027] (3.2) Record and are the AOA and AOD of the path with ID q in the RSRP image of the pth frame in the user's motion trajectory, The AOA and AOD coordinates of the path with ID q in all frames are put into the set middle,

[0028] Furthermore, in step 4, the identification method of the LOS path and the NLOS path is as follows:

[0029] (3) Record The latest and second latest AOA-AOD coordinates are and Where i>j, the LOS and NLOS identification of the qth path is performed by calculating the following formula:

[0030]

[0031] If Z>0, the qth path is identified as a LOS path, and if Z<0, it is an NLOS path.

[0032] Furthermore, the method of step 5 is as follows:

[0033] (1) The NLOS path moves along a straight line with a slope of -1. Scatterers with different NLOS paths are located at the same azimuth angle α of the base station. Straight lines with a slope of -1 and different intercepts are used to represent different NLOS path trajectories, so as to distinguish scatterers in the environment. Assuming that the qth path is the NLOS path, its AOA and AOD motion trajectories are expressed as:

[0034] Φ=-Θ+Δ q

[0035] where Θ, Φ and Δ q are the AOA, AOD and intercept of the qth NLOS path respectively;

[0036] (2) Least squares method is applied to All AOA-AOD coordinates in Δ q ;

[0037] (3) Based on Θ NLOS =2α-Φ NLOS + kπ, we get Δ q =2α q+ kπ, thereby estimating the azimuth angle α of the virtual base station on the scatterer of the qth NLOS path relative to the base station q ; Φ NLOS is the departure angle AOD of the NLOS path; k is a constant;

[0038] (4) The positions of the base stations and the environmental scatterers in the environment are fixed. The LOS path and the NLOS path trajectory generated by the scatterers have characteristics in the RSRP image. A mapping between the scatterer position and the user position is constructed. The AOA and AOD of each multipath reflect the position information of the user and the environment. Combined with the user motion state information output by the inertial measurement unit IMU, the user position is estimated.

[0039] (5) In the RSRP image, the motion trajectory of the LOS path is located at Θ LOS =Φ LOS +kπ, consider Θ LOS ∈[0,2π),Φ LOS ∈[0,2π), the LOS path trajectory is divided into Θ in the RSRP image LOS =Φ LOS -π,Θ LOS ∈[0,π) and Θ LOS =Φ LOS +π,Θ LOs ∈[π,2π) two trajectories; Φ LOS is the departure angle AOD of the LOS path;

[0040] (6) In the RSRP image, if the virtual base station generated by a scatterer is located at an azimuth angle α of the base station, without considering the angle range, the motion trajectory of the NLOS is located at θ NLOS =2α-Φ NLOS +π, Considering that the angle value range in the RSRP image is [0,2π), when as well as When , since part of the line segment exceeds the angle value range of the RSRP image [0,2π), the part exceeding the range [0,2π) is modulo;

[0041] (7) Combining the LOS and NLOS path motion trajectory expressions in step (5) and step (6), the RSRP image is divided into regions according to the position of the virtual base station generated by the scatterer relative to the base station. On this basis, a mapping codebook between different regions of the RSRP image and the position of the scatterer in the environment is constructed;

[0042] (8) The azimuth angle between the virtual base station generated by the scatterer and the base station is located according to the mapping codebook in step (7).

[0043] In addition, the present invention proposes an integrated communication and environmental perception device based on deep learning, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the methods when executing the computer program.

[0044] In addition, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the methods when executed by a processor.

[0045] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0046] The technical solution proposed in the present invention can directly perform channel estimation and perception in the beam management process; it does not rely on parameters such as time delay that require high clock synchronization and large bandwidth to complete LOS and NLOS identification; compared with the existing LOS and NLOS scene identification, LOS and NLOS identification can be further performed on each multipath; the mapping relationship between the motion characteristics of the multipath in the AOA-AOD space and the environmental geometry can be used to complete the perception of scatterers in the environment; traditional SLAM algorithms (such as BP-SLAM, PHD-SLAM, etc.) use probability to perform data association with a large amount of calculation, while the environmental feature association calculation of this solution is small. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic diagram of the geometric relationship between a base station, a user and a reflector in an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of the trajectory of the LOS path in the RSRP image in an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of the trajectory of the NLOS path in the RSRP image in an embodiment of the present invention;

[0050] Figure 4 A schematic diagram of the region division and codebook of the RSRP image according to the relationship between the scatterer and the environment in an embodiment of the present invention;

[0051] Figure 5 A flowchart of a method for integrating communication and environmental perception based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures are provided to provide a thorough understanding of the present invention. However, it should be clear to those skilled in the art that the present invention may also be implemented in other embodiments without these specific details.

[0053] In order to make the purpose, technical solution and advantages of the present invention more clear, the following will be described in detail with reference to the accompanying drawings. The embodiment of the present invention provides a method for integrating communication and environmental perception based on deep learning, such as Figure 5 As shown, the method comprises the following steps:

[0054] Step 101: During the beam scanning process, the base station and the user obtain the signal receiving strength RSRP on different beam pairs and convert it into an RSRP image.

[0055] It should be noted that:

[0056] (1) The base station and the user use their own beam codebooks to perform beam scanning and measure the RSRP on different beam pairs during the scanning process. i,j It indicates that the base station uses the i-th beam to transmit, and the user uses the j-th beam to receive the measured RSRP. When all base stations and all user beams have completed RSRP measurement, an RSRP matrix R can be obtained, which is expressed as

[0057]

[0058] Among them, M UE and M BS are the number of beams in the codebook used by the user and base station for beam scanning, and the angles corresponding to different beam numbers in the codebook increase with the increasing beam number. Different beam numbers correspond to different angle domain directions, so the RSRP matrix covers the discrete range of AOA and AOD.

[0059] Based on the RSRP matrices measured for different base stations and user orientations at the same test point, an RSRP image with a larger AOA and AOD angle coverage is obtained.

[0060] 1) First, create an initialized RSRP image I, the two dimensions of which represent AOA and AOD respectively, and set all elements to zero.

[0061] 2) Secondly, in order to prevent the beam pattern distortion that may be caused by the RSRP image looping at the 0° and 360° boundaries, the ranges of the AOA and AOD of the RSRP image are extended according to the angular range of the beam scanning.

[0062] 3) Next, in order to align the RSRP matrix with the AOA and AOD angle indexes of RSRP image I, the RSRP matrix measured for different base station-user orientation pairs is interpolated. The number of rows and columns of the interpolated RSRP matrix is ​​consistent with the number of rows and columns of the submatrix with the same AOA-AOD range in RSRP image I when the AOA-AOD coverage range remains unchanged.

[0063] 4) Finally, the interpolated RSRP matrix is ​​mapped to the RSRP image I according to the respective AOA and AOD angle coverage range, and the RSRP image I is normalized to a normalized RSRP matrix with a value range of 0-255 The specific expression is:

[0064]

[0065] Among them, I max is the maximum value in I, is the floor rounding operator. Normalized RSRP matrix Copy the three RGB channels of the image as the generated RSRP image. The generated RSRP image has a size of P rows and Q columns, and the minimum and maximum coverage ranges of AOA are Θ min and θ max , the minimum and maximum coverage of AOD are Φ min and Φ max .

[0066] Step 102: Use a deep learning-based target detection method to detect the characteristic pattern generated by the multipath in the RSRP image, and estimate the arrival angle AOA and departure angle AOD of the detected multipath; the multipath includes a direct path LOS and an indirect path NLOS.

[0067] It should be noted that:

[0068] (1) The beam patterns generated by channel multipath are sparsely distributed in the RSRP image and can be regarded as multiple targets in the image. Channel multipath estimation is transformed into a target detection problem.

[0069] (2) YOLO is a real-time target detection algorithm based on deep learning. When the RSRP image is input into the pre-trained YOLO network, the number of multipath beam patterns detected is Each detected object is represented by a detection box. The label of the detection box of the lth object is (class l ,c x,l ,c y,l ,w l ,h l ), where class l is the target type, and the normalized horizontal and vertical coordinates of the target center point in the RSRP image coordinate system (the ratio of the center point pixel position to Q or P) is c x,l ,c y,l , the normalized width and height (the ratio of the two sides of the detection box to Q or P) is w l ,h l .

[0070] (3) Estimate AOA and AOD based on the labels of the detection box, and find the position of the element with the maximum RSRP, i.e., the maximum value, in the detection box containing the lth path beam pattern (x l,max ,y l,max ), the AOA and AOD of the lth path are estimated to be:

[0071]

[0072] (4) All estimated RSRP images The AOA and AOD of the strip are put into the set The AOA-AOD coordinates of the lth path are

[0073] Step 103: Using a multi-target tracking algorithm, based on the estimated values ​​of multipaths at different positions during the user's movement, the multipath angle parameters during the user's movement are tracked and associated to obtain the change process of AOA and AOD of the same multipath at different positions during the user's movement.

[0074] It should be noted that:

[0075] (3.1) Using the tracking algorithm ByteTrack, based on the estimation of AOA and AOD of RSRP image multipath, the position change process of multipath in continuous RSRP image during user movement is tracked as follows:

[0076] (1) The RSRP image I at a certain position in the user's motion is regarded as a frame. In the user's motion trajectory, the RSRP images at different positions arranged in chronological order constitute a continuous RSRP image sequence

[0077] (2) For The kth frame I k , the detection frame detected by YOLO is divided into a set of detection frames above the confidence threshold τ And the set of detection boxes below the threshold

[0078] (3) The set of trajectories tracked in the kth frame is defined as

[0079] (4) In the first frame I 1 In the middle, high confidence detection boxes are initialized as trajectories

[0080] (5) In subsequent frames, each multipath trajectory is k+1 New location First, the Kalman filter is based on Make predictions;

[0081] (6) For the k+1th frame The first step of trajectory association is performed by calculating the intersection over union (IoU). Detection box of trajectory prediction and trajectory of lost tracking These two and The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and and lost track of the tracks Conduct trajectory association;

[0082] (7) After the first association, the successfully associated trajectory is added In the above example, the unassociated predicted trajectory and detection box are denoted as as well as

[0083] (8) The second step is to associate as well as The intersection over union (IoU) is used to calculate The detection box and trajectory prediction in The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and Conduct trajectory association;

[0084] (9) After the second association, the successfully associated trajectories are in the set middle, The unassociated detection frames will no longer be retained. Unassociated tracks will be stored in middle;

[0085] (10) will be associated in subsequent frames. Tracks that cannot be re-tracked within a certain number of frames will be deleted;

[0086] (11) High confidence detection box that is not matched will be initialized as a new trajectory and stored in middle;

[0087] (12) For each multipath in the k+1th frame, based on the results of the current frame tracking Assign an identification ID to each path;

[0088] (3.2) Record and are the AOA and AOD of the path with ID q in the RSRP image of the pth frame in the user's motion trajectory, The AOA and AOD coordinates of the path with ID q in all frames are put into the set middle,

[0089] Step 104: Using the tracked multipath AOA and AOD trajectories and their changing processes, perform LOS / NLOS identification on each multipath according to the different changing rules of AOA and AOD of the LOS path and the NLOS path.

[0090] It should be noted that:

[0091] (1) Note Θ NLOS With Φ NLOS are the AOA and AOD of the NLOS path, Θ LOS With Φ LOS The AOA and AOD of the LOS path are respectively, and the AOA and AOD of the NLOS path satisfy the following constraints

[0092] sin(2α-θ NLOS -Φ NLOS )=0

[0093] Among them, Figure 1 As shown, α is the direction angle between the base station and the virtual base station generated by the NLOS path on the scatterer.

[0094] (2) The angular relationship between the LOS path and the NLOS path can be further expressed as

[0095] Θ NLOS =2α-Φ NLOS +kπΘ LOS =Φ LOS +kπ

[0096] Therefore, the beam patterns of the LOS path and the NLOS path move along the straight lines with slopes of 1 and -1 respectively in the RSRP image, that is, the AOA and AOD change rates of the LOS path and the NLOS path are

[0097]

[0098] (3) Record The latest and second latest AOA-AOD coordinates are and Where i>j. The LOS and NLOS identification of the qth path can be performed by calculating the following formula:

[0099]

[0100] If Z>0, the qth path is identified as a LOS path, and if Z<0, it is an NLOS path.

[0101] Step 105: Fit the AOA and AOD motion trajectory points of each multipath in the user's motion process, use the fitted parameters to represent the different scatterers that generate multipaths in the environment, extract the environment and user location information contained in the RSRP image, and combine the sensor information on the user side to complete the estimation of the azimuth angle between the virtual base station and the base station in the environment and the user's position.

[0102] It should be noted that:

[0103] (1) The NLOS path moves along a straight line with a slope of -1. Since the scatterers that generate different NLOS paths are located at different azimuth angles α of the base station, a slope of -1 and different intercepts Δ q The straight lines represent different NLOS path trajectories, that is, distinguishing the scatterers in the environment. Assuming that the qth path is the NLOS path, its AOA and AOD motion trajectories are expressed as

[0104] Φ=-Θ+Δ q

[0105] (2) The least squares method can be applied to All AOA-AOD coordinates in Δ q .

[0106] (3) Based on Θ NLOS =2α-Φ NLOs +kπ, we can deduce Δ q =2α q +kπ, from which the azimuth angle α of the virtual base station on the scatterer of the qth NLOS path at the base station can be estimated q .

[0107] (4) Since the positions of the base stations and the environmental scatterers in the environment are fixed, the LOS paths and the NLOS paths generated by the scatterers have characteristics in the RSRP image, and a mapping between the RSRP image and environmental information (scatterer positions and user positions, etc.) can be constructed. At the same time, the positions of the LOS and NLOS paths in the RSRP image, that is, the AOA and AOD of each multipath, reflect the position information of the user and the environment. Combined with the user motion state information output by the inertial measurement unit (IMU), the user position can be estimated.

[0108] (5) Figure 2 As shown, in the RSRP image, the motion trajectory of the LOS path is located at Θ LOS =Φ LOS +kπ, consider Θ LOS ∈[0,2π),Φ LOS ∈[0,2π), the LOS path trajectory is divided into Θ in the RSRP image LOS=Φ LOS -π,Θ LOS ∈[0,π) and Θ LOS =Φ LOS +π,Θ LOS ∈[π,2π) two trajectories; Φ LOS is the departure angle AOD of the LOS path;

[0109] (6) In the RSRP image, if the virtual base station generated by a scatterer is located at an angle α in the direction of the base station, without considering the angle range, the motion trajectory of the NLOS is located at Θ NLOS =2α-Φ NLOS +π, Considering that the angle value range in the RSRP image is [0,2π), when as well as When , since part of the line segment exceeds the angle value range [0,2π) of the RSRP image, the part exceeding the range [0,2π) can be modulo, so the final NLOS path can present multiple tracks in the RSRP image. Figure 3 (a) and (b) show and The motion trajectory of the NLOS path generated by the scatterer in the RSRP image.

[0110] (7) Combining the LOS and NLOS path motion trajectory expressions in step (5) and step (6), the RSRP image is divided into regions according to the position of the virtual base station generated by the scatterer relative to the base station. On this basis, a mapping codebook between different regions of the RSRP image and the position of the scatterer in the environment is constructed, as follows: Figure 4 (a) shows the multipath AOA / AOD region that can be generated by scatterers with a value of 0≤α≤π. Figure 4 (b) shows the multipath AOA and AOD regions that can be generated by scatterers with π≤α≤2π. On this basis, the divided regions shown in the figure can be further divided to form a mapping codebook between different regions of the RSRP image and the positions of scatterers in the environment.

[0111] (8) The azimuth angle between the virtual base station generated by the scatterer and the base station is located according to the mapping codebook in step (7).

[0112] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for integrating communication and environmental perception based on deep learning, characterized in that: The method specifically comprises the following steps: Step 1: During the beam scanning process, the base station BS and the user obtain the signal receiving strength on different beam pairs and convert it into a signal receiving strength RSRP image; Step 2, using a target detection method based on deep learning, detects the characteristic pattern generated by the multipath in the RSRP image, and estimates the arrival angle AOA and departure angle AOD of the detected multipath; the multipath includes a direct path LOS and an indirect path NLOS; Step 3: Using a multi-target tracking algorithm, based on the estimated values ​​of multipaths at different positions during the user's movement, the multipath angle parameters during the user's movement are tracked and associated to obtain the change process of AOA and AOD of the same multipath at different positions during the user's movement. Step 4: Using the tracked multipath AOA and AOD trajectories and their changing processes, LOS and NLOS are identified for each multipath according to the different changing rules of AOA and AOD of the LOS path and the NLOS path. Step 5: Fit the AOA and AOD motion trajectory points of each multipath in the user's motion process, use the fitted parameters to represent the different scatterers that generate multipath in the environment, extract the environment and user location information contained in the RSRP image, and combine the sensor information on the user side to complete the estimation of the azimuth angle of the virtual base station vBS and the base station in the environment and the user's position.

2. According to claim 1, a method for integrating communication and environmental perception based on deep learning is characterized in that: In step 1, the beam scanning method is any one of an exhaustive beam scanning scheme and a layered codebook beam scanning scheme; the method of converting into an RSRP image is: using RSRP values ​​of different beam pairs to generate a color heat map or a grayscale image.

3. The method for integrating communication and environmental perception based on deep learning according to claim 1, characterized in that: In step 2, the algorithm for estimating the arrival angle AOA and the departure angle AOD of the detected multipath is any one of the YOLO algorithm, MUISC, ESPRIT, and OMP algorithms based on deep learning.

4. The method for integrating communication and environmental perception based on deep learning according to claim 1, characterized in that: The specific method in step 3 is as follows: (3.1) Using the tracking algorithm ByteTrack, based on the estimation of AOA and AOD of RSRP image multipath, the position change process of multipath in continuous RSRP image during user movement is tracked as follows: (1) The RSRP image I at a certain position in the user's motion is regarded as a frame. In the user's motion trajectory, the RSRP images at different positions arranged in chronological order constitute a continuous RSRP image sequence (2) For The kth frame I k , the detection frame detected by YOLO is divided into a set of detection frames above the confidence threshold τ And the set of detection boxes below the threshold (3) The set of trajectories tracked in the kth frame is defined as (4) In the first frame I1, the high confidence detection box is initialized as the trajectory (5) In subsequent frames, each multipath trajectory is k+1 New location The Kalman filter is based on Make predictions; (6) For the k+1th frame The first step of trajectory association is performed by calculating the intersection over union (IoU). Detection box of trajectory prediction and trajectory of lost tracking These two and The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and and lost track of the tracks Conduct trajectory association; (7) After the first association, the successfully associated trajectory is added In the above example, the unassociated predicted trajectory and detection box are denoted as as well as (8) The second step is to associate as well as Between, calculated by intersection over union (IoU) The detection box and trajectory prediction in The similarity of the detection box in the Hungarian algorithm is based on the similarity pair and Conduct trajectory association; (9) After the second association, the successfully associated trajectories are in the set middle, The unassociated detection frames are no longer retained. Unassociated tracks will be stored in middle; (10) will be associated in subsequent frames. Tracks that cannot be re-tracked within a certain number of frames will be deleted; (11) High confidence detection box that is not matched will be initialized as a new trajectory and stored in middle; (12) For each multipath in the k+1th frame, based on the results of the current frame tracking Assign an identification ID to each path; (3.2) Record and are the AOA and AOD of the path with ID q in the RSRP image of the pth frame in the user's motion trajectory, The AOA and AOD coordinates of the path with ID q in all frames are put into the set middle, 5. The method for integrating communication and environmental perception based on deep learning according to claim 4, characterized in that: In step 4, the identification method of LOS path and NLOS path is as follows: (3) Record The latest and second latest AOA-AOD coordinates are and Where i>j, the LOS and NLOS identification of the qth path is performed by calculating the following formula: If Z>0, the qth path is identified as a LOS path, and if Z<0, it is an NLOS path.

6. The method for integrating communication and environmental perception based on deep learning according to claim 5, characterized in that: The method for step 5 is as follows: (1) The NLOS path moves along a straight line with a slope of -1. Scatterers with different NLOS paths are located at the same azimuth angle α of the base station. Straight lines with a slope of -1 and different intercepts are used to represent different NLOS path trajectories, so as to distinguish scatterers in the environment. Assuming that the qth path is the NLOS path, its AOA and AOD motion trajectories are expressed as: F=-Θ+D q where Θ, Φ and Δ q are the AOA, AOD and intercept of the qth NLOS path respectively; (2) Apply the least squares method to All AOA-AOD coordinates in Δ q ; (3) Based on Θ NLOS =2α-Φ NLOS + kπ, we get Δ q =2α q + kπ, thereby estimating the azimuth angle α of the virtual base station on the scatterer of the qth NLOS path relative to the base station q ; The Φ NLOS is the departure angle AOD of the NLOS path; k is a constant; (4) The positions of the base stations and the environmental scatterers are fixed. The LOS paths and the NLOS paths generated by the scatterers have characteristics in the RSRP image. A mapping between the scatterer positions and the user positions is constructed. The user position is estimated based on the position information of the user and the environment reflected by the AOA and AOD of each multipath, combined with the user motion state information output by the inertial measurement unit (IMU). (5) In the RSRP image, the motion trajectory of the LOS path is located at Θ LOS =Φ LOS +kπ, consider Θ LOS ∈[0,2π),Φ LOS ∈[0, 2π), the LOS path trajectory is divided into Θ in the RSRP image LOS =Φ LOS -π, Θ LOS ∈[0,π) and Θ LOS =Φ LOS +π,Θ LOS ∈[π, 2π) two trajectories; the Φ LOS is the departure angle AOD of the LOS path; (6) In the RSRP image, if a virtual base station generated by a scatterer is located at an azimuth angle α of the base station, without considering the angle range, the motion trajectory of the NLOS is located at θ NLOS =2α-Φ NLOS +π, Considering that the angle value range in the RSRP image is [0, 2π), when as well as When , since part of the line segment exceeds the angle value range of the RSRP image [0, 2π), the part exceeding the range [0, 2π) is modulo; (7) Combining the LOS and NLOS path motion trajectory expressions in step (5) and step (6), the RSRP image is divided into regions according to the position of the virtual base station generated by the scatterer relative to the base station. On this basis, a mapping codebook between different regions of the RSRP image and the position of the scatterer in the environment is constructed; (8) The azimuth angle between the virtual base station generated by the scatterer and the base station is located according to the mapping codebook in step (7).

7. A communication and environment perception integrated device based on deep learning, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images