A visual perception-based v2v millimeter wave beam alignment method

Abstract

The application discloses a kind of V2V millimeter wave beam alignment methods based on visual perception, it is related to wireless communication field, it is related to V2V communication, millimeter wave, beam alignment etc. Research direction, first, the real-time scene image collected by vehicle-mounted monocular camera is estimated by FCOS3D target detection algorithm, the three-dimensional position of vehicle target in scene;Then, the spatial angle and the distance from the vehicle body are obtained by coordinate transformation to the vehicle target position, and the simulated beam forming vector is derived therefrom, to realize beam alignment.The application adopts the above-mentioned one kind of V2V millimeter wave beam alignment methods based on visual perception, can be completed with higher precision beam alignment in the case of lower hardware overhead by high-precision target position estimation, with beam alignment precision high, small spectrum overhead, hardware cost is low and the like advantage.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication, in particular to a V2V millimeter wave beam alignment method based on visual perception. BACKGROUND

[0002] V2V communication can effectively avoid traffic accidents through information transmission between vehicles, and is a key technology for improving road traffic safety and is widely used in the fields of automatic driving and intelligent transportation. With the rapid development of Internet of Vehicles technology, the number of intelligent networked vehicles is increasing year by year, and the communication traffic will increase significantly, which puts higher requirements on network bandwidth and network delay. Millimeter wave has a broad spectrum of 30GHz-300GHz, which can alleviate the spectrum congestion problem in traditional communication systems and provide higher transmission support for V2V communication. However, due to the high frequency of millimeter wave signals, the path loss in transmission is large, and the transmission depends on the direct path, which is easily affected by objects such as human bodies, leaves and buildings, causing a large signal attenuation. Therefore, a large-scale antenna array technology is usually used to form a narrow high-gain beam to compensate for the rapid attenuation of millimeter wave signals in transmission. However, due to the strong mobility of vehicles, it is difficult to align the narrow beam in real time, which brings difficulties to V2V millimeter wave beam alignment.

[0003] Traditional beam alignment methods include beam search with a specific codebook to maximize the received signal-to-noise ratio (SNR), or directly calculating the beamforming matrix according to the estimated channel matrix. However, this method has a large amount of calculation, which will result in a large amount of delay and spectrum overhead. In recent years, sensor-assisted beam alignment methods have received extensive attention from the academic community. These methods use effective out-of-band information collected by sensors to indicate the spatial characteristics of the communication environment, and assist in beam alignment. Since almost no frequency band resources of the system are occupied, the beam search overhead can be significantly reduced.

[0004] Since the vehicle-mounted monocular camera sensor has high detection accuracy and a long detection distance, it can provide rich perception detail information and has good performance in the field of computer vision and automatic driving. The present application believes that it has the feasibility to assist in completing beam alignment. SUMMARY

[0005] The purpose of the present application is to provide a V2V millimeter wave beam alignment method based on visual perception, which uses the scene perception ability of the vehicle-mounted monocular camera and combines computer vision perception technology to complete relatively accurate beam alignment while having the advantage of low hardware overhead.

[0006] To achieve the above purpose, the present application provides a V2V millimeter wave beam alignment method based on visual perception, comprising the following steps:

[0007] Step S1: design a fixed DFT codebook matrix, simulate beamforming;

[0008] Step S2: use the vehicle-mounted monocular camera to shoot the scene image, and use the FCOS3D target detection algorithm to detect the target vehicle in the scene, and obtain the three-dimensional position of the target in the camera coordinate system;

[0009] Step S3: coordinate transformation, transform the coordinate position of the target in the camera coordinate system to the antenna coordinate system, and then to the plane polar coordinate system, and calculate the distance and azimuth of the target from the vehicle body;

[0010] Step S4: according to the target azimuth, generate a beamforming vector using the DFT codebook matrix, and complete the beam alignment.

[0011] 3、Preferably, in step S1, specifically comprising:

[0012] The departure angle grid points in the channel are represented as , and satisfy , where is the number of departure angle grid points, is the number of transmitting end antennas, and specifically, the angle range of the departure angle and the arrival angle is , and the departure angle on the grid point can be represented as

[0013] Assuming , the array steering vectors corresponding to different angles are combined in a matrix to obtain the DFT codebook matrix of the transmitter as follows:

[0014] Wherein, each column of the DFT codebook matrix represents the corresponding antenna response of a departure angle or arrival angle.

[0015] Preferably, the coordinate system used in step S3 specifically includes:

[0016] (1) Camera coordinate system

[0017] The camera coordinate system takes the vehicle-mounted camera optical axis as the z-axis, and the x-axis and y-axis are parallel to the x-axis and y-axis of the image coordinate system, wherein the y-axis is parallel to the gravity axis, and the z-axis coordinate value represents the depth information of the target.

[0018] (2) Antenna coordinate system

[0019] The antenna coordinate system takes the midpoint of the vehicle body ULA antenna as the coordinate origin, the x-axis is parallel to the x-axis of the camera coordinate system, the y-axis is parallel to the z-axis of the camera coordinate system, and the z-axis is upward and parallel to the y-axis of the camera coordinate system.

[0020] (3) Plane polar coordinate system

[0021] The plane polar coordinate system is in the antenna coordinate system The plane polar coordinate system is in the antenna coordinate system

[0022] Preferably, in step S3, the coordinate transformation process is as follows:

[0023] (1) Transform from the camera coordinate system to the antenna coordinate system

[0024] Rotate the camera coordinate system by 90 degrees around the x-axis, and then translate T along the positive direction of the y-axis y m, and translate T along the positive direction of the z-axis z m, and the target coordinate in the camera coordinate system is , let , and the transformation formula from the camera coordinate system to the antenna coordinate system is as follows:

[0025] After transforming from the camera coordinate system to the antenna coordinate system, the position coordinates of the target are .

[0026] (2) Transform from the vehicle-mounted ULA antenna coordinate system to the plane polar coordinate system

[0027] Since the height of the vehicle body and other vehicles in the V2V communication scene is approximately equal, the pitch angle of the beam is ignored, and only the beam in the two-dimensional polar coordinate system is considered. According to the polar coordinate calculation formula:

[0028] Substitute the target coordinates in the antenna coordinate system into, and the distance from the target to the vehicle body and the beam departure angle are:

[0029] .

[0030] Preferably, in step S4, the target azimuth angle obtained in step S3 is used , the antenna array response corresponding to the angle in the DFT codebook matrix is selected, that is, the beamforming vector for the target is obtained, and the beam alignment is completed.

[0031] Therefore, the V2V millimeter wave beam alignment method based on visual perception has the following beneficial effects:

[0032] (1) First, the real-time scene image collected by the vehicle-mounted monocular camera is used to estimate the three-dimensional position of the vehicle target in the scene through the FCOS3D target detection algorithm;

[0033] (2) Then, the vehicle target position is transformed to obtain the spatial angle and the distance from the vehicle body, and the simulated beamforming vector is derived to realize beam alignment;

[0034] (3) By estimating the target position with high precision, high-precision beam alignment can be achieved with low hardware overhead. It has advantages such as high beam alignment accuracy, low spectrum overhead and low hardware cost.

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of a V2V millimeter-wave beam alignment scenario according to the present invention;

[0037] Figure 2 This is a schematic diagram of the coordinate system and coordinate system transformations defined in this invention;

[0038] Figure 3 This is the target position estimation and beam alignment result of the present invention;

[0039] Figure 4 (a)(b)(c) show the beam alignment and beam tracking results of the present invention in continuous scenes. Detailed Implementation Example

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0041] The communication scenario considered in this invention is V2V millimeter-wave communication on urban roads, as shown in the following scenario. Figure 1 As shown. Assume a vehicle is traveling along a lane, its direction of travel parallel to the right side of the lane. Assume a monocular camera sensor, receiving antenna, and transmitting antenna are mounted on the top of the vehicle, and a receiving antenna is mounted on the top of the target vehicle. The implementation includes the following steps:

[0042] Step 1: Design a fixed DFT codebook matrix for simulating beamforming. This invention addresses the beam alignment problem by employing a fixed DFT codebook to complete beamforming and generate a beam with a defined direction. The starting angle in the channel is represented as a grid. And satisfy ,here The number of grid points at the starting angle. This refers to the number of transmitting antennas. Specifically, the angle range considering the departure angle and arrival angle is as follows: Then the departure angle at the grid point can be expressed as follows:

[0043] Assume , the array steering vectors corresponding to different angles are written in a matrix, and the DFT codebook matrix of the transmitter is obtained as follows:

[0044] Where each column of the DFT codebook matrix represents the corresponding antenna response of an angle of departure or angle of arrival. If you want to point the beam to the direction corresponding to a grid point, the beamforming vector should be a column of the matrix.

[0045] Step two, use the vehicle-mounted monocular camera to shoot the scene image, and use the fully convolutional one-stage monocular three-dimensional target detection (FCOS3D) algorithm to detect the target vehicle in the scene, and obtain the three-dimensional position of the target in the camera coordinate system.

[0046] Step three, coordinate transformation is performed to obtain the distance and azimuth angle of the target from the vehicle body. Since the detection result of the FCOS3D algorithm is the three-dimensional position of the target in the camera coordinate system, this method first transforms the coordinate position of the target in the camera coordinate system to the antenna coordinate system, and then to the plane polar coordinate system, and calculates the distance and azimuth angle of the target from the vehicle body. The coordinate system used in this paper is as follows, and the coordinate transformation relationship is shown in Figure 2 :

[0047] (1) Camera coordinate system

[0048] The camera coordinate system takes the vehicle-mounted camera optical axis as the z-axis, and the x-axis and y-axis are parallel to the x-axis and y-axis of the image coordinate system. Among them, the y-axis is parallel to the gravity axis, and the z-axis coordinate value represents the depth information of the target.

[0049] (2) Antenna coordinate system

[0050] The antenna coordinate system takes the midpoint of the vehicle ULA antenna as the coordinate origin, the x-axis is parallel to the x-axis of the camera coordinate system, the y-axis is parallel to the z-axis of the camera coordinate system, and the z-axis is upward parallel to the y-axis of the camera coordinate system.

[0051] (3) Plane polar coordinate system

[0052] The plane polar coordinate system is in the antenna coordinate system on the plane, taking the origin as the polar coordinate pole and the x-axis as the polar axis.

[0053] The position and coordinate system transformation relationship diagram of the vehicle ULA antenna and the vehicle-mounted camera is shown in Figure 2 ULA antenna and vehicle-mounted camera are located on the vehicle centerline. According to the position relationship between the defined coordinate systems, the coordinate transformation process of this method is as follows:

[0054] (1) Transform from camera coordinate system to antenna coordinate system

[0055] Rotate the camera coordinate system by 90° around the x-axis, and then translate along the positive direction of the y-axis , and translate along the positive direction of the z-axis , the antenna coordinate system can be obtained. Let the coordinates of the target in the camera coordinate system be , let , and the transformation formula from the camera coordinate system to the antenna coordinate system is as follows:

[0056] Therefore, after transforming from the camera coordinate system to the antenna coordinate system, the position coordinates of the target are .

[0057] (2) Transform from the vehicle-mounted ULA antenna coordinate system to the planar polar coordinate system

[0058] Since the height of the vehicle body and other vehicles in the V2V communication scene is approximately equal, the pitch angle of the beam is ignored, and only the beam in the two-dimensional polar coordinate system is considered. According to the polar coordinate calculation formula:

[0059] Substitute the coordinates of the target in the antenna coordinate system , and the distance from the target to the vehicle body and the beam departure angle are obtained as:

[0060] Step four, according to the target azimuth angle, use the DFT codebook matrix to generate the beamforming vector, and complete the beam alignment. Use the target azimuth angle obtained in step three , select the antenna array response corresponding to the angle in the DFT codebook matrix, that is, obtain the beamforming vector for the target, and complete the beam alignment.

[0061] Embodiment:

[0062] According to the above method, simulation verification is carried out:

[0063] Simulation conditions: The method proposed in this paper is simulated and verified using the nuScenes dataset. This dataset uses a collection vehicle equipped with 5 remote radars and 6 cameras to collect driving data in more than 100 driving scenes. Each object in the scene is labeled with a 3D box and annotated with attributes. The scene is rich and complex. This embodiment tests and verifies the proposed method using images in different scenes, and obtains the following results.

[0064] Figure 3 The beam alignment result in the scene where the vehicle is driving longitudinally is recorded in Table 1 Figure 3The position information of the vehicle target and the error of the beam pointing result in the scene. Figure 3 The left side of the figure is the scene image, in which the detected vehicle target is framed by the 3D bounding box. The right side of the picture is the polar coordinate diagram corresponding to the target, the square mark is located at the (0, 0) point, indicating the antenna position, the triangular mark indicates the detected vehicle, and the plus mark indicates the target position labeled in the data set. The length of the beam represents the distance from the target to the vehicle body.

[0065] Observation Figure 3 It can be seen that the four vehicle targets in the figure are all detected by the target detection algorithm, and the positions are marked by 3D bounding boxes. In the polar coordinate diagram, the estimated target position of the algorithm almost coincides with the true target position, the four beams are respectively pointed to the four targets, the beam length represents the distance from the vehicle body to the target, the corresponding beam forming vector is generated, and the beam pointing to the vehicle target is realized.

[0066] Observing the data in Table 1, the distance errors of targets at different positions are compared. The farther the target is from the vehicle body, the greater the distance estimation error is, but the overall distance error is small. The angle error is small overall, and the angle estimation accuracy is high. Comparing car1 and car2, their distances are approximately equal, but car2 is surrounded by more vehicles and is blocked by the front vehicle, while car1 is surrounded by fewer vehicles. The distance error and angle error of car2 are much larger than those of car1, indicating that the increase of other targets around the vehicle target blocks the effective path of beam pointing, which reduces the accuracy of beam pointing.

[0067]

[0068] Figure 4 The figure is the beam pointing result of multiple consecutive frame scene images in the vehicle lateral driving scene. The detected vehicle target in the scene image is framed by the 3D bounding box. In the polar coordinate diagram, the triangular mark indicates the detected vehicle, and the length of the beam represents the distance from the vehicle target to the vehicle body.

[0069] Observation Figure 4 (a), the vehicle targets in the figure are all marked by 3D detection boxes, and the marks coincide with the vehicle. In the polar coordinate diagram, the generated beam is pointed to the target direction. Comparing Figure 4 (a) and Figure 4 (b), it can be seen that the two vehicles in the figure have moved, but they are all detected by the algorithm and marked by 3D bounding boxes. In the polar coordinate diagram, the generated beam changes correspondingly with the change of the target position, and the beam direction is still pointed to the target. Comparing Figure 4 (b) and Figure 4(c), the third vehicle appearing in the figure is also detected by the algorithm. In the polar coordinate diagram, three beams are generated for the three targets. At the same time, the beams pointing to the first two targets continue to change according to the change of the target position. The simulation results show that the V2V millimeter wave beam alignment based on visual perception can complete relatively accurate beam alignment for multiple target scenarios.

[0070] Therefore, the application adopts the above-mentioned V2V millimeter wave beam alignment method based on visual perception, which can complete high-precision beam alignment with low hardware overhead, has the advantages of high beam alignment precision, low spectrum overhead, low hardware cost, etc.

[0071] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can still be modified or replaced by equivalents, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A visual perception based V2V millimeter wave beam alignment method, characterized in that, The method comprises the following steps: Step S1: design a fixed DFT codebook matrix, simulate beamforming; Step S2: use a vehicle-mounted monocular camera to shoot a scene image, and use an FCOS3D target detection algorithm to detect target vehicles in the scene to obtain the three-dimensional position of the target in the camera coordinate system; Step S3: perform coordinate transformation to transform the coordinate position of the target in the camera coordinate system to the antenna coordinate system and then to the plane polar coordinate system, and calculate the distance and azimuth of the target from the vehicle body; Step S4: according to the target azimuth, generate a beamforming vector using the DFT codebook matrix to complete beam alignment. 2.The V2V millimeter wave beam alignment method based on visual perception according to claim 1, characterized in that, In step S1, specifically comprising: The departure angle grid points in the channel are represented as , and satisfy , is the number of departure angle grid points, is the number of transmitting end antennas, and the angle range of the departure angle and the arrival angle is The departure angles on the grid points are respectively represented as: The array steering vectors corresponding to different angles are combined in a matrix to obtain the DFT codebook matrix of the transmitter as follows: where each column of the DFT codebook matrix represents a corresponding antenna response for an angle of departure or angle of arrival. 3.The V2V millimeter wave beam alignment method based on visual perception according to claim 1, characterized in that, The coordinate system used in step S3 specifically comprises: (1) Camera coordinate system The camera coordinate system takes the optical axis of the vehicle-mounted camera as the z-axis, and the x-axis and y-axis are parallel to the x-axis and y-axis of the image coordinate system, wherein the y-axis is parallel to the gravity axis, and the z-axis coordinate value represents the depth information of the target; (2) Antenna coordinate system The antenna coordinate system takes the midpoint of the vehicle body ULA antenna as the coordinate origin, the x-axis is parallel to the x-axis of the camera coordinate system, the y-axis is parallel to the z-axis of the camera coordinate system, and the z-axis is upward and parallel to the y-axis of the camera coordinate system; (3) Plane polar coordinate system The plane polar coordinate system is in the antenna coordinate system On the plane, the origin is the polar coordinate pole, and the x-axis is the polar axis.

4. The visual perception based V2V millimeter wave beam alignment method according to claim 1, wherein, In step S3, the coordinate transformation process is as follows: (1) Transform from the camera coordinate system to the antenna coordinate system Rotate the camera coordinate system 90° around the x-axis and then translate it T along the positive y-axis y m, translate it T along the positive z-axis z m to get the antenna coordinate system. Let the coordinates of the target in the camera coordinate system be , let The transformation formula from the camera coordinate system to the antenna coordinate system is as follows: After transforming from the camera coordinate system to the antenna coordinate system, the position coordinates of the target are (2) Transform from the vehicle-mounted ULA antenna coordinate system to the plane polar coordinate system In the V2V communication scene, considering the case of beam in the two-dimensional polar coordinate system, according to the polar coordinate calculation formula: The coordinates of the target in the antenna coordinate system Substituting, we get the distance from the target to the vehicle body and the beam departure angle as: 。 5. The visual perception based V2V millimeter wave beam alignment method according to claim 1, characterized in that: In step S4, the target azimuth angle obtained in step S3 is used. By selecting the antenna array response corresponding to the angle in the DFT codebook matrix, the beamforming vector for the target is obtained, and beam alignment is completed.

Images