Target tracking method and device based on 5d millimeter wave radar, and storage medium

Abstract

This application provides a target tracking method, device, and storage medium based on 5D millimeter-wave radar. The millimeter-wave radar includes a three-transmit, four-receive millimeter-wave radar utilizing TDM MIMO technology. The method processes the transmitted and received signals from the radar to acquire 5D pedestrian information, including target range, azimuth, elevation, radial velocity, and tangential velocity. Target point detection is performed based on this 5D pedestrian information, and target tracking and data updates are performed using Kalman filtering. The method in this application is more sophisticated in acquiring velocity information, applying Kalman filtering to target radial and tangential velocities for tracking and updating, thereby better estimating and tracking the target trajectory.

Description

Technical Field

[0001] This application relates to radar signal processing technology, and more particularly to a target tracking method, device and storage medium based on 5D millimeter-wave radar. Background Technology

[0002] With rapid economic development and the continuous improvement of people's living standards, the car ownership rate is gradually increasing, but at the same time, traffic accidents are also on the rise. Therefore, the demand for Advanced Driver Assistance Systems (ADAS) in the field of safer, more intelligent vehicles has increased significantly. ADAS can proactively assess and prevent emergencies before the driver notices them, thereby reducing harm caused by driver error and improving driving safety.

[0003] In ADAS systems, pedestrians, as typical vulnerable road users, are of paramount importance in environmental perception. ADAS detects pedestrians by collecting data reflected from the surrounding environment by sensors installed in the vehicle, thereby predicting potential road hazards and taking appropriate protective measures. Commonly used sensors in ADAS include cameras, ultrasonic radar, lidar, and 4D millimeter-wave radar. However, the first few types of sensors have various drawbacks. Cameras are particularly susceptible to light conditions; ultrasonic radar has a long detection cycle, short maximum detection range, and is greatly affected by temperature and environment; lidar is bulky, expensive, has rapid attenuation, and has an extremely narrow beam, limiting its target search and acquisition capabilities to a small area. Millimeter-wave radar, on the other hand, is gradually gaining an increasingly important position among ADAS sensors due to its all-weather operation, immunity to light conditions, long detection range, high accuracy, low cost, and small size.

[0004] However, traditional 4D millimeter-wave vehicle radar is weak in sensing motion direction and speed information, thus making it unable to accurately detect and track targets. Summary of the Invention

[0005] This application provides a target tracking method, device, and storage medium based on 5D millimeter-wave radar to solve the problem of inaccurate target tracking in the prior art.

[0006] Firstly, this application provides a target tracking method based on 5D millimeter-wave radar. The 5D millimeter-wave radar includes three transmitting antennas arranged horizontally and four receiving antennas arranged horizontally. The horizontal distance between two adjacent transmitting antennas is λ, and the vertical distance between the second transmitting antenna in the middle and the first and third transmitting antennas is [missing information]. The vertical distance between the first and third transmitting antennas is 0; the horizontal distance between two adjacent receiving antennas is λ / 2; the method includes:

[0007] The carrier frequency is transmitted through three transmit antennas using a time-division multiplexing method. 77GHz FMCW signal ( , (At the speed of light), and acquire the received signals from the four receiving antennas for each transmitting antenna, and perform frequency mixing processing on the corresponding transmitted signals based on the received signals to obtain the beat signals corresponding to the 12 virtual array elements. Where i represents the i-th transmitting antenna and j represents the j-th receiving antenna, and i and j are positive integers;

[0008] For beat signals Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ;

[0009] According to enhanced beat signal and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ;

[0010] For beat signals Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ;

[0011] Based on the vertical array arrangement of the three transmitting antennas, the target elevation angle is obtained using the phase comparison angle measurement method. ;

[0012] Based on the target distance R and azimuth angle and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points;

[0013] Based on the target distance R and azimuth angle Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

[0014] In one possible design, the enhancement of the beat signal... and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ,include:

[0015] Enhanced beat signal and Doppler compensation is performed to obtain the compensated beat signal. and ;

[0016] Doppler-compensated beat signal and The emission interferometry is processed to obtain the relevant signals;

[0017] Perform FFT processing on the coherent signal to obtain the tangential Doppler frequency;

[0018] The target tangential velocity is obtained based on the tangential Doppler frequency. .

[0019] In one possible design, the distance to the target R and the azimuth angle are used as the basis for the design. Target radial velocity v r Target tangential velocity Target tracking is accomplished by updating the target state using Kalman filtering, including:

[0020] Set the state of the target at time k-1 as follows:

[0021]

[0022] According to the Kalman filter, the predicted state value for the next time step is:

[0023]

[0024] in Let be the state transition matrix, where It is the time interval between different frame signals. For process noise, where This is the transpose of the matrix.

[0025] In one possible design, for beat signals Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ,include:

[0026] Intermittent beat signal Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and enhance beat signal ;

[0027] For the enhanced beat signal and enhance beat signal Perform fast-time FFT processing to obtain a fast-time Fourier signal, extract the intermediate frequency related to the target distance from the fast-time Fourier signal, and obtain the target distance based on the intermediate frequency;

[0028] Perform slow-time FFT processing on the fast-time Fourier signal to obtain a slow-time Fourier signal. Obtain the radial Doppler frequency shift from the slow-time Fourier signal, and obtain the target radial velocity v from the Doppler frequency shift. r .

[0029] In one possible design, the target signal is... Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ,include:

[0030] For the eight virtual array elements corresponding to the first and third transmitting antennas, auxiliary beams are constructed and DBF coefficients are set. A pair of auxiliary beams are constructed to form seven sets of auxiliary beams. Angle discrimination curves are obtained based on the auxiliary beams.

[0031] Intermittent beat signal Sum and difference signals After performing 2D-CFAR, Doppler compensation is applied to obtain the compensated beat signal. Sum and difference signals ;

[0032] The compensated beat signal Sum and difference signals Multiply by the DBF coefficients and extract the maximum value and its subscript Idx to obtain the measurement angle range in which the target direction is within the Idx group of auxiliary beams;

[0033] Based on the measured angle range and the angle discrimination curve, the target azimuth angle is obtained using a lookup table method. .

[0034] In one possible design, the DBF coefficient dbfCoff is calculated using the following formula:

[0035]

[0036] in =1,…,8, where the beam pointing of 7 auxiliary beams is set. Grouped in pairs, the results are: (-45.5) -32.5 (-32.5) -19.5 (-19.5) -6.5 (-6.5) 6.5 (6.5) 19.5 (19.5) 32.5 ) and (32.5 45.5 ).

[0037] In one possible design, the target elevation angle is extracted using a phase comparison angle measurement method based on the vertical array arrangement of the three transmitting antennas. ,include:

[0038] Fourier beamforming is applied to the beat signals of the four virtual array elements corresponding to the second transmitting antenna to obtain the enhanced beat signal S. BB2 ;

[0039] Fourier beamforming is performed on the beat signals corresponding to the first transmitting antenna-third receiving antenna, the first transmitting antenna-fourth receiving antenna, the third transmitting antenna-first receiving antenna, and the third transmitting antenna-second receiving antenna to obtain enhanced beat signals. ;

[0040] According to the enhanced beat signal With enhanced beat signal phase difference Extract target pitch angle .

[0041] Secondly, this application provides a target tracking device based on a 5D millimeter-wave radar. The 5D millimeter-wave radar includes three transmitting antennas arranged horizontally in sequence and four receiving antennas arranged horizontally in sequence. The horizontal distance between two adjacent transmitting antennas is λ. The vertical distance between the second transmitting antenna (located in the middle) and the first and third transmitting antennas is λ / 2, and the vertical distance between the first and third transmitting antennas is 0. The horizontal distance between two adjacent receiving antennas is... The device includes:

[0042] The target distance and radial velocity acquisition module is used for beat signals. Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ;

[0043] The target tangential velocity acquisition module is used to obtain the enhanced beat signal. and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ;

[0044] The target azimuth acquisition module is used for beat signals. Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ;

[0045] The target elevation angle acquisition module is used to obtain the target elevation angle using the phase comparison angle measurement method based on the vertical array arrangement of the three transmitting antennas. ;

[0046] The target detection module is used to determine the target distance R and azimuth angle based on the target distance R. and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points;

[0047] The target tracking module is used to track the target distance R and azimuth angle. Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

[0048] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0049] The memory stores computer-executed instructions;

[0050] The processor executes computer execution instructions stored in the memory to implement a target tracking method based on 5D millimeter-wave radar.

[0051] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a target tracking method based on 5D millimeter-wave radar.

[0052] This application provides a target tracking method, device, and storage medium based on 5D millimeter-wave radar. The millimeter-wave radar includes a three-transmit, four-receive millimeter-wave radar utilizing TDM MIMO technology. The method processes the transmitted and received signals from the radar to acquire 5D pedestrian information, including target range, azimuth angle, target elevation angle, target radial velocity, and target tangential velocity. Target point detection is performed based on the 5D pedestrian information, and target tracking and data updating are performed using Kalman filtering. The following technical effects are achieved:

[0053] This invention can obtain two enhanced beat signals from the four beat signals corresponding to the first and third transmitting antennas respectively, achieving a 6dB receiving gain; thereby performing transmit interference and obtaining tangential velocity information; this invention also obtains two enhanced beat signals from the four beat signals corresponding to the second transmitting antenna and the middle four beat signals from the eight beat signals corresponding to the first and third transmitting antennas using Fourier beamforming technology, achieving a 6dB receiving gain; utilizing the vertical array arrangement of the four beat signals corresponding to the second transmitting antenna and the aforementioned middle four beat signals, elevation angle information is extracted from the vertical phase difference, and the obtained elevation angle information is more accurate; in addition, this invention uses transmit interference to obtain tangential velocity information, making the target velocity information more complete, and based on the tangential velocity and other pedestrian information, Kalman filtering is used for target tracking and updating, thereby better estimating and tracking the target trajectory. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a target tracking method based on 5D millimeter-wave radar provided in this application embodiment. Figure 1 ;

[0056] Figure 2A This is an array layout diagram of the transmitting antenna in the millimeter-wave radar used in the embodiments of this application;

[0057] Figure 2B This is an array layout diagram of the receiving antenna in the millimeter-wave radar used in the embodiments of this application;

[0058] Figure 3A This is a schematic diagram of the angle and distance information in the 5D pedestrian information obtained in the embodiments of this application;

[0059] Figure 3B This is a schematic diagram of speed information in 5D pedestrian information obtained in an embodiment of this application.

[0060] Figure 4 The auxiliary beam obtained in the embodiments of this application;

[0061] Figure 5 Angle identification curves obtained for embodiments of this application;

[0062] Figure 6 This is an array arrangement diagram of 12 virtual array elements in an embodiment of this application;

[0063] Figure 7A This is a 3D target output diagram showing the target moving along a back-to-forward trajectory in the embodiments of this application.

[0064] Figure 7B This is a 2D target output diagram showing the target moving along a back-to-forward trajectory in the embodiments of this application.

[0065] Figure 8A This is a 3D target output diagram showing the target moving along a trajectory from left to right in the embodiments of this application.

[0066] Figure 8B This is a 2D target output diagram showing the target moving along a trajectory from left to right in the embodiments of this application.

[0067] Figure 9A This is a 3D target output diagram showing the target moving along a clockwise circular trajectory in the embodiments of this application.

[0068] Figure 9B This is a 2D target output diagram showing the target moving along a clockwise circular trajectory in the embodiments of this application.

[0069] Figure 10A This is a 3D target output diagram showing the target moving along a trajectory of route W in the embodiments of this application.

[0070] Figure 10B This is a 2D target output diagram showing the target moving along a trajectory of route W in the embodiments of this application.

[0071] Figure 11 A schematic diagram of the structure of a target tracking device based on 5D millimeter-wave radar provided in this application embodiment;

[0072] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0074] First, the relevant concepts or terms involved in this application will be explained:

[0075] TDM (Time Division) technology: a technology in which multiple transmitting antennas take turns transmitting signals at different times.

[0076] MIMO (Multiple Input Multiple Output) technology: A radar transmission technology that includes multiple transmitting antennas and multiple receiving antennas to perform multiple inputs and multiple outputs.

[0077] FMCW (Frequency Modulated Continuous Wave): FMCW technology is a technique used in high-precision radar ranging. Its basic principle is that the transmitted wave is a high-frequency continuous wave, and the received echo frequency changes in the same way as the transmitted frequency, which is a triangular wave or a sawtooth wave. This invention is a sawtooth wave, which has only one time difference. The target distance can be calculated using this tiny time difference.

[0078] This method can be integrated into an Advanced Driver Assistance System (ADAS), using a 77GHz millimeter-wave radar as the ADAS sensor to detect and track far-field targets. The vehicle following control method provided in this application embodiment is described in detail below with reference to the accompanying drawings.

[0079] Figure 1 A schematic flowchart of a target tracking method based on 5D millimeter-wave radar provided in this application embodiment. Figure 1 .

[0080] Figure 2 shows the antenna array layout of the millimeter-wave radar used in this embodiment of the application. The millimeter-wave radar used in this embodiment includes three transmitting antennas and four transmitting antennas. Figure 2A This is a diagram showing the array layout of the transmitting antennas in a millimeter-wave radar. Figure 2BFigure 2 shows the array layout of the receiving antennas in a millimeter-wave radar. T1, T2, and T3 are the first, second, and third transmitting antennas arranged sequentially in the horizontal direction, respectively. R1, R2, R3, and R4 are four receiving antennas arranged at equal intervals in the horizontal direction. This millimeter-wave radar includes three transmitting antennas and four receiving antennas arranged sequentially in the horizontal direction. The horizontal distance between two adjacent transmitting antennas is λ. The vertical distance between the second transmitting antenna (located in the middle) and the first and third transmitting antennas is [missing information]. The vertical distance between the first and third transmitting antennas is 0; the horizontal distance between two adjacent receiving antennas is... ;

[0081] The method in this embodiment utilizes MIMO technology, employing the array arrangement and transmission carrier frequency shown in Figure 2. The 77GHz millimeter-wave radar transmits FMCW signals in a time-division multiplexing (TDM) manner. The transmitted signals are mixed according to the received signals to obtain the beat signals corresponding to 12 virtual array elements.

[0082] Based on the beat signals corresponding to the 12 virtual array elements, the 5D pedestrian information shown in Figure 3 is obtained through the method of this embodiment. Figure 3 shows the 5D pedestrian information from the millimeter-wave radar using the method of this invention. Figure 3A This is a schematic diagram of the angle and distance information in the 5D pedestrian information obtained in the embodiments of this application, including the target distance R and azimuth angle. and target pitch angle information ; Figure 3B This is a schematic diagram of velocity information in 5D pedestrian information obtained in an embodiment of this application, including the target radial velocity v. r Target tangential velocity Based on the 5D pedestrian information obtained above, the Kalman filter method is used for processing to complete the detection and tracking of the target.

[0083] The method in this application is more sophisticated in acquiring velocity information. It uses Kalman filtering based on the target's radial and tangential velocities to track and update the target, thereby better estimating and tracking the target trajectory.

[0084] like Figure 1 As shown, the method includes:

[0085] S110 uses time-division multiplexing to transmit carrier frequencies through three transmitting antennas. 77GHz FMCW signal ( , (At the speed of light), and acquire the received signals from the four receiving antennas for each transmitting antenna, and perform frequency mixing processing on the corresponding transmitted signals based on the received signals to obtain the beat signals corresponding to the 12 virtual array elements. Where i represents the i-th transmitting antenna and j represents the j-th receiving antenna, and i and j are positive integers;

[0086] c is the speed of light

[0087] Specifically, the processing method in step 110 includes:

[0088] Let T be the transmitted signal of the i-th transmitting antenna. i The calculation is performed using the following formula:

[0089] (1)

[0090] in For the center frequency, For frequency modulation slope, , The pulse repetition period, For time within a single Chirp; then the first... The transmitted signal of a Chirp is represented as:

[0091] (2)

[0092] Four receiving antennas receive the echo signal reflected by the target. Assuming the far-field target is relative to the first... Transmitting antenna and the The distance of the receiving antenna is The radial velocity is The delay of the echo signal for:

[0093] (3)

[0094] Where c is the speed of light. The initial distance to the target is denoted as , and the received signals from the four receiving antennas are denoted as . Obtained through the following formula

[0095] (4)

[0096] The beat signal obtained by mixing the i-th transmitting antenna and the j-th receiving antenna for:

[0097] (5)

[0098] in for The conjugate signal, This indicates that the signal has passed through a low-pass filter.

[0099] S120, for beat signals Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and To enhance beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ;

[0100] Specifically, step 120 includes the following steps:

[0101] S121, Counter-beat signal Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and enhance beat signal ;

[0102] Specifically, Fourier beamforming technology can be used to analyze beat signals. Sum and difference signals Phase compensation is performed so that the four received signals are in the same phase when they arrive at the radar receiver, thereby enhancing the received signal and achieving a 6dB receiving gain.

[0103] As shown in step S101, the transmission is performed through the first transmitting antenna, and the echo signals from the four receiving antennas are as follows:

[0104] (6)

[0105] in , The delay time caused by the arrangement of two adjacent receiving antennas. The baseline length between the two receiving antennas. It is the azimuth angle. For pitch angle, and The beat signal obtained by mixing the signal transmitted by the first transmitting antenna is:

[0106] (7)

[0107] Using beamforming technology, phase compensation is performed on the four beat signals corresponding to the first transmitting antenna to obtain an enhanced signal after summation in space:

[0108] (8)

[0109] Similarly, the four received signals corresponding to the third transmitting antenna are mixed with the transmitted signal to obtain the beat signal, and then Fourier beamforming is performed to obtain the enhanced signal:

[0110] (9)

[0111] in , The baseline lengths of the first and third transmitting antennas.

[0112] S122, Enhanced beat signal and enhance beat signal Perform fast-time FFT processing to obtain the fast-time Fourier signal, extract the intermediate frequency related to the target distance from the fast-time Fourier signal, and obtain the target distance based on the intermediate frequency;

[0113] Specifically, fast-time FFT is used to solve for the target distance R, and the time variable of the time FFT is t. s The beat signal is processed by a fast-time FFT to obtain a frequency domain signal, from which the intermediate frequency (IF) can be extracted. The obtained IF is then used as the frequency domain signal. The target distance can be obtained through formula conversion. .

[0114] S123. Perform slow-time FFT processing on the fast-time Fourier signal to obtain the slow-time Fourier signal. Obtain the radial Doppler frequency shift from the slow-time Fourier signal, and obtain the target radial velocity v from the Doppler frequency shift. r .

[0115] Specifically, slow-time FFT processing is used to solve for the target radial velocity v. r For the frequency domain signal obtained in step S122, a time variable nT is set, and a slow-time FFT is performed on nT to obtain the Doppler frequency shift, which is: The target radial velocity can be obtained through formula conversion. For each frame of radar echo signal, the velocity information is assumed to remain unchanged.

[0116] S130, Based on the enhanced beat signal and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. .

[0117] Specifically, step S130 includes the following steps:

[0118] S131, Enhanced beat signal and Doppler compensation is performed to obtain the compensated beat signal. and ;

[0119] Specifically, Fourier beamforming technology is used for Doppler compensation processing, where the compensation factor is... ,in , The frequency is the Doppler frequency.

[0120] S132, Beat signal after Doppler compensation and By performing emission interferometry processing, a coherent signal is obtained;

[0121] Specifically, its coherent signal is:

[0122] (10)

[0123] in for Conjugate signal.

[0124] S133. Perform FFT processing on the coherent signal to obtain the tangential Doppler frequency, and obtain the target tangential velocity based on the tangential Doppler frequency. .

[0125] The coherent frequency shift caused by angular velocity can be expressed by the phase term in equation (10). It is obtained by the time derivative, and its time reciprocal is:

[0126] (11)

[0127] in , Let be the angular velocity, and under small angular approximation... The target tangential velocity can be obtained from equation (11). .

[0128] S140, for beat signals Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ;

[0129] Specifically, S140 includes the following steps:

[0130] S141. Construct auxiliary beams for the eight virtual array elements corresponding to the first and third transmitting antennas and set DBF coefficients. Construct a set of auxiliary beams in pairs to form seven sets of auxiliary beams. Obtain the angle discrimination curve based on the auxiliary beams.

[0131] The DBF coefficient of the auxiliary beam is,

[0132] (12)

[0133] in =1,…,8; =-45.5 :13 45.5 Set the pointing direction of the two auxiliary beams in the first group to (-45.5). -32.5 Then the beam pointing of each group shifts 13 degrees to the right. Therefore, the beam directions of the 7 auxiliary beams are as follows: (-45.5) -32.5 (-32.5) -19.5 (-19.5) -6.5 (-6.5) 6.5 (6.5) 19.5 (19.5) 32.5 ) and (32.5 45.5 This simultaneously generates an angle-measuring curve, with an angle measurement range of -52°. ~52 The final result is 7 sets of auxiliary beams, such as Figure 4 As shown, the angle curve is as follows Figure 5 As shown.

[0134] S142, Counter-beat signal Sum and difference signals After performing 2D-CFAR, Doppler compensation is applied to obtain the compensated beat signal. Sum and difference signals ;

[0135] Specifically, after performing 2D-CFAR, the system iterates through the range and Doppler dimensions, extracting the maximum values ​​of the eight virtual array element signals for each range and velocity. It then determines whether the maximum amplitude exceeds a threshold. If it does, the current target range and velocity information is stored, and Doppler compensation is applied to the virtual array element beat signals. The compensation factor is... for, The frequency is the Doppler frequency.

[0136] S143, the compensated beat signal Sum and difference signals Multiply by the DBF coefficients and extract the maximum value and its subscript Idx to obtain the measurement angle range in which the target direction is within the Idx group of auxiliary beams;

[0137] S144. Based on the measured angle range and the angle identification curve, obtain the target azimuth using the table lookup method. .

[0138] S150. Based on the vertical array arrangement of the three transmitting antennas, the target elevation angle is obtained using the phase comparison angle measurement method. ;

[0139] Let the first Transmitting antenna and the The virtual array element synthesized by the receiving antenna is denoted as As shown in Figure 2, based on the arrangement of the millimeter-wave radar in this embodiment, the array of 12 virtual array elements is arranged as follows: Figure 6 As shown, the first transmitting antenna corresponds to 4 virtual array elements, and the third transmitting antenna corresponds to the middle 4 of the 8 virtual array elements. The phase difference between two adjacent virtual array elements in the horizontal direction is ,in The four virtual array elements corresponding to the second transmitting antenna The four virtual array elements in the middle of the eight beat signals corresponding to the first and third transmitting antennas. There is a phase difference in the vertical direction between each pair. ,in Here we use the phase difference in the vertical direction to solve for the pitch angle. .

[0140] Specifically, step S150 can be achieved through the following steps:

[0141] S151. Perform Fourier beamforming on the beat signals of the four virtual array elements corresponding to the second transmitting antenna to obtain enhanced beat signals. ;

[0142] S152. Perform Fourier beamforming on the beat signals corresponding to the first transmitting antenna-third receiving antenna, the first transmitting antenna-fourth receiving antenna, the third transmitting antenna-first receiving antenna, and the third transmitting antenna-second receiving antenna to obtain enhanced beat signals. ;

[0143] Specifically, assuming the target is in The signal generated at that location is , For amplitude, The phase is represented by the first and third transmitting antennas forming an 8-element virtual array.

[0144] (13)

[0145] The four virtual array elements corresponding to the second transmitting antenna can be represented by vectors as follows:

[0146] (14)

[0147] in The vertical virtual array phase difference, where The 3rd to 6th elements are ,and The phase difference of the four elements is .

[0148] right and Fourier beamforming is performed to obtain an enhanced signal. and Achieve a 6dB receive gain.

[0149] S153, Based on the enhanced beat signal With enhanced beat signal phase difference Extract target pitch angle .

[0150] Specifically, by enhancing the signal and Able to obtain its phase difference And the target pitch angle The solution can be obtained using the following formula:

[0151] (15)

[0152] S160, Based on target distance R and azimuth angle and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points;

[0153] Specifically, based on the above steps, 5D millimeter-wave radar point cloud data of each frame of radar signal is obtained, namely target range, target radial velocity, target azimuth angle, target elevation angle and target tangential velocity. Each frame of 5D millimeter-wave radar point cloud data can be obtained, and the target range, target azimuth angle and target elevation angle are converted into (x, y, z) in three-dimensional coordinates and output according to the following formula.

[0154]

[0155] (16)

[0156]

[0157] Then, the centroid algorithm is performed on each dimension of the 5D millimeter-wave radar point cloud data of each frame of radar signal. The dimensional information after the centroid algorithm is performed is used as the 5D point cloud data of the target at the trajectory point in the current frame, and is converted into three-dimensional coordinates (x, y, z) for output.

[0158] S170, Based on target distance R and azimuth angle Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

[0159] Specifically, after acquiring the 5D millimeter-wave radar point cloud data of each frame of trajectory points, the 5D radar point cloud data of the trajectory points is updated using Kalman filtering and converted into two-dimensional coordinates (x, y) for output, as shown in the right figure of Figure 8. As shown, the prediction and tracking of the target trajectory are completed. The detailed operation is as follows: since only the two-dimensional trajectory of the target is tracked, the pitch angle factor is not considered. At time... The state of -1 can be represented as,

[0160] (17)

[0161] Its next state prediction value is,

[0162] (18)

[0163] in Let be the state transition matrix, where It is the time interval between different frame signals.

[0164] (19)

[0165] Kalman filtering consists of two parts: the first part is the prediction process, and the second part is the update process.

[0166] The prediction process is as follows:

[0167] (20)

[0168] (twenty one)

[0169] The update process includes:

[0170] (twenty two)

[0171] (twenty three)

[0172] (twenty four)

[0173] in Let k be the estimation error at time k-1. The estimation error for the next time step. Represents the optimal Kalman gain. For For process noise with covariance, in a continuous white noise accelerated motion model, Follows the following distribution:

[0174] (25)

[0175] Process noise Determined by the target's motion characteristics, the observation matrix for

[0176] (26)

[0177] After each frame of 5D millimeter-wave radar point cloud data for a trajectory point is updated, the distance and azimuth angles are used to convert it into a two-dimensional trajectory for output. The conversion formula is as follows:

[0178]

[0179] (27)

[0180] In this embodiment, the method of this invention was used to detect and track the target when it moved along four different trajectories. Figures 7-10 show the target output results when the target moved along trajectories from back to front, from left to right, clockwise in a circle, and along a W-shaped path, respectively. Figure A shows the 3D target output result during target detection, and Figure B shows the 2D target output result during target tracking. The output represents the three-dimensional coordinates of the 5D millimeter-wave radar point cloud for each frame of signal. The output of the 5D millimeter-wave radar point cloud representing the target trajectory points in each frame of the signal; Figure B. The output represents the two-dimensional coordinates of the 5D millimeter-wave radar point cloud representing the target trajectory points after Kalman filtering in each frame of the signal.

[0181] The method provided in this embodiment utilizes TDM MIMO technology for a three-transmit, four-receive millimeter-wave radar. It processes the radar's transmitted and received signals to acquire 5D pedestrian information, including target range, azimuth, elevation, radial velocity, and tangential velocity. Target point detection is performed based on this 5D pedestrian information, and target tracking and data updates are achieved using Kalman filtering. The following technical effects are achieved:

[0182] This invention can obtain two enhanced beat signals from the four beat signals corresponding to the first and third transmitting antennas respectively, achieving a 6dB receiving gain; thereby performing transmit interference and obtaining tangential velocity information; this invention also obtains two enhanced beat signals from the four beat signals corresponding to the second transmitting antenna and the middle four beat signals from the eight beat signals corresponding to the first and third transmitting antennas using Fourier beamforming technology, achieving a 6dB receiving gain; utilizing the vertical array arrangement of the four beat signals corresponding to the second transmitting antenna and the aforementioned middle four beat signals, elevation angle information is extracted from the vertical phase difference, and the obtained elevation angle information is more accurate; in addition, this invention uses transmit interference to obtain tangential velocity information, making the target velocity information more complete, and based on the tangential velocity and other pedestrian information, Kalman filtering is used for target tracking and updating, thereby better estimating and tracking the target trajectory.

[0183] In this embodiment of the invention, electronic devices or main control devices can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0184] Figure 11 This is a schematic diagram of a target tracking device based on a 5D millimeter-wave radar, provided as an embodiment of this application. The 5D millimeter-wave radar includes three transmitting antennas arranged horizontally and four receiving antennas arranged horizontally. The horizontal distance between two adjacent transmitting antennas is λ, and the vertical distance between the second transmitting antenna in the middle and the first and third transmitting antennas is [missing information]. The vertical distance between the first and third transmitting antennas is 0; the horizontal distance between two adjacent receiving antennas is... ;like Figure 11 As shown, the device 1100 includes:

[0185] Mixer module 1101 is used to transmit carrier frequencies through three transmit antennas in a time-division multiplexing manner. 77GHz FMCW signal ( , (At the speed of light), and acquire the received signals from the four receiving antennas for each transmitting antenna, and perform frequency mixing processing on the corresponding transmitted signals based on the received signals to obtain the beat signals corresponding to the 12 virtual array elements. Where i represents the i-th transmitting antenna and j represents the j-th receiving antenna, and i and j are positive integers;

[0186] Target distance and target radial velocity acquisition module 1102 is used for beat signals Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and To enhance beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ;

[0187] Target tangential velocity acquisition module 1103 is used to acquire the target tangential velocity based on the enhanced beat signal. and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ;

[0188] Target azimuth acquisition module 1104 is used for beat signals Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ;

[0189] The target elevation angle acquisition module 1105 is used to acquire the target elevation angle using the phase comparison angle measurement method based on the vertical array arrangement of the three transmitting antennas. ;

[0190] Target detection module 1106 is used to detect targets based on their distance R and azimuth angle. and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points;

[0191] Target tracking module 1107 is used to track targets based on their distance R and azimuth angle. Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

[0192] Furthermore, the target tangential velocity acquisition module 1103 is specifically used for:

[0193] Enhanced beat signal and Doppler compensation is performed to obtain the compensated beat signal. and ;

[0194] Doppler-compensated beat signal and The emission interferometry is processed to obtain the relevant signals;

[0195] Perform FFT processing on the coherent signal to obtain the tangential Doppler frequency;

[0196] The target tangential velocity is obtained from the tangential Doppler frequency. .

[0197] Furthermore, the target tracking module 1107 is specifically used for:

[0198] Set the state of the target at time k-1 as follows:

[0199] (17)

[0200] According to the Kalman filter, the predicted state value for the next time step is:

[0201] (18)

[0202] in Let be the state transition matrix, where It is the time interval between different frame signals. For process noise, This is the transpose of the matrix.

[0203] Furthermore, the target distance and target radial velocity acquisition module 1102 is specifically used for:

[0204] Intermittent beat signal Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and enhance beat signal ;

[0205] Enhanced beat signal and enhance beat signal Perform fast-time FFT processing to obtain the fast-time Fourier signal, extract the intermediate frequency related to the target distance from the fast-time Fourier signal, and obtain the target distance based on the intermediate frequency;

[0206] Perform slow-time FFT processing on the fast-time Fourier signal to obtain the slow-time Fourier signal. Obtain the radial Doppler frequency shift from the slow-time Fourier signal, and then obtain the target radial velocity v from the Doppler frequency shift. r .

[0207] Furthermore, the target azimuth angle acquisition module 1104 is specifically used for:

[0208] For the eight virtual array elements corresponding to the first and third transmitting antennas, auxiliary beams are constructed and DBF coefficients are set. A pair of auxiliary beams are constructed to form seven sets of auxiliary beams. Angle discrimination curves are obtained based on the auxiliary beams.

[0209] Intermittent beat signal Sum and difference signals After performing 2D-CFAR, Doppler compensation is applied to obtain the compensated beat signal. Sum and difference signals ;

[0210] The compensated beat signal Sum and difference signals Multiply by the DBF coefficients and extract the maximum value and its subscript Idx to obtain the measurement angle range in which the target direction is within the Idx group of auxiliary beams;

[0211] Based on the measured angle range and the angle discrimination curve, the target azimuth angle is obtained using a lookup table method. .

[0212] Furthermore, the target azimuth angle acquisition module 1104 is also used for:

[0213] The DBF coefficient dbfCoff is calculated using the following formula:

[0214] (12)

[0215] in =1,…,8, where the beam pointing of 7 auxiliary beams is set. Grouped in pairs, the results are: (-45.5) -32.5 (-32.5) -19.5 (-19.5) -6.5 (-6.5) 6.5 (6.5) 19.5 (19.5) 32.5 ) and (32.5 45.5 ).

[0216] Furthermore, the target pitch angle acquisition module 1105 is specifically used for:

[0217] Fourier beamforming is applied to the beat signals of the four virtual array elements corresponding to the second transmitting antenna to obtain enhanced beat signals. ;

[0218] Fourier beamforming is performed on the beat signals corresponding to the first transmitting antenna-third receiving antenna, the first transmitting antenna-fourth receiving antenna, the third transmitting antenna-first receiving antenna, and the third transmitting antenna-second receiving antenna to obtain enhanced beat signals. ;

[0219] According to enhanced beat signal With enhanced beat signal phase difference Extract target pitch angle .

[0220] This embodiment provides a target tracking method based on 5D millimeter-wave radar, which can execute the target tracking method based on 5D millimeter-wave radar in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0221] In the aforementioned specific implementation of a target tracking device based on 5D millimeter-wave radar, each module can be implemented as a processor. The processor can execute computer execution instructions stored in the memory, thereby enabling the processor to execute the aforementioned target tracking method based on 5D millimeter-wave radar.

[0222] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 12 As shown, the electronic device 1200 includes at least one processor 1201 and a memory 1202. The electronic device 1200 also includes a communication component 1203. The processor 1201, the memory 1202, and the communication component 1203 are connected via a bus 1204.

[0223] In the specific implementation process, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to execute a target tracking method based on 5D millimeter-wave radar as executed on the electronic device side as described above.

[0224] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0225] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0226] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0227] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0228] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0229] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the target tracking method based on 5D millimeter-wave radar as described above.

[0230] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0231] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0232] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the scheme provided in any of the above embodiments.

[0233] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target tracking method based on 5D millimeter-wave radar, characterized in that, The millimeter-wave radar includes three transmitting antennas arranged in a horizontal direction and four receiving antennas arranged in a horizontal direction. The horizontal distance between two adjacent transmitting antennas is λ. The vertical distance between the second transmitting antenna in the middle position and the first and third transmitting antennas is λ / 2. The vertical distance between the first and third transmitting antennas is 0. The horizontal distance between two adjacent receiving antennas is λ / 2; the method includes: The carrier frequency is transmitted through three transmit antennas using a time-division multiplexing method. It is a 77GHz FMCW signal, in which , The speed of light is used, and the received signals from the four receiving antennas are obtained for each transmitting antenna. The received signals are then mixed with the corresponding transmitted signals to obtain the beat signals for the 12 virtual array elements. Where i represents the i-th transmitting antenna and j represents the j-th receiving antenna, and i and j are positive integers; For beat signals Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ; According to enhanced beat signal and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ; For beat signals Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ; Based on the vertical array arrangement of the three transmitting antennas, the target elevation angle is obtained using the phase comparison angle measurement method. ; Based on the target distance R and azimuth angle and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points; Based on the target distance R and azimuth angle Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

2. The method according to claim 1, characterized in that, The method based on enhanced beat signal and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ,include: Enhanced beat signal and Doppler compensation is performed to obtain the compensated beat signal. and ; Doppler-compensated beat signal and The emission interferometry is processed to obtain the relevant signals; Perform FFT processing on the coherent signal to obtain the tangential Doppler frequency; The target tangential velocity is obtained based on the tangential Doppler frequency. .

3. The method according to claim 1, characterized in that, The target distance R and target azimuth angle are used as the basis. Target radial velocity v r Target tangential velocity Target tracking is accomplished by updating the target state using Kalman filtering, including: Set the state of the target at time k-1 as follows: According to the Kalman filter, the predicted state value for the next time step is: in Let be the state transition matrix, where It is the time interval between different frame signals. For process noise, where [] T This is the transpose of the matrix.

4. The method according to claim 1, characterized in that, The target of beat signal Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ,include: Intermittent beat signal Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and enhance beat signal ; For the enhanced beat signal and enhance beat signal Perform fast-time FFT processing to obtain a fast-time Fourier signal, extract the intermediate frequency related to the target distance from the fast-time Fourier signal, and obtain the target distance based on the intermediate frequency; Perform slow-time FFT processing on the fast-time Fourier signal to obtain a slow-time Fourier signal. Obtain the radial Doppler frequency shift from the slow-time Fourier signal, and obtain the target radial velocity v from the Doppler frequency shift. r .

5. The method according to claim 1, characterized in that, The target of beat signal Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ,include: For the eight virtual array elements corresponding to the first and third transmitting antennas, auxiliary beams are constructed and DBF coefficients are set. A pair of auxiliary beams are constructed to form seven sets of auxiliary beams. Angle discrimination curves are obtained based on the auxiliary beams. Intermittent beat signal Sum and difference signals After performing 2D-CFAR, Doppler compensation is applied to obtain the compensated beat signal. Sum and difference signals ; The compensated beat signal Sum and difference signals Multiply by the DBF coefficients and extract the maximum value and its subscript Idx to obtain the measurement angle range in which the target direction is within the Idx group of auxiliary beams; Based on the measured angle range and the angle discrimination curve, the target azimuth angle is obtained using a lookup table method. .

6. The method according to claim 5, characterized in that, The DBF coefficient dbfCoff is calculated using the following formula: in =1,…,8, where the beam pointing of 7 auxiliary beams is set. Grouped in pairs, the results are: (-45.5) -32.5 (-32.5) -19.5 (-19.5) -6.5 (-6.5) 6.5 (6.5) 19.5 (19.5) 32.5 ) and (32.5 45.5 ).

7. The method according to claim 6, characterized in that, The target elevation angle is extracted using the phase comparison angle measurement method based on the vertical array arrangement of the three transmitting antennas. ,include: Fourier beamforming is applied to the beat signals of the four virtual array elements corresponding to the second transmitting antenna to obtain enhanced beat signals. ; Fourier beamforming is performed on the beat signals corresponding to the first transmitting antenna-third receiving antenna, the first transmitting antenna-fourth receiving antenna, the third transmitting antenna-first receiving antenna, and the third transmitting antenna-second receiving antenna to obtain enhanced beat signals. ; According to the enhanced beat signal With enhanced beat signal phase difference Extract target pitch angle .

8. A target tracking device based on 5D millimeter-wave radar, characterized in that, The 5D millimeter-wave radar includes three transmitting antennas arranged in a horizontal direction and four receiving antennas arranged in a horizontal direction. The horizontal distance between two adjacent transmitting antennas is λ. The vertical distance between the second transmitting antenna in the middle position and the first and third transmitting antennas is λ / 2. The vertical distance between the first and third transmitting antennas is 0. The horizontal distance between two adjacent receiving antennas is λ / 2; the device includes: The mixer module is used to transmit carrier frequencies through three transmit antennas in a time-division multiplexing manner. It is a 77GHz FMCW signal. , The speed of light is used, and the received signals from the four receiving antennas are obtained for each transmitting antenna. The received signals are then mixed with the corresponding transmitted signals to obtain the beat signals for the 12 virtual array elements. Where i represents the i-th transmitting antenna and j represents the j-th receiving antenna, and i and j are positive integers; The target distance and radial velocity acquisition module is used for beat signals. Sum and difference signals The signals were processed using Fourier beamforming technology to obtain enhanced beat signals. and For the enhanced beat signal and Perform 2D-FFT processing to obtain the target distance R and the target radial velocity v. r ; The target tangential velocity acquisition module is used to obtain the enhanced beat signal. and After performing emission interferometry to obtain the coherent signal, time-frequency analysis is then performed to obtain the target tangential velocity. ; The target azimuth acquisition module is used for beat signals. Sum and difference signals The target azimuth angle is obtained by the sum-difference beam amplitude angle measurement method. ; The target elevation angle acquisition module is used to obtain the target elevation angle using the phase comparison angle measurement method based on the vertical array arrangement of the three transmitting antennas. ; The target detection module is used to determine the target distance R and azimuth angle based on the target distance R. and target pitch angle information Three-dimensional coordinate transformation is performed to obtain the target detection points; The target tracking module is used to track the target distance R and azimuth angle. Target radial velocity v r Target tangential velocity Kalman filtering is used to update the target state and complete target tracking.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images