Training method of occupancy grid prediction network and vehicle environment perception method and device

Abstract

The present application relates to the field of environmental perception technology for intelligent driving, and provides a training method for an occupancy grid prediction network and a vehicle environment perception method and device, which are easy to deploy and can ensure the quality of multi-view occupancy grid prediction tasks. The method inputs multi-view image samples into an image feature extraction unit to obtain multi-view image feature samples; inputs multi-view image feature samples into a position encoding unit to obtain multi-view position encoding feature samples; inputs multi-view image feature samples and multi-view position encoding feature samples into a fusion unit to obtain three-dimensional perception visual feature samples, and inputs them into a multi-layer perceptron perspective transformation unit to obtain bird's-eye view feature samples; inputs bird's-eye view feature samples into an occupancy grid prediction unit to obtain occupancy grid prediction result samples; and adjusts the parameters of each unit to complete the training according to the difference between the occupancy grid prediction result samples and the occupancy grid label results.

Description

Technical Field

[0001] The present application relates to the field of environmental perception technology for intelligent driving, and in particular to a training method and device for an occupancy grid prediction network, a vehicle environment perception method and device, a computer device, a storage medium, and a computer program product. Background Art

[0002] The occupancy grid prediction result output by the occupancy grid prediction network can be used to form the vehicle's environmental perception result. The occupancy grid prediction network can output the occupancy grid prediction result based on the BEV feature. The full name of BEV is Bird's Eye View, and the Chinese name of BEV is Bird's Eye View. Therefore, the BEV feature can be called a Bird's Eye View feature.

[0003] The occupancy grid prediction network based on BEV features can use lift-splat operators or multi-layer perceptrons for perspective transformation. In the solution using lift-splat operators for perspective transformation, lift-splat operators include special operators such as cumsum, gather / gatherND, and scather / scatherND, which are difficult to deploy on general NPUs (the full name of NPU is Neural Network Processing Unit, and the Chinese name of NPU is embedded neural network processor), and the deployment is not friendly. Although the solution using multi-layer perceptron for perspective transformation is more friendly to deploy, this type of solution is more mature in the single-view occupancy grid prediction task, and the effect of direct migration to multi-view occupancy grid prediction task is poor. Summary of the invention

[0004] Based on this, it is necessary to provide a training method and device for an occupancy grid prediction network, a vehicle environment perception method and device, a computer device, a storage medium and a computer program product to address the above technical problems.

[0005] The present application provides a training method for an occupancy grid prediction network, wherein the occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit, and an occupancy grid prediction unit; the method includes:

[0006] Inputting the multi-view image samples of the sample vehicle into the image feature extraction unit to obtain multi-view image feature samples;

[0007] Inputting the multi-view image feature samples into the position encoding unit to obtain multi-view position encoding feature samples;

[0008] Inputting the multi-view image feature samples and the multi-view position coding feature samples into the fusion unit to obtain three-dimensional perception visual feature samples;

[0009] Inputting the three-dimensional perception visual feature samples into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample;

[0010] Inputting the bird's-eye view feature sample into the occupancy grid prediction unit to obtain an occupancy grid prediction result sample;

[0011] According to the difference between the occupancy grid prediction result sample and the occupancy grid label result, the parameters of each unit in the occupancy grid prediction network are adjusted to complete the training.

[0012] In one embodiment, the multi-view image feature samples are input into the position encoding unit to obtain multi-view position encoding feature samples, including:

[0013] The multi-view image feature samples are input into the position encoding unit so that the position encoding unit performs the following steps: expanding the depth dimension of each image feature coordinate in the multi-view image feature samples to obtain multi-view cone point cloud coordinate samples, and obtaining multi-view position encoding feature samples based on the multi-view cone point cloud coordinate samples.

[0014] In one embodiment, obtaining multi-view position coding feature samples according to the multi-view cone point cloud coordinate samples includes:

[0015] Convert the multi-view frustum point cloud coordinate samples from the image coordinate system to the ego-vehicle coordinate system of the sample vehicle;

[0016] The channels of the multi-view frustum point cloud coordinate samples in the vehicle coordinate system are aligned with the channels of the multi-view image feature samples to obtain multi-view position coding feature samples.

[0017] In one embodiment, the occupancy grid prediction network further includes an upsampling unit and a convolution unit;

[0018] The three-dimensional perception visual feature samples are input into the multi-layer perceptron perspective transformation unit to obtain bird's-eye view feature samples, including:

[0019] Inputting the three-dimensional perception visual feature samples into a multi-layer perceptron perspective transformation unit to obtain an initial bird's-eye view feature sample;

[0020] The initial bird's-eye view feature sample is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample.

[0021] The present application provides a training device for an occupancy grid prediction network, wherein the occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit, and an occupancy grid prediction unit. The device includes:

[0022] A first feature extraction module, used for inputting a multi-view image sample of a sample vehicle into the image feature extraction unit to obtain a multi-view image feature sample;

[0023] A first position coding module, used for inputting the multi-view image feature samples into the position coding unit to obtain multi-view position coding feature samples;

[0024] A first fusion module, used for inputting the multi-view image feature samples and the multi-view position coding feature samples into the fusion unit to obtain a three-dimensional perception visual feature sample;

[0025] A first perspective transformation module, used for inputting the three-dimensional perception visual feature sample into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample;

[0026] A first occupancy grid prediction module, used for inputting the bird's-eye view feature sample into the occupancy grid prediction unit to obtain an occupancy grid prediction result sample;

[0027] The first parameter adjustment module is used to adjust the parameters of each unit in the occupied grid prediction network according to the difference between the occupied grid prediction result sample and the occupied grid label result to complete the training.

[0028] The present application provides a vehicle environment perception method, the method comprising:

[0029] Obtaining a target occupancy grid prediction network; the target occupancy grid prediction network is: the occupancy grid prediction network is trained according to the above occupancy grid prediction network training method and the position encoding unit is removed;

[0030] Inputting the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features;

[0031] Get the multi-view position encoding feature samples obtained in the last iteration of the training process;

[0032] Inputting the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain a three-dimensional perception visual feature;

[0033] Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature;

[0034] Inputting the bird's-eye view feature into an occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result;

[0035] According to the occupancy grid prediction result, the environment perception result of the target vehicle is obtained.

[0036] In one embodiment, the object occupancy grid prediction network includes: an upsampling unit and a convolution unit;

[0037] Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature includes:

[0038] Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain an initial bird's-eye view perspective feature;

[0039] The initial bird's-eye view feature is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature.

[0040] The present application provides a vehicle environment perception device, the device comprising:

[0041] A network acquisition module is used to acquire a target occupancy grid prediction network; the target occupancy grid prediction network is obtained by removing the position encoding unit after the occupancy grid prediction network is trained according to the above occupancy grid prediction network training method;

[0042] A second feature extraction module is used to input the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features;

[0043] A position coding acquisition module is used to obtain multi-view position coding feature samples obtained in the last iteration during the training process;

[0044] A second fusion module is used to input the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain a three-dimensional perception visual feature;

[0045] A second perspective transformation module is used to input the three-dimensional perception visual feature into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature;

[0046] A second occupancy grid prediction module, used for inputting the bird's-eye view feature into an occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result;

[0047] The perception result acquisition module is used to obtain the environmental perception result of the target vehicle according to the occupancy grid prediction result.

[0048] The present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0049] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.

[0050] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0051] The solution provided in the present application first performs position encoding on multi-view image features to obtain multi-view position encoding features, then fuses the multi-view position encoding features with the multi-view image features to obtain three-dimensional perception visual features, and then uses a multi-layer perceptron to perform perspective transformation, inputs the three-dimensional perception visual features into the perspective transformation unit of the multi-layer perceptron to obtain bird's-eye view features (also called BEV features), which improves deployment friendliness while enhancing the expressiveness of bird's-eye view features, thereby achieving efficient deployment while ensuring the quality of occupancy grid prediction tasks in multi-view scenarios; and, the target occupancy grid prediction network used in subsequent reasoning scenarios removes the position encoding unit, and directly uses the multi-view position encoding feature samples obtained in the last iteration of the training process for fusion during reasoning, which can speed up reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A schematic diagram of a flow chart of a training method for an occupancy grid prediction network in one embodiment;

[0054] Figure 2 A schematic diagram of a flow chart of a training method for an occupancy grid prediction network in another embodiment;

[0055] Figure 3 is a schematic flow chart of a vehicle environment perception method in one embodiment;

[0056] Figure 4 is a flow chart of a vehicle environment perception method in another embodiment;

[0057] Figure 5 is a structural block diagram of a training device for an occupancy grid prediction network in one embodiment;

[0058] Figure 6 is a structural block diagram of a vehicle environment perception device in one embodiment;

[0059] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The training method of the occupancy grid prediction network provided in the present application can be executed by a computer device. The occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit and an occupancy grid prediction unit.

[0062] The image feature extraction unit and the position encoding unit can be regarded as two branches of the occupancy grid prediction network. The training method includes Figure 1 In order to distinguish the steps shown from the method used during reasoning, the features involved in the training method can be added with the word "sample". For example, the vehicles involved in the training are called sample vehicles, and the multi-view images involved are called multi-view image samples.

[0063] Step S101: input a multi-view image sample of a sample vehicle into an image feature extraction unit to obtain a multi-view image feature sample.

[0064] The sample vehicle may be equipped with 6 panoramic cameras corresponding to different viewing angles. The images collected by these panoramic cameras may be called multi-view image samples. In addition, a corresponding occupancy grid label result may be constructed for each multi-view image sample by labeling. After obtaining the multi-view image sample, the multi-view image sample may be input into the image feature extraction unit to obtain a multi-view image feature sample.

[0065] The image feature extraction unit may include an image encoder backbone unit and a 1×1 convolution unit. For example, the size of the multi-view image sample may be denoted as N c ×H1×W1×3; where N c Matching the number of panoramic cameras, H1 represents the length of the image, W1 represents the width of the image, and 3 represents the RGB colors of the image. After the multi-view image samples are input into the image encoder backbone unit, a size of N c×(H1 / 16)×(W1 / 16)×C1, where C1 represents the number of channels. Inputting the initial multi-view image feature samples into a 1×1 convolution unit, we can obtain a convolution unit of size N. c ×(H1 / 16)×(W1 / 16)×C1 multi-view image feature samples.

[0066] Step S102: input the multi-view image feature samples into a position encoding unit to obtain multi-view position encoding feature samples.

[0067] After obtaining the multi-view image feature samples, the multi-view image feature samples may be input into a position encoding unit to perform position encoding on each image feature to obtain multi-view position encoded feature samples.

[0068] Step S103: input the multi-view image feature samples and the multi-view position coding feature samples into a fusion unit to obtain three-dimensional perception visual feature samples.

[0069] The multi-view image feature samples and the multi-view position coding feature samples are aligned on the channel, so the multi-view image feature samples and the multi-view position coding feature samples can be input into the fusion unit, and the multi-view image feature samples and the multi-view position coding feature samples are fused, and the obtained fusion result can be called a three-dimensional perception visual feature sample. The fusion method can be an addition method.

[0070] If the multi-view image feature samples and the multi-view position coding feature samples are aligned, the sizes of both are N c ×(H1 / 16)×(W1 / 16)×C1, the size of the obtained three-dimensional perception visual feature sample is also N c ×(H1 / 16)×(W1 / 16)×C1.

[0071] Step S104: input the three-dimensional perception visual feature samples into the multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample.

[0072] The present application utilizes a multi-layer perceptron for perspective transformation, and therefore, the multi-layer perceptron may be referred to as a multi-layer perceptron perspective transformation unit.

[0073] After obtaining the three-dimensional perception visual feature samples, the three-dimensional perception visual feature samples can be input into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample.

[0074] Step S105: input the bird's-eye view feature samples into an occupied grid prediction unit to obtain occupied grid prediction result samples.

[0075] After obtaining the bird's-eye view feature samples, the bird's-eye view feature samples may be input into an occupancy grid prediction unit to obtain occupancy grid prediction result samples.

[0076] Step S106, adjusting the parameters of each unit in the occupancy grid prediction network according to the difference between the occupancy grid prediction result sample and the occupancy grid label result to complete the training.

[0077] If the size of the difference between the occupancy grid prediction result sample and the occupancy grid label result does not meet the requirements, the parameters of each unit in the occupancy grid prediction network can be adjusted according to the difference, thereby completing an iteration and entering the next iteration. The processing flow of the next iteration is the same as that of this iteration. When the size of the difference between the occupancy grid prediction result sample obtained in the iteration and the occupancy grid label result meets the requirements, the iteration can be stopped, the training is completed, and the trained occupancy grid prediction network is obtained.

[0078] The occupancy grid prediction network obtained by the above training method can first position encode the multi-view image features to obtain multi-view position encoding features, and then fuse the multi-view position encoding features with the multi-view image features to obtain three-dimensional perception visual features. Then, a multi-layer perceptron is used for perspective transformation, and the three-dimensional perception visual features are input into the perspective transformation unit of the multi-layer perceptron to obtain bird's-eye view features (also called BEV features). While improving deployment friendliness, the expression ability of the bird's-eye view features is enhanced, thereby achieving efficient deployment while ensuring the quality of the occupancy grid prediction task in multi-view scenarios.

[0079] In one embodiment, a multi-view image feature sample is input into a position encoding unit to obtain a multi-view position encoding feature sample, including: inputting the multi-view image feature sample into the position encoding unit so that the position encoding unit performs the following steps: expanding the depth dimension of each image feature coordinate in the multi-view image feature sample to obtain a multi-view cone point cloud coordinate sample, and obtaining a multi-view position encoding feature sample based on the multi-view cone point cloud coordinate sample.

[0080] With size N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image feature sample is taken as an example:

[0081] After getting the size N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image feature samples, the multi-view image feature samples can be input into the position encoding unit, and the position encoding unit can perform the following steps: expand the depth dimension of each image feature coordinate in the multi-view image feature sample, that is, a total of N c The coordinates of the image features of size N × (H1 / 16) × (W1 / 16) need to be expanded in depth dimension. c×(H1 / 16)×(W1 / 16)×D×3 multi-view frustum point cloud coordinate samples; c ×(H1 / 16)×(W1 / 16)×D×3 multi-view frustum point cloud coordinate samples are processed to obtain a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view position encoding feature samples.

[0082] In one embodiment, a multi-perspective position coding feature sample is obtained based on a multi-perspective view cone point cloud coordinate sample, including: converting the multi-perspective view cone point cloud coordinate sample from an image coordinate system to a self-vehicle coordinate system of the sample vehicle; aligning the channel of the multi-perspective view cone point cloud coordinate sample in the self-vehicle coordinate system to the channel of the multi-perspective image feature sample to obtain a multi-perspective position coding feature sample.

[0083] With size N c ×(H1 / 16)×(W1 / 16)×D×3 multi-view frustum point cloud coordinate sample is taken as an example:

[0084] The multi-view cone point cloud coordinate samples are in the image coordinate system of each periscopic camera. Therefore, the multi-view cone point cloud coordinate samples can be converted from the image coordinate system to the self-vehicle coordinate system of the sample vehicle. After the coordinate system conversion, the multi-view cone point cloud coordinate samples are still N c ×(H1 / 16)×(W1 / 16)×D×3.

[0085] The size of the multi-view image feature sample is N c ×(H1 / 16)×(W1 / 16)×C1, the channels of the multi-view frustum point cloud coordinate samples in the ego-vehicle coordinate system can be aligned to the channels of the multi-view image feature samples, that is, D×3 is converted to C1, thus obtaining a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view position encoding feature samples.

[0086] Specifically, the position encoding unit includes a 1×1 convolution unit, a ReLU activation function, and a 1×1 convolution unit connected in sequence. The multi-view frustum point cloud coordinate samples in the ego-vehicle coordinate system can be reshaped (shape transformed) and then processed through a 1×1 convolution unit, a ReLU activation function, and a 1×1 convolution unit to achieve channel alignment, thereby obtaining a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view position encoding feature samples. Among them, ReLU is the full name of Linearrectification function in English, and its Chinese name is linear rectification function.

[0087] In one embodiment, the occupancy grid prediction network also includes an upsampling unit and a convolution unit; the three-dimensional perception visual feature samples are input into the multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample, including: the three-dimensional perception visual feature samples are input into the multi-layer perceptron perspective transformation unit to obtain an initial bird's-eye view feature sample; the initial bird's-eye view feature sample is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample.

[0088] After the three-dimensional perception visual feature samples are input into the multi-layer perceptron perspective transformation unit, they are processed by reshape and pemute to obtain down-sampled bird's-eye view feature samples, which can be called initial bird's-eye view feature samples. The initial bird's-eye view feature samples are processed by the upsampling unit and the convolution unit, and the results obtained are called bird's-eye view feature samples.

[0089] In order to better understand the above method, an application example of the training method of the occupancy grid prediction network of the present application is described in detail below. In this application example, the image feature extraction unit may include an image encoder backbone unit and a 1×1 convolution unit. The position encoding unit includes a 1×1 convolution unit, a ReLU activation function, and a 1×1 convolution unit connected in sequence.

[0090] The size of the multi-view image sample can be recorded as N c ×H1×W1×3; where N c Matching the number of panoramic cameras, H1 represents the length of the image, W1 represents the width of the image, and 3 represents the RGB colors of the image.

[0091] Reference Figure 2 After inputting the multi-view image samples into the image encoder backbone unit, we can get a size of N c ×(H1 / 16)×(W1 / 16)×C1, where C1 represents the number of channels. Inputting the initial multi-view image feature samples into a 1×1 convolution unit, we can obtain a convolution unit of size N. c ×(H1 / 16)×(W1 / 16)×C1 multi-view image feature samples.

[0092] The multi-view image feature sample is input into the position encoding unit, and the position encoding unit can perform the following steps: expand the depth dimension of each image feature coordinate in the multi-view image feature sample, that is, a total of N c The coordinates of the image features of size N × (H1 / 16) × (W1 / 16) need to be expanded in depth dimension. c×(H1 / 16)×(W1 / 16)×D×3 multi-view cone point cloud coordinate samples, D represents the depth, 3 represents that each cone point cloud coordinate is determined by 3-dimensional information; according to the camera parameters of the sample vehicle's periscopic camera, the multi-view cone point cloud coordinate samples can be converted from the image coordinate system to the sample vehicle's own vehicle coordinate system. After the coordinate system conversion, the multi-view cone point cloud coordinate samples are still N c ×(H1 / 16)×(W1 / 16)×D×3.

[0093] The multi-view frustum point cloud coordinate samples in the ego-vehicle coordinate system can be reshaped and processed through a 1×1 convolution unit, a ReLU activation function, and a 1×1 convolution unit to achieve channel alignment, thereby obtaining a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view position encoding feature samples.

[0094] Set the size to N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image feature samples and multi-view position coding feature samples are input into the fusion unit to obtain a three-dimensional perception visual feature sample. The size of the three-dimensional perception visual feature sample is N c ×(H1 / 16)×(W1 / 16)×C1.

[0095] For a size of N c ×(H1 / 16)×(W1 / 16)×C1 three-dimensional perception visual feature samples are reshaped and pemute (rearranged) to obtain flattened three-dimensional perception visual feature samples. The size of the three-dimensional perception visual feature samples is increased from N c ×(H1 / 16)×(W1 / 16)×C1 becomes C1×[N c ×(H1 / 16)×(W1 / 16)].

[0096] After the flattened 3D perception visual feature samples are input into the multi-layer perceptron perspective transformation unit, the initial bird's-eye view feature samples can be obtained after reshape and pemute processing. The size of the initial bird's-eye view feature samples is (H1 / scale)×(W1 / scale)×C b , C b Indicates the number of channels, C b The size of is different from that of C1. scale is the scaling ratio, which is a preset value.

[0097] The initial bird's-eye view feature sample is processed by the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample. The size of the bird's-eye view feature sample is H b ×W b ×C b , Hb Represents the length of three-dimensional space, W b Indicates the width of three-dimensional space.

[0098] Next, the bird's-eye view feature sample can be input into the occupancy grid prediction unit to obtain the initial occupancy grid prediction result sample. The size of the initial occupancy grid prediction result sample is C b Z×H b ×W b , Z is the height of the three-dimensional space. The initial occupancy grid prediction result sample is stretched to obtain an occupancy grid prediction result sample with a length, width and height of the three-dimensional space. The size of the occupancy grid prediction result sample is C b ×H b ×W b ×Z.

[0099] If the size of the difference between the occupancy grid prediction result sample and the occupancy grid label result does not meet the requirements, the parameters of each unit in the occupancy grid prediction network can be adjusted according to the difference, thereby completing an iteration and entering the next iteration. The processing flow of the next iteration is the same as that of this iteration. When the size of the difference between the occupancy grid prediction result sample obtained in the iteration and the occupancy grid label result meets the requirements, the iteration can be stopped, the training is completed, and the trained occupancy grid prediction network is obtained.

[0100] The occupancy grid prediction network obtained by the above training method can first perform position encoding on the multi-view image features to obtain multi-view position encoding features, and then fuse the multi-view position encoding features with the multi-view image features to obtain three-dimensional perception visual features. Then, a multi-layer perceptron is used for perspective transformation, and the three-dimensional perception visual features are input into the perspective transformation unit of the multi-layer perceptron to obtain bird's-eye view features (also called BEV features). It gets rid of the dependence on special operators and can be obtained with the help of linear operators such as convolution. While improving deployment friendliness, it enhances the expression ability of bird's-eye view features, thereby achieving efficient deployment while ensuring the quality of occupancy grid prediction tasks in multi-view scenarios.

[0101] The present application also provides a vehicle environment perception method, which can be executed by a computer device, the method comprising: Figure 3 The steps shown are:

[0102] Step S301, obtaining a target occupancy grid prediction network.

[0103] After the occupancy grid prediction network is trained according to the above training method, the position encoding unit can be removed to obtain the target occupancy grid prediction network. At this time, the target occupancy grid prediction network includes an image feature extraction unit, a fusion unit, a multi-layer perceptron perspective transformation unit and an occupancy grid prediction unit.

[0104] Step S302: input the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features.

[0105] The target vehicle may be equipped with 6 panoramic cameras corresponding to different viewing angles, and the images collected by these panoramic cameras may be called multi-view images. The multi-view images are input into the image feature extraction unit to obtain multi-view image features.

[0106] The image feature extraction unit may include an image encoder backbone unit and a 1×1 convolution unit. For example, the size of the multi-view image may be denoted as N c ×H1×W1×3; where N c Matching the number of panoramic cameras, H1 represents the length of the image, W1 represents the width of the image, and 3 represents the RGB colors of the image. After the multi-view image is input into the image encoder backbone unit, a size of N c ×(H1 / 16)×(W1 / 16)×C1, where C1 represents the number of channels. Inputting the initial multi-view image features into a 1×1 convolution unit, we can obtain a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image features.

[0107] Step S303, obtaining a multi-view position coding feature sample obtained in the last iteration during the training process.

[0108] When the camera parameters of the target vehicle's surround camera are fixed to the camera parameters of the sample vehicle's surround camera, the multi-view position coding feature sample obtained in the last iteration of the multiple iterations involved in the above training process can be retained. The multi-view position coding feature sample can be applied in the reasoning scenario. The size of the multi-view position coding feature sample is N c ×(H1 / 16)×(W1 / 16)×C1, which is consistent with the size of the multi-view image features.

[0109] Step S304: input the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain three-dimensional perception visual features.

[0110] The multi-view image features and the multi-view position encoding features are aligned on the channel. For example, both have a size of N c ×(H1 / 16)×(W1 / 16)×C1, the multi-view image features and the multi-view position coding features can be input into the fusion unit, and the multi-view image features and the multi-view position coding features are fused, and the obtained fusion result can be called a three-dimensional perception visual feature.

[0111] Step S305, inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature.

[0112] After obtaining the three-dimensional perception visual features, the three-dimensional perception visual features can be input into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view perspective feature.

[0113] Step S306: input the bird's-eye view feature into the occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result.

[0114] After obtaining the bird's-eye view feature, the bird's-eye view feature may be input into an occupancy grid prediction unit to obtain an occupancy grid prediction result.

[0115] Step S307, obtaining the environment perception result of the target vehicle according to the occupancy grid prediction result.

[0116] The occupancy grid prediction result describes whether each grid in the three-dimensional space is occupied. Therefore, the environmental perception result of the target vehicle can be obtained based on the occupancy grid prediction result.

[0117] In the above-mentioned vehicle environment perception method, the multi-view image features are first position-encoded to obtain multi-view position coding features, and then the multi-view position coding features are fused with the multi-view image features to obtain three-dimensional perception visual features. Then, a multi-layer perceptron is used for perspective transformation, and the three-dimensional perception visual features are input into the perspective transformation unit of the multi-layer perceptron to obtain bird's-eye view features (also called BEV features). While improving deployment friendliness, the expression ability of bird's-eye view features is enhanced, thereby achieving efficient deployment while ensuring the quality of occupancy grid prediction tasks in multi-view scenarios; and, the target occupancy grid prediction network used in the reasoning scenario removes the position coding unit, and directly uses the multi-view position coding feature samples obtained in the last iteration of the training process for fusion during reasoning, which can speed up the reasoning speed.

[0118] In one embodiment, the target occupancy grid prediction network includes: an upsampling unit and a convolution unit; inputting the three-dimensional perception visual feature into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view feature, including: inputting the three-dimensional perception visual feature into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain an initial bird's-eye view feature; inputting the initial bird's-eye view feature into the upsampling unit and the convolution unit to obtain a bird's-eye view feature.

[0119] After the three-dimensional perception visual features are input into the multi-layer perceptron perspective transformation unit, they are processed by reshape and pemute to obtain downsampled bird's-eye view features, which can be called initial bird's-eye view features. The initial bird's-eye view features are processed by upsampling units and convolution units, and the result obtained is called bird's-eye view features.

[0120] In order to better understand the above method, an application example of the vehicle environment perception method of the present application is described in detail below. In this application example, the image feature extraction unit may include an image encoder backbone unit and a 1×1 convolution unit. The position encoding unit includes a 1×1 convolution unit, a ReLU activation function, and a 1×1 convolution unit connected in sequence.

[0121] The size of the multi-view image can be recorded as N c ×H1×W1×3; where N c Matching the number of panoramic cameras, H1 represents the length of the image, W1 represents the width of the image, and 3 represents the RGB colors of the image.

[0122] Reference Figure 4 After the multi-view image is input into the image encoder backbone unit, a size of N can be obtained. c ×(H1 / 16)×(W1 / 16)×C1, where C1 represents the number of channels. Inputting the initial multi-view image features into a 1×1 convolution unit, we can obtain a size of N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image features.

[0123] Get the multi-view position coding feature sample obtained in the last iteration of the training process. The size of the multi-view position coding feature sample is N c ×(H1 / 16)×(W1 / 16)×C1.

[0124] Set the size to N c ×(H1 / 16)×(W1 / 16)×C1 multi-view image features and multi-view position coding feature samples are input into the fusion unit to obtain a three-dimensional perception visual feature. The size of the three-dimensional perception visual feature is N c ×(H1 / 16)×(W1 / 16)×C1.

[0125] For a size of N c ×(H1 / 16)×(W1 / 16)×C1 are reshaped and pemute (rearranged) to obtain the flattened three-dimensional perception visual features. The size of the three-dimensional perception visual features is increased from N c ×(H1 / 16)×(W1 / 16)×C1 becomes C1×[N c×(H1 / 16)×(W1 / 16)].

[0126] After the flattened 3D perception visual features are input into the multi-layer perceptron perspective transformation unit, the initial bird's-eye view features can be obtained after reshape and pemute processing. The size of the initial bird's-eye view features is (H1 / scale)×(W1 / scale)×C b , C b Indicates the number of channels, C b The size of is different from that of C1.

[0127] The initial bird's-eye view feature is processed by the upsampling unit and the convolution unit to obtain the bird's-eye view feature. The size of the bird's-eye view feature is H b ×W b ×C b , H b Represents the length of three-dimensional space, W b Indicates the width of three-dimensional space.

[0128] Next, the bird's-eye view feature can be input into the occupancy grid prediction unit to obtain the initial occupancy grid prediction result. The size of the initial occupancy grid prediction result is C b Z×H b ×W b , Z is the height of the three-dimensional space. The initial occupancy grid prediction result is stretched to obtain an occupancy grid prediction result with a three-dimensional length, width and height. The size of the occupancy grid prediction result is C b ×H b ×W b ×Z.

[0129] The environmental perception results of the target vehicle can be obtained based on the occupancy grid prediction results.

[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0131] Based on the same inventive concept, the embodiment of the present application also provides a training device for an occupancy grid prediction network for implementing the training method for the occupancy grid prediction network involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the training device for one or more occupancy grid prediction networks provided below can refer to the limitations of the training method for the occupancy grid prediction network above, and will not be repeated here.

[0132] In one embodiment, Figure 5 As shown, a training device for an occupancy grid prediction network is provided, wherein the occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit and an occupancy grid prediction unit, and the device includes:

[0133] A first feature extraction module 501 is used to input a multi-view image sample of a sample vehicle into the image feature extraction unit to obtain a multi-view image feature sample;

[0134] A first position coding module 502, configured to input the multi-view image feature samples into the position coding unit to obtain multi-view position coding feature samples;

[0135] A first fusion module 503, configured to input the multi-view image feature samples and the multi-view position coding feature samples into the fusion unit to obtain a three-dimensional perception visual feature sample;

[0136] A first perspective transformation module 504, configured to input the three-dimensional perception visual feature sample into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample;

[0137] A first occupancy grid prediction module 505, configured to input the bird's-eye view feature sample into the occupancy grid prediction unit to obtain an occupancy grid prediction result sample;

[0138] The first parameter adjustment module 506 is used to adjust the parameters of each unit in the occupancy grid prediction network according to the difference between the occupancy grid prediction result sample and the occupancy grid label result to complete the training.

[0139] In one embodiment, the first position encoding module 502 is also used to: input the multi-view image feature samples into the position encoding unit, so that the position encoding unit performs the following steps: expand the depth dimension of each image feature coordinate in the multi-view image feature samples to obtain multi-view cone point cloud coordinate samples, and obtain multi-view position encoding feature samples based on the multi-view cone point cloud coordinate samples.

[0140] In one embodiment, the first position encoding module 502 is also used to: convert the multi-perspective view cone point cloud coordinate samples from the image coordinate system to the vehicle coordinate system of the sample vehicle; align the channels of the multi-perspective view cone point cloud coordinate samples located in the vehicle coordinate system to the channels of the multi-perspective image feature samples to obtain multi-perspective position encoding feature samples.

[0141] In one embodiment, the occupancy grid prediction network also includes an upsampling unit and a convolution unit; the first perspective transformation module 504 is also used to: input the three-dimensional perception visual feature sample into the multi-layer perceptron perspective transformation unit to obtain an initial bird's-eye view feature sample; input the initial bird's-eye view feature sample into the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample.

[0142] Each module in the training device of the occupancy grid prediction network can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0143] Based on the same inventive concept, the embodiment of the present application also provides a vehicle environment perception device for implementing the vehicle environment perception method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle environment perception device embodiments provided below can refer to the limitations of the vehicle environment perception method above, and will not be repeated here.

[0144] In one embodiment, Figure 6 As shown, a vehicle environment perception device is provided, comprising:

[0145] The network acquisition module 601 is used to acquire a target occupancy grid prediction network; the target occupancy grid prediction network is obtained by removing the position encoding unit after the occupancy grid prediction network is trained according to the above occupancy grid prediction network training method;

[0146] A second feature extraction module 602 is used to input the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features;

[0147] The position coding acquisition module 603 is used to acquire the multi-view position coding feature samples obtained in the last iteration during the training process;

[0148] A second fusion module 604 is used to input the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain a three-dimensional perception visual feature;

[0149] A second perspective transformation module 605 is used to input the three-dimensional perception visual feature into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature;

[0150] A second occupancy grid prediction module 606, configured to input the bird's-eye view feature into an occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result;

[0151] The perception result acquisition module 607 is used to obtain the environment perception result of the target vehicle according to the occupancy grid prediction result.

[0152] In one embodiment, the target occupancy grid prediction network includes: an upsampling unit and a convolution unit; the second perspective transformation module 605 is also used to: input the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain initial bird's-eye view features; input the initial bird's-eye view features into the upsampling unit and the convolution unit to obtain bird's-eye view features.

[0153] Each module in the above vehicle environment perception device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0154] In an exemplary embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the above method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for an occupancy grid prediction network or a vehicle environment perception method is implemented.

[0155] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0158] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0160] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0161] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0162] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A training method for an occupancy grid prediction network, characterized in that: The occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit and an occupancy grid prediction unit; the method includes: Inputting the multi-view image samples of the sample vehicle into the image feature extraction unit to obtain multi-view image feature samples; The multi-view image feature sample is input into the position encoding unit, so that the position encoding unit performs the following steps: expanding the depth dimension of each image feature coordinate in the multi-view image feature sample to obtain a multi-view view cone point cloud coordinate sample, converting the multi-view view cone point cloud coordinate sample from the image coordinate system to the vehicle coordinate system of the sample vehicle; aligning the channel of the multi-view view cone point cloud coordinate sample in the vehicle coordinate system to the channel of the multi-view image feature sample to obtain a multi-view position encoding feature sample; Inputting the multi-view image feature samples and the multi-view position coding feature samples into the fusion unit to obtain three-dimensional perception visual feature samples; Inputting the three-dimensional perception visual feature samples into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample; Inputting the bird's-eye view feature sample into the occupancy grid prediction unit to obtain an occupancy grid prediction result sample; According to the difference between the occupancy grid prediction result sample and the occupancy grid label result, the parameters of each unit in the occupancy grid prediction network are adjusted to complete the training.

2. The method according to claim 1, characterized in that The occupancy grid prediction network also includes an upsampling unit and a convolution unit; The three-dimensional perception visual feature samples are input into the multi-layer perceptron perspective transformation unit to obtain bird's-eye view feature samples, including: Inputting the three-dimensional perception visual feature samples into a multi-layer perceptron perspective transformation unit to obtain an initial bird's-eye view feature sample; The initial bird's-eye view feature sample is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample.

3. A vehicle environment perception method, characterized in that: The method comprises: Obtain a target occupancy grid prediction network; the target occupancy grid prediction network is: an occupancy grid prediction network obtained by removing a position encoding unit after training according to the training method of claim 1 or 2; Inputting the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features; Get the multi-view position encoding feature samples obtained in the last iteration of the training process; Inputting the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain a three-dimensional perception visual feature; Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature; Inputting the bird's-eye view feature into an occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result; According to the occupancy grid prediction result, the environment perception result of the target vehicle is obtained.

4. The method according to claim 3, characterized in that The target occupancy grid prediction network includes: an upsampling unit and a convolution unit; Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature includes: Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain an initial bird's-eye view perspective feature; The initial bird's-eye view feature is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature.

5. A training device for an occupancy grid prediction network, characterized in that: The occupancy grid prediction network includes an image feature extraction unit, a position encoding unit, a fusion unit, a multi-layer perceptron perspective transformation unit and an occupancy grid prediction unit, and the device includes: A first feature extraction module, used for inputting a multi-view image sample of a sample vehicle into the image feature extraction unit to obtain a multi-view image feature sample; The first position encoding module is used to input the multi-view image feature sample into the position encoding unit, so that the position encoding unit performs the following steps: expanding the depth dimension of each image feature coordinate in the multi-view image feature sample to obtain a multi-view view cone point cloud coordinate sample, converting the multi-view view cone point cloud coordinate sample from the image coordinate system to the vehicle coordinate system of the sample vehicle; aligning the channel of the multi-view view cone point cloud coordinate sample in the vehicle coordinate system to the channel of the multi-view image feature sample to obtain a multi-view position encoding feature sample; A first fusion module, used for inputting the multi-view image feature samples and the multi-view position coding feature samples into the fusion unit to obtain a three-dimensional perception visual feature sample; A first perspective transformation module, used for inputting the three-dimensional perception visual feature sample into a multi-layer perceptron perspective transformation unit to obtain a bird's-eye view feature sample; A first occupancy grid prediction module, used for inputting the bird's-eye view feature sample into the occupancy grid prediction unit to obtain an occupancy grid prediction result sample; The first parameter adjustment module is used to adjust the parameters of each unit in the occupied grid prediction network according to the difference between the occupied grid prediction result sample and the occupied grid label result to complete the training.

6. The device according to claim 5, characterized in that The occupancy grid prediction network also includes an upsampling unit and a convolution unit; The first perspective transformation module is also used for: Inputting the three-dimensional perception visual feature samples into a multi-layer perceptron perspective transformation unit to obtain an initial bird's-eye view feature sample; The initial bird's-eye view feature sample is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature sample.

7. A vehicle environment perception device, characterized in that: The device comprises: A network acquisition module is used to acquire a target occupancy grid prediction network; the target occupancy grid prediction network is: an occupancy grid prediction network is trained according to the training method of claim 1 or 2, and the position encoding unit is removed; A second feature extraction module is used to input the multi-view image of the target vehicle into the image feature extraction unit of the target occupancy grid prediction network to obtain multi-view image features; A position coding acquisition module is used to obtain multi-view position coding feature samples obtained in the last iteration during the training process; A second fusion module is used to input the multi-view image features and the multi-view position coding feature samples into a fusion unit of the target occupancy grid prediction network to obtain a three-dimensional perception visual feature; A second perspective transformation module is used to input the three-dimensional perception visual feature into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain a bird's-eye view perspective feature; A second occupancy grid prediction module, used for inputting the bird's-eye view feature into an occupancy grid prediction unit of the target occupancy grid prediction network to obtain an occupancy grid prediction result; The perception result acquisition module is used to obtain the environmental perception result of the target vehicle according to the occupancy grid prediction result.

8. The device according to claim 7, characterized in that The target occupancy grid prediction network includes: an upsampling unit and a convolution unit; The second perspective transformation module is also used for: Inputting the three-dimensional perception visual features into the multi-layer perceptron perspective transformation unit of the target occupancy grid prediction network to obtain an initial bird's-eye view perspective feature; The initial bird's-eye view feature is input into the upsampling unit and the convolution unit to obtain a bird's-eye view feature.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images