Image processing method, electronic device, storage medium and program product

Abstract

The embodiment of the invention provides an image processing method, electronic equipment, a storage medium and a program product. The method comprises the steps that multiple view angle images of a target vehicle are acquired, a current feature map is determined according to the multiple view angle images, and the current feature map comprises multiple sampling points; for any sampling point, the processing logic of each layer of image processing corresponding to the sampling point is determined, and the processing logic comprises at least one of the following items: a time self-attention mechanism, a space cross attention mechanism and a feedforward neural network; performing data processing on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain image feature values corresponding to the sampling points; and determining a feature vector group of the current aerial view of the target vehicle according to the plurality of image feature values corresponding to the plurality of sampling points. And the safety of automatic driving of the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, electronic equipment, storage medium and program product. Background Art

[0002] The vehicle can achieve autonomous driving. During the autonomous driving process, the autonomous driving system needs to respond in a timely manner to ensure the safety of autonomous driving.

[0003] In the prior art, multiple perspective images can be acquired, the current feature maps corresponding to the multiple perspective images can be determined, and the current feature maps can be processed to obtain a bird's-eye view of the vehicle at the current moment. However, the current feature map may include multiple sampling points, and each sampling point of the current feature map needs to be processed by multiple layers of image processing, and each layer of image processing also needs to perform multiple time and space attention calculations, which makes the efficiency of determining the bird's-eye view low, resulting in low safety of vehicle autonomous driving. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, an electronic device, a storage medium and a program product to solve the technical problem of low safety of vehicle autonomous driving.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire multiple perspective images of the target vehicle, and determine a current feature map based on the multiple perspective images, wherein the current feature map includes multiple sampling points;

[0007] For any sampling point, determine the processing logic of each layer of image processing corresponding to the sampling point, wherein the processing logic includes at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network;

[0008] According to the processing logic of each layer of image processing and the current feature map, data processing is performed on the sampling point to obtain an image feature value corresponding to the sampling point;

[0009] A feature vector group of the current bird's-eye view of the target vehicle is determined according to the multiple image feature values ​​corresponding to the multiple sampling points.

[0010] In a possible implementation, determining the processing logic of each layer of image processing corresponding to the sampling point includes:

[0011] Determining the processing logic of the first layer of image processing corresponding to the sampling point includes: a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network;

[0012] For the i-th layer of image processing corresponding to the sampling point, the processing logic of the i-th layer of image processing is determined according to the processing information of the i-1-th layer of image processing, wherein i is 2, 3, ..., N in sequence, N is a positive integer greater than 1, and N is the number of layers of image processing corresponding to the sampling point.

[0013] In a possible implementation, determining the processing logic of the i-th layer image processing according to the processing information of the i-1-th layer image processing includes:

[0014] Determine whether the processing information of the i-1th layer image processing includes at least one of the following: a first similarity and a second similarity, the first similarity being the similarity between the input feature value and the output feature value of the temporal self-attention mechanism, and the second similarity being the similarity between the input feature value and the output feature value of the spatial cross-attention mechanism;

[0015] If yes, determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing;

[0016] If not, it is determined that the processing logic of the i-th layer image processing includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0017] In a possible implementation, determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing includes:

[0018] If the processing information includes a first similarity, determining the processing logic of the i-th layer image processing according to the first similarity and a first threshold;

[0019] If the processing information includes a second similarity, determining the processing logic of the i-th layer image processing according to the second similarity and a second threshold;

[0020] If the processing information includes a first similarity and a second similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the second similarity.

[0021] In a possible implementation, determining the processing logic of the i-th layer image processing according to the first similarity and the second similarity includes:

[0022] If the first similarity is greater than the first threshold, and the second similarity is greater than the second threshold, determining that the processing logic includes the feedforward neural network;

[0023] If the first similarity is greater than the first threshold and the second similarity is less than or equal to the second threshold, determining that the processing logic includes the spatial cross-attention mechanism and the feedforward neural network;

[0024] If the first similarity is less than or equal to the first threshold and the second similarity is greater than the second threshold, determining that the processing logic includes the temporal self-attention mechanism and the feedforward neural network;

[0025] If the first similarity is less than or equal to the first threshold and the second similarity is less than or equal to the second threshold, it is determined that the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0026] In a possible implementation, data processing is performed on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain image feature values ​​corresponding to the sampling points, including:

[0027] Performing data processing on the sampling point through the processing logic of the first layer of image processing to obtain a first intermediate eigenvalue of the sampling point;

[0028] By means of the processing logic of the i-th layer image processing, data processing is performed on the i-1th intermediate eigenvalue according to the current feature map to obtain the i-th intermediate eigenvalue of the sampling point;

[0029] The value of i is 2, 3, ..., N, until i is N, and the Nth intermediate eigenvalue is determined as the image eigenvalue.

[0030] In a possible implementation, for the processing logic of any layer of images; the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network, wherein,

[0031] Performing data processing on the i-1th intermediate eigenvalue through the temporal self-attention mechanism of the i-th layer image processing to obtain the i-th temporal eigenvalue;

[0032] Performing data processing on the i-th temporal feature value through the spatial cross attention mechanism of the i-th layer of image processing to obtain the i-th spatial feature value;

[0033] The i-th spatial eigenvalue is processed by the feedforward neural network of the i-th layer of image processing to obtain the i-th intermediate eigenvalue.

[0034] In a possible implementation, data processing is performed on the i-1th intermediate eigenvalue through the temporal self-attention mechanism to obtain the i-th temporal eigenvalue, including:

[0035] Determine a plurality of dynamic sampling points corresponding to the (i-1)th intermediate eigenvalue, and a temporal attention weight of each dynamic sampling point;

[0036] Determine the dynamic sampling point whose temporal attention weight is greater than the first weight threshold as a target sampling point, so as to determine at least one target sampling point;

[0037] The i-th time feature value is determined according to the at least one target sampling point and the time attention weight of each target sampling point.

[0038] In a possible implementation, it further includes:

[0039] Determine the similarity between the i-1th intermediate feature value and the i-th time feature value as a first similarity corresponding to the temporal self-attention mechanism of the i-th layer image processing;

[0040] The similarity between the i-th temporal feature value and the i-th spatial feature value is determined as a second similarity corresponding to the spatial cross-attention mechanism of the i-th layer image processing.

[0041] In a possible implementation manner, before obtaining the target feature vector corresponding to the current feature map, the method further includes:

[0042] Obtaining a preset transfer matrix, and determining a transposed matrix of the preset transfer matrix;

[0043] The product of the preset transfer matrix and the transposed matrix is ​​determined as the target feature vector.

[0044] In a second aspect, the present application provides an image processing device, including an acquisition module, a first determination module, a data processing module, and a second determination module:

[0045] The acquisition module is used to acquire multiple perspective images of the target vehicle and determine a current feature map based on the multiple perspective images, wherein the current feature map includes multiple sampling points;

[0046] The first determination module is used to determine, for any sampling point, a processing logic for image processing of each layer corresponding to the sampling point, wherein the processing logic includes at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network;

[0047] The data processing module is used to perform data processing on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain image feature values ​​corresponding to the sampling points;

[0048] The second determination module is used to determine a feature vector group of the current bird's-eye view of the target vehicle according to a plurality of image feature values ​​corresponding to the plurality of sampling points.

[0049] In a possible implementation manner, the first determining module is specifically configured to:

[0050] Determining the processing logic of the first layer of image processing corresponding to the sampling point includes: a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network;

[0051] For the i-th layer of image processing corresponding to the sampling point, the processing logic of the i-th layer of image processing is determined according to the processing information of the i-1-th layer of image processing, wherein i is 2, 3, ..., N in sequence, N is a positive integer greater than 1, and N is the number of layers of image processing corresponding to the sampling point.

[0052] In a possible implementation manner, the first determining module is specifically configured to:

[0053] Determine whether the processing information of the i-1th layer image processing includes at least one of the following: a first similarity and a second similarity, the first similarity being the similarity between the input feature value and the output feature value of the temporal self-attention mechanism, and the second similarity being the similarity between the input feature value and the output feature value of the spatial cross-attention mechanism;

[0054] If yes, determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing;

[0055] If not, it is determined that the processing logic of the i-th layer image processing includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0056] In a possible implementation manner, the first determining module is specifically configured to:

[0057] If the processing information includes a first similarity, determining the processing logic of the i-th layer image processing according to the first similarity and a first threshold;

[0058] If the processing information includes a second similarity, determining the processing logic of the i-th layer image processing according to the second similarity and a second threshold;

[0059] If the processing information includes a first similarity and a second similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the second similarity.

[0060] In a possible implementation manner, the first determining module is specifically configured to:

[0061] If the first similarity is greater than the first threshold, and the second similarity is greater than the second threshold, determining that the processing logic includes the feedforward neural network;

[0062] If the first similarity is greater than the first threshold and the second similarity is less than or equal to the second threshold, determining that the processing logic includes the spatial cross-attention mechanism and the feedforward neural network;

[0063] If the first similarity is less than or equal to the first threshold and the second similarity is greater than the second threshold, determining that the processing logic includes the temporal self-attention mechanism and the feedforward neural network;

[0064] If the first similarity is less than or equal to the first threshold and the second similarity is less than or equal to the second threshold, it is determined that the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0065] In a possible implementation manner, the data processing module is specifically used to:

[0066] Performing data processing on the sampling point through the processing logic of the first layer of image processing to obtain a first intermediate eigenvalue of the sampling point;

[0067] By means of the processing logic of the i-th layer image processing, data processing is performed on the i-1th intermediate eigenvalue according to the current feature map to obtain the i-th intermediate eigenvalue of the sampling point;

[0068] The value of i is 2, 3, ..., N, until i is N, and the Nth intermediate eigenvalue is determined as the image eigenvalue.

[0069] In a possible implementation, for the processing logic of any layer of images; the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network, and the data processing module is specifically used for:

[0070] Performing data processing on the i-1th intermediate eigenvalue through the temporal self-attention mechanism of the i-th layer image processing to obtain the i-th temporal eigenvalue;

[0071] Performing data processing on the i-th temporal feature value through the spatial cross attention mechanism of the i-th layer of image processing to obtain the i-th spatial feature value;

[0072] The i-th spatial eigenvalue is processed by the feedforward neural network of the i-th layer of image processing to obtain the i-th intermediate eigenvalue.

[0073] In a possible implementation manner, the data processing module is specifically used to:

[0074] Determine a plurality of dynamic sampling points corresponding to the (i-1)th intermediate eigenvalue, and a temporal attention weight of each dynamic sampling point;

[0075] Determine the dynamic sampling point whose temporal attention weight is greater than the first weight threshold as a target sampling point, so as to determine at least one target sampling point;

[0076] The i-th time feature value is determined according to the at least one target sampling point and the time attention weight of each target sampling point.

[0077] In a possible implementation manner, the data processing module is specifically used to:

[0078] Determine the similarity between the i-1th intermediate feature value and the i-th time feature value as a first similarity corresponding to the temporal self-attention mechanism of the i-th layer image processing;

[0079] The similarity between the i-th temporal feature value and the i-th spatial feature value is determined as a second similarity corresponding to the spatial cross-attention mechanism of the i-th layer image processing.

[0080] In a possible implementation, the second acquisition module, the third determination module and the fourth determination module are further included:

[0081] The second acquisition module is used to acquire a preset transfer matrix;

[0082] The third determination module is used to determine the transposed matrix of the preset transfer matrix;

[0083] The fourth determination module is used to determine the product of the preset transfer matrix and the transposed matrix as the target feature vector.

[0084] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0085] The memory stores computer-executable instructions;

[0086] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0087] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0088] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0089] The image processing method, electronic device, storage medium and program product provided in the embodiments of the present application can reduce the processing logic in each layer of image processing corresponding to any sampling point in the current feature map after acquiring the current feature map corresponding to multiple perspective images, thereby improving the efficiency of determining the bird's-eye view and improving the safety of vehicle autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0091] Figure 1 A schematic diagram of the application scenario provided for this application;

[0092] Figure 2 A flowchart of an image processing method provided in this application;

[0093] Figure 3 A flowchart of a determination processing logic provided for this application;

[0094] Figure 4 A dynamic execution schematic diagram provided for this application;

[0095] Figure 5 A schematic diagram of an image processing process provided by this application;

[0096] Figure 6 A schematic diagram of the architecture of dynamic screening of sampling points provided in this application;

[0097] Figure 7 A schematic diagram of the structure of an image processing device provided by the present application;

[0098] Figure 8 A schematic diagram of the structure of an electronic device provided in this application.

[0099] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0100] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0101] First, the terms involved in this application are explained:

[0102] Bird's Eye View (BEV): refers to a two-dimensional image viewed from above. It can be generated by fusing and processing multiple sensor data around the vehicle (such as cameras, radars, lidars, etc.). The bird's eye view can provide a panoramic view of the vehicle's surroundings, helping the autonomous driving system to better understand and analyze the surrounding situation, thereby making safer and more accurate driving decisions.

[0103] Figure 1 A schematic diagram of the application scenario provided for this application. Figure 1 The specific application scenario of the present application is that the vehicle 101 is driving on the road. The vehicle 101 may include an automatic driving system, and the vehicle may perform automatic driving through the automatic driving system. Multiple cameras may be installed in the vehicle 101, and multiple perspective images around the vehicle 101 may be captured by the multiple cameras. The automatic driving system in the vehicle 101 may obtain multiple perspective images, determine the current bird's-eye view of the surrounding environment of the vehicle 101 at the current moment based on the multiple perspective images, and the automatic driving system may control the vehicle 101 to perform automatic driving based on the current bird's-eye view.

[0104] In the prior art, multiple perspective images can be acquired, the current feature maps corresponding to the multiple perspective images can be determined, and the current feature maps can be processed to obtain a bird's-eye view of the vehicle at the current moment. However, the current feature map may include multiple sampling points, and each sampling point of the current feature map needs to be processed by multiple layers of image processing, and each layer of image processing also needs to perform time and space attention calculations, which makes the efficiency of determining the bird's-eye view low, resulting in low safety of vehicle autonomous driving.

[0105] The image processing method provided in the embodiment of the present application can reduce the processing logic in each layer of image processing corresponding to any sampling point in the current feature map after acquiring the current feature map corresponding to multiple perspective images, thereby improving the efficiency of determining the bird's-eye view and improving the safety of vehicle autonomous driving.

[0106] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0107] Figure 2 A flowchart of an image processing method provided in this application. Figure 2 , the method may include:

[0108] S201, acquiring multiple perspective images of a target vehicle.

[0109] The executing entity of the embodiment of the present application may be an autonomous driving system, or an image processing device provided in the autonomous driving system. The image processing device may be implemented by software, or by a combination of software and hardware.

[0110] Multiple cameras can be installed on the target vehicle, and multiple perspective images can be acquired through the multiple cameras.

[0111] S202: Determine a current feature map based on multiple viewing angle images.

[0112] A preset model may be used to perform feature extraction processing on multiple perspective images to obtain a current feature map, wherein the current feature map may include features of the surrounding environment of the target vehicle at the current moment, and the current feature map may include multiple sampling points.

[0113] S203: For any sampling point, determine the processing logic of each layer of image processing corresponding to the sampling point.

[0114] Multi-layer image processing may be performed on each sampling point in the current feature map to obtain a feature vector group of the current bird's-eye view of the target vehicle at the current moment.

[0115] For any layer of image processing, the processing logic of the image processing may include at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network.

[0116] For any sampling point, the processing logic of the first layer of image processing corresponding to the sampling point may include a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network.

[0117] For the i-th layer of image processing corresponding to the sampling point, the processing logic of the i-th layer of image processing is determined according to the processing information of the i-1-th layer of image processing, where i is 2, 3, ..., N in sequence, N is a positive integer greater than 1, and N is the number of layers of image processing corresponding to the sampling point.

[0118] That is, in the processing logic of image processing after the first layer of image processing, the temporal self-attention mechanism and / or the spatial cross-attention mechanism can be skipped according to the processing information of the previous image processing, which can improve the efficiency of determining the bird's-eye view and thereby improve the safety of vehicle autonomous driving.

[0119] The processing logic of the i-th layer image processing can correspond to the following four cases: Case 1: The processing logic may include a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network. Case 2: The processing logic may include a temporal self-attention mechanism and a feedforward neural network. Case 3: The processing logic may include a spatial cross-attention mechanism and a feedforward neural network. Case 4: The processing logic may include a feedforward neural network.

[0120] The processing information of the i-1th layer of image processing may include a first similarity and a second similarity, wherein the first similarity may be the similarity between the input feature value and the output feature value of the temporal self-attention mechanism, and the second similarity may be the similarity between the input feature value and the output feature value of the spatial cross-attention mechanism.

[0121] S204: Perform data processing on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain image feature values ​​corresponding to the sampling points.

[0122] The sampling point can be processed by the processing logic of the first layer of image processing to obtain the first intermediate eigenvalue of the sampling point; and the i-1th intermediate eigenvalue can be processed by the processing logic of the i-th layer of image processing according to the current feature map to obtain the i-th intermediate eigenvalue of the sampling point.

[0123] Among them, i takes values ​​of 2, 3, ..., N, until i reaches N, and the Nth intermediate eigenvalue is determined as the image eigenvalue.

[0124] S205 . Determine a feature vector group of the current bird's-eye view of the target vehicle according to a plurality of image feature values ​​corresponding to a plurality of sampling points.

[0125] The feature position corresponding to each image feature value is determined, and multiple image feature values ​​are spliced ​​according to the feature position corresponding to each image feature value, so as to obtain a feature vector group of the current bird's-eye view.

[0126] The image processing method provided in the embodiment of the present application can determine the current feature map corresponding to multiple perspective images of the target vehicle. For any sampling point in the current feature map, the processing logic of each layer of image processing corresponding to the sampling point can be determined. The processing logic can include at least one feedforward neural network. According to the processing logic of each layer of image processing and the current feature map, the sampling point can be processed to obtain the image feature value corresponding to the sampling point. According to the multiple image feature values ​​corresponding to the multiple sampling points, the feature vector group of the current bird's-eye view of the target vehicle can be determined. In the above process, the processing logic in each layer of image processing corresponding to the sampling point can be reduced, the efficiency of determining the bird's-eye view can be improved, and the safety of the vehicle's automatic driving can be improved.

[0127] Based on the above embodiments, Figure 4 , the execution process of the processing logic for determining the i-th layer image processing of the sampling point provided in the embodiment of the present application is described, where i can be 2, 3, ..., N.

[0128] Figure 3 A flowchart of determining the processing logic provided for this application. Please refer to Figure 3 , the method may include:

[0129] S301. Determine whether processing information of the i-1th layer image processing includes at least one of the following: a first similarity and a second similarity.

[0130] The processing logic of the i-1 layer image processing includes the i-1 layer temporal self-attention mechanism, and the processing information of the i-1 layer image processing includes the first similarity.

[0131] If the processing logic of the i-1th layer of image processing includes the i-1th spatial cross-attention mechanism, then the processing information of the i-1th layer of image processing includes the second similarity.

[0132] S302: If yes, determine the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing.

[0133] Processing information may include the following three situations:

[0134] Case 1: If the processing information includes the first similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the first threshold.

[0135] In the i-1 layer image processing, if the input eigenvalue and the output eigenvalue of the temporal self-attention mechanism have a large similarity, the processing of the temporal self-attention mechanism has a small impact on the eigenvalue, that is, there is less time information in the sampling point. Then, when the first similarity of the i-1 layer is greater than the first threshold, the temporal self-attention mechanism can be skipped in the i-layer image processing.

[0136] If the first similarity is included in the processing information, it can be determined that the processing logic of the i-th layer image processing includes a spatial cross-attention mechanism and a feedforward neural network.

[0137] If the first similarity is greater than the first threshold, the temporal self-attention mechanism can be skipped, and then it is determined that the processing logic of the i-th layer image processing includes the spatial cross-attention mechanism and the feedforward neural network; if the first similarity is less than or equal to the first threshold, the temporal self-attention mechanism cannot be skipped, and the processing logic of the i-th layer image processing includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0138] Case 2: If the processing information includes the second similarity, the processing logic of the i-th layer image processing is determined according to the second similarity and the second threshold.

[0139] In the i-1 layer image processing, if the input eigenvalue and the output eigenvalue of the spatial cross-attention mechanism have a large similarity, the processing of the spatial cross-attention mechanism has a small impact on the eigenvalue, that is, there is less spatial information in the sampling point. Then, when the second similarity of the i-1 layer is greater than the first threshold, the temporal self-attention mechanism can be skipped in the i-layer image processing.

[0140] If the second similarity is included in the processing information, it can be determined that the processing logic of the i-th layer image processing includes a temporal self-attention mechanism and a feedforward neural network.

[0141] If the second similarity is greater than the second threshold, the spatial cross-attention mechanism can be skipped, and then it is determined that the processing logic of the i-th layer image processing includes the temporal self-attention mechanism and the feedforward neural network; if the first similarity is less than or equal to the first threshold, the spatial cross-attention mechanism cannot be skipped, and the processing logic of the i-th layer image processing includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

[0142] Case 3: If the processing information includes the first similarity and the second similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the second similarity.

[0143] If the processing information includes the first similarity and the second similarity, it can be determined that the processing logic of the i-th layer image processing includes a feedforward neural network.

[0144] If the first similarity is greater than a first threshold and the second similarity is greater than a second threshold, the i-th layer image processing skips the temporal self-attention mechanism and the spatial cross-attention mechanism, and the processing logic includes a feedforward neural network.

[0145] If the first similarity is greater than the first threshold and the second similarity is less than or equal to the second threshold, the i-th layer image processing skips the temporal self-attention mechanism, and the processing logic includes the spatial cross-attention mechanism and the feedforward neural network.

[0146] If the first similarity is less than or equal to the first threshold and the second similarity is greater than the second threshold, the i-th layer image processing skips the spatial cross-attention mechanism, and the processing logic includes the temporal self-attention mechanism and the feedforward neural network.

[0147] If the first similarity is less than or equal to the first threshold and the second similarity is less than or equal to the second threshold, the processing logic includes a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network.

[0148] S303: If not, determine that the processing logic of the i-th layer image processing includes a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network.

[0149] If the processing information of the i-1 layer image processing does not include the first similarity and the second similarity, the temporal self-attention mechanism and the spatial cross-attention mechanism are not performed in the i-1 layer image processing, and the temporal self-attention mechanism and the spatial cross-attention mechanism are required in the i layer image processing.

[0150] The image processing method provided in the embodiment of the present application can determine whether to skip the temporal self-attention mechanism and spatial cross-attention mechanism of the current image processing based on the processing information of the previous layer of image processing, which can reduce the processing time of the sampling points and improve the efficiency of determining the bird's-eye view.

[0151] Figure 4 A dynamic execution diagram provided for this application. Figure 4 In the example, for any sampling point of the current feature map, three layers of image processing can be performed on the sampling point, namely the first layer image processing, the second layer image processing and the third layer image processing.

[0152] The processing logic of the first layer of image processing may include a first temporal self-attention mechanism (Temporal Self-Attention, TSA), a second spatial cross-attention mechanism (Spatial Cross-Attention, SCA) and a third feed-forward neural network (Feed-Forward Network, FFN).

[0153] The processing logic of the second layer image processing may include a second SCA and a second FFN. If the first similarity between the input and output of the first TSA is greater than a first threshold, the second TSA of the second layer image processing is skipped.

[0154] The processing logic of the third-layer image processing may include a third TSA and a third FFN. If the first similarity corresponding to the second TSA does not exist in the second-layer image processing, the processing logic of the third-layer image processing includes the third TSA; if the second similarity between the input and output of the second SCA in the second-layer image processing is greater than the second threshold, the second SCA of the third-layer image processing is skipped.

[0155] Based on the above embodiments, Figure 5 , the embodiment of the present application provides a further explanation of the execution process of performing data processing on the i-1th intermediate eigenvalue to obtain the i-th intermediate eigenvalue when the processing logic of the i-th layer image processing includes a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network, wherein i can be 2, 3, ..., N.

[0156] Figure 5 A schematic diagram of an image processing process provided for this application. Figure 5 , the method may include:

[0157] S501. Perform data processing on the i-1th intermediate eigenvalue through the temporal self-attention mechanism of the i-th layer image processing to obtain the i-th temporal eigenvalue.

[0158] The target attention operator in the temporal self-attention mechanism and the spatial cross-attention mechanism can be determined by the deformable attention operator (Deformable Attention) and bilinear interpolation. The target attention operator can be shown as follows:

[0159]

[0160] Among them, x i,j,k is the kth dynamic sampling point of the jth sampling point in the i-th layer, t i,j,k is the weight of the dynamic sampling point, A i,j is the attention weight of the jth sampling point in the i-th layer, W i is the preset transfer matrix of the current feature image, W i ′ is the transposed matrix of the preset transfer matrix.

[0161] For any sampling point, each time the attention mechanism is calculated, W i ·W i ′By performing calculations, a preset transfer matrix of the current feature map can be determined before image processing is performed on the current feature map, and a transposed matrix of the preset transfer matrix can be determined; the product of the preset transfer matrix and the transposed matrix can be determined as the target feature vector.

[0162] The target feature vector can be stored and directly obtained during the attention mechanism calculation, which can convert part of the computational overhead into memory access overhead, reduce the computational overhead, and improve the efficiency of image processing.

[0163] Furthermore, due to the attention weight A i,j = 0, then in the attention calculation, the corresponding calculation content (t i,j,k ·A i,j )·(W i ·W i ′ ·x i,j,k ) can be omitted. Therefore, if A i,j Approaching 0, sampling point x i,j,k The corresponding calculation results (t i,j,k ·A i,j )·(W i ·W i ′ ·x i,j,k ) has little influence on feature extraction, that is, it can be ignored.

[0164] Therefore, the embodiment of the present application configures a first weight threshold corresponding to the temporal self-attention mechanism and a second weight threshold corresponding to the spatial cross-attention mechanism.

[0165] Multiple dynamic sampling points corresponding to the i-1th intermediate eigenvalue and the time attention weight of each dynamic sampling point can be determined; the dynamic sampling points whose time attention weights are greater than the first weight threshold are determined as target sampling points to determine at least one target sampling point; the i-th time eigenvalue is determined based on the at least one target sampling point and the time attention weight of each target sampling point.

[0166] A plurality of dynamic sampling points can be screened by a first weight threshold, and dynamic sampling points having a smaller impact on the calculation of the temporal self-attention mechanism can be filtered out, thereby reducing the amount of calculation and improving the efficiency of image processing.

[0167] S502. Perform data processing on the i-th temporal eigenvalue through the spatial cross-attention mechanism of the i-th layer of image processing to obtain the i-th spatial eigenvalue.

[0168] The specific execution process of S502 can refer to the execution process of S501, which will not be repeated here.

[0169] S503 , performing data processing on the i th spatial eigenvalue through the i th layer of image processing feedforward neural network to obtain the i th intermediate eigenvalue.

[0170] Feedforward neural networks can be used for feature extraction and information processing.

[0171] The image processing method provided in the embodiment of the present application can screen multiple dynamic sampling points through attention weights and weight thresholds during the processing of the temporal self-attention mechanism and the spatial cross-attention mechanism, and determine at least one target sampling point, which can reduce the processing process of the attention mechanism, improve the efficiency of image processing, and thereby improve the safety of autonomous driving.

[0172] Figure 6 A schematic diagram of the architecture of dynamic screening of sampling points provided for this application. Figure 6 , the five dynamic sampling points that can be sampled are dynamic sampling point 1, dynamic sampling point 2, dynamic sampling point 3, dynamic sampling point 4 and dynamic sampling point 5, among which the time attention weight of dynamic sampling point 1 is 0.97, the time attention weight of dynamic sampling point 2 is 0.01, the time attention weight of dynamic sampling point 3 is 0.74, the time attention weight of dynamic sampling point 4 is 0.03, and the time attention weight of dynamic sampling point 5 is 0.85. The first weight threshold of the time attention mechanism is 0.1, and dynamic sampling point 1, dynamic sampling point 3, and dynamic sampling point 5 are determined as target sampling points. Image processing is performed on dynamic sampling point 1, dynamic sampling point 3, and dynamic sampling point 5 to obtain image feature values.

[0173] It is worth noting that in the embodiment of the present application, the temporal self-attention mechanism and the spatial cross-attention mechanism of all layers of image processing after the first layer of image processing can be skipped through the first similarity and the second similarity in the first layer of image processing. The specific execution process can be referred to the execution process in the above embodiment, which will not be repeated here.

[0174] For example, assuming that there are three layers of image processing, namely the first layer of image processing, the second layer of image processing and the third layer of image processing, and assuming that the first similarity corresponding to the temporal self-attention mechanism in the first layer of image processing is greater than the first threshold, and the spatial cross-attention mechanism is less than the second threshold, the temporal self-attention mechanism of the second layer of image processing and the third layer of image processing can be skipped, and the processing logic of the second layer of image processing includes the spatial cross-attention mechanism and the feedforward neural network.

[0175] Figure 7 This is a schematic diagram of the structure of an image processing device provided by this application. Figure 7 , the image processing device 10 may include an acquisition module 11, a first determination module 12, a data processing module 13 and a second determination module 14:

[0176] The acquisition module 11 is used to acquire multiple viewing angle images of the target vehicle and determine a current feature map based on the multiple viewing angle images, wherein the current feature map includes multiple sampling points;

[0177] The first determination module 12 is used to determine, for any sampling point, a processing logic for image processing of each layer corresponding to the sampling point, the processing logic including at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network;

[0178] The data processing module 13 is used to perform data processing on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain the image feature value corresponding to the sampling point;

[0179] The second determination module 14 is used to determine a feature vector group of the current bird's-eye view of the target vehicle according to a plurality of image feature values ​​corresponding to a plurality of sampling points.

[0180] The image processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0181] In a possible implementation manner, the first determining module 12 is specifically configured to:

[0182] The processing logic of the first layer of image processing corresponding to the sampling point is determined, including: temporal self-attention mechanism, spatial cross-attention mechanism and feed-forward neural network;

[0183] For the i-th layer of image processing corresponding to the sampling point, the processing logic of the i-th layer of image processing is determined according to the processing information of the i-1-th layer of image processing, where i is 2, 3, ..., N in sequence, N is a positive integer greater than 1, and N is the number of layers of image processing corresponding to the sampling point.

[0184] In a possible implementation manner, the first determining module 12 is specifically configured to:

[0185] Determine whether the processing information of the i-1th layer image processing includes at least one of the following: a first similarity and a second similarity, the first similarity being the similarity between the input feature value and the output feature value of the temporal self-attention mechanism, and the second similarity being the similarity between the input feature value and the output feature value of the spatial cross-attention mechanism;

[0186] If so, determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing;

[0187] If not, it is determined that the processing logic of the i-th layer image processing includes a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network.

[0188] In a possible implementation manner, the first determining module 12 is specifically configured to:

[0189] If the processing information includes a first similarity, determining a processing logic for image processing of the i-th layer according to the first similarity and a first threshold;

[0190] If the processing information includes a second similarity, determining a processing logic for image processing of the i-th layer according to the second similarity and a second threshold;

[0191] If the processing information includes the first similarity and the second similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the second similarity.

[0192] In a possible implementation manner, the first determining module 12 is specifically configured to:

[0193] If the first similarity is greater than a first threshold and the second similarity is greater than a second threshold, determining that the processing logic includes a feedforward neural network;

[0194] If the first similarity is greater than a first threshold and the second similarity is less than or equal to a second threshold, determining that the processing logic includes a spatial cross-attention mechanism and a feed-forward neural network;

[0195] If the first similarity is less than or equal to a first threshold and the second similarity is greater than a second threshold, determining that the processing logic includes a temporal self-attention mechanism and a feedforward neural network;

[0196] If the first similarity is less than or equal to the first threshold and the second similarity is less than or equal to the second threshold, it is determined that the processing logic includes a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network.

[0197] In a possible implementation manner, the data processing module 13 is specifically used to:

[0198] Through the processing logic of the first layer of image processing, the sampling point is processed to obtain the first intermediate eigenvalue of the sampling point;

[0199] Through the processing logic of the i-th layer image processing, according to the current feature map, the i-1th intermediate eigenvalue is processed to obtain the i-th intermediate eigenvalue of the sampling point;

[0200] Among them, i takes values ​​of 2, 3, ..., N, until i reaches N, and the Nth intermediate eigenvalue is determined as the image eigenvalue.

[0201] In a possible implementation, for the processing logic of any layer of images; the processing logic includes a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network, and the data processing module 13 is specifically used to:

[0202] Through the temporal self-attention mechanism of the i-th layer image processing, the i-1th intermediate eigenvalue is processed to obtain the i-th temporal eigenvalue;

[0203] The i-th temporal feature value is processed through the spatial cross-attention mechanism of the i-th layer image processing to obtain the i-th spatial feature value;

[0204] The i-th spatial eigenvalue is processed by the i-th layer of image processing feedforward neural network to obtain the i-th intermediate eigenvalue.

[0205] In a possible implementation manner, the data processing module 13 is specifically used to:

[0206] Determine multiple dynamic sampling points corresponding to the i-1th intermediate eigenvalue and the time attention weight of each dynamic sampling point;

[0207] Determine a dynamic sampling point whose temporal attention weight is greater than a first weight threshold as a target sampling point to determine at least one target sampling point;

[0208] An i-th temporal feature value is determined according to at least one target sampling point and a temporal attention weight of each target sampling point.

[0209] In a possible implementation manner, the data processing module 13 is specifically used to:

[0210] The similarity between the i-1th intermediate eigenvalue and the i-th temporal eigenvalue is determined as the first similarity corresponding to the temporal self-attention mechanism of the i-th layer image processing;

[0211] The similarity between the i-th temporal feature value and the i-th spatial feature value is determined as the second similarity corresponding to the spatial cross-attention mechanism of the i-th layer image processing.

[0212] In a possible implementation, the second acquisition module 14, the third determination module 15 and the fourth determination module 16 are further included:

[0213] The second acquisition module 14 is used to acquire a preset transfer matrix;

[0214] The third determination module 15 is used to determine the transposed matrix of the preset transfer matrix;

[0215] The fourth determination module 16 is used to determine the product of the preset transfer matrix and the transposed matrix as the target feature vector.

[0216] The image processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0217] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 8 The electronic device 20 may include a processor 21 and a memory 22. Exemplarily, the processor 21 and the memory 22 are interconnected via a bus 23.

[0218] The memory 22 stores computer executable instructions;

[0219] The processor 21 executes the computer-executable instructions stored in the memory 22, so that the processor 21 executes the image processing method shown in the above method embodiment.

[0220] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the image processing method of the above method embodiment.

[0221] Accordingly, an embodiment of the present application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the image processing method shown in the above method embodiment.

[0222] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0223] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0224] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0225] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0226] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0227] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0228] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0229] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0230] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire multiple perspective images of the target vehicle, and determine a current feature map based on the multiple perspective images, wherein the current feature map includes multiple sampling points; For any sampling point, determine the processing logic of each layer of image processing corresponding to the sampling point, wherein the processing logic includes at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network; According to the processing logic of each layer of image processing and the current feature map, data processing is performed on the sampling point to obtain an image feature value corresponding to the sampling point; A feature vector group of the current bird's-eye view of the target vehicle is determined according to the multiple image feature values ​​corresponding to the multiple sampling points.

2. The method according to claim 1, characterized in that Determining the processing logic of each layer of image processing corresponding to the sampling point includes: Determining the processing logic of the first layer of image processing corresponding to the sampling point includes: a temporal self-attention mechanism, a spatial cross-attention mechanism and a feedforward neural network; For the i-th layer of image processing corresponding to the sampling point, the processing logic of the i-th layer of image processing is determined according to the processing information of the i-1-th layer of image processing, wherein i is 2, 3, ..., N in sequence, N is a positive integer greater than 1, and N is the number of layers of image processing corresponding to the sampling point.

3. The method according to claim 2, characterized in that Determining the processing logic of the i-th layer image processing according to the processing information of the i-1th layer image processing includes: Determine whether the processing information of the i-1th layer image processing includes at least one of the following: a first similarity and a second similarity, the first similarity being the similarity between the input feature value and the output feature value of the temporal self-attention mechanism, and the second similarity being the similarity between the input feature value and the output feature value of the spatial cross-attention mechanism; If yes, determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing; If not, it is determined that the processing logic of the i-th layer image processing includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

4. The method according to claim 3, characterized in that Determining the processing logic of the i-th layer image processing according to the first similarity and / or the second similarity of the i-1-th layer image processing includes: If the processing information includes a first similarity, determining the processing logic of the i-th layer image processing according to the first similarity and a first threshold; If the processing information includes a second similarity, determining the processing logic of the i-th layer image processing according to the second similarity and a second threshold; If the processing information includes a first similarity and a second similarity, the processing logic of the i-th layer image processing is determined according to the first similarity and the second similarity.

5. The method according to claim 4, characterized in that Determining the processing logic of the i-th layer image processing according to the first similarity and the second similarity includes: If the first similarity is greater than the first threshold, and the second similarity is greater than the second threshold, determining that the processing logic includes the feedforward neural network; If the first similarity is greater than the first threshold and the second similarity is less than or equal to the second threshold, determining that the processing logic includes the spatial cross-attention mechanism and the feedforward neural network; If the first similarity is less than or equal to the first threshold and the second similarity is greater than the second threshold, determining that the processing logic includes the temporal self-attention mechanism and the feedforward neural network; If the first similarity is less than or equal to the first threshold and the second similarity is less than or equal to the second threshold, it is determined that the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network.

6. The method according to any one of claims 1 to 5, characterized in that: According to the processing logic of each layer of image processing and the current feature map, data processing is performed on the sampling point to obtain the image feature value corresponding to the sampling point, including: Performing data processing on the sampling point through the processing logic of the first layer of image processing to obtain a first intermediate eigenvalue of the sampling point; By means of the processing logic of the i-th layer image processing, data processing is performed on the i-1th intermediate eigenvalue according to the current feature map to obtain the i-th intermediate eigenvalue of the sampling point; The value of i is 2, 3, ..., N, until i is N, and the Nth intermediate eigenvalue is determined as the image eigenvalue.

7. The method according to claim 6, characterized in that The processing logic for any layer of images; the processing logic includes the temporal self-attention mechanism, the spatial cross-attention mechanism and the feedforward neural network, wherein, Performing data processing on the i-1th intermediate eigenvalue through the temporal self-attention mechanism of the i-th layer image processing to obtain the i-th temporal eigenvalue; Performing data processing on the i-th temporal feature value through the spatial cross attention mechanism of the i-th layer of image processing to obtain the i-th spatial feature value; The i-th spatial eigenvalue is processed by the feedforward neural network of the i-th layer of image processing to obtain the i-th intermediate eigenvalue.

8. The method according to claim 7, characterized in that Through the temporal self-attention mechanism, data processing is performed on the i-1th intermediate eigenvalue to obtain the i-th temporal eigenvalue, including: Determine a plurality of dynamic sampling points corresponding to the (i-1)th intermediate eigenvalue, and a temporal attention weight of each dynamic sampling point; Determine the dynamic sampling point whose temporal attention weight is greater than the first weight threshold as a target sampling point, so as to determine at least one target sampling point; The i-th time feature value is determined according to the at least one target sampling point and the time attention weight of each target sampling point.

9. The method according to claim 7, characterized in that: Also includes: Determine the similarity between the i-1th intermediate feature value and the i-th time feature value as a first similarity corresponding to the temporal self-attention mechanism of the i-th layer image processing; The similarity between the i-th temporal feature value and the i-th spatial feature value is determined as a second similarity corresponding to the spatial cross-attention mechanism of the i-th layer image processing.

10. The method according to any one of claims 1 to 9, characterized in that: Before obtaining the target feature vector corresponding to the current feature map, the method further includes: Obtaining a preset transfer matrix, and determining a transposed matrix of the preset transfer matrix; The product of the preset transfer matrix and the transposed matrix is ​​determined as the target feature vector.

11. An image processing device, characterized in that: It includes an acquisition module, a first determination module, a data processing module and a second determination module: The acquisition module is used to acquire multiple perspective images of the target vehicle and determine a current feature map based on the multiple perspective images, wherein the current feature map includes multiple sampling points; The first determination module is used to determine, for any sampling point, a processing logic for image processing of each layer corresponding to the sampling point, wherein the processing logic includes at least one of the following: a temporal self-attention mechanism, a spatial cross-attention mechanism, and a feedforward neural network; The data processing module is used to perform data processing on the sampling points according to the processing logic of each layer of image processing and the current feature map to obtain image feature values ​​corresponding to the sampling points; The second determination module is used to determine a feature vector group of the current bird's-eye view of the target vehicle according to a plurality of image feature values ​​corresponding to the plurality of sampling points.

12. An electronic device, characterized in that: include: Memory and processor, The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the image processing method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 10 is implemented.

Images