Radar echo pattern fitting method, system and terminal based on multimodality and attention mechanism

Abstract

The present invention discloses a radar echo map fitting method, system and terminal based on multimodality and attention mechanism, the method comprising: obtaining a scene graph and a priori echo map, supplementing the scene graph into the priori echo map, the scene graph contains key scene information that affects electromagnetic wave propagation, and has consistent format features with the priori echo map; introducing a cross-attention mechanism in a data-driven model, inputting the scene graph and the priori echo map into the data-driven model, the data-driven model extracts and fuses target key features from the scene graph and the priori echo map using the cross-attention mechanism, and outputs a fitted radar echo map. The present invention generates a scene graph based on the scene truth value, and performs feature extraction and fusion based on the attention mechanism, thereby improving the ability to extract echo map features, realizing efficient fusion of scene graph and priori echo map information, and effectively improving the prediction accuracy of radar echo maps.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a radar echo pattern fitting method, system, terminal and computer-readable storage medium based on multimodality and attention mechanism. Background Art

[0002] The existing modeling method reduces the amount of calculation at the expense of accuracy in order to achieve real-time calculation speed. This method is not suitable for high-precision simulation. Although the echo map output by the mechanism model can be used as a priori echo map and further improved through data-driven methods. However, the conventional data-driven model uses the priori echo map as input and cannot fit the relationship between the priori echo map input by the model and the real echo map output well, so it is impossible to accurately fit the real echo map.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of the present invention is to provide a radar echo pattern fitting method, system, terminal and computer-readable storage medium based on multimodality and attention mechanism, aiming to solve the problem that the real echo pattern cannot be accurately fitted in the prior art.

[0005] To achieve the above object, the present invention provides a radar echo pattern fitting method based on multimodality and attention mechanism, and the radar echo pattern fitting method based on multimodality and attention mechanism comprises the following steps:

[0006] Acquire a scene graph and a priori echo graph, and add the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph;

[0007] A cross-attention mechanism is introduced into the data-driven model, and the scene graph and the prior echo graph are input into the data-driven model. The data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo graph, and outputs a fitted radar echo graph.

[0008] Optionally, the radar echo map fitting method based on multimodality and attention mechanism, wherein the step of acquiring a scene map and a priori echo map and supplementing the scene map to the priori echo map, specifically includes:

[0009] Acquire scene information, acquire a scene truth value according to the scene information, and generate a scene graph in an echogram format according to the scene truth value;

[0010] The scene graph is modified according to the format of the priori echo graph, and the modified scene graph is added to the priori echo graph.

[0011] Optionally, in the radar echogram fitting method based on multimodality and attention mechanism, the scene graph includes a scene truth value, and the scene truth value includes the shape, electromagnetic parameters, position and speed information of the scene target;

[0012] ;

[0013] in, represents the electric field received by the radar, represents the reflection coefficient of the object surface, represents the incident electromagnetic wave, represents an imaginary number, represents the propagation constant, and They represent the propagation direction of electromagnetic waves. Axis coordinates and Axis direction coordinates, Represents the angle of incidence.

[0014] Optionally, in the radar echo map fitting method based on multimodality and attention mechanism, the step of modifying the scene graph according to the format of the prior echo map specifically includes:

[0015] Generate a scene graph that matches the two prior echo graphs, so that the two scene graphs correspond to the range-angle echo graph and the range-Doppler echo graph, respectively;

[0016] Adjusting the coordinates of the two scene graphs into the formats of the range-angle echo graph and the range-Doppler echo graph, so that the two scene graphs match the features of the prior echo graph;

[0017] The scene graph at the bird's-eye view is orthogonally expanded in the angle dimension to generate the same coordinate format and resolution as the prior echo graph.

[0018] Optionally, the radar echo map fitting method based on multimodality and attention mechanism, wherein the cross attention mechanism is introduced into the data-driven model, the scene graph and the prior echo map are input into the data-driven model, and the data-driven model uses the cross attention mechanism to extract and fuse target key features from the scene graph and the prior echo map, and outputs the fitted radar echo map, specifically comprising:

[0019] Introducing a cross-attention mechanism in the data-driven model in advance, and inputting the priori echo map that has supplemented the scene map into the data-driven model;

[0020] The data-driven model establishes connections between different input data through the cross-attention mechanism, and extracts key target features that complement each other in the priori echo graph and the scene graph;

[0021] The data-driven model fuses the target key features through the cross-attention mechanism and outputs a fitted radar echo map.

[0022] Optionally, in the radar echo image fitting method based on multimodality and attention mechanism, the cross-attention mechanism includes a spatial attention mechanism and a channel attention mechanism; the spatial attention mechanism is used to capture the spatial influence of different targets in the scene on the propagation of electromagnetic waves; the channel attention mechanism is used to explore the importance of different feature dimensions in the scene.

[0023] Optionally, in the radar echo image fitting method based on multimodality and attention mechanism, the data-driven model uses a Unet network to utilize a cross-attention mechanism to extract complementary target key features from the prior echo image and the scene image.

[0024] In addition, to achieve the above-mentioned purpose, the present invention also provides a radar echo image fitting system based on multimodality and attention mechanism, wherein the radar echo image fitting system based on multimodality and attention mechanism includes:

[0025] A scene information supplementation module, used to obtain a scene graph and a priori echo graph, and supplement the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph;

[0026] A feature extraction and fusion module is used to introduce a cross-attention mechanism into the data-driven model, input the scene graph and the prior echo map into the data-driven model, and the data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo map, and output a fitted radar echo map.

[0027] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a radar echo image fitting program based on multimodality and attention mechanism stored in the memory and run on the processor, and the radar echo image fitting program based on multimodality and attention mechanism is executed by the processor to implement the steps of the radar echo image fitting method based on multimodality and attention mechanism as described above.

[0028] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a radar echo image fitting program based on multimodality and attention mechanism, and when the radar echo image fitting program based on multimodality and attention mechanism is executed by a processor, the steps of the radar echo image fitting method based on multimodality and attention mechanism as described above are implemented.

[0029] In the present invention, a scene graph and a priori echo graph are obtained, and the scene graph is supplemented into the prior echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format characteristics with the prior echo graph; a cross-attention mechanism is introduced into the data-driven model, and the scene graph and the prior echo graph are input into the data-driven model, and the data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo graph, and outputs a fitted radar echo graph. The present invention generates a scene graph based on the scene truth value, and performs feature extraction and fusion based on the attention mechanism, thereby improving the ability to extract echo graph features, realizing efficient fusion of scene graph and prior echo graph information, and can effectively improve the prediction accuracy of radar echo graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a preferred embodiment of the radar echogram fitting method based on multimodality and attention mechanism of the present invention;

[0031] Figure 2 It is a schematic diagram of calculating the scene truth value in a preferred embodiment of the radar echo map fitting method based on multimodality and attention mechanism of the present invention;

[0032] Figure 3 It is a scene graph generated according to the scene truth value in a preferred embodiment of the radar echo graph fitting method based on multimodality and attention mechanism of the present invention;

[0033] Figure 4 It is a schematic diagram of image expansion of a scene graph in a preferred embodiment of the radar echo graph fitting method based on multimodality and attention mechanism of the present invention;

[0034] Figure 5 It is a scene graph corresponding to the distance-angle echo graph in a preferred embodiment of the radar echo graph fitting method based on multimodality and attention mechanism of the present invention;

[0035] Figure 6 It is a scene graph corresponding to a range-Doppler echo graph in a preferred embodiment of the radar echo graph fitting method based on multimodality and attention mechanism of the present invention;

[0036] Figure 7It is a schematic diagram of the Unet network structure in the data-driven model in a preferred embodiment of the radar echogram fitting method based on multimodality and attention mechanism of the present invention;

[0037] Figure 8 It is a schematic diagram of output result comparison and verification in a preferred embodiment of the radar echogram fitting method based on multimodality and attention mechanism of the present invention;

[0038] Fig. 9 It is a structural diagram of a preferred embodiment of a radar echo pattern fitting system based on multimodality and attention mechanism of the present invention;

[0039] Fig.10 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] The radar echo pattern fitting method based on multimodality and attention mechanism described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the radar echo map fitting method based on multimodality and attention mechanism includes the following steps:

[0042] Step S10, obtaining a scene graph and a priori echo graph, and adding the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph.

[0043] Specifically, scene information is obtained, a scene truth value is obtained according to the scene information, a scene graph in an echo map format is generated according to the scene truth value, the scene graph is modified according to the format of the prior echo map, and the modified scene graph is added to the prior echo map.

[0044] Since only using the prior echo map as input may cause errors in the input information of the data-driven model and miss important scene features. To solve this problem, the present invention supplements the scene information in the input data to ensure that comprehensive scene information is input. Considering that high-quality data similar to the output features helps to improve the fitting effect of the data-driven model, the present invention supplements the scene information in the form of pictures. The supplementary image needs to meet the following two requirements: 1) Contain key scene information that affects the propagation of electromagnetic waves; 2) Have consistent format features with the radar echo map.

[0045] The scene graph includes the scene truth value, which includes the shape, electromagnetic parameters, position and velocity information of the scene target. In terms of scene information, the influence of the scene target on the electromagnetic wave propagation process is mainly reflected in the physical processes such as reflection and scattering of electromagnetic waves by objects, such as Figure 2 As shown in formula (1):

[0046] ; (1)

[0047] in, represents the electric field received by the radar, Represents the reflection coefficient of the object surface, which is determined by the electrical parameters and the angle of incidence. Represents the incident electromagnetic wave (i.e., the incident electric field), which is determined by the relative position of the target and the radar. represents an imaginary number, represents the propagation constant, which is determined by the frequency of the incident wave and the speed of the reflecting surface, and They represent the propagation direction of electromagnetic waves. Axis coordinates and Axis direction coordinates, It represents the incident angle, which is determined by the normal vector of the incident surface and the relative position between the radar and the target. It can be seen that the impact of the target on the electromagnetic wave propagation process is reflected in the target's shape, electromagnetic parameters, position and speed. Therefore, the scene graph should contain the target's shape, electromagnetic parameters, position and speed information.

[0048] Based on the target’s shape, electromagnetic parameters, and location information, a scene graph is generated. Figure 3 As shown in the figure, the top view of the vehicle is used to reflect the appearance characteristics of the target, and RGB is used to represent the electromagnetic parameters of the target. (Electromagnetic parameters are used to calculate the reflection coefficient The coordinates of the target on the way represent its position, and the depth value can also be used to represent its speed.

[0049] In terms of the image format, the radar's prior echo map is an expanded map that satisfies special coordinates of certain length and width. In order to keep the scene map consistent with the features of the prior echo map, the scene map needs to be modified according to the format of the prior echo map. The modifications include:

[0050] (1) Generate a scene graph that matches the two prior echo maps: Ensure that the scene graph can correspond to echo maps in the range-angle-map and range-doppler-map formats respectively.

[0051] (2) Coordinate format conversion: The coordinates of the scene graph are adjusted to the formats of range-angle-map and range-doppler-map to match the characteristics of the prior echo map.

[0052] (3) Image expansion: The scene graph from the top-down angle (scene top view) is expanded orthogonally in the angle dimension to generate the same coordinate format and resolution as the echo graph, such as Figure 4 As shown ( Figure 4 In represents the incident angle, R represents the radius).

[0053] According to the above method, the prior echo map is generated as follows Figure 5 and Figure 6 As shown ( Figure 5 In Rx Represents the horizontal distance, Figure 5 and Figure 6 middle R Indicates the vertical distance, Figure 6 In V Indicates speed), it can be seen that the scene graph and the radar’s prior echo graph have similar coordinates and representation methods. Figure 5 and Figure 6 The target contains all the information in the scene that affects the radar echo pattern.

[0054] Step S20: introduce a cross-attention mechanism into the data-driven model, input the scene graph and the prior echo graph into the data-driven model, and the data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo graph, and outputs a fitted radar echo graph.

[0055] Specifically, a cross-attention mechanism is introduced into the data-driven model in advance, and the prior echo map that has supplemented the scene map is input into the data-driven model; the data-driven model establishes connections between different input data through the cross-attention mechanism, and extracts target key features that complement each other in the prior echo map and the scene map; the data-driven model fuses the target key features through the cross-attention mechanism, and outputs a fitted radar echo map.

[0056] In response to the needs of feature extraction and fusion, the present invention uses an attention mechanism to extract the target key features of the scene graph and the prior echo graph before the scene graph is input into the data-driven model (for example, the relative position relationship between targets in the scene, etc., which have a great impact on the radar detection results, belong to the target key features). In order to better fuse the features of the prior echo graph and the scene graph, the present invention introduces a cross attention mechanism (CA) in the data-driven model. This mechanism extracts complementary features of the prior echo graph and the scene graph by establishing connections between different input data, thereby enhancing the data-driven model's ability to understand the input information, because the scene information contained in the prior echo graph is not very accurate, and some scene information is missing, which can be compensated by the scene graph. The features of the scene graph are very different from those of the echo graph, and the prior echo graph compensates for this.

[0057] The cross attention mechanism includes spatial attention mechanism and channel attention mechanism. The spatial attention mechanism is used to capture the spatial impact of different targets in the scene on electromagnetic wave propagation, while the channel attention mechanism is used to explore the importance of different feature dimensions in the scene. By combining the two, the information that is crucial to electromagnetic wave propagation (i.e., target key features) in the scene graph can be more effectively extracted.

[0058] In addition, in order to reflect the multipath propagation effect caused by the interaction of electromagnetic waves in different airspaces in the scene space and better fit the features of different dimensions, the present invention introduces a spatial and channel attention mechanism in the Skip Connections module of the Unet network (a classic deep learning model). Through this improvement, the features in the encoder and decoder can be effectively fused, so that the data-driven model can more accurately correct the details of the echo map while maintaining the consistency of the backbone features.

[0059] According to the above method, the present invention improves the Unet network structure and constructs CDA-Unet (Cross Attention and Dual Attention Unet) based on the prior signal level radar model. Figure 7 CDA-Unet uses the attention mechanism to extract and fuse key information from the scene graph and the prior echo graph, and achieves high-precision prediction of the real echo graph through the improved Unet structure (the spatial and channel attention mechanisms are also added to the Unet to better fit the input and output data).

[0060] The modeling method using multimodal fusion as input and the optimization method based on attention mechanism proposed in this invention can effectively improve the prediction accuracy of radar echo map. This method can significantly improve the confidence of radar model in different simulations and promote the development of simulation tests such as autonomous driving and drones.

[0061] To prove the effectiveness of the method of the present invention, radar data were collected and a data-driven model was established using a conventional prior modeling method and a data-driven model was established using the modeling method proposed by the present invention. The training hyperparameters of the method of the present invention are shown in the following table:

[0062]

[0063] The effect of the data-driven model in the training set on the test set is as follows Figure 8 shown.

[0064] The radar echo map output by CDA-Unet is highly consistent with the actual radar test results. From the prediction results, it can be seen that CDA-Unet can accurately predict the echo distribution area and intensity of the target in multiple scenarios. Compared with the Unet network, CDA-Unet shows a significant improvement in prediction accuracy. The MSE (Mean Squared Error) of its range-angle map and range-doppler map is reduced by 57.3% and 65.4% respectively compared with the Unet network.

[0065] The CDA-Unet proposed in this paper effectively compensates for the shortcomings of the prior echo map in detail features by combining scene information and attention mechanism, accurately predicts the echo characteristics of targets and multipath signals, achieves the research goal of signal-level radar modeling, and demonstrates excellent prediction performance and wide applicability.

[0066] The present invention can be used in application scenarios of high-precision modeling of sensors, including simulation of sensors such as radar, lidar and camera, communication channels, interference simulation, optical simulation, etc. The present invention can effectively improve calculation efficiency and calculation accuracy.

[0067] Furthermore, if Fig. 9 As shown, based on the above-mentioned radar echo image fitting method based on multimodality and attention mechanism, the present invention also provides a radar echo image fitting system based on multimodality and attention mechanism, wherein the radar echo image fitting system based on multimodality and attention mechanism includes:

[0068] A scene information supplementation module 51 is used to obtain a scene graph and a priori echo graph, and supplement the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph;

[0069] The feature extraction and fusion module 52 is used to introduce a cross-attention mechanism into the data-driven model, input the scene graph and the prior echo map into the data-driven model, and the data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo map, and outputs a fitted radar echo map.

[0070] Furthermore, if Fig.10 As shown, based on the above-mentioned radar echo pattern fitting method and system based on multimodality and attention mechanism, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Fig.10 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0071] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a radar echo image fitting program 40 based on multimodality and attention mechanism is stored on the memory 20, and the radar echo image fitting program 40 based on multimodality and attention mechanism can be executed by the processor 10, thereby realizing the radar echo image fitting method based on multimodality and attention mechanism in the present application.

[0072] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the radar echo pattern fitting method based on multimodality and attention mechanism.

[0073] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The processor 10, the memory 20, and the display 30 of the terminal communicate with each other via a system bus.

[0074] In one embodiment, when the processor 10 executes the radar echo pattern fitting program 40 based on multimodality and attention mechanism in the memory 20, the following steps are implemented:

[0075] Acquire a scene graph and a priori echo graph, and add the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph;

[0076] A cross-attention mechanism is introduced into the data-driven model, and the scene graph and the prior echo graph are input into the data-driven model. The data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo graph, and outputs a fitted radar echo graph.

[0077] The acquiring of the scene graph and the priori echo graph, and supplementing the scene graph to the priori echo graph, specifically includes:

[0078] Acquire scene information, acquire a scene truth value according to the scene information, and generate a scene graph in an echogram format according to the scene truth value;

[0079] The scene graph is modified according to the format of the priori echo graph, and the modified scene graph is added to the priori echo graph.

[0080] Wherein, the scene graph includes scene truth values, and the scene truth values ​​include shape, electromagnetic parameters, position and velocity information of scene targets;

[0081] ;

[0082] in, represents the electric field received by the radar, represents the reflection coefficient of the object surface, represents the incident electromagnetic wave, represents an imaginary number, represents the propagation constant, and They represent the propagation direction of electromagnetic waves. Axis coordinates and Axis direction coordinates, Represents the angle of incidence.

[0083] The step of modifying the scene graph according to the format of the priori echo graph specifically includes:

[0084] Generate a scene graph that matches the two prior echo graphs, so that the two scene graphs correspond to the range-angle echo graph and the range-Doppler echo graph, respectively;

[0085] Adjusting the coordinates of the two scene graphs into the formats of the range-angle echo graph and the range-Doppler echo graph, so that the two scene graphs match the features of the prior echo graph;

[0086] The scene graph at the bird's-eye view is orthogonally expanded in the angle dimension to generate the same coordinate format and resolution as the prior echo graph.

[0087] The method of introducing a cross attention mechanism into the data-driven model, inputting the scene graph and the prior echo graph into the data-driven model, and extracting and fusing target key features from the scene graph and the prior echo graph using the cross attention mechanism, and outputting a fitted radar echo graph, specifically includes:

[0088] Introducing a cross-attention mechanism in the data-driven model in advance, and inputting the priori echo map that has supplemented the scene map into the data-driven model;

[0089] The data-driven model establishes connections between different input data through the cross-attention mechanism, and extracts key target features that complement each other in the priori echo graph and the scene graph;

[0090] The data-driven model fuses the target key features through the cross-attention mechanism and outputs a fitted radar echo map.

[0091] Among them, the cross-attention mechanism includes a spatial attention mechanism and a channel attention mechanism; the spatial attention mechanism is used to capture the spatial impact of different targets in the scene on the propagation of electromagnetic waves; the channel attention mechanism is used to explore the importance of different feature dimensions in the scene.

[0092] The data-driven model uses the Unet network to utilize the cross-attention mechanism to extract the key target features that complement each other from the prior echo map and the scene map.

[0093] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a radar echo image fitting program based on multimodality and attention mechanism, and when the radar echo image fitting program based on multimodality and attention mechanism is executed by a processor, the steps of the radar echo image fitting method based on multimodality and attention mechanism as described above are implemented.

[0094] In summary, the present invention provides a radar echo map fitting method, system, terminal and computer-readable storage medium based on multimodality and attention mechanism, the method comprising: obtaining a scene graph and a priori echo map, supplementing the scene graph to the priori echo map, the scene graph contains key scene information that affects electromagnetic wave propagation, and has consistent format characteristics with the priori echo map; introducing a cross-attention mechanism in a data-driven model, inputting the scene graph and the priori echo map into the data-driven model, the data-driven model extracts and fuses target key features from the scene graph and the priori echo map using the cross-attention mechanism, and outputs a fitted radar echo map. The present invention generates a scene graph based on the scene truth value, and performs feature extraction and fusion based on the attention mechanism, thereby improving the ability to extract echo map features, realizing efficient fusion of scene graph and priori echo map information, and can effectively improve the prediction accuracy of radar echo maps.

[0095] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or terminal including the element.

[0096] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0097] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A radar echo pattern fitting method based on multimodality and attention mechanism, characterized in that: The radar echo map fitting method based on multimodality and attention mechanism includes: Acquire a scene graph and a priori echo graph, and add the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph; Introducing a cross attention mechanism into a data-driven model, inputting the scene graph and the prior echo graph into the data-driven model, wherein the data-driven model extracts and fuses target key features from the scene graph and the prior echo graph using the cross attention mechanism, and outputs a fitted radar echo graph; The acquiring of the scene graph and the priori echo graph, and supplementing the scene graph to the priori echo graph, specifically includes: Acquire scene information, acquire a scene truth value according to the scene information, and generate a scene graph in an echogram format according to the scene truth value; Modifying the scene graph according to the format of the priori echo graph, and adding the modified scene graph to the priori echo graph; The modifying the scene graph according to the format of the priori echo graph specifically includes: Generate a scene graph that matches the two prior echo graphs, so that the two scene graphs correspond to the range-angle echo graph and the range-Doppler echo graph, respectively; Adjusting the coordinates of the two scene graphs into the formats of the range-angle echo graph and the range-Doppler echo graph, so that the two scene graphs match the features of the prior echo graph; The scene graph at the bird's-eye view is orthogonally expanded in the angle dimension to generate the same coordinate format and resolution as the prior echo graph.

2. The radar echo pattern fitting method based on multimodality and attention mechanism according to claim 1, characterized in that: The scene graph includes scene truth values, which include shape, electromagnetic parameters, position and velocity information of scene targets.

3. The radar echo pattern fitting method based on multimodality and attention mechanism according to claim 1, characterized in that: The cross attention mechanism is introduced into the data-driven model, the scene graph and the prior echo graph are input into the data-driven model, and the data-driven model uses the cross attention mechanism to extract and fuse target key features from the scene graph and the prior echo graph, and outputs a fitted radar echo graph, specifically including: Introducing a cross-attention mechanism in the data-driven model in advance, and inputting the priori echo map that has supplemented the scene map into the data-driven model; The data-driven model establishes connections between different input data through the cross-attention mechanism, and extracts key target features that complement each other in the priori echo graph and the scene graph; The data-driven model fuses the target key features through the cross-attention mechanism and outputs a fitted radar echo map.

4. The radar echo pattern fitting method based on multimodality and attention mechanism according to claim 3 is characterized in that: The cross-attention mechanism includes a spatial attention mechanism and a channel attention mechanism; the spatial attention mechanism is used to capture the spatial impact of different targets in the scene on the propagation of electromagnetic waves; the channel attention mechanism is used to explore the importance of different feature dimensions in the scene.

5. The radar echo pattern fitting method based on multimodality and attention mechanism according to claim 3 is characterized in that: The data-driven model uses a Unet network to utilize a cross-attention mechanism to extract complementary target key features from the prior echo map and the scene map.

6. A radar echo pattern fitting system based on multimodality and attention mechanism, characterized in that: The radar echo image fitting system based on multimodality and attention mechanism is applied to the radar echo image fitting method based on multimodality and attention mechanism according to any one of claims 1 to 5, and the radar echo image fitting system based on multimodality and attention mechanism includes: A scene information supplementation module, used to obtain a scene graph and a priori echo graph, and supplement the scene graph to the priori echo graph, wherein the scene graph contains key scene information that affects electromagnetic wave propagation and has consistent format features with the priori echo graph; A feature extraction and fusion module is used to introduce a cross-attention mechanism into the data-driven model, input the scene graph and the prior echo map into the data-driven model, and the data-driven model uses the cross-attention mechanism to extract and fuse target key features from the scene graph and the prior echo map, and output a fitted radar echo map.

7. A terminal, characterized in that: The terminal includes: a memory, a processor, and a radar echo image fitting program based on multimodality and attention mechanism stored in the memory and executable on the processor. When the radar echo image fitting program based on multimodality and attention mechanism is executed by the processor, the steps of the radar echo image fitting method based on multimodality and attention mechanism as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a radar echo image fitting program based on multimodality and attention mechanism. When the radar echo image fitting program based on multimodality and attention mechanism is executed by a processor, the steps of the radar echo image fitting method based on multimodality and attention mechanism as described in any one of claims 1 to 5 are implemented.

Images