Radar image decoding method and device based on communication perception fusion system

Abstract

The invention belongs to the technical field of wireless communication and semantic coding, and discloses a radar image decoding method and device based on a communication perception fusion system, and the method comprises the steps: obtaining a target radar image from a radar system through a wireless channel; executing a first decoding operation on the target radar image through a neural network channel decoder module to obtain multiple pieces of image feature information of different sizes and scales of the target radar image; performing a second decoding operation on the plurality of pieces of image feature information with different sizes and scales through a semantic decoder module to obtain semantic feature information of the target radar image; and obtaining a detection result according to the semantic feature information. Therefore, the communication perception fusion system can integrate the radar image recognition process and the communication process of the communication system, so that the utilization rate of computing resources and transmission resources is improved, and the image recognition result and the category label of the radar image can be accurately predicted based on the semantic feature information of the radar image.

Description

Technical Field

[0001] The invention belongs to the technical field of wireless communication and semantic coding. Background Art

[0002] SAR (Synthetic Aperture Radar) is an active remote sensing technology that uses a mobile synthetic aperture method to achieve high-resolution, all-weather and weather-independent ground imaging by mounting the radar system on a mobile platform. This technology is very valuable for Earth observation and helps monitor surface deformation, monitor geological disasters, and track changes in ice and snow cover. The all-weather capability and high-resolution positioning of synthetic aperture radar are crucial for detecting major surface changes.

[0003] After acquiring a SAR image, i.e., a radar image, it is usually necessary to perform image recognition on the radar image. In the related art, the radar image recognition system is separated from the communication system, i.e., the radar image recognition process is separated from the communication process of the communication system. In this way, the radar image recognition system requires computing resources for the recognition process of the radar image, and the communication system also requires computing resources during the communication process, resulting in a large consumption of computing resources. Summary of the invention

[0004] The purpose of the present invention is to integrate the radar image recognition process with the communication process of the communication system through a communication perception fusion system, that is, to achieve the integration of communication and perception, thereby saving computing resources and transmission resources, that is, to improve the utilization rate of computing resources and transmission resources.

[0005] In a first aspect, an embodiment of the present invention provides a radar image decoding method based on a communication perception fusion system, which is applied to the communication perception fusion system. The communication perception fusion system includes a neural network channel decoder module and a semantic decoder module. The method includes:

[0006] Acquire target radar images from the radar system via a wireless channel;

[0007] Performing a first decoding operation on the target radar image by the neural network channel decoder module to obtain image feature information of multiple different sizes of the target radar image;

[0008] Performing a second decoding operation on the plurality of image feature information of different sizes through the semantic decoder module to obtain semantic feature information of the target radar image;

[0009] A detection result is obtained according to the semantic feature information, where the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0010] Optionally, the neural network channel decoder module includes a redundancy submodule and a multi-scale fusion submodule;

[0011] The performing a first decoding operation on the target radar image by the neural network channel decoder module to obtain image feature information of multiple different scales of the target radar image includes:

[0012] Performing a first decoding sub-operation on the target radar image through the redundancy sub-module to obtain a first radar image; the first decoding operation is used to enhance feature information of the target radar image;

[0013] The second decoding sub-operation is performed on the first radar image through the multi-scale fusion sub-module to obtain image feature information of multiple different scales; the second decoding sub-operation includes an upsampling operation and a decoding operation.

[0014] Optionally, the semantic decoder module includes a detection head submodule, which includes multiple detectors; the loss functions corresponding to the multiple detectors include a first type of loss function and a second type of loss function; the first type of loss function is used to regress the positions of the predicted bounding box and the true bounding box of the target radar image; the second type of loss function makes the predicted category label corresponding to the target radar image the same as the true category label of the target radar image.

[0015] Optionally, performing a second decoding operation on the plurality of image feature information of different size scales by the semantic decoder module to obtain the semantic feature information of the target radar image includes:

[0016] Using the multiple detectors to detect the multiple image feature information of different sizes in sequence;

[0017] Performing a weighted summation on the function value of the first type of loss function and the function value of the second type of loss function to obtain a total loss function value corresponding to the semantic decoder module;

[0018] When the total loss function value is less than a preset value, the semantic feature information of the target radar image is determined.

[0019] Optionally, acquiring a target radar image from a radar system through a wireless channel includes:

[0020] Determine an original radar image corresponding to the target radar image in the radar system;

[0021] Obtaining a channel gain and Gaussian white noise corresponding to the wireless channel;

[0022] The target radar image is determined based on the original radar image, the channel gain and the Gaussian white noise.

[0023] In a second aspect, an embodiment of the present invention provides a communication perception fusion system, including: an image acquisition module, a neural network channel decoder module, a semantic decoder module and a prediction module;

[0024] The image acquisition module is used to acquire a target radar image from the radar system through a wireless channel;

[0025] The neural network channel decoder module is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image;

[0026] The semantic decoder module is used to perform a second decoding operation on the plurality of image feature information of different sizes to obtain the semantic feature information of the target radar image;

[0027] The prediction module is used to obtain a detection result according to the semantic feature information, and the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0028] In a third aspect, an embodiment of the present invention provides a radar image decoding device based on a communication perception fusion system, which is applied to the communication perception fusion system, and the device includes:

[0029] An image acquisition module, used for acquiring a target radar image from a radar system through a wireless channel;

[0030] A neural network channel decoder module is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image;

[0031] A semantic decoder module, configured to perform a second decoding operation on the plurality of image feature information of different sizes to obtain semantic feature information of the target radar image;

[0032] A prediction module is used to obtain a detection result according to the semantic feature information, wherein the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0033] In a fourth aspect, an embodiment of the present invention provides an electronic device, including:

[0034] at least one processor;

[0035] a memory for storing the at least one processor-executable instruction;

[0036] The at least one processor is configured to execute the instructions to implement the method described in any one of the first aspects.

[0037] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any method described in the first aspect.

[0038] In a sixth aspect, an embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0039] According to the technical solution provided by the embodiment of the present invention, the radar system and the communication perception fusion system can communicate, so that the radar system can send a radar image to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through a neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0040] It can be seen that the communication-perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, it realizes the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. Moreover, based on the semantic feature information of the radar image, the image recognition results and category labels of the radar image can be accurately predicted. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A structural block diagram of a communication perception fusion system provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the structure of a neural network channel decoder module provided by an embodiment of the present invention;

[0043] Figure 3 for Figure 2 A schematic diagram of specific steps for the redundant submodule in performing decoding operation;

[0044] Figure 4 A schematic diagram of a real bounding box, a predicted bounding box and a minimum enclosing box of a radar image provided by an embodiment of the present invention;

[0045] Figure 5 A flowchart of a radar image decoding method based on a communication perception fusion system provided in an embodiment of the present disclosure;

[0046] Figure 6 The present invention provides a schematic diagram of detection results of wireless channel signal-to-noise ratios of 10 decibels, 15 decibels, and 20 decibels, respectively.

[0047] Figure 7 A schematic diagram of the structure of a radar image decoding device based on a communication perception fusion system provided in an embodiment of the present disclosure;

[0048] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The present invention will be described in detail below through examples.

[0050] SAR (Synthetic Aperture Radar) is an active remote sensing technology that uses a mobile synthetic aperture method to achieve high-resolution, all-weather and weather-independent ground imaging by mounting the radar system on a mobile platform. This technology is very valuable for Earth observation and helps monitor surface deformation, monitor geological disasters, and track changes in ice and snow cover. The all-weather capability and high-resolution positioning of synthetic aperture radar are crucial for detecting major surface changes.

[0051] After acquiring a SAR image, i.e., a radar image, it is usually necessary to perform image recognition on the radar image. In the related art, the radar image recognition system is separated from the communication system, i.e., the radar image recognition process is separated from the communication process of the communication system. In this way, the radar image recognition system requires computing resources for the recognition process of the radar image, and the communication system also requires computing resources during the communication process, resulting in a large consumption of computing resources. Moreover, both the radar image recognition process and the communication process of the communication system require transmission resources, resulting in a large consumption of transmission resources.

[0052] Based on the above technical problems existing in the related technologies, the embodiments of the present invention provide a radar image decoding method based on a communication perception fusion system, a radar image decoding device based on a communication perception fusion system, and a communication perception fusion system.

[0053] In order to describe the solution clearly, a communication perception fusion system provided by an embodiment of the present invention will be introduced in detail below in combination with specific application scenarios.

[0054] like Figure 1 As shown, a communication perception fusion system provided by an embodiment of the present invention may include: an image acquisition module 110, a neural network channel decoder module 120, a semantic decoder module 130 and a prediction module 140.

[0055] The image acquisition module 110 is used to acquire a target radar image from a radar system through a wireless channel.

[0056] Specifically, the radar system can communicate with the communication perception fusion system. After generating the radar image, the radar system can send the radar image to the communication perception fusion system through a wireless channel. In order to describe the solution clearly, the radar image is called a target radar image.

[0057] In practical applications, the radar system may be affected by Rayleigh fading and white noise when transmitting the original radar image to the communication perception fusion system. Therefore, the wireless channel can be modeled to improve the image quality of the target radar image.

[0058] Assume that X is the original radar image, X' is the target radar image, h is the Rayleigh fading channel gain, η represents the mean value of 0 and the variance of σ 2 Gaussian noise, that is, η~(0,σ 2 ),η~(0,σ 2 ) means that η has a mean of 0 and a variance of σ 2 Then,

[0059]

[0060] Among them, for AWGN channel is the channel model representation under Gaussian channel, and for fading channel is the channel model representation under Rayleigh fading channel.

[0061] The neural network channel decoder module 120 is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image.

[0062] Specifically, after the communication perception fusion system receives the target radar image, the neural network channel decoder module can perform a first decoding operation on the target radar image. The first decoding operation is used to extract image feature information of multiple different scales of the target radar image. This is beneficial for the semantic decoder module to accurately extract the semantic feature information of the target radar image based on the image feature information of multiple different scales in the subsequent steps.

[0063] In practical applications, the neural network channel decoder module can include a redundant submodule and a multi-scale fusion submodule, such as Figure 2 The redundant submodule can perform a first decoding sub-operation on the target radar image to obtain a first radar image X1'; the first decoding operation is used to enhance the characteristic information of the target radar image. The specific steps of the redundant submodule performing the first decoding sub-operation can be as follows: Figure 3As shown, specifically: the target radar image is sequentially passed through the global pooling layer, the fully connected layer, the ReLU activation function, the fully connected layer and the Sigmoid activation function, the proportional coefficient is output from the Sigmoid activation function, the feature information to be enhanced in the target radar image is multiplied by the proportional coefficient, and the feature information after multiplication by the proportional coefficient is subtracted to obtain the first radar image after the feature information is enhanced. Among them, the ReLU (Rectified Linear Unit) activation function and the Sigmoid activation function are both commonly used activation functions in deep learning.

[0064] After obtaining the first radar image X1' through the redundancy submodule, the multi-scale fusion submodule performs a second decoding sub-operation on the first radar image to obtain image feature information of multiple different scales, namely M1', M2' and M3'.

[0065]

[0066] Among them, F U is an upsampling operation, the superscript -1 is a reverse operation opposite to the encoding process, and M1', M2' and M3' are feature information of three different scales of X'. Of course, the feature information of three different scales of X' is extracted here for example. In practical applications, feature information of more scales can be extracted, which is not specifically limited here.

[0067] The semantic decoder module 130 is used to perform a second decoding operation on image feature information of multiple different sizes to obtain semantic feature information of the target radar image.

[0068] Specifically, after obtaining image feature information of multiple different scales through the neural network channel decoder module, the second decoding operation can be performed on the image feature information of multiple different scales through the semantic decoder module to obtain semantic feature information of the target radar image.

[0069] Among them, the semantic decoder module mainly includes a detection head submodule, and the detection head submodule may include multiple detectors, for example, it may include 3 detectors, and each detector may include multiple convolutional layers; the loss functions used in the processing process of multiple detectors mainly include first-type loss functions and second-type loss functions; the first-type loss function is used to regress the positions of the predicted bounding box and the true bounding box of the target radar image, that is, to make the predicted bounding box and the true bounding box of the radar image as close as possible to the same position, so as to improve the accuracy of the predicted position; the second-type loss function is used to make the predicted category label corresponding to the target radar image the same as the true category label of the target radar image, that is, to improve the accuracy of the predicted category.

[0070] In practical applications, the first type of loss function can include complete intersection loss and distribution focus loss, that is, the first type of loss function is a combination of complete intersection loss and distribution focus loss, which is mainly used for object position regression. The complete intersection loss and distribution focus loss will be introduced below.

[0071] The complete intersection loss can be expressed as follows:

[0072]

[0073] Among them, b represents the center point of the box, Θ represents the Euclidean distance, and w and h represent the width and height of the corresponding box, such as Figure 4 As shown, A represents the predicted bounding box of the radar image, and B represents the real bounding box of the radar image. In the above formula, A is the area of ​​the predicted bounding box, and B is the area of ​​the real bounding box; E is the minimum enclosed box between A and B, and b gt is the center point of the predicted bounding box, W gt is the width of the predicted bounding box, h gt is the height of the predicted bounding box, b p is the center of the real bounding box, W p is the width of the actual border box, h p is the height of the actual border box, W e is the width of the minimum enclosing box, h e is the height of the minimum enclosing box.

[0074] Distribution Focus Loss is an enhanced version of the Focus Loss and is used as a loss function to regress the distance from a point to a bounding box. The label y represents the location of the true label. When the predicted probability distribution is flat around the label y, it indicates that there may be a lack of sufficient certainty in predicting that particular class. In order to speed up the network's ability to locate values ​​near the label y, two points y are explicitly added. i and i+1 The probability of being closer to label y. This enables the detector to quickly focus on values ​​near object y and improve the accuracy of category prediction. The distribution focus loss function can be expressed as follows:

[0075]

[0076] Among them, y is the position of the true label, y i and i+1 is the position of the two added points.

[0077]

[0078] Moreover, in practical applications, the second type of loss function can be a focal loss function, which is an object loss function that returns a value indicating the accuracy of object classification. It can be expressed as follows:

[0079]

[0080] in, represents the sigmoid function, the total number of samples is represented by n, and y i represents the i-th sample, x i represents the predicted probability of the i-th sample.

[0081] The total loss function corresponding to the processing of multiple detectors can be expressed as:

[0082] loss = ε·l box +η·L1+ι·L2

[0083] That is, the total loss function is obtained by weighted summing the above three loss functions. The weight coefficients of the three loss functions in the formula can be set according to actual needs, and the embodiment of the present invention does not specifically limit this. In addition, when the function value of the total loss function is less than a preset value, it is determined that the semantic feature information of the target radar image is obtained.

[0084] The prediction module 140 is used to obtain a detection result according to the semantic feature information, where the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0085] Specifically, after obtaining the semantic feature information of the target radar image through the semantic decoder module, the semantic feature information can be input into the prediction module, namely the prediction layer, to obtain the image recognition result of the target radar image and the category label corresponding to the target radar image.

[0086] From the above description, it can be seen that the radar system and the communication perception fusion system can communicate, so that the radar system can send radar images to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through the neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0087] It can be seen that the technical solution provided by the embodiment of the present invention, the communication perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, realize the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. In addition, based on the semantic feature information of the radar image, the image recognition result and category label of the radar image can be accurately predicted.

[0088] After introducing a communication perception fusion system provided by an embodiment of the present invention, a radar image decoding method based on the communication perception fusion system provided by an embodiment of the present invention will be described in detail below.

[0089] like Figure 5 As shown, an embodiment of the present invention provides a radar image decoding method based on a communication perception fusion system. The method can be applied to the communication perception fusion system described above. The communication perception fusion system may include a neural network channel decoder module and a semantic decoder module. The method may include the following steps:

[0090] S510, acquiring a target radar image from a radar system through a wireless channel.

[0091] Specifically, the radar system can communicate with the communication perception fusion system. After generating the radar image, the radar system can send the radar image to the communication perception fusion system through a wireless channel. In order to describe the solution clearly, the radar image is called a target radar image.

[0092] In one implementation, acquiring a target radar image from a radar system through a wireless channel may include the following steps, namely, step a1 to step a3:

[0093] Step a1, determining an original radar image corresponding to a target radar image in a radar system.

[0094] Step a2: Obtain the channel gain and Gaussian white noise corresponding to the wireless channel.

[0095] Step a3, determining the target radar image based on the original radar image, channel gain and Gaussian white noise.

[0096] In practical applications, when the radar system transmits the original radar image to the communication perception fusion system, it may be affected by Rayleigh fading and white noise. Therefore, the channel gain and white noise of the wireless channel can be obtained to model the original radar image and the white noise wireless channel pair, thereby improving the image quality of the target radar image.

[0097] Specifically, assume that X is the original radar image, X' is the target radar image, h is the Rayleigh fading channel gain, η represents the mean value of 0 and the variance of σ 2 Gaussian noise, that is, η~(0,σ 2 ),η~(0,σ 2 ) means that η has a mean of 0 and a variance of σ 2 Then,

[0098]

[0099] Among them, for AWGN channel is the channel model representation under Gaussian channel, and for fading channel is the channel model representation under Rayleigh fading channel.

[0100] S520, performing a first decoding operation on the target radar image through a neural network channel decoder module to obtain image feature information of multiple different scales of the target radar image.

[0101] Specifically, after the communication perception fusion system receives the target radar image, the neural network channel decoder module can perform a first decoding operation on the target radar image. The first decoding operation is used to extract image feature information of multiple different scales of the target radar image. This is beneficial for the semantic decoder module to accurately extract the semantic feature information of the target radar image based on the image feature information of multiple different scales in the subsequent steps.

[0102] In one embodiment, the neural network channel decoder module includes a redundancy submodule and a multi-scale fusion submodule;

[0103] S520, performing a first decoding operation on the target radar image by a neural network channel decoder module to obtain image feature information of multiple different scales of the target radar image, may include the following steps, namely step b1 and step b2:

[0104] Step a1, performing a first decoding sub-operation on the target radar image through the redundant sub-module to obtain a first radar image; the first decoding operation is used to enhance the feature information of the target radar image.

[0105] Step a2, performing a second decoding sub-operation on the first radar image through a multi-scale fusion sub-module to obtain image feature information of multiple different scales; the second decoding sub-operation includes an upsampling operation and a decoding operation.

[0106] Since step b1 and step b2 have been described in detail in the above embodiment, they will not be described again here.

[0107] S530, performing a second decoding operation on the image feature information of multiple different size scales through a semantic decoder module to obtain semantic feature information of the target radar image.

[0108] In one embodiment, the semantic decoder module includes a detection head submodule, which includes multiple detectors; the loss functions corresponding to the multiple detectors include a first type of loss function and a second type of loss function; the first type of loss function is used to regress the positions of the predicted bounding box and the true bounding box of the target radar image, that is, to make the predicted bounding box and the true bounding box of the radar image as close as possible to the same position to improve the accuracy of the predicted position; the second type of loss function is used to make the predicted category label corresponding to the target radar image the same as the true category label of the target radar image, that is, to improve the accuracy of the predicted category.

[0109] It should be noted that the first type of loss function and the second type of loss function have been explained in detail in the above embodiments and will not be repeated here.

[0110] At this time, S530, performing a second decoding operation on the image feature information of multiple different sizes through the semantic decoder module to obtain the semantic feature information of the target radar image may include the following steps, namely, step c1 to step c3:

[0111] Step c1, using multiple detectors to detect image feature information of multiple different sizes in sequence.

[0112] Step c2: perform weighted summation on the function value of the first type of loss function and the function value of the second type of loss function to obtain the total loss function value corresponding to the semantic decoder module.

[0113] Step c3, when the total loss function value is less than a preset value, determining the semantic feature information of the target radar image.

[0114] Specifically, the total loss function of the multiple detectors included in the semantic decoder module is obtained by weighted summing the first type of loss function and the second type of loss function. The weight coefficients corresponding to the first type of loss function and the second type of loss function can be set according to actual needs, and the embodiment of the present invention does not specifically limit this. In addition, when the function value of the total loss function is less than a preset value, it is determined that the semantic feature information of the target radar image is obtained.

[0115] S540, obtaining a detection result according to the semantic feature information, where the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0116] Specifically, after obtaining the semantic feature information of the target radar image through the semantic decoder module, the semantic feature information can be input into the prediction module, namely the prediction layer, to obtain the image recognition result of the target radar image and the category label corresponding to the target radar image.

[0117] like Figure 6As shown in FIG. 1 , the detection results of the signal-to-noise ratio of the wireless channel are 10 dB, 15 dB, and 20 dB respectively. Figure 6 It can be seen that when the signal-to-noise ratio is 10 decibels, the detection result shows: ship 0.87, that is, the probability of the ship category label is 0.87; when the signal-to-noise ratio is 15 decibels, the detection result shows: ship 0.89, that is, the probability of the ship category label is 0.89; when the signal-to-noise ratio is 20 decibels, the detection result shows: ship 0.93, that is, the probability of the ship category label is 0.93. It can be seen that the radar image decoding method provided by the embodiment of the present invention has a high accuracy of the detection result when the signal-to-noise ratio is low, and as the signal-to-noise ratio of the wireless channel increases, the accuracy of the detection effect also increases.

[0118] From the above description, it can be seen that the radar system and the communication perception fusion system can communicate, so that the radar system can send radar images to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through the neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0119] It can be seen that the technical solution provided by the embodiment of the present invention, the communication perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, realize the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. In addition, based on the semantic feature information of the radar image, the image recognition result and category label of the radar image can be accurately predicted.

[0120] The embodiment of the present invention also provides a radar image decoding device based on a communication perception fusion system, which is applied to a communication perception fusion system, such as Figure 7 Said device comprises:

[0121] An image acquisition module 710 is used to acquire a target radar image from a radar system through a wireless channel;

[0122] A neural network channel decoder module 720 is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image;

[0123] A semantic decoder module 730 is used to perform a second decoding operation on the plurality of image feature information of different sizes to obtain semantic feature information of the target radar image;

[0124] The prediction module 740 is used to obtain a detection result according to the semantic feature information, where the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

[0125] According to the technical solution provided by the embodiment of the present invention, the radar system and the communication perception fusion system can communicate, so that the radar system can send a radar image to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through a neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0126] It can be seen that the communication-perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, it realizes the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. Moreover, based on the semantic feature information of the radar image, the image recognition results and category labels of the radar image can be accurately predicted.

[0127] Optionally, the neural network channel decoder module includes a redundancy submodule and a multi-scale fusion submodule;

[0128] The neural network channel decoder module is specifically used for:

[0129] Performing a first decoding sub-operation on the target radar image through the redundancy sub-module to obtain a first radar image; the first decoding operation is used to enhance feature information of the target radar image;

[0130] The second decoding sub-operation is performed on the first radar image through the multi-scale fusion sub-module to obtain image feature information of multiple different scales; the second decoding sub-operation includes an upsampling operation and a decoding operation.

[0131] Optionally, the semantic decoder module includes a detection head submodule, which includes multiple detectors; the loss functions corresponding to the multiple detectors include a first type of loss function and a second type of loss function; the first type of loss function is used to regress the positions of the predicted bounding box and the true bounding box of the target radar image; the second type of loss function is used to make the predicted category label corresponding to the target radar image the same as the true category label of the target radar image.

[0132] Optionally, the semantic decoder module is specifically used to:

[0133] Using the multiple detectors to detect the multiple image feature information of different sizes in sequence;

[0134] Performing a weighted summation on the function value of the first type of loss function and the function value of the second type of loss function to obtain a total loss function value corresponding to the semantic decoder module;

[0135] When the total loss function value is less than a preset value, the semantic feature information of the target radar image is determined.

[0136] Optionally, the image acquisition module is specifically used to:

[0137] Determine an original radar image corresponding to the target radar image in the radar system;

[0138] Obtaining a channel gain and Gaussian white noise corresponding to the wireless channel;

[0139] The target radar image is determined based on the original radar image, the channel gain and the Gaussian white noise.

[0140] The embodiment of the present invention further provides an electronic device 800, including:

[0141] at least one processor 801;

[0142] a memory 802 for storing the at least one processor executable instruction;

[0143] Among them, the at least one processor is configured to execute the instructions to implement the above-mentioned radar image decoding method based on the communication perception fusion system.

[0144] According to the technical solution provided by the embodiment of the present invention, the radar system and the communication perception fusion system can communicate, so that the radar system can send a radar image to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through a neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0145] It can be seen that the communication-perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, it realizes the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. Moreover, based on the semantic feature information of the radar image, the image recognition results and category labels of the radar image can be accurately predicted.

[0146] An embodiment of the present invention also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to execute the above-mentioned radar image decoding method based on the communication perception fusion system.

[0147] According to the technical solution provided by the embodiment of the present invention, the radar system and the communication perception fusion system can communicate, so that the radar system can send a radar image to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through a neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0148] It can be seen that the communication-perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, it realizes the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. Moreover, based on the semantic feature information of the radar image, the image recognition results and category labels of the radar image can be accurately predicted.

[0149] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned radar image decoding method based on the communication perception fusion system when executed by a processor.

[0150] According to the technical solution provided by the embodiment of the present invention, the radar system and the communication perception fusion system can communicate, so that the radar system can send a radar image to the communication perception fusion system, and the communication perception fusion system can extract image feature information of multiple different scales of the target radar image through a neural network channel decoder module, and the semantic decoder module can obtain the semantic feature information of the target radar image through the image feature information of multiple different scales, and then accurately obtain the image recognition result and category label based on the semantic feature information of the target radar image.

[0151] It can be seen that the communication-perception fusion system can integrate the radar image recognition process with the communication process of the communication system, that is, it realizes the integration of communication and perception, thereby saving computing resources and transmission resources, that is, improving the utilization rate of computing resources and transmission resources. Moreover, based on the semantic feature information of the radar image, the image recognition results and category labels of the radar image can be accurately predicted.

[0152] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A radar image decoding method based on a communication perception fusion system, applied to the communication perception fusion system, wherein the communication perception fusion system includes a neural network channel decoder module and a semantic decoder module, characterized in that: The method comprises: Acquire target radar images from the radar system via a wireless channel; Performing a first decoding operation on the target radar image by the neural network channel decoder module to obtain image feature information of multiple different sizes of the target radar image; Performing a second decoding operation on the plurality of image feature information of different sizes through the semantic decoder module to obtain semantic feature information of the target radar image; A detection result is obtained according to the semantic feature information, where the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

2. The method according to claim 1, characterized in that The neural network channel decoder module includes a redundancy submodule and a multi-scale fusion submodule; The performing a first decoding operation on the target radar image by the neural network channel decoder module to obtain image feature information of multiple different scales of the target radar image includes: Performing a first decoding sub-operation on the target radar image through the redundancy sub-module to obtain a first radar image; the first decoding operation is used to enhance feature information of the target radar image; The second decoding sub-operation is performed on the first radar image through the multi-scale fusion sub-module to obtain image feature information of multiple different scales; the second decoding sub-operation includes an upsampling operation and a decoding operation.

3. The method according to claim 1, characterized in that: The semantic decoder module includes a detection head submodule, which includes multiple detectors; the loss functions corresponding to the multiple detectors include a first type of loss function and a second type of loss function; the first type of loss function is used to regress the positions of the predicted bounding box and the true bounding box of the target radar image; the second type of loss function is used to make the predicted category label corresponding to the target radar image the same as the true category label of the target radar image.

4. The method according to claim 3, characterized in that The step of performing a second decoding operation on the plurality of image feature information of different sizes by the semantic decoder module to obtain the semantic feature information of the target radar image includes: Using the multiple detectors to detect the multiple image feature information of different sizes in sequence; Performing a weighted summation on the function value of the first type of loss function and the function value of the second type of loss function to obtain a total loss function value corresponding to the semantic decoder module; When the total loss function value is less than a preset value, the semantic feature information of the target radar image is determined.

5. The method according to any one of claims 1 to 4, characterized in that: The step of acquiring a target radar image from a radar system through a wireless channel comprises: Determine an original radar image corresponding to the target radar image in the radar system; Obtaining a channel gain and Gaussian white noise corresponding to the wireless channel; The target radar image is determined based on the original radar image, the channel gain and the Gaussian white noise.

6. A communication perception fusion system, characterized in that: include: Image acquisition module, neural network channel decoder module, semantic decoder module and prediction module; The image acquisition module is used to acquire a target radar image from the radar system through a wireless channel; The neural network channel decoder module is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image; The semantic decoder module is used to perform a second decoding operation on the plurality of image feature information of different sizes to obtain the semantic feature information of the target radar image; The prediction module is used to obtain a detection result according to the semantic feature information, and the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

7. A radar image decoding device based on a communication perception fusion system, applied to a communication perception fusion system, characterized in that: The device comprises: An image acquisition module, used for acquiring a target radar image from a radar system through a wireless channel; A neural network channel decoder module is used to perform a first decoding operation on the target radar image to obtain image feature information of multiple different sizes of the target radar image; A semantic decoder module, configured to perform a second decoding operation on the plurality of image feature information of different sizes to obtain semantic feature information of the target radar image; A prediction module is used to obtain a detection result according to the semantic feature information, wherein the detection result includes an image recognition result of the target radar image and / or a category label corresponding to the target radar image.

8. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

Images