Semantic feature coding method and device under intelligent network connection automobile detection task

Abstract

The invention belongs to the technical field of intelligent network connection automobiles, and provides a semantic feature coding method and device under an intelligent network connection automobile detection task. The method comprises the following steps: performing semantic feature extraction on an input image by adopting a weak supervision semantic feature extraction model; and coding each extracted semantic feature by using an adaptive semantic coding model to obtain a semantic code of the input image. According to the method, semantic feature extraction can be carried out on the product input image, and wireless channel transmission is carried out after channel coding, so that the transmission data volume can be greatly reduced, and the method can adapt to a low signal-to-noise ratio environment in an actual detection industrial scene; the semantic feature mining precision of the input image is improved, and the transmission adaptability of the semantic features in a complex channel scene is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent connected vehicles, and in particular to a semantic feature encoding method and device for an intelligent connected vehicle detection task. Background Art

[0002] Manufacturing is a pillar industry of the national economy, and its high-quality development is crucial to the construction of a modern economic system. With the development of the intelligent connected vehicle manufacturing industry and the intensification of competition, enterprises have increasing requirements for the automation and efficiency of production lines. Traditional manual inspection methods are time-consuming and labor-intensive, and are easily affected by subjective factors. The introduction of intelligent connected vehicle product inspection technology can achieve fast and accurate inspection and improve the automation level of the production line. However, the defects in vehicle products will have a direct impact on the quality of the product and the use of users. Therefore, strengthening its inspection capabilities and improving its manufacturing quality are of great significance to promoting the innovative development of my country's intelligent connected vehicle manufacturing industry. Vehicle product surface quality inspection is an important part of quality inspection. The visual quality inspection of vehicle products focuses on key technologies such as high-quality sensors, high-stability light sources, high-precision recognition models, and low-latency recognition to improve the accuracy and real-time recognition of vehicle product surface defects. With the continuous advancement of computer technology, especially the development of image processing and some deep learning algorithms, computers can better process and analyze image data. This provides strong support for the development of intelligent connected vehicle product inspection technology, enabling detection algorithms to more accurately identify and classify product defects.

[0003] The complex noise environment at industrial sites can cause serious distortion in the process of vehicle product image acquisition and transmission, greatly affecting the accuracy of product defect detection. In addition, the acquisition and transmission of a large number of images input into the recognition network takes up a lot of bandwidth resources.

[0004] However, most existing semantic communication transmission systems for specific tasks have adaptability problems when the channel changes, and ignore the impact of the configuration, layout and other characteristics of the inspection product on the probability of defect location, which leads to inaccurate semantic feature extraction and affects the accuracy of product quality inspection. Summary of the invention

[0005] In view of the defects in the prior art, the present invention provides a semantic feature encoding method and device for intelligent connected vehicle detection tasks, so as to reduce the amount of transmitted data, improve the accuracy of semantic feature mining of input images and the adaptability of semantic feature transmission in complex channel scenarios.

[0006] A semantic feature encoding method for an intelligent connected vehicle detection task, characterized by comprising:

[0007] A weakly supervised semantic feature extraction model is used to extract semantic features from the input image;

[0008] The extracted semantic features are encoded using an adaptive semantic coding model to obtain the semantic coding of the input image.

[0009] Furthermore, the weakly supervised semantic feature extraction model is trained using the weakly supervised training data.

[0010] Furthermore, weakly supervised training data is obtained in the following way:

[0011] Use a pre-trained convolutional neural network to extract features from the input image to obtain a first feature map;

[0012] Perform probability map sampling on the first feature map, and use the self-attention mechanism to extract features from the sampled first feature map to obtain a second feature map;

[0013] Sort the second feature map by importance to obtain the feature importance probability score of each channel;

[0014] Multiply the feature importance probability score of each channel by the first feature map element by element to obtain a third feature map;

[0015] Perform channel summation on the third feature map to generate a feature importance map;

[0016] Convert the feature importance map into a binary weak label map;

[0017] The weak label map is paired with the input image to form weakly supervised training data.

[0018] Furthermore, probability map sampling includes:

[0019] Use object detector to detect the input image and get the product category;

[0020] According to the product category and the prior probability of defective parts of the predefined product category, the posterior probability of defective parts of the product category is obtained;

[0021] The first feature map is sampled according to the posterior probability of the defective part of the product category to obtain the sampled first feature map.

[0022] Furthermore, the second feature map is sorted by importance to obtain the feature importance probability score of each channel, including:

[0023] Perform global average pooling on the second feature map to obtain the average value of the feature map of each channel;

[0024] Use the activation function ReLU to map the average value of the feature map of each channel to the non-negative space to obtain the mapped channel feature map;

[0025] Normalizing the mapped channel feature map to obtain a normalized channel feature map;

[0026] The softmax function is used to calculate the probability distribution of the normalized channel feature map pixel values ​​to obtain the feature importance probability score of each channel.

[0027] Furthermore, the semantic features include defect location semantic features, image global semantic features and channel statistical semantic features.

[0028] Furthermore, the adaptive semantic coding model is obtained in the following way:

[0029] The semantic features of the defect part, the global semantic features of the image and the statistical semantic features of the channel are fused and reconstructed to obtain the first multimodal features;

[0030] Adding a signal-to-noise ratio condition, compressing the first multimodal feature, and obtaining a second multimodal feature;

[0031] The second multimodal feature is encoded using a channel encoder to obtain a semantic code of the input image.

[0032] Furthermore, the semantic features of the defect part, the global semantic features of the image and the statistical semantic features of the channel are fused and reconstructed to obtain the first multimodal features, including:

[0033] The semantic features of the defect part, the global semantic features of the image and the statistical semantic features of the channel are weighted to obtain the weighted RGB features;

[0034] After aligning the weighted RGB features with the global semantic features of the image, convolution fusion is performed to obtain the convolution fusion features;

[0035] The residual connection operation is used to fuse the RGB features with the convolutional fusion features and reconstruct the details to obtain the first multimodal features.

[0036] Furthermore, a signal-to-noise ratio condition is added to compress the first multimodal feature to obtain a second multimodal feature, including:

[0037] Performing a pooling operation on the first multimodal feature using a global average pooling function;

[0038] The pooled first multimodal features are added with different signal-to-noise ratio conditions to obtain contextual input information based on the signal-to-noise ratio;

[0039] The context information is fed into a fully connected neural network to generate a scaling factor, and the generated scaling factor is multiplied by the first multimodal feature to obtain the first multimodal feature with different proportional weights;

[0040] The second multimodal features are obtained by sorting according to the proportional weights.

[0041] An adaptive semantic feature encoding device for intelligent connected vehicle detection tasks includes an extraction unit and an encoding unit, wherein:

[0042] An extraction unit, used for extracting semantic features from an input image using a weakly supervised semantic feature extraction model;

[0043] The encoding unit is used to encode the extracted semantic features using an adaptive semantic encoding model to obtain a semantic encoding of the input image.

[0044] Beneficial effects of the present invention: To address the problem of image transmission distortion caused by large image noise and poor channel quality in vehicle product visual quality inspection scenarios, which affects quality inspection accuracy, the present invention innovatively adopts a semantic communication architecture, integrates quality inspection task characteristics and product layout knowledge, and proposes a weakly supervised semantic feature extraction method to estimate the parts of the input image that are prone to defects. Furthermore, a channel-adaptive semantic feature encoding method is designed, which not only reduces the amount of transmitted data, but also improves the semantic feature mining accuracy of the input image and the adaptability of semantic feature transmission in complex channel scenarios by fusing multiple channel model statistical distribution characteristics and combining the perceived real-time signal-to-noise ratio status. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific embodiments of the present invention, the following will briefly introduce the drawings required for the specific embodiments or the prior art description. In all the drawings, each element or part is not necessarily drawn according to the actual scale.

[0046] Figure 1 A flowchart of a semantic feature encoding method for a smart connected vehicle detection task provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the structure of a semantic feature encoding device for an intelligent connected vehicle detection task provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0049] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0050] In one embodiment, the present invention provides a semantic feature coding method for intelligent connected vehicle inspection tasks, which can extract semantic features from product input images and transmit them through wireless channels after channel coding, which can not only greatly reduce the amount of transmitted data, but also adapt to the low signal-to-noise ratio environment in actual inspection industrial scenes. After the product input image is extracted through semantic features and transmitted through channel coding, it can be input into the quality inspection model to determine whether the vehicle product image has defects.

[0051] A semantic feature encoding method for an intelligent connected vehicle detection task, comprising:

[0052] 1. Use a weakly supervised semantic feature extraction model to extract semantic features from the input image;

[0053] In one embodiment, the weakly supervised semantic feature extraction model is trained using weakly supervised training data.

[0054] Specifically, weakly supervised training data is obtained in the following way:

[0055] (1) Using a pre-trained convolutional neural network to extract features from an input image to obtain a first feature map;

[0056] (2) performing probability map sampling on the first feature map, and using a self-attention mechanism to extract features from the sampled first feature map to obtain a second feature map;

[0057] The self-attention mechanism is used to further extract features from the first feature map after sampling. Each pixel (or position) of the first feature map after sampling is compared with all other pixels to evaluate the degree of correlation between them, which can better capture global information. In this way, the weakly supervised semantic feature extraction model can identify which areas are most important for understanding the entire input image.

[0058] In one embodiment, probability map sampling includes:

[0059] ① Use object detector to detect the input image and get the product category;

[0060] ②According to the product category and the prior probability of defective parts of the predefined product category, the posterior probability of defective parts of the product category is obtained;

[0061] The prior probability of defective parts is the probability distribution of defective parts of product categories based on existing knowledge, experience or assumptions, rather than relying on currently observed data or evidence. Specifically, it refers to the possibility of occurrence of specific defect categories (including dents, scratches, cracks, etc.), insertion positions (referring to the specific positions of defective parts in the input image) or scales (referring to the area occupied by defective parts in the entire input image) in product defect detection.

[0062] For example, if we know that a certain type of automotive product is more likely to have defects in a specific area, then the prior probability of defects in this specific area will be higher; the posterior probability is the probability distribution updated in combination with the actual observed data. By observing the specific features in the input image, the initial prior probability can be updated to obtain a more accurate posterior probability. Using the Bayesian theorem method, the prior probability is updated in combination with the actual observed data to obtain the posterior probability.

[0063] According to the input image I, the category c, insertion position p and scale s of the defect part on the input image I are estimated a priori.

[0064] Category refers to defects such as dents, scratches, cracks, etc.; insertion position refers to the specific location of the defect in the input image, usually represented by a coordinate, such as (x, y); scale refers to the area occupied by the defect in the entire input image.

[0065] The product prior probability o is defined as a vector o={p, s, c} composed of the parameters of the defective part to be detected (category c, insertion position p and scale s).

[0066] Input image I and product category label of input image is c fg , initialize the defective part posterior probability P(oI,c fg ), perform image feature map sampling with a priori probability on the first feature map, i.e., P(o|O′).

[0067] Assuming that the detected defective parts and the defective parts to be detected are independent of each other, the posterior probability P(o|O′) of the defective parts of the product category of the input image can be decomposed into:

[0068] P(oO′)=P(p,s,cO′)=P(cO′)P(p,sO′,c)=P(cO′)P(pO′,c)P(sO′,c,p)

[0069] ③ Sampling the first feature map according to the posterior probability of the defective parts of the product category to obtain the sampled first feature map.

[0070] ④ Use the self-attention mechanism to extract features from the first feature map after sampling to obtain the second feature map

[0071] (3) Sort the second feature map by importance to obtain the feature importance probability score of each channel;

[0072] In one embodiment, obtaining a feature importance probability score for each channel includes:

[0073] ① Perform global average pooling on the second feature map to obtain the average value of the feature map of each channel;

[0074] ② Use the activation function ReLU to map the average value of the feature map of each channel to the non-negative space to obtain the mapped channel feature map;

[0075] In one embodiment, the position characteristics of the mixed Gaussian distribution can also be used to constrain the positions of the defective parts to be detected and the defective parts of the detected defective parts. Assume that the center of the c-type defect is relative to the defect to be detected The posterior probability of the position satisfies the bimodal mixed Gaussian distribution. Since the natural image is still real after being flipped left and right, the relative position probability distribution of the C-type defect part and the defect part to be detected usually has the characteristic of left-right symmetry with the reference (detected defect part) object as the axis, further constraining the mixed Gaussian distribution about the defect part to be detected Bilateral symmetry, expressed as:

[0076]

[0077] In the formula, F2 is the normalization factor, and is the mean of two left-right symmetrical Gaussian components, is the corresponding covariance matrix, B is the image background area, N a (,) is Gaussian distribution.

[0078] ③ Normalize the mapped channel feature map to obtain a normalized channel feature map;

[0079] ④ Use the softmax function to calculate the probability distribution of the normalized channel feature map pixel values ​​and obtain the feature importance probability score of each channel, which indicates the importance of the channel in defect detection.

[0080] (4) multiplying the feature importance probability score of each channel by the first feature map element by element to obtain a third feature map;

[0081] (5) Channel summation is performed on the third feature map to generate a feature importance map; based on the feature importance map, the defect-related areas in the input image can be located, so that the weakly supervised semantic feature extraction model pays more attention to the areas where defects may appear in the input image.

[0082] (6) Convert the feature importance map into a binary weak label map;

[0083] (7) Pair the weak label map with the input image to form weakly supervised training data. Determine the defect area and the normal area based on the pixel values ​​of the feature importance map, and convert the feature importance map into a binary weak label map. Use the generated weak label map to pair it with the input image to form weakly supervised training data.

[0084] In one embodiment, the feature importance region and the defect part can be matched in the input image by minimizing the cross entropy loss function, and the feature importance region and the defect part can be associated, thereby improving the accuracy of defect detection. Specifically, it is expressed as:

[0085]

[0086] Where N is the number of pixels in the input image, and y i is the true label of the i-th pixel (the label of the defective part), p i is the predicted probability of the ith pixel (the value of the feature importance map).

[0087] In the initial stage, the pre-trained weakly supervised semantic feature extraction model parameters are used for training to improve the convergence speed and stability of the model.

[0088] Subsequently, the acquired weakly supervised training data is gradually introduced, and the weakly supervised semantic feature extraction model is continuously optimized through iterative training, thereby completing the training optimization of the semantic feature extraction module of the input image.

[0089] 2. Encode the extracted semantic features using an adaptive semantic coding model to obtain the semantic coding of the input image.

[0090] In one of the embodiments, the semantic features include defect location semantic features, image global semantic features, and channel statistical semantic features.

[0091] In one embodiment, the adaptive semantic coding model is obtained in the following manner:

[0092] (1) fusing and reconstructing the semantic features of the defect part, the global semantic features of the image, and the statistical semantic features of the channel to obtain the first multimodal features;

[0093] In one embodiment, obtaining the first multimodal feature includes:

[0094] ①Weight the semantic features of the defect part, the global semantic features of the image, and the statistical semantic features of the channel to obtain the weighted RGB features;

[0095] Channel statistical semantic features mainly refer to the statistical characteristics of the channel, which describe the behavior of the channel when transmitting information.

[0096] Combined with the semantic features of the defect part F p , image global semantic feature F g , channel statistical semantic features C a Get the structural attention score A t .

[0098] From the image global semantic feature F g The values ​​of all spatial positions are averaged to obtain a statistic for each channel, and the statistical information of the channel is converted into a channel statistical semantic feature C a ; Channel statistical semantic features C a and the global semantic feature F of the image g and the semantic feature of the defect part F p The cascaded result is the attention score At.

[0099] Image global semantic feature F g These are color features extracted from the entire input image, which reflect the overall color distribution and properties of the input image.

[0100] RGB feature F i It refers to the semantic feature of the defect part F p , image global semantic feature F g The RGB features of each pixel.

[0101] RGB feature F i Combined with attention A t Generate weighted RGB features F wi It can be expressed as:

[0102]

[0103] in, and represents element-wise multiplication and concatenation operations, f E Indicates the expansion operation, so that F g With F p Same spatial dimension.

[0104] The weighted RGB features mainly suppress the weights of unimportant features in the input image and increase the weights of important features. They can also make the adaptive semantic coding model more robust to local noise or occlusion.

[0105] ② After aligning the weighted RGB features with the global semantic features of the image, perform convolution fusion to obtain the convolution fusion features;

[0106] ③ Using the residual connection operation, the RGB features are fused with the convolutional fusion features to reconstruct the details and obtain the first multimodal features.

[0107] Further weighted RGB feature F wi and the semantic feature of the defect part F p After alignment, convolution fusion is performed; then, the weighted RGB features are re-added to fuse and reconstruct details; finally, after residual connection, the first multimodal feature F m It can be expressed as:

[0108]

[0109] where f RES is the residual connection operation.

[0110] In one embodiment, the first multimodal feature F m and support mask y s Generate the basic foreground prototype p through mask average pooling operation b , which can be expressed as:

[0111]

[0112] Here, i[·] represents the indicator function, (x, y) represents the spatial position of the pixel, and c represents the product defect category.

[0113] (2) adding a signal-to-noise ratio condition to compress the first multimodal feature to obtain a second multimodal feature;

[0114] In one embodiment, obtaining the second multimodal feature includes:

[0115] ①Use the global average pooling function to perform pooling operation on the first multimodal feature;

[0116] ② Add different signal-to-noise ratio conditions to the pooled first multimodal features to obtain contextual input information based on the signal-to-noise ratio;

[0117] ③ Sending the context information into a fully connected neural network to generate a scaling factor, multiplying the generated scaling factor by the first multimodal feature to obtain the first multimodal feature with different proportional weights;

[0118] The fully connected neural network mainly consists of two fully connected layers.

[0119] ④ Sort by proportional weight to obtain the second multimodal feature.

[0120] The first multimodal features with the highest proportion weights are selected, and the remaining weakly correlated first multimodal features are compressed to obtain the second multimodal features.

[0121] In order to adapt to the complex channel environment at the vehicle quality inspection site and solve the problem of excessive consumption of computing resources caused by multiple training of appropriate signal-to-noise ratio conditional models, a soft attention mechanism is added to solve the cross-modal fusion problem of semantic features of defect parts, global semantic features of images and channel statistical semantics under different signal-to-noise ratios.

[0122] (3) A channel encoder is used to encode the second multimodal features to obtain a semantic code of the input image.

[0123] The second multimodal feature is encoded by a channel encoder, and an additive white Gaussian noise (AWGN) channel is considered. The AWGN channel is an independent and identically distributed Gaussian white noise channel, and the noise samples are independent of each other. This independence makes channel analysis more convenient and more reliable.

[0124] The compressed second multimodal features are transmitted through the wireless channel and corrupted by additive white Gaussian noise, and then received by the receiver. The channel features and image semantic features are cross-modally fused through the soft attention mechanism, so that the extracted semantic features can adapt to the dynamically changing actual channel conditions, thereby improving the reliability of the transmission of the semantic features of the input image.

[0125] After the receiver receives the signal, it is input into the fully connected classifier to map the signal to a predefined defect category. Then, the Softmax function is used to convert the output of the fully connected layer into a probability distribution, indicating the probability of each image being judged as a defect category, thereby completing the intelligent connected vehicle product inspection task.

[0126] Aiming at the problem that image transmission distortion caused by large image noise and poor channel quality affects the quality inspection accuracy in vehicle product visual quality inspection scenarios, the present invention innovatively adopts a semantic communication architecture, integrates quality inspection task characteristics and product layout knowledge, proposes a weakly supervised semantic feature extraction method, estimates the parts of the input image that are prone to defects, and then designs a channel-adaptive semantic feature encoding method. By fusing the statistical distribution characteristics of multiple channel models and combining the perceived real-time signal-to-noise ratio status, the present invention solves the problems of low accuracy in semantic feature mining of input images and poor adaptability to complex channel scenarios.

[0127] In one embodiment, if Figure 2 As shown, an adaptive semantic feature encoding device for intelligent connected vehicle detection tasks is also provided, including an extraction unit and an encoding unit, wherein:

[0128] An extraction unit, used for extracting semantic features from an input image using a weakly supervised semantic feature extraction model;

[0129] The encoding unit is used to encode the extracted semantic features using an adaptive semantic encoding model to obtain a semantic encoding of the input image.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0131] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A semantic feature encoding method for intelligent connected vehicle detection tasks, characterized in that: include: A weakly supervised semantic feature extraction model is used to extract semantic features from the input image; The extracted semantic features are encoded using an adaptive semantic coding model to obtain a semantic coding of the input image.

2. The semantic feature encoding method for intelligent connected vehicle detection task according to claim 1, characterized in that: The weakly supervised semantic feature extraction model is trained using weakly supervised training data.

3. The semantic feature encoding method for a smart connected vehicle detection task as claimed in claim 2, characterized in that: The weakly supervised training data is obtained in the following manner: Use a pre-trained convolutional neural network to extract features from the input image to obtain a first feature map; Performing probability map sampling on the first feature map, and performing feature extraction on the sampled first feature map using a self-attention mechanism to obtain a second feature map; Sorting the second feature maps by importance to obtain a feature importance probability score for each channel; Multiplying the feature importance probability score of each channel by the first feature map element by element to obtain a third feature map; Performing channel summation on the third feature map to generate a feature importance map; Converting the feature importance map into a binary weak label map; The weak label map is paired with the input image to form weakly supervised training data.

4. The semantic feature encoding method for a smart connected vehicle detection task as claimed in claim 3, characterized in that: The probability map sampling includes: Use object detector to detect the input image and get the product category; Obtaining a posterior probability of defective parts of the product category according to the product category and a predefined prior probability of defective parts of the product category; The first feature map is sampled according to the posterior probability of the defective part of the product category to obtain the sampled first feature map.

5. The semantic feature encoding method for a smart connected vehicle detection task as claimed in claim 3, characterized in that: The importance sorting of the second feature map to obtain the feature importance probability score of each channel includes: Performing global average pooling on the second feature map to obtain an average value of the feature map of each channel; Use the activation function ReLU to map the average value of the feature map of each channel to a non-negative space to obtain a mapped channel feature map; Normalizing the mapped channel feature map to obtain a normalized channel feature map; The softmax function is used to calculate the probability distribution of the pixel values ​​of the normalized channel feature map to obtain the feature importance probability score of each channel.

6. The semantic feature encoding method for a smart connected vehicle detection task according to claim 1, characterized in that: The semantic features include defect location semantic features, image global semantic features and channel statistical semantic features.

7. The semantic feature encoding method for intelligent connected vehicle detection task according to claim 1, characterized in that: The adaptive semantic coding model is obtained in the following way: fusing and reconstructing the semantic features of the defect part, the global semantic features of the image, and the statistical semantic features of the channel to obtain a first multimodal feature; Adding a signal-to-noise ratio condition, compressing the first multimodal feature to obtain a second multimodal feature; The second multimodal feature is encoded using a channel encoder to obtain a semantic code of the input image.

8. The semantic feature encoding method for a smart connected vehicle detection task as claimed in claim 7, characterized in that: The fusing and reconstructing the semantic features of the defect part, the global semantic features of the image and the statistical semantic features of the channel to obtain the first multimodal features includes: Weighting the semantic features of the defect part, the global semantic features of the image and the statistical semantic features of the channel to obtain a weighted RGB feature; After aligning the weighted RGB features with the global semantic features of the image, performing convolution fusion to obtain convolution fusion features; A residual connection operation is adopted to fuse the RGB features with the convolutional fusion features and reconstruct the details to obtain the first multimodal features.

9. The semantic feature encoding method for a smart connected vehicle detection task according to claim 7, wherein the adding of a signal-to-noise ratio condition to compress the first multimodal feature to obtain the second multimodal feature comprises: Performing a pooling operation on the first multimodal feature using a global average pooling function; The pooled first multimodal features are added with different signal-to-noise ratio conditions to obtain contextual input information based on the signal-to-noise ratio; Sending the context information to a fully connected neural network to generate a scaling factor, and multiplying the generated scaling factor by the first multimodal feature to obtain first multimodal features with different proportional weights; The second multimodal features are obtained by sorting according to the proportional weights.

10. An adaptive semantic feature encoding device for intelligent connected vehicle detection tasks, characterized in that: It includes an extraction unit and an encoding unit, wherein: An extraction unit, used for extracting semantic features from an input image using a weakly supervised semantic feature extraction model; The encoding unit is used to encode the extracted semantic features using an adaptive semantic encoding model to obtain a semantic encoding of the input image.

Images