Visual prompt generation method, image recognition method, device and system for intended functional safety

Abstract

The application discloses a visual prompt generation method, an image recognition method, a device and a system for expected functional safety, the visual prompt generation method comprising: obtaining a feature vector from an image, performing layer normalization and dimension elevation projection on the feature vector to obtain a high-dimensional feature in a high-dimensional feature space; performing reverse compression on the high-dimensional feature through a preset dimension reduction mapping function to generate a reconstructed reference feature; calculating a weighted difference value between the high-dimensional feature and the reconstructed reference feature to obtain a residual error; and mapping the residual error back to the high-dimensional feature space and performing weighted fusion with the high-dimensional feature to obtain a visual prompt feature. The application can improve expected functional safety and enhance the robustness of a perception system.

Description

Technical Field

[0001] This invention relates to the fields of intended functional safety and computer vision, and in particular to a method, image recognition method, apparatus and system for generating visual cues for intended functional safety. Background Technology

[0002] With the rapid development of autonomous driving technology, deep neural network-based visual perception systems have become a core component of environmental understanding. However, in practical applications, especially in scenarios with poor lighting conditions such as low light, nighttime, and tunnel entrances / exits, the performance of such systems faces severe challenges. From the perspective of expected functional safety (SOTIF, ISO 21448), this performance degradation caused by environmental factors is not due to system hardware failure, but rather a typical performance limitation scenario. In such scenarios, the image signal-to-noise ratio is significantly reduced, and key structural information such as the texture and edges of the target is easily submerged by background noise, making it difficult for deep neural networks to extract effective discriminative features. This directly leads to problems such as target feature loss and a sharp drop in detection accuracy, potentially causing autonomous driving perception systems to fall into known or unknown unsafe scenarios, seriously threatening driving safety.

[0003] To address the aforementioned issues, existing technologies primarily employ data augmentation, image preprocessing, or end-to-end low-light enhancement networks to improve input image quality. In recent years, visual cue learning methods have been proposed as a lightweight solution. The core idea is to overlay a set of learnable cue vectors onto the network input to guide the model to focus on and enhance target features in low-light conditions. However, under the SOTIF framework, this approach suffers from the following shortcomings.

[0004] Feature dilution risk: During the forward propagation of the signal in the network, it undergoes multiple layers of convolution and nonlinear transformations. The structural information injected in the shallow layers is smoothed and diluted layer by layer in the deep network. This results in the enhanced features not being effectively transmitted to the end of the network responsible for high-level semantic understanding, leaving the perception system functionally insufficient in critical decision-making stages (such as object classification and trajectory prediction), failing to fundamentally solve the performance limitation problem.

[0005] Noise amplification risk: Existing cue learning methods lack the ability to effectively distinguish between the true structure of the target and background noise. In the process of amplifying weak signals, inherent random interference such as shot noise and readout noise in the sensor can easily be amplified simultaneously, introducing false responses into the feature map. This not only increases the false detection rate of the system but also reduces the confidence level of the perception results, violating the fundamental requirements of expected functional safety for predictable and robust system behavior.

[0006] In summary, existing technologies suffer from the problem that deep feature dilution leads to persistent functional deficiencies in perception systems in low-light scenarios, failing to meet real-time and accuracy requirements. Furthermore, they lack effective cross-layer structure preservation and noise suppression mechanisms, failing to meet the SOTIF standard's requirements for robustness and security of perception systems.

[0007] The disclosure of the above background technical content is only for the purpose of assisting in understanding the concept and technical solution of this application, and does not necessarily provide technical instruction. Summary of the Invention

[0008] The purpose of this invention is to provide a visual cue generation method, image recognition method, apparatus, and system for expected functional safety. This invention can improve expected functional safety and enhance the robustness of the perception system.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for generating visual cues, characterized by comprising the following steps: For the feature vector obtained from the image Layer normalization and updimensional projection are performed on the feature to obtain high-dimensional features in the high-dimensional feature space. ; The high-dimensional features are mapped using a preset dimensionality reduction function. Perform reverse compression to generate reconstructed baseline features ; Calculate the high-dimensional features and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. ; The residual Mapping back to the high-dimensional feature space and with the high-dimensional features Perform weighted fusion to obtain visual cue features .

[0010] Furthermore, following any one or a combination of the aforementioned technical solutions, the residual... and visual cue features The calculation formula is as follows: ; in, This is the first balance coefficient.

[0011] Furthermore, based on any one or a combination of the aforementioned technical solutions, visual cueing features The calculation formula is as follows: ; in, This is the second balance coefficient. This is the projection function that maps the residuals back to the high-dimensional feature space.

[0012] Furthermore, based on any one or a combination of the aforementioned technical solutions, the high-dimensional features The calculation formula is as follows:

[0013] in, This represents the dimension-scaling function, and Norm represents the normalization operation. For the generated high-dimensional features.

[0014] Furthermore, based on any one or a combination of the aforementioned technical solutions, the baseline features are reconstructed. The calculation formula is:

[0015] in, represents the dimension reduction mapping function, and Norm represents the normalization operation.

[0016] Furthermore, following any one or a combination of the aforementioned technical solutions, the generation of the visual cue features is further constrained in the following ways: Pre-generated feature vectors The upper-level network obtains shallow features For the shallow features Perform convolution operations to obtain shallow anchor point features. ; The visual cue features and the shallow anchor point features are calculated using a preset structural consistency loss function. If the deviation between the two values ​​is less than a preset threshold, then the visual cue feature is output.

[0017] Furthermore, based on any one or a combination of the aforementioned technical solutions, the structural consistency loss function The calculation formula is as follows: Where N is the total number of feature points in the image, and j is an integer from 1 to N. This represents the L2 norm.

[0018] Furthermore, following any one or a combination of the aforementioned technical solutions, the shallow features are processed using a 1×1 convolution. Perform spatial alignment operations to obtain the shallow anchor point features. .

[0019] Furthermore, following any one or a combination of the aforementioned technical solutions, the visual cue features are obtained using a preset visual cue generation module, including the following steps: The image is input into a preset neural network model to extract feature vectors from the image using the neural network model. , the feature vector Input the visual cue generation module to output the visual cue features; The visual cue features include a nonlinear expansion unit, an inverse reconstruction unit, a residual module, and a mapping module; wherein, the nonlinear expansion unit is configured to modify the feature vector. Perform layer normalization and up-dimensional projection to obtain high-dimensional features. ; The reverse reconstruction unit is configured to perform a dimensionality reduction mapping function on the high-dimensional features. Perform reverse compression to generate reconstructed baseline features ; The residual module is configured to compute the high-dimensional features. and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. ; The mapping module is configured to map the residual Mapping back to the high-dimensional feature space and with the high-dimensional features Perform weighted fusion to obtain visual cue features .

[0020] Furthermore, based on any one or a combination of the aforementioned technical solutions, the visual cue generation module is trained in the following manner: Obtain a training set, which includes multiple images with real labels; Select a neural network model, input the training set into the neural network model, use the preceding layers of the neural network model to output shallow features, and use the subsequent layers of the neural network model to output deep features. Construct a basic module, which includes the nonlinear extension unit, the inverse reconstruction unit, the residual module, and the mapping module; The shallow features and the deep features are input into the base module and the base module is trained as follows: the visual cue generation module is trained using a preset structural consistency loss function until the structural consistency loss function converges. The structural consistency loss function is related to the deviation of the visual cue features predicted by the base module relative to the shallow features. The trained base module is then used as the visual cue generation module.

[0021] According to another aspect of the present invention, an image recognition method is provided for identifying target feature information from an image, comprising the following steps: The image to be identified is acquired and input into a pre-built image recognition model to output target feature information; The image recognition model is constructed in the following way: Determine a neural network model suitable for image recognition, and a basic module for generating visual cue features; Obtain a training set, which includes multiple images with real labels; Inputting the training set into the neural network model for model training includes: the prior network layers of the neural network model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. ; The shallow features and the feature vector The input is the basic module, which predicts visual cue features based on the visual cue generation method described above, or a combination of the above technical solutions. The neural network model is trained using a preset total loss function until the total loss function converges. The trained neural network model is then used as the image recognition model. The total loss function is related to the deviation of the visual cue features predicted by the base module relative to the shallow features, as well as the deviation between the prediction results of the neural network model and the true labels.

[0022] According to another aspect of the present invention, an image recognition method is provided for identifying target feature information from an image, comprising the following steps: The image to be identified is acquired and input into a pre-built image recognition model to output target feature information; The image recognition model is constructed in the following way: Identify a basic image recognition model that is suitable for image recognition and is pre-trained, as well as a basic module for generating visual cue features; Obtain a training set, which includes multiple images with real labels; With the parameters of the basic image recognition model fixed, the training set is input into the basic image recognition model for model training, including: the prior network layers of the basic image recognition model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. ; The shallow features and the feature vector The input is the basic module, which predicts visual cue features based on the visual cue generation method described above, or a combination of the above technical solutions. The basic image recognition model is trained using a preset structural consistency loss function until the structural consistency loss function converges. The structural consistency loss function is related to the deviation of the visual cue features predicted by the basic module relative to the shallow features. The trained basic image recognition model is then used as the image recognition model.

[0023] Furthermore, based on any one or a combination of the aforementioned technical solutions, the expression for the structural consistency loss function is as follows: ;in, Based on the total number of feature points in the image, This represents the L2 norm.

[0024] According to another aspect of the present invention, an image recognition apparatus is provided, including a processor configured to execute the image recognition method described in any one of the above technical solutions or a combination of multiple technical solutions to obtain target feature information in an image to be recognized.

[0025] According to another aspect of the present invention, an intelligent driving system is provided, characterized in that it includes an image recognition device as described in any one or a combination of the above technical solutions.

[0026] The beneficial effects of the technical solution provided by this invention are as follows: a. Enhance the expected functional safety: Ensure that the deep features of the perception system retain the target structure in low-light scenarios, reducing the risk of missed detections due to feature dilution; b. Enhance the robustness of the perception system: effectively suppress random noise, reduce false alarms, and support SOTIF coverage of known unsafe scenarios; c. Plug and play: It can be embedded into mainstream backbone networks, making it easy to implement SOTIF enhancements in existing autonomous driving perception architectures. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 An architecture diagram of a visual prompting module provided as an exemplary embodiment of the present invention; Figure 2 A flowchart illustrating the training method for existing image recognition models; Figure 3 A flowchart illustrating a training method for an image recognition model provided as an exemplary embodiment of the present invention; Figure 4 This is a flowchart illustrating how existing technologies utilize image recognition models to identify target feature information in images. Figure 5 A flowchart illustrating the identification of target feature information in an image using an image recognition model and a visual cueing module, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0031] To address the shortcomings of existing technologies, this application aims to provide a visual feature enhancement method that can effectively maintain the geometric integrity of deep features in low-light environments, while accurately purifying target signals and suppressing random noise, thereby improving the expected functional safety of autonomous driving perception systems in performance-limited scenarios.

[0032] In one embodiment of the present invention, a method for generating cross-layer feature-aligned visual cues for expected functional safety is provided, applicable to autonomous driving systems. See [link to relevant documentation]. Figure 1 The method includes the following steps: For obtaining feature vectors from images The execution layer normalizes and performs updimensional projection to project it onto a high-dimensional feature space to obtain high-dimensional features. ; The high-dimensional features are mapped using a preset dimensionality reduction function. Perform reverse compression to generate reconstructed baseline features ; Calculate the high-dimensional features and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. ; The residual Mapping back to the high-dimensional feature space and with the high-dimensional features Weighted fusion is performed to obtain visual cue features. These visual cue features serve as visual cue signals to enable the neural network to perceive target structural information from the image, thereby allowing the neural network to more accurately identify target objects, especially for images in low light or low illumination.

[0033] Compared to existing technologies that extract feature vectors from acquired images using pre-set neural networks... And based on feature vectors To obtain target structure information in an image, this method is based on the feature vector. Before obtaining the target structure information, the feature vector is first used as a basis. The visual cue features are obtained by aligning visual features / signals across layers, and then combined with feature vectors. Obtain high-dimensional features Extracting target structure information from images can reduce the risk of perception failure due to feature decay and improve the sensitivity and robustness of neural networks in perceiving target structure information in images.

[0034] In one embodiment of the present invention, the visual cue generation method for cross-layer feature alignment oriented towards expected functional safety specifically includes the following steps.

[0035] Step S1, Nonlinear Signal Expansion: First, obtain the feature vector extracted from the image by the neural network. The neural network can use ViT, ResNet, or other detection / tracking model backbones. In a specific application, the feature vector... The output of the feature encoder at the (i-1)th layer of the baseline deep neural network, where i is an integer not less than 2, is then processed by a preset nonlinear expansion unit to transform the feature vector. Projecting onto a high-dimensional feature space to activate latent weak signals yields the high-dimensional features. .

[0036] In this context, the high-dimensional feature space refers to the feature representation space formed in the reciprocal residual multilayer perceptron architecture by mapping the input features from D dimensions to a 2D intermediate latent representation through a channel-extended linear layer. This intermediate high dimension is used to enhance the linear separability of features and the response of latent target structures in low-light images. After GELU activation and channel compression, the output features are restored to D dimensions consistent with the input features, enabling fusion with subsequent network layers.

[0037] The high-dimensional features The calculation formula is as follows:

[0038] in, Norm represents the dimension-scaling mapping function (e.g., composed of a linear layer and the GELU activation function), and Norm represents the normalization operation. For the generated high-dimensional features.

[0039] This step aims to amplify the linear separability of features in a high-dimensional space. By increasing the dimensionality, the weak target structure signal is decoupled from the mixed background noise in the feature space, providing redundancy for subsequent denoising and purification.

[0040] Step S2, Inverse Reconstruction and Benchmark Acquisition: Inverse compression is performed on the high-dimensional features to remove noise signals and generate reconstructed benchmark features. Because noise signals lack structural support, they are smoothed during compression.

[0041] The reconstructed benchmark features The calculation formula is:

[0042] in, represents the dimension reduction mapping function, and Norm represents the normalization operation.

[0043] Optionally, the dimensionality reduction mapping function is composed of a layer normalization layer, a channel expansion linear layer, a GELU activation layer, and a channel compression linear layer connected in sequence. The layer normalization layer is used to normalize the input high-dimensional features, the channel expansion linear layer is used to expand the channel dimension of the normalized features from D to 2D, the GELU activation layer is used to perform non-linear activation on the expanded features, and the channel compression linear layer is used to compress the channel dimension of the activated features from 2D back to D to generate reconstructed baseline features.

[0044] This step simulates a signal loss process. Because random noise lacks semantic structure support, it becomes naturally smooth in the nonlinear mapping of compression-reconstruction due to the inability to obtain neighborhood structural relationships. However, target information with physical structure can be partially preserved, thus generating the reconstruction baseline features. This serves as a denoising reference benchmark containing only the basic skeleton.

[0045] Step S3, Weighted Difference and Visual Cue Generation: Calculate the weighted difference between the reconstructed state and the expanded state, and extract the indelible stubborn structural information as the residual. And ultimately generate visual cue features. The residual and visual cue features The calculation formula is as follows: ; ; in, The first balance coefficient, This is the second balance coefficient. This is a projection function that maps the residuals back to the cue space, i.e., the aforementioned high-dimensional feature space. It can be seen that the output visual cue features are formed by fusing the original high-frequency response with the purified structural residuals.

[0046] Optionally, the projection function consists of a layer normalization layer, a channel expansion linear layer, a GELU activation layer, and a channel compression linear layer; wherein, the channel expansion linear layer is used to expand the channel dimension of the back-projected residual from D to 2D, and the channel compression linear layer is used to compress the channel dimension of the residual feature after GELU activation from 2D back to D, so as to generate residual cue components.

[0047] Unlike existing technologies that simply superimpose feature vectors extracted from images, this application employs a specific structural sieve algebraic architecture, namely subtraction logic. This precisely extracts the target edge information (i.e., "residuals") that cannot be erased in step S2. In this way, the cross-layer feature alignment visual cue generation method provided in this application, when applied to an intelligent driving system, can effectively suppress false responses caused by interference from streetlights and reflections at night, cutting off the propagation path of noise from the underlying algorithm. The resulting... With an extremely high signal-to-noise ratio, it ensures the decision confidence of the perception system under low-light conditions, directly compensating for the limitations of perception performance in the intended functional safety (SOTIF).

[0048] In one embodiment of the present invention, the visual cue feature is generated by a pre-constructed visual cue generation module and a back projection difference generation module, and the feature vector is obtained from the original deep feature signal of the image. Input a pre-built bidirectional projection differential generation module to output a prompt signal .

[0049] The visual cue generation module is constructed in the following way.

[0050] Pre-build basic modules, see Figure 1 The basic module includes a nonlinear extension unit, an inverse reconstruction unit, a residual module, and a mapping module. The nonlinear extension unit is configured to process the feature vector... Perform layer normalization and up-dimensional projection to obtain high-dimensional features. The inverse reconstruction unit is configured to perform a dimensionality reduction mapping function on the high-dimensional features. Perform reverse compression to generate reconstructed baseline features The residual module is configured to compute the high-dimensional features. and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. The mapping module is configured to map the residual Mapping back to the high-dimensional feature space and with the high-dimensional features Perform weighted fusion to obtain visual cue features .

[0051] Prepare the training set for the basic modules. In this specific embodiment, it is a low-light dataset, consisting of multiple low-light images with real labels. The labels are determined based on the specific task, such as classification labels, detection boxes, segmentation annotations, or tracking boxes. The images do not need to be enhanced or brightened beforehand.

[0052] A neural network model is selected, which may use ViT, ResNet, or other detection / tracking model backbones. The training set is input into the neural network model. The model outputs shallow features using the preceding layers and deep features using the following layers. It should be noted that in this application, the preceding and following layers of the neural network model are relative descriptions. For example, following the input-to-output direction, if the neural network model includes n1 and n2 layers, where n2 > n1, then the n1 layer is the preceding layer and the n2 layer is the following layer.

[0053] The shallow features and the deep features are input into the base module and the base module is trained as follows.

[0054] Shallow features are processed using a 1×1 convolutional layer (Conv1x1). Perform convolution operations to obtain shallow anchor point features. Specifically, a 1×1 convolutional layer can be set in the base module, or a 1×1 convolutional layer can be set in the neural network model, or an external 1×1 convolutional layer can be used for shallow features. Perform convolution operations to obtain shallow anchor point features. And output it to the basic module.

[0055] The nonlinear extension unit, inverse reconstruction unit, residual module, and mapping module in the basic module perform the following calculations sequentially based on the feature vector to obtain the visual cue features. : ; ; ; .

[0056] according to Aligned shallow anchor point features Calculate structural consistency loss The formula for calculating structural consistency loss is: Where N is the total number of feature points in the image, and j is an integer from 1 to N. This represents the visual cue feature corresponding to the j-th feature point. This represents the shallow anchor point feature corresponding to the j-th feature point. This represents the L2 norm.

[0057] The training of the basic module is completed when the structural consistency loss function converges, and the trained basic module is used as the visual cue generation module.

[0058] Using the trained visual cue generation module, the feature vectors obtained from the image are... When input is sent to the visual cue generation module, the module generates visual cue signals, or visual cue features, that are of reference value. , will prompt features By inputting a neural network model for image recognition, more accurate predictions can be obtained in low-light scenes.

[0059] The visual cue generation module can be integrated into a neural network model for image recognition, or it can be used as an add-on module in conjunction with an existing trained basic image recognition model to obtain an image recognition model that provides recognition results and robustness in dark environments. Unlike... Figure 4 The prior art image recognition model is used in a way that is not shown in the present application. This application provides an image recognition model that combines a visual cues generation module, such as... Figure 5 As shown.

[0060] The training process of existing neural network models for image recognition is as follows: Figure 2 As shown, the training process of the image recognition model combined with the visual cues generation module provided in this application is as follows: Figure 3 As shown.

[0061] In one embodiment of the present invention, an image recognition method is provided, which acquires an image to be recognized and inputs it into a pre-built image recognition model to output target feature information.

[0062] The image recognition model is constructed by: determining a neural network model suitable for image recognition, and a basic module for generating visual cue features; and obtaining a training set, which includes multiple images with real labels.

[0063] Inputting the training set into the neural network model for model training includes: the prior network layers of the neural network model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. The shallow features and the feature vector The basic module is input, as described in the above embodiment, which includes a nonlinear extension unit, an inverse reconstruction unit, a residual module, and a mapping module. The basic module outputs predicted visual cue features.

[0064] The neural network model is trained using a preset total loss function until the total loss function converges. The trained neural network model is then used as the image recognition model. The total loss function is positively correlated with the deviation of the visual cue features predicted by the base module relative to the shallow features, and also positively correlated with the deviation between the prediction results of the neural network model and the true labels. The expression for the total loss function is: = Task Loss + ;in It is the balance coefficient. During training, backpropagation is used to primarily update the parameters of the visual cue generation module and the cross-layer alignment projection layer.

[0065] Existing technologies typically focus only on the accuracy of the final classification, while this application, by adding this penalty to the total loss function, physically forces that the cue features generated in the deep layers must geometrically approximate the clear edges of the shallow layers.

[0066] In this application, the structural consistency loss function is used during image recognition model training. This is added as a penalty term to the total loss. Minimize this function (i.e.) This means forcing the deep generator to adjust its parameters. This allows the generated deep features to geometrically approximate the clear edges of the shallow layers, thus physically achieving lossless feature transfer across layers. Therefore, addressing the feature dilution problem in deep neural networks during multi-layer transfer (i.e., deep semantic layers losing shallow physical details), the cross-layer constraint mechanism constructed in this invention forces lossless feature alignment.

[0067] This optimization mechanism enables cross-layer memory of features, preventing key dark-light cues from being smoothed out deep within the backbone network. This fundamentally solves the problem of perception systems failing due to unclear details in complex low-light environments, providing a robust algorithmic safety guarantee for the all-weather operation of autonomous driving systems under non-fault conditions.

[0068] In one embodiment of the present invention, an image recognition method is provided for identifying target feature information from an image, comprising the following steps: acquiring the image to be recognized and inputting it into a pre-built image recognition model to output target feature information.

[0069] The image recognition model is constructed as follows: a pre-trained base image recognition model suitable for image recognition is determined, along with a base module for generating visual cue features. A training set is obtained, comprising multiple images with real labels.

[0070] With the parameters of the basic image recognition model fixed, the training set is input into the basic image recognition model for model training, including: the prior network layers of the basic image recognition model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. .

[0071] The shallow features and the feature vector The input is the basic module, which predicts visual cue features based on the visual cue generation method described above.

[0072] The basic image recognition model is trained using a preset structural consistency loss function until the structural consistency loss function converges. The structural consistency loss function is related to the deviation of the visual cue features predicted by the basic module relative to the shallow features. The trained basic image recognition model is then used as the image recognition model.

[0073] This application uses a cross-layer geometric anchoring mechanism to ensure that deep perception features are geometrically aligned with shallow features, reducing the risk of perception failure due to feature attenuation; and uses a projection difference structure to purify the target signal, suppress irregular noise, and improve the expected functional safety of the system in low-light scenarios.

[0074] In one embodiment of the present invention, an image recognition apparatus is provided, including a processor configured to execute the image recognition method as described in any of the above embodiments to obtain target feature information in an image to be recognized.

[0075] According to another aspect of the present invention, an intelligent driving system is provided, characterized in that it includes an image recognition device as described in any of the above embodiments.

[0076] It should be noted that the image recognition method embodiments, image recognition device embodiments, intelligent driving system embodiments, and the above-mentioned visual prompt generation method embodiments are based on the same inventive concept. All contents of the visual prompt generation method embodiments are incorporated into the image recognition method embodiments, image recognition device embodiments, and intelligent driving system embodiments by reference.

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0078] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating visual cues, characterized in that, Includes the following steps: For the feature vector obtained from the image Layer normalization and updimensional projection are performed on the feature to obtain high-dimensional features in the high-dimensional feature space. ; The high-dimensional features are mapped using a preset dimensionality reduction function. Perform reverse compression to generate reconstructed baseline features ; Calculate the high-dimensional features and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. ; The residual Mapping back to the high-dimensional feature space and with the high-dimensional features Perform weighted fusion to obtain visual cue features .

2. The visual cue generation method according to claim 1, characterized in that, The residual and visual cue features The calculation formula is as follows: ； in, This is the first balance coefficient.

3. The visual cue generation method according to claim 1, characterized in that, Visual cue features The calculation formula is as follows: ； in, This is the second balance coefficient. This is the projection function that maps the residuals back to the high-dimensional feature space.

4. The visual cue generation method according to claim 1, characterized in that, The high-dimensional features The calculation formula is as follows: ； in, This represents the dimension-scaling function, and Norm represents the normalization operation. For the generated high-dimensional features.

5. The visual cue generation method according to claim 1, characterized in that, Reconstructing the baseline features The calculation formula is: ； in, represents the dimension reduction mapping function, and Norm represents the normalization operation.

6. The visual cue generation method according to claim 1, characterized in that, It also includes constraining the generation of the visual cue features in the following ways: Pre-generated feature vectors The upper-level network obtains shallow features For the shallow features Perform convolution operations to obtain shallow anchor point features. ; The visual cue features and the shallow anchor point features are calculated using a preset structural consistency loss function. If the deviation between the two values ​​is less than a preset threshold, then the visual cue feature is output.

7. The visual cue generation method according to claim 6, characterized in that, The structural consistency loss function The calculation formula is as follows: Where N is the total number of feature points in the image, and j is an integer from 1 to N. This represents the L2 norm.

8. The visual cue generation method according to claim 6, characterized in that, The shallow features are processed by 1×1 convolution. Perform spatial alignment operations to obtain the shallow anchor point features. .

9. The visual cue generation method according to claim 1, characterized in that, The visual cue features are obtained using a preset visual cue generation module, including the following steps: The image is input into a preset neural network model to extract feature vectors from the image using the neural network model. , the feature vector Input the visual cue generation module to output the visual cue features; The visual cue features include a nonlinear expansion unit, an inverse reconstruction unit, a residual module, and a mapping module; wherein, the nonlinear expansion unit is configured to modify the feature vector. Perform layer normalization and up-dimensional projection to obtain high-dimensional features. ; The reverse reconstruction unit is configured to perform a dimensionality reduction mapping function on the high-dimensional features. Perform reverse compression to generate reconstructed baseline features ; The residual module is configured to compute the high-dimensional features. and the reconstructed benchmark features The weighted difference between them is used to obtain the residual. ; The mapping module is configured to map the residual Mapping back to the high-dimensional feature space and with the high-dimensional features Perform weighted fusion to obtain visual cue features .

10. The visual cue generation method according to claim 9, characterized in that, The visual cue generation module is trained in the following way: Obtain a training set, which includes multiple images with real labels; Select a neural network model, input the training set into the neural network model, use the preceding layers of the neural network model to output shallow features, and use the subsequent layers of the neural network model to output deep features. Construct a basic module, which includes the nonlinear extension unit, the inverse reconstruction unit, the residual module, and the mapping module; The shallow features and the deep features are input into the base module and the base module is trained as follows: the visual cue generation module is trained using a preset structural consistency loss function until the structural consistency loss function converges. The structural consistency loss function is related to the deviation of the visual cue features predicted by the base module relative to the shallow features. The trained base module is then used as the visual cue generation module.

11. An image recognition method, characterized in that, The method for identifying target feature information from an image includes the following steps: The image to be identified is acquired and input into a pre-built image recognition model to output target feature information; The image recognition model is constructed in the following way: Determine a neural network model suitable for image recognition, and a basic module for generating visual cue features; Obtain a training set, which includes multiple images with real labels; Inputting the training set into the neural network model for model training includes: the prior network layers of the neural network model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. ; The shallow features and the feature vector The input is the basic module, which predicts visual cue features based on the visual cue generation method as described in any one of claims 1-9; The neural network model is trained using a preset total loss function until the total loss function converges. The trained neural network model is then used as the image recognition model. The total loss function is related to the deviation of the visual cue features predicted by the base module relative to the shallow features, as well as the deviation between the prediction results of the neural network model and the true labels.

12. An image recognition method, characterized in that, The method for identifying target feature information from an image includes the following steps: The image to be identified is acquired and input into a pre-built image recognition model to output target feature information; The image recognition model is constructed in the following way: Identify a basic image recognition model that is suitable for image recognition and is pre-trained, as well as a basic module for generating visual cue features; Obtain a training set, which includes multiple images with real labels; With the parameters of the basic image recognition model fixed, the training set is input into the basic image recognition model for model training, including: the prior network layers of the basic image recognition model generating shallow features based on the training set. And its generation of feature vectors based on the training set in the later network layers. ; The shallow features and the feature vector The input is the basic module, which predicts visual cue features based on the visual cue generation method as described in any one of claims 1-9; The basic image recognition model is trained using a preset structural consistency loss function until the structural consistency loss function converges. The structural consistency loss function is related to the deviation of the visual cue features predicted by the basic module relative to the shallow features. The trained basic image recognition model is then used as the image recognition model.

13. An image recognition device, characterized in that, The device includes a processor configured to perform the image recognition method as described in any one of claims 11 to 12 to obtain target feature information in an image to be recognized.

14. An intelligent driving system, characterized in that, Includes the image recognition device as described in claim 13.

Images