Method for detecting visual angle icon of mechanical arm for vehicle central control

Abstract

The invention discloses a vehicle central control-oriented mechanical arm visual angle icon detection method, which comprises the following steps of: S1, data set processing: collecting vehicle central control screen icon data, preprocessing and marking the collected data, and dividing a data set according to a proportion; s2, constructing and training an improved model: constructing an ECPA module by taking DEIM-D-FINE as a reference model, introducing a C2fSPD module in combination with a C2f module and an SPDConv module, and optimizing a feedforward network structure by adopting a ConvolutionalGLU module to obtain the improved model; training parameters are set, the improved model is trained, and the weight of the improved model is obtained; s3, detection and verification: inputting to-be-detected icon data of the central control screen of the vehicle, and performing image detection to obtain a detection result; and verifying the model performance. The method has good accuracy, icon details can be fully captured, key features can be adaptively enhanced, and irrelevant responses can be inhibited.

Description

A method for detecting robotic arm view icons for vehicle central control Technical Field

[0001] This invention belongs to the field of new artificial intelligence technology, specifically relating to a method for detecting icons from the perspective of a robotic arm for vehicle central control. Background Technology

[0002] Currently, icon detection in in-vehicle central control interfaces mainly relies on deep learning methods based on computer vision. However, in actual vehicle testing, the similarity in icon design and poor resolution often lead to insufficient model perception of subtle differences from the robotic arm's acquisition perspective, resulting in class confusion and increased false detection rates.

[0003] Most existing research directly applies detection frameworks designed for general scenarios, failing to fully consider the unique challenges posed by icon similarity and image resolution issues from the perspective of robotic arm testing. Specifically, traditional detection methods may be insufficient in capturing subtle discriminative features, easily leading to category confusion between similar icons; at the same time, when faced with inputs of poor resolution, they exacerbate the loss of icon detail features, thus limiting detection accuracy in practical detection scenarios.

[0004] Chinese patent document CN116954428A discloses a method, apparatus, device, and medium for displaying and recognizing vehicle icons. The method includes: in response to a trigger event for a preset function of a vehicle, acquiring an image of a display area on the vehicle's dashboard corresponding to the preset function; processing the display area image according to a target binarization processing algorithm corresponding to the preset function to obtain a binarized image of the display area image; determining the display geometric moment values ​​of the maximum points in the binarized image at different orders; and identifying whether there is an anomaly in the display state of the vehicle icon in the display area image based on the determined display geometric moment values ​​and the standard geometric moment values ​​of each function. This prior art solves the problem that changes in the icon's background prevent the setting of a threshold, thus hindering the recognition of vehicle icon displays, achieving accurate recognition of vehicle icons in a liquid crystal instrument panel.

[0005] Chinese patent document CN120298978A discloses a machine vision-based method and system for measuring coupler force in heavy-haul trains, belonging to the field of railway train operation monitoring. The method includes: acquiring images of the coupler area of ​​a heavy-haul train, and performing decontamination and enhancement processing on the coupler area images in conjunction with operating condition information to obtain enhanced coupler area images; identifying coupler feature targets in the enhanced coupler area images based on an improved DEIM-DFine network model trained with dedicated data augmentation, and obtaining the pixel coordinates of the coupler feature targets by combining dynamic temporal filtering; and quantitatively identifying the coupler force using a CNN-ResMLP network model based on the pixel coordinates of the coupler feature targets and the corresponding operating condition information to obtain the coupler force identification result. This prior art improves the accuracy and environmental adaptability of coupler force measurement through multi-stage collaborative processing, providing a referenceable technical guarantee solution for the safe operation of heavy-haul trains.

[0006] In vehicle testing, traditional electronic center console screen testing primarily relies on manual methods. This process requires testers to use tools or manually click on various function icons on the screen and observe their feedback. While this testing process offers some flexibility, in the face of large-scale industrial production, this manual approach inevitably limits production efficiency. With the increasing complexity of in-vehicle functions, the number and types of center console icons have also increased significantly, making testing more challenging. Furthermore, prolonged repetitive testing can cause visual fatigue for testers, leading to decreased attention spans and inevitably impacting the repeatability and reliability of the tests.

[0007] In the field of automated inspection, existing research has attempted to introduce deep learning technology into vehicle testing. Researchers have proposed a convolutional neural network-based instrument panel icon recognition method, demonstrating the feasibility of this approach. However, this method suffers from limitations in detection accuracy and faces certain difficulties in practical deployment. Other researchers have proposed an improved method based on YOLOv5, introducing an efficient channel attention mechanism, a dynamic attention mechanism detection head, and the CARAFE upsampling operator to enhance the detection accuracy of vehicle LCD instrument panel icons. However, its ability to differentiate between the complex and diverse center console icons, especially those with high similarity, needs further evaluation.

[0008] In automated vehicle testing, the effective detection and discrimination of icons on the vehicle's central control screen is fundamental to enabling robotic arms to replace manual inspection. Only by effectively identifying the differences between icons and achieving accurate icon detection can the robotic arm perform further click tests on various functional modules according to test instructions. However, current research on vehicle central control screen icon detection is limited, and datasets directly applicable to this research are also scarce. Furthermore, in actual testing, icons on the central control screen exhibit a certain degree of similarity, and poor screen resolution further exacerbates the difficulty of icon feature recognition, placing higher demands on the accuracy and stability of the model.

[0009] To address the aforementioned issues, this invention proposes an improved DEIM-D-FINE network-based method for detecting in-vehicle center console icons. This method effectively distinguishes highly similar center console icons, significantly reducing false positives and false negatives, and improving detection accuracy. Furthermore, even with poor image resolution during testing, this method maintains stable detection performance, providing reliable technical support for automated testing of vehicle center console screens and meeting the high demands of industrial production for testing efficiency and quality. This allows the model to achieve better detection performance in the vehicle center console icon detection testing phase, providing more reliable technical support for automated testing of in-vehicle center consoles. Summary of the Invention

[0010] The technical problem this invention aims to solve is to provide a method for detecting icons from the perspective of a robotic arm in vehicle central control systems. This method approaches the problem from both channel and spatial dimensions. It achieves lightweight channel feature enhancement through the cascading of depthwise separable convolutions and efficient channel attention, while incorporating pixel-level attention and designing an ECPA module to strengthen the extraction of subtle differences between icons. Furthermore, it introduces and improves the C2f structure in YOLOv8 to C2f_SPD, aiming to alleviate the loss of detail during downsampling and improve the model's detection stability under low-resolution and high-similarity icon conditions. Finally, it employs ConvolutionalGLU to optimize the feedforward network structure, enabling adaptive selection and enhancement of discriminative features. This allows the model to achieve better detection performance in the detection and testing of vehicle central control system icons, providing more reliable technical support for automated testing of in-vehicle central control systems.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for detecting icons from the perspective of a robotic arm facing the vehicle's central control, specifically including the following steps:

[0012] S1 Dataset Processing: Collect icon data from the vehicle's central control screen, preprocess and label the collected data, and then divide the dataset proportionally;

[0013] S2: Construct and train the improved model: Using DEIM-D-FINE as the baseline model, construct the ECPA module, combine it with the C2f and SPDConv modules, introduce the C2f_SPD module, and use the ConvolutionalGLU module to optimize the feedforward network structure to obtain the improved model; set the training parameters, train the improved model, and obtain the weights of the improved model.

[0014] S3 Detection and Verification: Input the icon data of the vehicle's central control screen to be detected, perform image detection, and obtain the detection results; at the same time, verify the model performance.

[0015] Preferably, in step S1, icon data of the vehicle's central control screen is acquired through an image acquisition device, and the data is cleaned to remove duplicate images and images with image quality below the sharpness threshold; the targets are labeled using Labelimg, and the labeled dataset is divided into a training set, a validation set, and a test set. Images with image quality below the sharpness threshold are low-quality images that affect the clear recognition of icons due to screen interface switching ghosting caused by the timing of the shooting, blurring, distortion, and occlusion caused by the movement of the robotic arm, as well as image noise.

[0016] Preferably, the specific steps of step S2 are as follows:

[0017] S21 Building an improved model: Build the DEIM-D-FINE object detection algorithm framework, use the vehicle center console icon dataset for training, and obtain the initial model training weights;

[0018] By combining the ECA attention mechanism with pixel attention, and employing residual connections and adaptive channel mapping, an ECPA module is constructed.

[0019] During model downsampling, the C2f module was improved using SPDConv and applied to the DEIM-DFINE baseline model;

[0020] In the improvement of the feedforward neural network, the Convolutional GLU module is used to optimize the multilayer perceptron mlp to obtain an improved model;

[0021] S22: Set training parameters and use the vehicle center console icon dataset for training. Traverse all vehicle center console icon images in the training set to obtain the weights of the improved model. The training iterations are 300 rounds, the base learning rate is set to 0.0008, the worker size is 4, and the existing loss function MAL (Matchability-Aware Loss / Multi-Anchor Learning Loss) is used. By fusing IoU (localization quality) and classification confidence (classification quality), dynamic loss weights are assigned to each matching sample to achieve the goal of "stable optimization of high-quality matching and effective utilization of low-quality matching".

[0022] The algorithm is improved by adopting the above technical solution and combining ECA and pixel attention to obtain the ECPA module. ECPA first enhances the response of key channels, and then, combined with the features of the original input image, it is input to pixel attention for refined weighting in the spatial dimension. This allows the model to learn target features by combining channel and spatial information, enhancing its ability to capture subtle discriminative features between icons, thereby improving the accuracy of distinguishing similar targets. C2f_SPD is introduced to optimize the sampling process. Specifically, SPDConv is first introduced into the C2f structure and then applied to the downsampling process of the network model. This avoids the loss of details during downsampling and ensures the integrity of icon detail features in the model's forward propagation, thus directly improving the model's ability to represent small icons. The ConvolutionalGLU module is used to optimize the feedforward network structure, enabling the network to adaptively suppress irrelevant features and enhance key details, achieving adaptive selection and enhancement of discriminative features.

[0023] Preferably, the ECPA module constructed in step S21 consists of two branches, namely a first branch and a second branch. The first branch extracts local details by using depthwise separable convolution, and then uses the ECA attention mechanism module to generate channel weights, enhance the channel response related to the target features, and suppress noise channels.

[0024] Meanwhile, the first branch also retains the features obtained by depthwise separable convolution. The retained features are concatenated with the feature information obtained by ECA to form a guide graph containing global information and channel importance. The ECA module first performs global average pooling on the input features to obtain channel vectors, then adaptively calculates the size k of the one-dimensional convolution kernel based on the number of channels C, and further generates weights. Then, by multiplying with the original features channel by channel, dynamic recalibration is achieved, thereby enhancing the target-related channel response and improving feature discrimination.

[0025] The formula for calculating the adaptive kernel size k used in the ECA attention mechanism module is shown in (1):

[0026] (1);

[0027] Where k is the adaptive kernel size, C is the number of input channels, γ=2, and b=1;

[0028] The second branch is the residual branch, which preserves the original input image and provides original detailed features for subsequent processing;

[0029] The information from the first and second branches will be further input into pixel attention to extract detailed features in the spatial dimension. After processing by the ECPA module, the model can improve its ability to discriminate differences between targets and suppress interference from the background.

[0030] Using the above technical solution, an ECPA module integrating dual attention was constructed. The ECPA module combines ECA attention and pixel attention through a residual connection structure. The module employs a two-branch processing flow: one branch extracts local features using depthwise separable convolution, then adaptively enhances the key channel response through an ECA attention mechanism; the other branch preserves the original feature representation through convolution. After concatenating the features from the two branches, spatial dimension features are filtered by pixel-level attention, and finally, feature enhancement is achieved through residual connections. This structure can simultaneously improve the model's perception of channel importance and spatial details, strengthening the discriminative feature extraction of similar vehicle center console icons.

[0031] Preferably, step S21 uses SPDConv to improve the C2f module, specifically as follows:

[0032] After the input feature map enters the C2fSPD module, it will first generate an intermediate feature map and split it. The feature data on the main branch will be passed to multiple Bottleneck structures containing SPDConv for processing to obtain more complex detailed features. The other branches only pass the original feature information. Finally, all information is merged by Concat, so as to retain the original information at the bottom level while fusing the complex information at the high level, so that the model can comprehensively utilize multi-scale and multi-level feature information. In the Bottleneck part, for the feature map with input (S,S,C1), where S is the height and width of the image and C1 is the number of channels, the SPDConv module will first divide it from the x and y directions and process it. The formulas for this process are shown in (2)-(7):

[0033] (2);

[0034] (3);

[0035] (4);

[0036] (5);

[0037] (6);

[0038] (7);

[0039] Where X is the input feature map, and scale is the downsampling factor, which is 2 in SPDConv; The input feature map (S, S, C1) is processed by SPD to obtain four sub-feature maps, each with dimensions (S / 2, S / 2, C1). These four sub-feature maps are then concatenated along the channel direction, further changing the dimensions to (S / 2, S / 2, 4C1). A 1×1 convolution then transforms the channel size from 4C1 to C2. While the spatial dimensions are reduced, no information is lost. This transformation allows subsequent convolutions to quickly capture complete and detailed features. This enables the acquisition of subtle differences between icons, improving the model's detection performance.

[0040] By adopting the above technical solution, SPDConv is introduced to improve the downsampling process. By combining SPDConv with the C2f structure, a C2f_SPD module is constructed and applied to the downsampling process. This reduces the loss of detail in the detection of vehicle center console icons, improves the model's ability to distinguish differences between icons and its adaptability to low-resolution situations, and ensures that it can maintain accurate detection performance even in complex testing environments.

[0041] Preferably, in step S21, the optimization of the multilayer perceptron (MLP) using the Convolutional GLU module in the improvement of the feedforward neural network specifically involves:

[0042] For the input features, the Convolutional GLU module first performs a linear transformation on the input features, and then generates branches for feature value propagation and control signals.

[0043] In the branch that includes the generation of control signals, the feature data is processed by a 3×3 depthwise separable convolution and then an activation function is used to generate a gating signal; this design allows each pixel location to have its own unique gating signal, thereby enabling adaptive filtering of local detail features;

[0044] Subsequently, the outputs of the eigenvalue propagation branch and the control signal branch are combined by point-by-point multiplication to suppress useless features.

[0045] The above technical solution incorporates a Convolutional GLU module to optimize MLP. Convolutional GLU generates spatially aware gating signals through 3×3 depthwise convolutions, enabling the model to weight features based on the local context of each pixel location, achieving fine-grained feature selection. Its unique gating mechanism enhances the model's ability to filter discriminative features. In the vehicle center console icon detection task, the use of this module strengthens the model's perception of subtle differences between icons, improves the accuracy of distinguishing similar icons, and allows the detection model to maintain stable performance in complex testing environments.

[0046] Preferably, in step S3, the vehicle central control screen icon data from the test set is used for testing, and the model is evaluated using a validation set; the evaluation metrics include precision, recall, F1 score, mAP@50, and AP for small objectives. small with AR small The calculation process is shown in formulas (8)-(12);

[0047] (8);

[0048] (9);

[0049] (10);

[0050] (11);

[0051] (12);

[0052] Where Precision is the target value; Recall is the recall value; TP represents true positives, FP represents false positives, and FN represents false negatives; AP is the area under the Precision-Recall curve, and N is the number of target classes. Furthermore, AP is used as an evaluation metric. small with AR small These figures respectively reflect the model's precision and recall for detecting small targets.

[0053] Using the aforementioned technical solution, the model first processes the dataset images through various data augmentation strategies to generate multiple versions of samples to increase the density of the supervisory signal. In subsequent data processing, the ECPA module is constructed by combining ECA attention and pixel attention to extract detailed image features. The ECA module enhances the response of feature channels by adaptively calibrating channel weights, while pixel attention further refines the feature responses. The two work synergistically, enabling ECPA to enhance spatial detail features while locking onto the most discriminative feature channels, thereby improving the model's ability to capture subtle differences between icons and increasing its resolution accuracy for target images. In the feature extraction stage, an improved C2f_SPD structure is introduced to optimize the sampling process, effectively mitigating the problem of detail loss during sampling and improving the model's adaptability to low-resolution icons. Subsequently, the feedforward network structure is optimized through the ConvolutionalGLU module, enabling the network to adaptively suppress irrelevant features and enhance key details, achieving adaptive selection and enhancement of discriminative features. Compared to the original model, this method improves the accuracy of similar icon detection and performance in complex testing environments, providing a valuable technical solution for vehicle central control automation testing.

[0054] Preferably, in step S1, the training set, validation set, and test set are divided in a ratio of 8:1:1.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] (1) Accuracy: DEIM-DFINE itself has good accuracy. The dual attention mechanism of the ECPA module can effectively enhance the perception of subtle icon features; the improved C2f_SPD structure ensures the integrity of features during downsampling; the gating mechanism of Convolutional GLU realizes the adaptive enhancement of discriminative features. These innovative designs together improve the model's ability to distinguish similar icons and reduce false positives and false negatives.

[0057] (2) Detail preservation capability: In view of the characteristics of images from the perspective of the robotic arm, the model effectively alleviates the problem of detail loss during downsampling by constructing a feature preservation mechanism through SPDConv, ensuring the feature integrity in similar icons and low-resolution cases, enabling the model to fully capture the discriminative details of icons and provide a reliable feature basis for accurate detection.

[0058] (3) Feature selection capability: The Convolutional GLU module implements feature selection through a gating mechanism. This enables the model to adaptively enhance key features and suppress irrelevant responses, significantly improving the model's sensitivity to subtle differences between icons and enabling it to maintain stable detection performance in complex testing environments.

[0059] (4) Versatility: The ECPA module is a plug-and-play module that can be adaptively embedded into the network structure and replaced. The overall model can detect and recognize icons in the central control screen of fixed types of vehicles. By adjusting the training data, the model can be adapted to detect central control icons of different types of vehicles. Attached Figure Description

[0060] Figure 1 is a flowchart of the detection method for the robot arm perspective icon for vehicle central control according to the present invention;

[0061] Figure 2 is a structural diagram of the ECPA module constructed in the detection method of the robot arm perspective icon for vehicle central control of the present invention;

[0062] Figure 3 is a structural diagram of the C2f SPD module in the detection method of the robot arm perspective icon for vehicle central control of the present invention;

[0063] Figure 4 is a structural diagram of the ConvolutionalGLU module in the detection method of the robot arm perspective icon for vehicle central control according to the present invention. Detailed Implementation

[0064] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present invention more clearly, and should not be used to limit the scope of protection of the present invention.

[0065] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0066] Definition of the noun:

[0067] HGStem module: Full name is High-performance Generalizable Stem feature enhancement module;

[0068] HGBlock module: short for High-performance Generalizable Block, which is a high-performance, highly generalizable module;

[0069] c2f: refers to the C2f module or C2f feature fusion module, which stands for CSP Bottleneck with 2convolutions and feature fusion.

[0070] The prior art content upon which this invention is based specifically includes:

[0071] (1) DEIM target detection algorithm:

[0072] DEIM is a novel object detection method proposed to address the slow convergence speed caused by sparse supervision information and low-quality matching in the DETR object detection framework. It proposes new improvements to address the shortcomings of DETR.

[0073] Dense O2O Strategy:

[0074] DETR's O2O matching mechanism assigns only one positive sample to each target during model training, achieving a one-to-one mapping between predicted targets and ground truth labels. While this simplifies the process by eliminating redundant bounding boxes, the reduced number of positive samples significantly diminishes the amount of supervisory information, making early training and optimization difficult and slowing convergence. DEIM's Dense O2O method, on the other hand, effectively increases the density of supervisory signals by increasing the number of positive samples, accelerating model training and convergence. Unlike traditional O2M matching strategies that simply assign multiple positive samples to detected targets in an image, Dense O2O uses various data augmentation methods, including Mosaic, to stitch a single target image into multiple target images, maintaining one positive sample for each detected target in each image. This method eliminates the need for a large number of positive and negative samples and significantly increases the number of matches. Compared to the O2M strategy, this method is less costly, and compared to the O2O strategy, it facilitates faster model convergence.

[0075] MAL loss:

[0076] As a commonly used loss function, VFL is widely applied in object detection algorithms. However, within the DETR framework, VFL suffers from difficulties in convergence due to the low IoU matching degree, making loss optimization challenging, and the problem of classifying samples with IoU=0 as negative samples. To address this, DEIM proposed MAL loss to improve matching quality. Especially in low-quality matching stages, MAL gradually increases the penalty as confidence increases, prompting the model to calibrate its confidence output and localization quality, thereby improving matching accuracy.

[0077] (2) ECA attention mechanism

[0078] ECA is an efficient channel attention mechanism. Its core idea is to effectively acquire information about interactions between channels without increasing the computational complexity of the model, thereby improving model performance. Its workflow mainly consists of three steps: global average pooling, efficient channel attention, and feature recalibration.

[0079] First, for the input image, ECA performs global average pooling on each channel to obtain channel description vectors. Second, ECA models the channel features using one-dimensional convolutions, generating channel attention weights. In this process, ECA employs adaptive convolution kernels, allowing it to automatically select appropriate interaction ranges even when the number of network channels changes. This step is also relatively simple, as the use of one-dimensional convolutions results in low computational cost and high efficiency. Finally, by applying the generated channel attention weights to the input feature map, channel recalibration is achieved, thereby enhancing the expressive power of important feature channels.

[0080] (3) SPDConv

[0081] The idea behind SPDConv is to perform efficient and lossless downsampling of feature maps without losing information, thereby improving the detection performance of the model when dealing with images with poor resolution or small targets. This module consists of a spatial-to-depth layer, namely the SPD layer, and a non-strut convolution.

[0082] Faced with the input feature image, the SPD layer first transforms it from a spatial dimension to a depth dimension. This operation preserves the spatial information of the feature image, ensuring downsampling without loss of information. To address the change in the number of channels resulting from the transformation, SPDConv further reduces the number of channels using non-strut convolutions, while simultaneously processing the features using learnable parameters.

[0083] SPDConv's design effectively preserves pixel information, which is crucial for retaining small targets and detailed features. By using this design, the model can more effectively preserve the detailed features of targets, improving the detection accuracy of similar targets.

[0084] (4) Convolutional GLU

[0085] ConvolutionalGLU combines channel attention with convolutional feature extraction, enabling the model to maintain high computational efficiency while possessing stronger feature representation capabilities. Specifically, the convolutional feedforward part uses 3×3 convolutions to extract neighborhood features and obtain effective information; the gating mechanism can dynamically adjust the information flow to determine more important features in the input feature map, thereby enhancing the model's ability to process complex data.

[0086] Example: As shown in Figure 1, the method for detecting icons from the perspective of a robotic arm facing the vehicle's central control unit specifically includes the following steps:

[0087] The method for detecting icons from the perspective of a robotic arm facing the vehicle's central control unit includes the following steps:

[0088] S1 Dataset Processing: Collect icon data from the vehicle's central control screen, preprocess and label the collected data, and then divide the dataset proportionally;

[0089] In step S1, icon data of the vehicle's central control screen is acquired through an image acquisition device, and the data is cleaned to remove duplicate images and images with image quality below the sharpness threshold. Labelimg is used to annotate the targets, and the annotated dataset is divided into training, validation, and test sets. Images with image quality below the sharpness threshold are low-quality images that affect clear icon recognition due to screen interface switching ghosting caused by the timing of the capture, blurring, distortion, and occlusion caused by robotic arm movement, and image noise.

[0090] S2: Construct and train the improved model: Using DEIM-D-FINE as the baseline model, construct the ECPA module, combine it with the C2f and SPDConv modules, introduce the C2f_SPD module, and use the ConvolutionalGLU module to optimize the feedforward network structure to obtain the improved model; set the training parameters, train the improved model, and obtain the weights of the improved model.

[0091] The specific steps of step S2 are as follows:

[0092] S21 Build an improved model: Build the DEIM-D-FINE object detection algorithm framework, use the vehicle center console icon dataset for training, and obtain the initial model training weights; the training environment uses Python 3.8, CUDA version 11.8, and RTX4090 GPU to accelerate training;

[0093] By combining ECA attention mechanism and pixel attention, and employing residual connections and adaptive channel mapping, an ECPA module is constructed. In its structural design, this module integrates channel attention (ECA) and pixel attention, and uses residual connections and adaptive channel mapping, which can be directly applied to the network structure to improve the target detection capability.

[0094] During model downsampling, the C2f module was improved using SPDConv and applied to the DEIM-DFINE baseline model;

[0095] In the improvement of the feedforward neural network, the Convolutional GLU module is used to optimize the multilayer perceptron mlp to obtain an improved model;

[0096] S22: Set training parameters and train using the vehicle center console icon dataset to obtain the weights of the improved model; set training parameters and train using the vehicle center console icon dataset, traversing all vehicle center console icon images in the training set to obtain the weights of the improved model. The training iterations are 300 rounds, the base learning rate is set to 0.0008, the worker size is 4, and the existing loss function MAL (Matchability-Aware Loss) is used. By fusing IoU (localization quality) and classification confidence (classification quality), dynamic loss weights are assigned to each matching sample to achieve the goal of "stable optimization of high-quality matching and effective utilization of low-quality matching."

[0097] The loss function used in the training process is the existing Matchability-Aware Loss (MAL). MAL improves the sensitivity to low-quality matches by directly incorporating matching quality into the loss function, thereby enhancing the ability to handle low-quality matches. The formula for calculating the MAL loss function is as follows:

[0098] (13);

[0099] Where q is the Intersection over Union (IoU) between the bounding box and the target box, p is the confidence score, and y is the class label, where y=1 represents foreground and y=0 represents background. The value is 1.5.

[0100] The above technical solution is adopted, and the algorithm is improved by combining ECA and pixel attention to obtain the ECPA module. ECPA first enhances the response of key channels, and then, combined with the features of the original input image, it is input to pixel attention for refined weighting in the spatial dimension. This allows the model to learn target features by combining channel and spatial information, enhancing its ability to capture subtle discriminative features between icons, thereby improving the accuracy of distinguishing similar targets. C2f_SPD is introduced to optimize the sampling process. Specifically, SPDConv is first introduced into the C2f structure and then applied to the downsampling process of the network model. This avoids the loss of details during downsampling and ensures the integrity of icon detail features in the model's forward propagation, thus directly improving the model's ability to represent small icons. The ConvolutionalGLU module is used to optimize the feedforward network structure, enabling the network to adaptively suppress irrelevant features and enhance key details, achieving adaptive selection and enhancement of discriminative features.

[0101] As shown in Figure 2, the ECPA module constructed in step S21 consists of two branches, namely the first branch and the second branch. The first branch extracts local details by using depthwise separable convolution, and then uses the ECA attention mechanism module to generate channel weights, enhance the channel response related to the target features, and suppress noise channels.

[0102] Meanwhile, the first branch also retains the features obtained by depthwise separable convolution. The retained features are concatenated with the feature information obtained by ECA to form a guide graph containing global information and channel importance. The ECA module first performs global average pooling on the input features to obtain channel vectors, then adaptively calculates the size k of the one-dimensional convolution kernel based on the number of channels C, and further generates weights. Then, by multiplying with the original features channel by channel, dynamic recalibration is achieved, thereby enhancing the target-related channel response and improving feature discrimination.

[0103] The formula for calculating the adaptive kernel size k used in the ECA attention mechanism module is shown in (1):

[0104] (1);

[0105] Where k is the adaptive kernel size, C is the number of input channels, γ=2, and b=1;

[0106] The second branch is the residual branch, which preserves the original input image and provides original detailed features for subsequent processing;

[0107] The information from the first and second branches will be further input into pixel attention to extract detailed features in the spatial dimension; after processing by the ECPA module, the model can improve its ability to distinguish differences between targets and suppress interference from the background.

[0108] Using the above technical solution, an ECPA module integrating dual attention was constructed. The ECPA module combines ECA attention and pixel attention through a residual connection structure. The module employs a two-branch processing flow: one branch extracts local features using depthwise separable convolution, then adaptively enhances the key channel response through an ECA attention mechanism; the other branch preserves the original feature expression through convolution. After the features from the two branches are concatenated, spatial dimension features are filtered by pixel-level attention, and finally, feature enhancement is achieved through residual connections. This structure can simultaneously improve the model's ability to perceive channel importance and spatial details, strengthening the discriminative feature extraction of similar vehicle center console icons.

[0109] As shown in Figure 3, step S21 uses SPDConv to improve the C2f module, specifically as follows:

[0110] After the input feature map enters the C2fSPD module, it will first generate an intermediate feature map and split it. The feature data on the main branch will be passed to multiple Bottleneck structures containing SPDConv for processing to obtain more complex detailed features. The other branches only pass the original feature information. Finally, all information is merged by Concat, so as to retain the original information at the bottom level while fusing the complex information at the high level, so that the model can comprehensively utilize multi-scale and multi-level feature information. In the Bottleneck part, for the feature map with input (S,S,C1), where S is the height and width of the image and C1 is the number of channels, the SPDConv module will first divide it from the x and y directions and process it. The formulas for this process are shown in (2)-(7):

[0111] (2);

[0112] (3);

[0113] (4);

[0114] (5);

[0115] (6);

[0116] (7);

[0117] Where X is the input feature map, and scale is the downsampling factor, which is 2 in SPDConv; The input feature map (S, S, C1) is processed by SPD to obtain four sub-feature maps, each with dimensions (S / 2, S / 2, C1). These four sub-feature maps are then concatenated along the channel direction, further changing the dimensions to (S / 2, S / 2, 4C1). A 1×1 convolution then transforms the channel size from 4C1 to C2. While the spatial dimensions are reduced, no information is lost. This transformation allows subsequent convolutions to quickly capture complete and detailed features. This enables the acquisition of subtle differences between icons, improving the model's detection performance.

[0118] By adopting the above technical solution, SPDConv is introduced to improve the downsampling process. By combining SPDConv with the C2f structure, a C2f_SPD module is constructed and applied to the downsampling process. This reduces the loss of detail in the detection of vehicle center console icons, improves the model's ability to distinguish differences between icons and its adaptability to low-resolution situations, and ensures that it can maintain accurate detection performance even in complex testing environments.

[0119] As shown in Figure 4, in step S21, the optimization of the multilayer perceptron (MLP) using the Convolutional GLU module in the improvement of the feedforward neural network specifically involves:

[0120] For the input features, the Convolutional GLU module first performs a linear transformation on the input features, and then generates branches for feature value propagation and control signals.

[0121] In the branch that includes the generation of control signals, the feature data is processed by a 3×3 depthwise separable convolution and then an activation function is used to generate a gating signal; this design allows each pixel location to have its own unique gating signal, thereby enabling adaptive filtering of local detail features;

[0122] Subsequently, the outputs of the eigenvalue propagation branch and the control signal branch are combined by point-by-point multiplication to suppress useless features.

[0123] The above technical solution incorporates a Convolutional GLU module to optimize MLP. A Gated Linear Unit (GLU) is a special type of channel mixer containing two linear projections, capable of feature transfer and gating signal generation, respectively. The core idea of ​​GLU is to use gating mechanisms to control the flow of information, thereby enhancing the model's expressive power. Convolutional GLU combines 3×3 depthwise convolutions into the GLU's gating branch. Convolutional GLU generates spatially aware gating signals through 3×3 depthwise convolutions, enabling the model to perform feature weighting based on the local context of each pixel location, achieving fine-grained feature selection. Its unique gating mechanism enhances the model's ability to filter discriminative features. In the vehicle center console icon detection task, the use of this module strengthens the model's perception of subtle differences between icons, improves the accuracy of distinguishing similar icons, and allows the detection model to maintain stable performance in complex testing environments.

[0124] S3 Detection and Verification: Input the icon data of the vehicle's central control screen to be detected, perform image detection, and obtain the detection results; at the same time, verify the model performance;

[0125] In step S3, the vehicle's central control screen icon data from the test set is used for testing, and the model is evaluated using a validation set. Evaluation metrics include precision, recall, F1 score, mAP@50, and AP for smaller objectives. small with AR small The calculation process is shown in formulas (8)-(12);

[0126] (8);

[0127] (9);

[0128] (10);

[0129] (11);

[0130] (12);

[0131] Where Precision is the target value; Recall is the recall value; TP represents true positives, FP represents false positives, and FN represents false negatives; AP is the area under the Precision-Recall curve, and N is the number of target classes. Furthermore, AP is used as an evaluation metric. small with AR small These figures respectively reflect the model's precision and recall for detecting small targets.

[0132] The improved model of this invention was compared with the original model, and the data are shown in Table 1.

[0133] Table 1 Comparison of effects before and after improvement

[0134] Using the above technical solution, the model first processes the dataset images through various data augmentation strategies to generate multiple versions of samples to increase the density of the supervision signal. In subsequent data processing, the ECPA module is constructed by combining ECA attention mechanism and pixel attention to extract image detail features. The ECA module enhances the response of feature channels by adaptively calibrating channel weights, while pixel attention further refines the feature response. The two work synergistically, enabling ECPA to lock in the most discriminative feature channels while strengthening spatial detail features, thereby enhancing the model's ability to capture subtle differences between icons and improving its resolution accuracy for target images. In the feature extraction stage, an improved C2f_SPD structure is introduced to optimize the sampling process, effectively alleviating the problem of detail loss during sampling and improving the model's adaptability to low-resolution icons. Subsequently, the feedforward network structure is optimized through the ConvolutionalGLU module, enabling the network to adaptively suppress irrelevant features and enhance key details, achieving adaptive selection and enhancement of discriminative features. Compared with the original model, this method improves the accuracy of similar icon detection and performance in complex testing environments, providing a valuable technical solution for vehicle central control automation testing; specific advantages include:

[0135] (1) In the feature extraction stage, the ECGA module is proposed. Through the synergistic effect of channel attention and pixel attention, the model can autonomously focus on the discriminative subtle features between icons, realize the distinction between highly similar vehicle control icons, and alleviate the problem of false detection and missed detection caused by feature confusion.

[0136] (2) An improved C2f_SPD structure is adopted during the downsampling process. The feature preservation mechanism alleviates the problem of detail loss in traditional downsampling, enhances the model's adaptability to small-sized and low-resolution icons, avoids information loss caused by traditional downsampling operations, improves the model's feature preservation ability for low-resolution icons and small-sized targets, and ensures detection stability under the changing perspectives of the robotic arm;

[0137] (3) A Convolutional GLU gating mechanism is introduced into the feedforward network. Through a spatially adaptive feature selection strategy, the model's ability to perceive key details is enhanced, while noise interference in the feature transmission process is suppressed, thereby comprehensively improving the robustness and generalization ability of the model in complex testing environments. Adaptive selection and enhancement of local features are realized, thereby improving the robustness and discriminative feature extraction ability of the model in complex testing environments;

[0138] (4) Closely combined with the actual needs of vehicle testing, solutions were designed to address core challenges such as the high similarity of central control icons and resolution fluctuations from the perspective of the robotic arm, reflecting a problem-oriented development approach for industrial scenarios.

[0139] For those skilled in the art, the specific embodiments are merely exemplary descriptions of the present invention. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for detecting icons from the perspective of a robotic arm in a vehicle's central control system, characterized in that, Specifically, the following steps are included: S1 Dataset Processing: Collect vehicle central control screen icon data, preprocess and label the collected data, and then divide the dataset proportionally; S2 Building and Training the Improved Model: Using DEIM-D-FINE as the baseline model, build the ECPA module, combine it with the C2f and SPDConv modules, introduce the C2f_SPD module, and use the ConvolutionalGLU module to optimize the feedforward network structure to obtain the improved model; set training parameters, train the improved model, and obtain the weights of the improved model; S3 Detection and Validation: Input the vehicle central control screen icon data to be detected, perform image detection, and obtain the detection results; at the same time, verify the model performance.

2. The method for detecting robotic arm view icons for vehicle central control as described in claim 1, characterized in that, In step S1, the icon data of the vehicle's central control screen is acquired through an image acquisition device, and the data is cleaned to remove duplicate images and images with image quality below the clarity threshold; the targets are labeled using Labelimg, and the labeled dataset is divided into training set, validation set and test set.

3. The method for detecting robotic arm view icons for vehicle central control as described in claim 1, characterized in that, The specific steps of step S2 are as follows: S21: Construct the improved model: Build the DEIM-D-FINE object detection algorithm framework, train it using the vehicle center console icon dataset, and obtain the initial model training weights; combine the ECA attention mechanism and pixel attention, and use residual connections and adaptive channel mapping to construct the ECPA module; during the model downsampling process, use SPDConv to improve the C2f module and apply it to the DEIM-DFINE baseline model; in the improvement of the feedforward neural network, use the Convolutional GLU module to optimize the multilayer perceptron mlp to obtain the improved model; S22: Set the training parameters, train it using the vehicle center console icon dataset, traverse all vehicle center console icon images in the training set, and obtain the weights of the improved model.

4. The method for detecting robotic arm view icons for vehicle central control as described in claim 3, characterized in that, The ECPA module constructed in step S21 consists of two branches: a first branch and a second branch. The first branch extracts local details using depthwise separable convolution, and then uses the ECA attention mechanism module to generate channel weights, enhancing the channel responses related to the target features and suppressing noisy channels. Simultaneously, the first branch also retains the features obtained from depthwise separable convolution. These retained features are concatenated with the features obtained from ECA to form a guiding graph containing global information and channel importance. The ECA module first performs global average pooling on the input features to obtain channel vectors, then adaptively calculates the one-dimensional convolution kernel size k based on the number of channels C, and further generates weights. Subsequently, it performs dynamic recalibration by multiplying the weights with the original features channel by channel, thereby enhancing the target-related channel responses and improving feature discrimination. The formula for calculating the adaptive kernel size k used in the ECA attention mechanism module is shown in (1): (1); where k is the adaptive kernel size, C is the number of input channels, γ=2, b=1; the second branch is the residual branch, which preserves the original input image and provides original detail features for subsequent processing; the information from the first branch and the second branch will be used to further extract spatial detail features from the input pixel attention.

5. The method for detecting robotic arm view icons for vehicle central control as described in claim 4, characterized in that, In step S21, the C2f module is improved using SPDConv. Specifically, after the input feature map enters the C2fSPD module, an intermediate feature map is first generated and split. The feature data on the main branch is passed to multiple Bottleneck structures containing SPDConv for processing to obtain more complex detailed features. The remaining branches only pass the original feature information. Finally, all information is merged by Concat, so as to retain the original information at the bottom level while fusing the complex information at the high level, so that the model can comprehensively utilize multi-scale and multi-level feature information. In the Bottleneck part, for the feature map with input (S,S,C1), where S is the height and width of the image and C1 is the number of channels, the SPDConv module will first divide and process it in the x and y directions. The formulas for this process are shown in (2)-(7): （2）； （3）； （4）； （5）； （6）； (7); where X is the input feature map, and scale is the downsampling factor, which is 2 in SPDConv; The input feature map (S, S, C1) is processed by SPD to obtain 4 sub-feature maps, all of which have dimensions of (S / 2, S / 2, C1). By concatenating the 4 sub-feature maps according to the channel direction, the dimensions are further changed to (S / 2, S / 2, 4C1). On this basis, the channels are transformed from 4C1 to C2 by 1×1 convolution.

6. The method for detecting robotic arm view icons for vehicle central control as described in claim 4, characterized in that, In step S21, the improvement of the feedforward neural network, specifically the optimization of the multilayer perceptron mlp using the Convolutional GLU module, involves the following steps: For the input features, the Convolutional GLU module first performs a linear transformation on the input features, and then generates branches for feature value propagation and control signals. In the branch containing the control signal generation, the feature data is processed by a 3×3 depthwise separable convolution and then a gating signal is generated using an activation function; subsequently, the outputs of the feature value propagation branch and the control signal branch are combined by pointwise multiplication to suppress useless features.

7. The method for detecting robotic arm view icons for vehicle central control as described in claim 4, characterized in that, In step S3, the vehicle's central control screen icon data from the test set is used for testing, and the model is evaluated using a validation set. Evaluation metrics include precision, recall, F1 score, mAP@50, and AP for smaller objectives. small with AR small The calculation process is shown in formulas (8)-(12); （8）； （9）； （10）； （11）； (12); where Precision is the precision rate; Recall is the recall rate; TP represents true positives, FP represents false positives, and FN represents false negatives; AP is the area under the Precision-Recall curve, and N is the number of target classes.

8. The method for detecting robotic arm view icons for vehicle central control as described in claim 4, characterized in that, In step S1, the training set, validation set, and test set are divided in a ratio of 8:1:1.

Images