Lightweight improvement method of yolox target detection based on Android platform

Abstract

A lightweight improved method for Yolox target detection based on the Android platform includes the following steps: Step 1: collecting and preparing annotated images and corresponding label data required for training, and preprocessing the data; Step 2: replacing the original BCE cross entropy loss function with the current varifocalloss loss function; Step 3: replacing the original CSPDarknet network with the MobileNet network; Step 4: using the Yolox algorithm to train the data set in Step 1, and continuously optimizing the model parameters through the backpropagation algorithm; Step 5: converting the model format: converting the trained MobileNet network format into a format readable by Android devices; Step 6: deploying the model: deploying the converted format on the Android device, and using the GPU acceleration technology of the Android platform to accelerate the model's reasoning process; Step 7: implementing target detection. The present invention can significantly reduce the model's computational and storage overhead while ensuring target detection accuracy, thereby improving target detection performance and efficiency on mobile devices.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection in images, and in particular relates to a lightweight improved method for Yolox target detection based on the Android platform. Background Art

[0002] With the continuous development of computer vision technology, object detection has become an important research direction in the field. Object detection technology can automatically identify and locate objects in images or videos, and is widely used in traffic monitoring, intelligent security, autonomous driving and other fields.

[0003] Currently, the mainstream approach to mobile object detection is to port pre-trained deep learning models to mobile devices for inference. However, deep learning models typically have large computational and storage overheads, which often leads to performance and efficiency issues on mobile devices.

[0004] Existing model lightweighting techniques for mobile devices primarily employ methods such as pruning, quantization, and distillation to reduce the number of model parameters and computational complexity. However, these methods can reduce model accuracy and robustness, and lack the flexibility and customization to meet the needs of diverse application scenarios and devices. Furthermore, these methods often require additional computing and storage resources for training and conversion, hindering real-time inference and deployment on mobile devices. Summary of the Invention

[0005] To overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a lightweight and improved method for Yolox target detection based on the Android platform. While ensuring target detection accuracy, it can significantly reduce the computational and storage overhead of the model, thereby improving the target detection performance and efficiency on mobile devices. At the same time, the method of the present invention can fully utilize the hardware resources on the mobile device, further improving the target detection efficiency, and has practical application value.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The lightweight improvement method of yolox target detection based on Android platform includes the following steps:

[0008] Step 1: Collect and prepare the annotated images and corresponding label data required for training, and preprocess the data;

[0009] Step 2: Based on the Yolox model, an attention mechanism is introduced between the Yolox backbone layer and the data enhancement layer, and the original BCE cross entropy loss function is replaced by the varifocalloss loss function;

[0010] Step 3: Replace the original CSPDarknet network with the MobileNet network to achieve network lightweighting, reduce model size and runtime computation, and facilitate deployment and operation on Android devices.

[0011] By introducing an attention mechanism between the Yolox backbone layer and the data augmentation layer and replacing the backbone network model, the network can be lightweighted, reducing the model size and runtime computational effort, thereby better adapting to the performance limitations of mobile devices. Furthermore, the introduction of the varifocalloss loss function can also improve the model's performance on mobile devices. Compared to existing model lightweighting techniques for mobile devices, the method proposed in this paper can reduce the model size and runtime computational effort while maintaining high detection accuracy, thereby improving object detection performance and efficiency on mobile devices.

[0012] Step 4: Train the lightweight and improved MobileNet network from step 3: Use the Yolox algorithm and the dataset from step 1 to continuously optimize the model parameters through the backpropagation algorithm so that the model can accurately identify and locate the target object.

[0013] Step 5: Convert the model format: Convert the trained MobileNet network format to a format readable by Android devices. After training with the lightweight MobileNet network structure in step 4, the trained model needs to be converted to a format readable by Android devices so that it can be deployed and run on mobile devices.

[0014] Specifically, converting the trained PTH model into an ONNX model is the first step in the conversion process. The ONNX model is then converted into an NCNN format using a scripting tool for deployment and operation on mobile devices. Therefore, step 5 is a key step in converting the trained model into a format readable by mobile devices, building on step 4.

[0015] Step 6: Deploy the model: After converting the format, deploy it to the Android device and use the GPU acceleration technology of the Android platform to accelerate the model's reasoning process;

[0016] Step 6 builds on Step 5, which involved converting the trained MobileNet network into an NCNN model for deployment and execution on mobile devices. Step 6 then deploys the converted NCNN model onto an Android device and leverages the Android platform's GPU acceleration technology to accelerate the model's inference process. Therefore, Step 6 is a continuation of Step 5 and the final step in object detection on mobile devices.

[0017] Step 7: Implement object detection: Run the deployed model on the Android device, perform object detection on the input image, and return the detection results.

[0018] Step 1 prepares the data set required for training, including the following steps:

[0019] (1) Data collection: The PASCAL VOC dataset, including JPEGImages, ImageSets, and Annotations, was collected from the official website of the open dataset. JPEGImages contains the training dataset, ImageSets contains the train.txt, trainval.txt, and val.txt files of each type, and Annotations contains the XML files of each type.

[0020] (2) Data preprocessing: Data preprocessing is to preprocess the collected original images to make them suitable for model training; it includes cropping, scaling, and enhancement to facilitate model training;

[0021] First, the original image is resized to the specified size for subsequent processing;

[0022] The original image size of the selected target detection is 416x416 pixels;

[0023] Next, convert the color image into a grayscale image; this can reduce the complexity of data storage and processing, and can reduce the time and computation required for model training;

[0024] Finally, the pixel values in the image are scaled to between 0 and 1. This makes the data more stable during processing and reduces the gradient explosion and gradient vanishing problems during training.

[0025] The step 2 comprises the following steps:

[0026] (1) Introduction of attention mechanism: An attention mechanism is introduced between the backbone layer and the data enhancement layer of target detection to improve the accuracy of the model by weighting the importance of different feature maps. Specifically, this application uses the CBAM attention model to implement the weighted operation of the feature map, and inserts the channel-spatial attention mechanism CBAM module between the yolox backbone layer and the data enhancement layer channel. The CBAM module is an implementation of the channel-spatial attention mechanism, which can effectively improve the accuracy of the model.

[0027] The attention model mainly consists of two parts: channel attention and spatial attention. First, through the channel attention mechanism, the feature maps extracted by the YOLOX backbone layer are subjected to average pooling and maximum pooling operations respectively to realize the spatial information of the aggregated feature maps. The generated average pooling features and maximum pooling features are passed through the shared network layer. After the shared network is applied to each feature, the average pooling features and the maximum pooling features are element-wise summed, and the merged features are output as the channel attention map Mc through the Sigmiod activation function. Spatial attention performs average pooling and maximum pooling operations on the feature map along the channel axis to compress the feature map in the channel dimension, and splices the two feature maps in the channel dimension to generate a valid feature map, which is then passed through a 7X7 convolution layer. Finally, the final channel attention map Ms is obtained through the Sigmiod function operation.

[0028] (2) Replace the original BCE cross entropy loss function with the VariFacalLoss loss function; in the process of replacing the BCE cross entropy loss function with the VariFocalLoss loss function, the output layer of the model needs to be modified; the output layer includes a category prediction branch, a confidence prediction branch, and a bounding box prediction branch. The category prediction branch is used to predict which category each object belongs to and outputs a vector of length C (the number of categories). The confidence prediction branch is used to predict the confidence of each object, that is, it is the probability of the object and outputs a value. The bounding box prediction branch is used to predict the bounding box position of each object and outputs 4 values.

[0029] The VariFocalLoss loss function introduces a learnable exponent γ and modifies the weights and reweights in the loss function calculation formula, placing greater emphasis on learning difficult samples. Therefore, the output layer needs to be modified accordingly to adapt to this change.

[0030] In the YOLOX object detection network, the output layer usually includes a classification branch and a regression branch. In the classification branch, each object needs to be classified, while in the regression branch, the position information of each object needs to be regressed. In order to adapt to the calculation of the VariFocalLoss loss function, the prediction results of each object in the classification branch need to be processed;

[0031] Specifically, the output of the classification branch needs to be processed by the sigmoid function first, and then converted into a predicted probability, and the VariFocalLoss loss function is calculated based on this probability;

[0032] In object detection tasks, it is generally necessary to classify and regress the targets in the image. In the design of the output layer, there are two branches, corresponding to the classification and regression tasks respectively. The output of the classification branch is generally compressed to the range of 0-1 through a sigmoid function, and then converted into the corresponding prediction probability.

[0033] When using the VariFocalLoss loss function, the loss value needs to be calculated based on the predicted probability, and the output layer needs to be modified accordingly. In the output layer, the output of the classification branch needs to be processed by the sigmoid function first, and then converted into a predicted probability. This predicted probability is used as the input of the VariFocalLoss loss function to calculate the corresponding loss value. Because the VariFocalLoss loss function introduces a learnable exponent γ and modifies the weight adjustment term in the calculation formula of the loss function, it pays more attention to the learning of difficult samples. Therefore, the output layer needs to be modified accordingly to adapt to this change;

[0034] In the regression branch, since the VariFocalLoss loss function only modifies the classification branch, the calculation method of the regression branch does not need to change.

[0035] The step 3 comprises the following steps:

[0036] (1) According to the requirements of the target detection task, determine the input and output size of the MobileNet network, and adjust the depth and width of the MobileNet network according to the actual situation to achieve better detection effect and computational efficiency;

[0037] The depth adjustment adjusts the depth of the MobileNet network by increasing or decreasing the number of network layers to enhance the network's expressiveness and receptive field. The width adjustment adjusts the width of the MobileNet network by increasing or decreasing the number of channels in each layer of the network to enhance the network's feature expression capability.

[0038] (2) According to the network structure, the MobileNet network is integrated into the target detection model, and the layers corresponding to the original CSPDarknet network are replaced with the layers corresponding to the MobileNet network;

[0039] The convolution operation of the convolution layer or residual block in the CSPDarknet network is split into two parts: depthwise convolution and pointwise convolution. These two parts are then replaced with the depthwise separable convolution layer in the MobileNet network. Since the MobileNet network uses depthwise separable convolution, its calculation method is different from that of ordinary convolution, so corresponding adjustments and compatibility are required.

[0040] (3) Train the replaced target detection model, and record and evaluate the performance indicators of the model on the test set; during the training process, the network's learning rate, batch size, optimizer, etc. can be adjusted and optimized as needed to improve the performance of the model; in the field of deep learning, the adjustment of the learning rate is a basic optimization strategy to enable the model to converge faster and obtain better performance during training. The choice of learning rate needs to be adjusted according to the specific task and network structure. Usually, methods such as learning rate decay and learning rate warm-up are used to optimize the choice of learning rate. In addition, parameters such as batch size and optimizer also need to be adjusted according to the specific situation to improve model performance.

[0041] This paper uses a learning rate decay method, gradually reducing the learning rate as training progresses, to avoid problems such as exploding or vanishing gradients during training. The batch size refers to the number of samples input during each training session and needs to be adjusted based on the GPU memory capacity and the complexity of the specific task. A larger batch size can better utilize the GPU's parallel computing capabilities, speeding up training, but it also increases the GPU's memory usage and the model's generalization error.

[0042] Step 4 trains the network model, including the following steps:

[0043] (1) Use the dataset preprocessed in step 1;

[0044] (2) Divide the dataset into a dataset and a validation set in a ratio of 8:2;

[0045] (3) Input the preprocessed data set into the YOLOX target detection network, and input the network's predicted value pred and the true value gt into the loss function L. The loss value is calculated by the following formula

[0046] Loss = L(pred,gt)

[0047] Where L represents the loss function, pred represents the predicted value of the network output, and gt represents the true value. The network parameters are optimized according to the loss function L, and the gradient descent method is used to update the neural network parameters. Assuming the current neural network parameters are , the update formula is:

[0048]

[0049] Among them, η represents the learning rate, represents the gradient of the loss function L with respect to the parameter θ, where θ represents the parameter value at the t-th time step, θ t+1 Represents the parameter value at the t+1th time step. By updating the neural network parameters through multiple iterations, the network performance is optimized and the accuracy and speed of target detection are improved;

[0050] (4) After a round of parameter updates, the model needs to be tested using a validation set to verify the model's generalization ability. Specifically, the validation set is input into the YOLOX network and the loss measure between the predicted result and the true result is calculated, that is, the validation set loss. Assuming the validation set size is N, the predicted box of the i-th sample is, and the true box is, then the validation set loss L is calculated as follows:

[0051]

[0052] Among them, S is the number of prediction boxes for each grid, C is the number of target categories, and represent the predicted value and true value of the cth category in the jth grid of the i-th sample, and represent the predicted value and true value of whether there is a target in the j-th grid of the i-th sample, and represent the confidence prediction value and true value in the j-th grid of the i-th sample, respectively. It represents the index set of the prediction box with the largest intersection-over-union ratio with the true box in the j-th grid of the i-th sample and are two weight coefficients used to balance the weights of grids with targets and grids without targets.

[0053] By calculating the validation set loss, you can evaluate the performance of the current model. If the loss is high, you need to continue training until the predetermined stopping condition is reached.

[0054] (5) After two iterations, the images in the dataset are input into the optimized YOLOX object detection network for training to obtain the accuracy of the model. The optimized YOLOX object detection network has higher detection accuracy and faster detection speed;

[0055] (6) Repeat the above steps until the epoch reaches 300 rounds, at which point the model has converged.

[0056] The step 5 comprises the following steps:

[0057] (1) Export the trained model: Export the network structure and parameters from the Yolox model trained in step 4.

[0058] (2) Select the target format: According to the requirements of the Android device and deployment framework, select the appropriate target format. In this invention, the ONNX format is selected.

[0059] (3) Format conversion: Use relevant tools to convert the exported ONNX model into NCNN format and optimize it to adapt to the hardware environment and deployment framework requirements of Android devices.

[0060] (4) Verify the conversion results: Verify the converted model to ensure that it is consistent with the output results of the original model and can be used for inference operations on Android devices.

[0061] The step 6 comprises the following steps:

[0062] (1) Determine the hardware environment of the Android device: Determine the appropriate model size, accuracy, and deployment method based on the hardware environment of the Android device, such as CPU, GPU, and memory.

[0063] (2) Select a deployment framework: Select a suitable deployment framework based on the hardware environment of the Android device and the target detection requirements. In this invention, the PyTorch Mobile framework is selected as the deployment framework.

[0064] (3) Convert the model format: Convert the trained model format to a format readable by Android devices. In the present invention, the model is converted to ONNX format and then to NCNN.

[0065] (4) Deploy the model to an Android device: Package the converted model and the deployment framework into an application and deploy it to the Android device.

[0066] (5) Perform performance testing and optimization: Test and optimize the deployed model on Android devices to improve the model's inference speed and accuracy.

[0067] The step 7 comprises the following steps:

[0068] (1) Loading the converted model: Load the deployed model onto the Android device and prepare the input data.

[0069] (2) Image preprocessing: Preprocess the input image, such as scaling, cropping, normalization, etc., to adapt to the format and size of the model input.

[0070] (3) Perform model inference: Input the preprocessed image into the model, perform target detection inference operations, and obtain the detection results.

[0071] (4) Post-processing: The detection results are post-processed, such as screening, non-maximum suppression (NMS), bounding box decoding, category prediction, etc., to obtain the final target detection results.

[0072] (5) Visualization results: Visualize the target detection results, such as drawing bounding boxes, labeling categories, etc., to facilitate user viewing and understanding.

[0073] Beneficial effects of the present invention:

[0074] This invention improves target detection efficiency by maintaining the original accuracy while further enhancing it. Furthermore, by replacing the original CSPDarknet model with the MobileNet model, this invention achieves lightweight design, reduces model storage space and computational complexity, and improves the model's real-time performance on mobile devices.

[0075] On the other hand, real-time target detection on mobile terminals is realized: the present invention is based on the Android platform and can realize real-time target detection on mobile devices without the need for cloud computing resources. This is very beneficial for some application scenarios with high real-time requirements, such as smart transportation, security monitoring and other fields. It can be applied to multiple fields, and target detection technology has a wide range of applications in many fields, such as smart home, smart medical care, smart retail, etc. The present invention is based on the Android platform and can realize target detection applications in multiple fields, expanding the application scope of the technology. The present invention realizes real-time target detection on mobile devices, which can provide users with a more convenient, efficient and safe smart service experience. This is very beneficial for some application scenarios with high requirements for user experience, such as smart home, smart retail and other fields.

[0076] In summary, the beneficial effects of the present invention include improving the accuracy of target detection, realizing real-time target detection on mobile terminals, being applicable to multiple fields, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is the overall flow chart of the present invention.

[0078] Figure 2 This is the original yolox deep learning network model structure diagram.

[0079] Figure 3 This is the structure diagram of the Yolox deep learning network model that introduces the attention mechanism.

[0080] Figure 4 This is the structure diagram of the Yolox deep learning network model after modifying the backbone network.

[0081] Figure 5 This is the model conversion flow chart of the present invention.

[0082] Figure 6 This is a flowchart of the operation of the present invention on the Android platform.

[0083] Figure 7 Schematic diagram of the target detection effect of the present invention. DETAILED DESCRIPTION

[0084] The present invention will be described in further detail below with reference to the accompanying drawings.

[0085] The scenario is as follows: a server with a GPU, the downloaded PASCAL VOC 2007 dataset, an Android phone, model conversion code, and Android Studio. The data to be tested consists of object categories in a workstation environment that are related to public datasets and a subset of images from the dataset.

[0086] like Figure 1 As shown, the implementation steps of the present invention are as follows:

[0087] 1) Server-side environment deployment

[0088] Install the GPU version of the PyTorch intelligent computing framework on the server and download and install the Android development tool Android Studio.

[0089] 2) Prepare the VOC dataset

[0090] The PASCAL VOC dataset is collected from the official open dataset website, including JPEGImages, ImageSets, and Annotations. JPEGImages contains the training dataset, ImageSets contains the train.txt, trainval.txt, and val.txt files for each type, and Annotations contains the XML files for each class.

[0091] 3) Improve the loss function and introduce the attention mechanism

[0092] First, a channel-spatial attention mechanism (CBAM) module is inserted between the backbone layer and the data enhancement layer channels. The CBAM module is inserted between the backbone layer and the data enhancement layer channels to process the features extracted from these two modules. In the CBAM module, the channel attention mechanism is first processed to increase the model's attention to the feature channel. Then the spatial attention mechanism is processed to enhance the ability to extract features at different spatial locations. The output feature map of the CBAM module is then fused with the features extracted from the backbone layer and the data enhancement layer, and finally used as the input for subsequent processing, such as Figure 3As shown in the figure, in the Yolox deep learning network model that incorporates the attention mechanism, the CBAM module is inserted between the backbone and data augmentation layer channels to process the features extracted from these two modules. Specifically, the CBAM module takes as input the feature maps from the backbone and data augmentation layers. After processing them using the channel-wise and spatial-attention mechanisms, the output feature maps are fused with the features extracted from the backbone and data augmentation layers. In this process, the CBAM module processes the input feature maps using a bottleneck structure. First, a global average pooling operation is performed to obtain the global average and maximum value of each channel. These two values are then used as input to two fully connected layers to obtain channel-wise and spatial-attention weights, respectively. These weights are then used to perform a weighted average of the input feature maps to obtain the feature maps processed by the attention mechanism. Finally, the feature maps processed by the attention mechanism are fused with the features extracted from the backbone and data augmentation layers to obtain the final feature maps that serve as input for subsequent processing. In this way, the attention mechanism allows the network to focus more on important feature channels and spatial locations, thereby improving model accuracy. Next, the original BCE cross entropy loss function is replaced with the VariFacalLoss loss function. After replacing the BCE cross entropy loss function with the VariFocalLoss loss function, the model will add weight values to positive and negative samples during training to control the shared weight of positive and negative samples to the total loss function value, so that the model can focus more on difficult-to-distinguish samples during training, thereby improving the accuracy of target detection.

[0093] 4) Lightweight the model by replacing the backbone network

[0094] like Figure 4As shown, the original CSPDarknet network is replaced with the MobileNet network to achieve network lightweighting, reducing model size and runtime computational effort for deployment and execution on Android devices. The original Yolox deep learning network uses the CSPDarknet network as the backbone network. However, to efficiently deploy and execute object detection applications on Android devices, this network needs to be replaced with the lightweight MobileNet network. Therefore, the backbone network portion of the original CSPDarknet network can be replaced with the MobileNet network, and the corresponding layers can be replaced as described above. This reduces model size and computational effort, improving model efficiency and inference speed, meeting the requirements for deploying and executing object detection applications on Android devices. Furthermore, due to its smaller model size and faster inference speed, the MobileNet network can run on lower computing resources. These measures are designed to enable efficient deployment and execution of object detection applications on Android devices. The MobileNet network is a lightweight network model with a smaller model size, faster inference speed, and the ability to run on lower computing resources. By replacing the CSPDarknet network with the MobileNet network, the model size and computational complexity can be significantly reduced, and the model's operating efficiency and inference speed can be improved, thus meeting the requirements for deploying and running object detection applications on Android devices.

[0095] Comparative experiments have shown that using MobileNet as the backbone network can reduce model size to approximately 1 / 10 of its original size, while significantly improving inference speed, enabling rapid object detection on Android devices. This lightweight approach can make object detection applications more popular and practical, improving user experience and the promotional value of applications.

[0096] 5) Train the YOLOX model

[0097] The dataset is fed into the YOLOX neural network. The network's predicted and true values are fed into the loss function to calculate the loss value. The neural network parameters are then updated using gradient descent. After each round of parameter updates, the validation set is fed into the YOLOX network for verification, and the validation loss is calculated. After two iterations, the images in the dataset are fed into the trained model to determine the model's accuracy. This process is repeated until 300 epochs have been reached, indicating model convergence.

[0098] 6) Model conversion

[0099] like Figure 5As shown, the trained model is converted from PyTorch to an ONNX model through code conversion, and then converted from ONNX to an NCNN format using a scripting tool. These conversions are designed to improve model portability and deployment efficiency across different platforms. By converting the trained PyTorch model to the ONNX format, it can be supported and used by a wider range of deep learning frameworks and hardware accelerators. Further conversion of the ONNX model to the NCNN format allows for better deployment and application on mobile devices and other embedded devices.

[0100] By converting a model from one format to another, we can achieve model compression and lightweighting, while also reducing the computational effort and memory usage during deployment and runtime. This helps us efficiently run object detection tasks even with limited hardware resources, resulting in better application performance and responsiveness. Model conversion also speeds up model deployment and application across different platforms, increasing the model's practical application value.

[0101] 7) Model Deployment

[0102] like Figure 6 As shown, the converted model format is deployed to an Android device and the model's inference process is accelerated using the Android platform's GPU acceleration technology. The purpose of deploying the converted model format to an Android device and accelerating the model's inference process using the Android platform's GPU acceleration technology is to improve the model's running speed and responsiveness, making target detection tasks on mobile devices smoother and more efficient. By utilizing GPU acceleration technology, the model's computing speed and capabilities can be significantly improved, thereby reducing inference time and latency, increasing real-time and interactivity, and improving the user experience. In addition, by performing the inference process on a mobile device, user privacy can be protected and the risk of data leakage can be avoided.

[0103] 8) Implementing target detection

[0104] Run the deployed model on the Android device, perform target detection on the input image, and return the detection results. During the detection process, you can use a variety of optimization strategies, such as image preprocessing, non-maximum suppression, etc., to improve the accuracy and efficiency of detection. For target detection running on the deployed model on the Android device, the input image needs to be preprocessed, such as image resizing, normalization, channel conversion and other operations to meet the input requirements of the model. At the same time, in order to improve the accuracy and efficiency of detection, you can also use optimization strategies such as non-maximum suppression (NMS) to post-process the detection results, remove overlapping boxes and boxes with low confidence, and retain the final detection results. The detection results are as follows: Figure 7 shown.

[0105] In step 1, the quality and quantity of the dataset are crucial to the training and performance of the model. Therefore, it is necessary to select a dataset related to the object detection task and preprocess and clean it to facilitate subsequent model training and evaluation.

[0106] In step 2, a channel-spatial attention mechanism (CBAM) module is inserted between the backbone layer and the data augmentation layer of the original lightweight model. This allows the model to automatically focus on important features related to target detection when processing images, improving the network's attention and anti-interference capabilities. The original BCE cross-entropy loss function is replaced with the VariFocal Loss loss function to improve the model's classification performance and its ability to identify difficult and easy samples. VariFocal Loss introduces a variable focus parameter into the loss function, allowing the model to pay more attention to difficult-to-classify samples, thereby improving the model's classification performance.

[0107] In step 3, for target detection tasks in resource-constrained scenarios such as mobile devices, lightweight and efficient operation is achieved by replacing the backbone network model. Replacing the CSPDarknet network with the MobileNet network can reduce model size and computational complexity, reduce target detection time, and improve detection speed and efficiency. The MobileNet network is a lightweight network structure based on depthwise separable convolutions. It has a small model size and computational complexity while ensuring high accuracy, making it very suitable for deployment and operation on mobile devices.

[0108] In step 4, the improved target detection model is trained using the dataset and the model's performance indicators are recorded. The performance indicators are accuracy, recall, and precision. Accuracy represents the proportion of positive samples correctly predicted by the model; recall represents the proportion of positive samples correctly detected by the model; and precision represents the proportion of all samples correctly predicted by the model. Simultaneously, the learning rate and weight decay are adjusted during training to improve the model's performance. During network training, batch normalization is used to standardize the input data in each batch to accelerate the model's convergence and improve its generalization ability.

[0109] In step 6, the deployment process needs to take into account the hardware performance and limitations of the device, and adopt appropriate frameworks and optimization techniques to improve the model's inference speed and efficiency. At the same time, the model's security and stability also need to be considered to ensure its correctness and reliability.

[0110] In step 7, during the detection process, various optimization strategies can be employed, such as image preprocessing and non-maximum suppression, to improve detection accuracy and efficiency. At the same time, user experience and feedback must be considered, and model optimization and improvement must be performed based on user needs and feedback to enhance the model's practical application value.

[0111] This paper proposes a method for object detection on mobile devices. This method utilizes a neural network to automatically identify and locate objects in images or videos, and employs model compression and simplification algorithms to reduce computational and storage overhead. This method significantly improves object detection performance and efficiency on mobile devices and has broad application in areas such as traffic monitoring, intelligent security, and autonomous driving.

[0112] YOLOX is an object detection method based on the YOLO family of algorithms. It boasts high efficiency, accuracy, and lightweight features, and is widely used in computer vision. With the increasing popularity of mobile devices, applying the YOLOX algorithm to the Android platform is of great significance for mobile object detection.

[0113] Therefore, the present invention proposes a YOLOX target detection lightweight method based on the Android platform, which replaces the original backbone network with a large number of parameters with a lightweight network with a small amount of parameter calculation to improve the target detection performance and efficiency on mobile devices.

[0114] The purpose of these optimization strategies is to improve detection efficiency, reduce computational effort and memory usage, and adapt to the computing resource constraints of Android devices while maintaining detection accuracy. By combining preprocessing and optimization strategies, we can achieve relatively fast and accurate target detection on Android devices, meeting practical application requirements.

[0115] The above disclosure is only an example of the present invention and certainly cannot be used to limit the scope of the present invention. Ordinary technicians in this field can understand the process of implementing the above example and make equivalent changes in accordance with the claims of the present invention.

Claims

1. A lightweight improvement method for yolox target detection based on the Android platform, characterized in that: The following steps are included: Step 1: Collect and prepare the annotated images and corresponding label data required for training, and preprocess the data; Step 2: Based on the Yolox model, an attention mechanism is introduced between the Yolox backbone layer and the data enhancement layer, and the original BCE cross entropy loss function is replaced by the varifocalloss loss function; Step 3: Replace the original CSPDarknet network with the MobileNet network Step 4: Train the lightweight and improved MobileNet network from step 3: Use the Yolox algorithm and the dataset from step 1 to continuously optimize the model parameters through the backpropagation algorithm so that the model can accurately identify and locate the target object. Step 5: Convert the model format: Convert the trained MobileNet network format to a format that can be read by Android devices; Step 6: Deploy the model: After converting the format, deploy it to the Android device and use the GPU acceleration technology of the Android platform to accelerate the model's reasoning process; Step 7: Implement object detection: Run the deployed model on the Android device, perform object detection on the input image, and return the detection results; The step 3 comprises the following steps: (1) Determine the input and output sizes of the MobileNet network based on the requirements of the target detection task, and adjust the depth and width of the MobileNet network according to actual conditions; The depth adjustment adjusts the depth of the MobileNet network by increasing or decreasing the number of layers in the network, and the width adjustment adjusts the width of the MobileNet network by increasing or decreasing the number of channels in each layer of the network; (2) According to the network structure, the MobileNet network is integrated into the target detection model, and the layers corresponding to the original CSPDarknet network are replaced with the layers corresponding to the MobileNet network; The convolution operation of the convolution layer or residual block in the CSPDarknet network is split into two parts: depthwise convolution and pointwise convolution. These two parts are then replaced with the depthwise separable convolution layer in the MobileNet network. Since the MobileNet network uses depthwise separable convolution, its calculation method is different from that of ordinary convolution, so corresponding adjustments and compatibility are required. (3) Train the replaced target detection model and record and evaluate the performance indicators of the model on the test set.

2. The yolox target detection lightweight improvement method based on the Android platform according to claim 1, wherein Step 1 prepares the data set required for training, including the following steps: (1) Data collection: The PASCAL VOC dataset, including JPEGImages, ImageSets, and Annotations, was collected from the official website of the open dataset. JPEGImages contains the training dataset, ImageSets contains the train.txt, trainval.txt, and val.txt files of each type, and Annotations contains the XML files of each type. (2) Data preprocessing: Data preprocessing is to preprocess the collected original images to make them suitable for model training; First, the original image is resized to the specified size, and the selected target is detected on the original image; Next, convert the color image to grayscale; Finally, the pixel values in the image are scaled.

3. The yolox target detection lightweight improvement method based on Android platform according to claim 1, wherein The step 2 comprises the following steps: (1) Use the CBAM attention model to implement weighted operations on feature maps, and insert the channel-spatial attention mechanism CBAM module between the yolox backbone layer and the data enhancement layer channel; The attention model mainly consists of two parts: channel attention and spatial attention. First, through the channel attention mechanism, the feature maps extracted by the YOLOX backbone layer are subjected to average pooling and maximum pooling operations respectively to realize the spatial information of the aggregated feature maps. The generated average pooling features and maximum pooling features are passed through the shared network layer. After the shared network is applied to each feature, the average pooling features and the maximum pooling features are element-wise summed, and the merged features are output as the channel attention map Mc through the Sigmiod activation function. Spatial attention performs average pooling and maximum pooling operations on the feature map along the channel axis to compress the feature map in the channel dimension, and splices the two feature maps in the channel dimension to generate a valid feature map, which is then passed through the convolution layer. Finally, the final channel attention map Ms is obtained through the Sigmiod function operation. (2) Replace the original BCE cross entropy loss function with the VariFacalLoss loss function; in the process of replacing the BCE cross entropy loss function with the VariFocalLoss loss function, the output layer of the model needs to be modified; the output layer includes a category prediction branch, a confidence prediction branch, and a bounding box prediction branch. The category prediction branch is used to predict which category each object belongs to and outputs a vector of length C (the number of categories). The confidence prediction branch is used to predict the confidence of each object, that is, it is the probability of the object and outputs a value. The bounding box prediction branch is used to predict the bounding box position of each object and outputs 4 values. The VariFocalLoss loss function introduces a learnable exponent γ and modifies the weight and re-adjustment terms in the loss function calculation formula; In the YOLOX target detection network, the output layer includes a classification branch and a regression branch. In the classification branch, each target needs to be classified, while in the regression branch, the position information of each target needs to be regressed. In order to adapt to the calculation of the VariFocalLoss loss function, the prediction results of each target in the classification branch need to be processed. The output of the classification branch is first processed by the sigmoid function, and then converted into a predicted probability, and the VariFocalLoss loss function is calculated based on this probability; When using the VariFocalLoss loss function, the loss value needs to be calculated based on the predicted probability. The output layer needs to be modified accordingly. In the output layer, the output of the classification branch needs to be processed by the sigmoid function first, and then converted into the predicted probability. This predicted probability is used as the input of the VariFocalLoss loss function to calculate the corresponding loss value. In the regression branch, since the VariFocalLoss loss function only modifies the classification branch, the calculation method of the regression branch does not need to change.

4. The yolox target detection lightweight improvement method based on Android platform according to claim 1, wherein The step (4) trains the network model, including the following steps: (1) Use the dataset preprocessed in step 1; (2) Divide the dataset into a dataset and a validation set according to the proportion; (3) Input the preprocessed data set into the YOLOX target detection network, and input the network's predicted value pred and the true value gt into the loss function L. The loss value is calculated by the following formula Loss = L(pred,gt) Where L represents the loss function, pred represents the predicted value of the network output, and gt represents the true value. The network parameters are optimized according to the loss function L, and the gradient descent method is used to update the neural network parameters. Assuming that the current neural network parameters are , the update formula is: Among them, η represents the learning rate, represents the gradient of the loss function L with respect to the parameter θ, where θ represents the parameter value at the t-th time step, θ t+1 Represents the parameter value at the t+1th time step; by updating the neural network parameters through multiple iterations, the network performance is optimized and the accuracy and speed of target detection are improved; (4) After a round of parameter update, the model needs to be tested using the validation set to verify the generalization ability of the model. Specifically, the validation set is input into the YOLOX network to calculate the loss metric between the predicted result and the true result, that is, the validation set loss. Assuming the validation set size is N, the predicted box of the i-th sample is , and the true box is , the validation set loss L is calculated as follows: Among them, S is the number of prediction boxes for each grid, C is the number of target categories, and represent the predicted value and true value of the cth category in the jth grid of the i-th sample, and represent the predicted value and true value of whether the target exists in the j-th grid of the i-th sample, and represent the confidence prediction value and true value in the j-th grid of the i-th sample, respectively. It represents the index set of the prediction box with the largest intersection-over-union ratio with the true box in the j-th grid of the i-th sample and are two weight coefficients used to balance the weights of grids with targets and grids without targets; Evaluate the performance of the current model by calculating the validation set loss. If the loss is high, continue training until the predetermined stopping condition is reached. (5) After two iterations, the images in the dataset are input into the optimized YOLOX object detection network for training to obtain the accuracy of the model; (6) Repeat the above steps until the model has converged.

5. The yolox target detection lightweight improvement method based on Android platform according to claim 1, wherein The step 5 comprises the following steps: (1) Export the trained model: Export the network structure and parameters from the Yolox model trained in step 4; (2) Select the target format: Select the appropriate target format based on the requirements of the Android device and deployment framework; (3) Format conversion: Use relevant tools to convert the exported ONNX model into NCNN format and optimize it; (4) Verify the conversion results: Verify the converted model to ensure that it is consistent with the output results of the original model and can be used for inference operations on Android devices.

6. The yolox target detection lightweight improvement method based on the Android platform according to claim 1, wherein The step 6 comprises the following steps: (1) Determine the hardware environment of the Android device: Determine the appropriate model size, accuracy, and deployment method based on the hardware environment of the Android device, such as CPU, GPU, and memory; (2) Select a deployment framework: Select an appropriate deployment framework based on the hardware environment of the Android device and the target detection requirements; (3) Convert model format: Convert the trained model format to a format that can be read by Android devices; (4) Deploy the model to an Android device: Package the converted model and the deployment framework into an application and deploy it to the Android device; (5) Perform performance testing and optimization: Test and optimize the deployed model on Android devices.

7. The yolox target detection lightweight improvement method based on the Android platform according to claim 1, wherein The step 7 comprises the following steps: (1) Load the converted model: Load the deployed model onto the Android device and prepare the input data; (2) Image preprocessing: preprocess the input image; (3) Perform model inference: Input the preprocessed image into the model, perform target detection inference operations, and obtain the detection results; (4) Post-processing: Post-process the detection results to obtain the final target detection results; (5) Visualization results: Visualize the target detection results.

Images