Rotating convolutional neural network optimization method based on low-power-consumption Internet of Things monitoring

Abstract

The invention relates to a rotating convolutional neural network optimization method based on low-power-consumption Internet of Things monitoring, and the method comprises the steps: obtaining input image data, carrying out the preprocessing of the input image data, and generating standard input data; performing feature extraction on the standard input data based on a rotating convolution unit, and fusing multi-angle features extracted after a convolution kernel rotates at multiple angles to generate a fused feature image; performing multi-scale feature processing on the fused feature image based on a lightweight convolutional network architecture to generate a target candidate region; and performing position and classification optimization on the target candidate region, and generating a target detection result in combination with non-maximum suppression. The method has the effect of improving the resource processing efficiency of the low-power-consumption equipment.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision, and in particular to a rotational convolutional neural network optimization method based on low-power Internet of Things monitoring. Background Art

[0002] Existing security monitoring equipment is centered on low-power design and integrates multimedia processing modules, image signal processing modules and basic ARM architecture processors. Its design principle is to achieve efficient processing of fixed functions through hardware acceleration modules (such as intelligent video engine IVE), such as edge detection, moving target tracking and other simple video analysis tasks. However, the front-end processing capabilities of these devices are limited and lack sufficient computing resources to execute advanced video analysis algorithms, which limits the deployment of deep learning technology in resource-constrained devices, making it difficult to meet the real-time and responsiveness requirements of low-power devices in intelligent monitoring scenarios.

[0003] The above-mentioned existing technical solutions have the following defects: the existing monitoring equipment cannot efficiently execute complex deep learning algorithms in a resource-constrained environment, resulting in difficulty in balancing computing performance and energy efficiency, so there is room for improvement. Summary of the invention

[0004] In order to improve the resource processing efficiency of low-power devices, the present application provides a rotational convolutional neural network optimization method based on low-power Internet of Things monitoring.

[0005] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A method for optimizing a rotational convolutional neural network based on low-power Internet of Things monitoring, the method comprising: Acquire input image data, preprocess the input image data, and generate standard input data; Extracting features from the standard input data based on a rotational convolution unit, and fusing multi-angle features extracted after the convolution kernel is rotated at multiple angles to generate a fused feature image; Performing multi-scale feature processing on the fused feature image based on a lightweight convolutional network architecture to generate a target candidate region; The target candidate region is optimized in position and classification, and a target detection result is generated in combination with non-maximum suppression.

[0006] By adopting the above technical solution, by obtaining the input image data and preprocessing it, it is possible to reduce the noise and redundancy in the data, normalize the input image, and improve the accuracy of subsequent feature extraction; by extracting multi-angle features through the rotation convolution unit and fusing them to generate a fused feature image, it is possible to capture multi-directional features while reducing the number of model parameters, thereby improving the target detection capability in complex scenarios; by performing multi-scale feature processing through a lightweight convolutional network architecture, it is possible to adapt to the detection needs of targets of different sizes and enhance the robustness of the model; by optimizing the position and classification of the target candidate area and combining it with non-maximum suppression, it is possible to eliminate redundant detection boxes and further improve the accuracy and reliability of the detection results.

[0007] In one example, the present application may be further configured as follows: extracting features from the standard input data based on the rotational convolution unit, and fusing multi-angle features extracted after the convolution kernel is rotated at multiple angles to generate a fused feature image, specifically including: By rearranging the parameters of the convolution kernel according to a predetermined rotation rule, multi-angle feature extraction is performed on the standard input data to obtain feature images corresponding to the multi-angles; The feature images at multiple angles are fused through an addition operation, and the fusion results are nonlinearly transformed through an activation function to generate the fused feature image.

[0008] By adopting the above technical solution, by rearranging the parameters of the convolution kernel according to predetermined rotation rules, it is possible to capture the feature information of the input image in multiple directions and significantly improve the directional adaptability of the model; by additively fusing multi-angle feature images and performing nonlinear transformation through activation functions, it is possible to enhance the expressiveness of features and provide high-quality feature input for subsequent detection tasks.

[0009] In one example, the present application may be further configured as follows: the parameters of the convolution kernel are rearranged according to a predetermined rotation rule, specifically including: Obtaining a parameter matrix of the convolution kernel; In the horizontal rotation, the row index of the parameter matrix of the convolution kernel is reversely arranged to generate a horizontally rotated convolution kernel; In the vertical rotation, the column index of the parameter matrix of the convolution kernel is reversely arranged to generate a vertically rotated convolution kernel; In other rotations, the row and column indexes of the parameter matrix of the convolution kernel are adjusted according to the rotation angle to achieve parameter position transformation corresponding to the rotation angle.

[0010] By adopting the above technical solution, by obtaining the convolution kernel parameter matrix, reliable initialization data can be provided for the rotation operation; horizontal rotation is achieved by reverse arrangement of row indices and vertical rotation is achieved by reverse arrangement of column indices, which can effectively extract image features in the horizontal and vertical directions; by adjusting the parameter position based on the rotation angle, feature information at any angle can be extracted, significantly enhancing the network's feature capture capability in complex directional scenarios.

[0011] In one example, the present application may be further configured as follows: performing multi-scale feature processing on the fused feature image based on a lightweight convolutional network architecture to generate a target candidate region specifically includes: Performing dimensionality reduction processing on the fused feature image through a pooling layer to reduce feature dimensions; Using an upsampling module to perform multi-scale reconstruction on the fused feature image after dimension reduction to generate a multi-scale feature image; Based on the multi-scale feature image, the target candidate region for the target detection task is extracted.

[0012] By adopting the above technical solution, the feature image is fused by reducing the dimension through the pooling layer, which can reduce the amount of calculation and retain the significant feature information, thereby improving the processing efficiency of the feature map; multi-scale reconstruction is performed through the upsampling module, which can adapt to the detection needs of targets of different sizes and optimize the comprehensiveness and applicability of target detection; by extracting target candidate areas based on multi-scale feature maps, the accuracy of target positioning can be improved, and high-quality candidate areas can be provided for subsequent optimization steps.

[0013] In one example, the present application may be further configured as follows: the dimensionality reduction processing of the fused feature image by the pooling layer specifically includes: Filter by maximum value The fusion feature image is reduced in dimension, wherein the I out is the fused feature image after 3×3 maximum filtering, i and j are the offset parameters in the convolution operation, x+i and y+j are other pixels in the 3×3 area around the current pixel position after offset, and I (x+i, y+j) is the value of all adjacent pixels in the area; A data rearrangement operation is performed on the fused feature image after the maximum value filtering based on direct memory access technology.

[0014] By adopting the above technical solution, the feature image is fused through dimensionality reduction by maximum filtering, which can reduce the feature resolution while retaining significant information and improve the quality of the features after dimensionality reduction; the data of the filtered feature map is rearranged through direct memory access technology, which can reduce the consumption of hardware resources and further improve computing efficiency.

[0015] In one example, the present application may be further configured as follows: the position and classification optimization of the target candidate area specifically includes: Using a position optimization function to perform regression optimization on the detection frame of the target candidate area; A classification optimization function is used to optimize the category of the target candidate region.

[0016] By adopting the above technical solution, the detection frame of the target candidate area is regressed and optimized through the position optimization function, which can reduce the deviation between the detection frame and the actual target and improve the positioning accuracy of target detection; by optimizing the target category through the classification optimization function, it can reduce misclassification and improve the classification performance of the detection task.

[0017] In one example, the present application may be further configured as follows: the use of the position optimization function to perform regression optimization on the detection frame of the target candidate area specifically includes: Based on the detection frame information of the target candidate area, obtaining the center point parameter data of the corresponding detection frame; The center point parameter data of the detection frame is regressed using the position optimization function to obtain optimized detection frame parameters.

[0018] By adopting the above technical solution, by obtaining the center point parameters based on the detection frame information, reliable basic data can be provided for subsequent regression calculations; by optimizing the detection frame center point parameters through regression calculations, the positioning accuracy and stability of the target frame can be further improved.

[0019] In one example, the present application may be further configured as follows: optimizing the category of the target candidate area using a classification optimization function specifically includes: Acquire feature information in the target candidate area, and calculate the category probability of the candidate box through a classification optimization function; A confidence score of the candidate box is generated according to the category probability of the candidate box to improve the classification accuracy of the target detection.

[0020] By adopting the above technical solution and calculating the category probability of the target candidate area through the classification optimization function, the distribution of the target category can be accurately evaluated and the classification performance of target detection can be improved; by generating the confidence score of the candidate box, the credibility of the candidate box can be effectively quantified, providing a basis for the subsequent removal of redundant boxes.

[0021] In one example, the present application may be further configured as follows: the step of generating a target detection result by combining non-maximum suppression includes: Sorting the target candidate regions according to the confidence scores of the candidate boxes; The overlap between the candidate box corresponding to the current highest confidence score and other candidate boxes is calculated in sequence, and the candidate areas whose overlap exceeds a preset selection threshold are removed until all the candidate areas are processed to obtain the target detection result.

[0022] By adopting the above technical solution, by sorting the candidate boxes based on the confidence scores, high-confidence boxes can be processed first, thereby improving the efficiency of candidate area screening; by calculating the overlap of candidate boxes and removing candidate boxes that exceed the threshold, redundant detection boxes can be effectively reduced, thereby optimizing the accuracy and stability of the final target detection results.

[0023] In summary, this application includes the following beneficial technical effects: 1. By obtaining input image data and preprocessing it, it is possible to reduce noise and redundancy in the data, normalize the input image, and improve the accuracy of subsequent feature extraction; by extracting multi-angle features through the rotation convolution unit and fusing them to generate a fused feature image, it is possible to capture multi-directional features while reducing the number of model parameters, and improve the target detection capability in complex scenes; by performing multi-scale feature processing through a lightweight convolutional network architecture, it is possible to adapt to the detection requirements of targets of different sizes and improve the robustness of the model; by optimizing the position and classification of the target candidate area and combining it with non-maximum suppression, it is possible to eliminate redundant detection frames and further improve the accuracy and reliability of the detection results; 2. By rearranging the parameters of the convolution kernel according to the predetermined rotation rules, the feature information of the input image in multiple directions can be captured, significantly improving the directional adaptability of the model; by performing additive fusion on multi-angle feature images and performing nonlinear transformation through activation functions, the feature expression ability can be enhanced, providing high-quality feature input for subsequent detection tasks; 3. By obtaining the convolution kernel parameter matrix, reliable initialization data can be provided for the rotation operation; horizontal rotation can be achieved by reverse arrangement of row indices and vertical rotation can be achieved by reverse arrangement of column indices, which can effectively extract image features in the horizontal and vertical directions; by adjusting the parameter position based on the rotation angle, feature information at any angle can be extracted, significantly enhancing the network's feature capture capability in complex directional scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of a rotational convolutional neural network optimization method based on low-power Internet of Things monitoring in one embodiment of the present application; Figure 2 It is a flowchart for implementing step S20 in a rotational convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application; Figure 3It is a flowchart for implementing step S21 in a rotational convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application; Figure 4 It is a flowchart for implementing step S30 in a rotational convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application; Figure 5 It is a flowchart for implementing step S31 in a rotational convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application; Figure 6 This is the implementation process of step S40 in the rotation convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application. Figure 1 ; Figure 7 It is a flowchart for implementing step S41 in the rotation convolutional neural network optimization method based on low-power Internet of Things monitoring in one embodiment of the present application; Figure 8 It is a flowchart for implementing step S42 in the rotation convolutional neural network optimization method based on low-power Internet of Things monitoring in one embodiment of the present application; Fig. 9 Another implementation process of step S40 in the rotation convolutional neural network optimization method based on low-power IoT monitoring in one embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0025] The present application is further described in detail below in conjunction with the accompanying drawings.

[0026] In one embodiment, if Figure 1 As shown, the present application discloses a rotation convolutional neural network optimization method based on low-power Internet of Things monitoring, which specifically includes the following steps: S10: Obtain input image data, preprocess the input image data, and generate standard input data. Specifically, the input image data can be acquired through camera acquisition. The acquired image may contain interference factors such as different brightness, contrast, and noise. It needs to be normalized to adjust the pixel value to a uniform range for subsequent feature extraction. In addition, the image can be denoised by a Gaussian filter, and the edge features can be enhanced by wavelet transform technology. After these preprocessing steps are completed, standard input data suitable for the subsequent convolutional neural network input format is generated.

[0027] S20: Extract features from the standard input data based on the rotational convolution unit, and generate a fused feature image by fusing the multi-angle features extracted after the convolution kernel is rotated at multiple angles.

[0028] Specifically, the design of the rotational convolution unit allows the same convolution kernel to extract image features after being rotated at multiple angles. This process is achieved by adjusting the positional relationship of the convolution kernel parameters. The extracted multi-angle feature image can more comprehensively capture the feature information in different directions. For example, when detecting the boundary contours of pedestrians and vehicles, features can be obtained from the horizontal, vertical and diagonal directions respectively. Finally, these multi-angle feature images are added pixel by pixel and the results are nonlinearly transformed through the ReLU activation function to generate a fused feature image containing rich information.

[0029] S30: Perform multi-scale feature processing on the fused feature image based on a lightweight convolutional network architecture to generate target candidate regions.

[0030] Specifically, the fused feature image is processed through multiple layers of rotational convolution units and pooling layers in the network to gradually extract features of different scales; the specific layer reconstructs the low-resolution feature map through the upsampling module, and fuses the high-resolution feature map with the upsampling result through a splicing operation to form a multi-scale feature map containing global information and local details; finally, on these multi-scale feature maps, possible target candidate areas are marked through a specific target extraction algorithm, providing a basis for subsequent target optimization.

[0031] S40: Optimize the position and classification of the target candidate area, and generate the target detection result in combination with non-maximum suppression.

[0032] Specifically, the detection frame of the target candidate area is precisely adjusted through the position optimization function and the classification optimization function respectively. The position optimization function regresses the position of the target frame according to the center point, width and height information of the detection frame, while the classification optimization function calculates the category probability according to the feature information in the target candidate area. At the same time, the non-maximum suppression technology is combined to deduplicate the detection results. By calculating the overlap of the candidate frames (such as IoU), the target frame with the highest confidence and no excessive overlap is selected to finally generate the target detection result.

[0033] Furthermore, the rotational convolutional neural network model of this scheme is trained on the Ubuntu 18.04 system. The training framework is developed based on the Darknet architecture. The RMSProp optimizer is used to accelerate network convergence. The initial learning rate is set to 0.045, and the decay rate of each epoch is 0.98. The ImageNet dataset is used in the model training process, and the training cycle is 50 epochs. In order to improve the performance of the embedded intelligent video engine (IVE), the input image size is optimized and adjusted, and the standard input size is increased from 224×224 to 256×256. Through the above training process, the model can maintain a high feature extraction capability while reducing the computational complexity, and effectively adapt to the hardware resource limitations of low-power devices. As shown in Tables 1 and 2, the performance of this model is tested on the Hisilicon HI3516EV200 and MSC313E platforms. The test results show that the model can achieve fast processing on these embedded chips while maintaining high accuracy. The performance comparison on the ImageNet dataset is shown in Table 1, which evaluates the speed and CPU occupancy of different networks on the EV200 and 313E platforms. The model based on the rotation convolution kernel design of the present invention (Our, INT8 mode) achieves 69.4% Top-1 accuracy on the EV200 platform, which is comparable to other lightweight networks (such as MobileNet and ShuffleNet). On the EV200 platform, the speed of this model reaches 760FPS (floating point mode) and 123FPS (INT8 mode), and the performance on the MSC313E platform also maintains a high operating efficiency. In addition, the performance comparison on specific detection datasets is shown in Table 2, which evaluates the accuracy and speed of pedestrian and vehicle detection tasks. The model of the present invention achieves 76.12% mAP in pedestrian detection tasks and 87.04% mAP in vehicle detection tasks, which is slightly lower than models such as YOLOv3-Tiny, but has significant advantages in speed and power consumption optimization. The model achieves 874FPS (INT8 mode) on the EV200 platform and completes the detection task while maintaining a CPU occupancy of 39%, showing high hardware adaptability.

[0034] Table 1: Performance on ImageNet

[0035] Table 2: Performance on the detection dataset

[0036] In one embodiment, if Figure 2As shown, in step S20, feature extraction is performed on the standard input data based on the rotation convolution unit, and multi-angle features extracted after the convolution kernel is rotated at multiple angles are fused to generate a fused feature image, which specifically includes: S21: Rearrange the parameters of the convolution kernel according to a predetermined rotation rule to perform multi-angle feature extraction on the standard input data to obtain feature images corresponding to the multi-angles.

[0037] Specifically, the parameters of the convolution kernel can be regarded as a matrix, and rotation is achieved by adjusting the row and column indexes of the matrix. The design of the rotation convolution kernel unit allows the convolution kernel to extract features in multiple directions (such as 0°, 90°, 180°, 270°). For example, in horizontal rotation, horizontal features can be extracted by reversing the index of each row in the parameter matrix; in vertical rotation, vertical features are captured by reversing the index of each column of the parameter matrix; in other rotation directions, arbitrary angles of rotation are achieved by combining the adjustment rules of row and column indexes. These rotation operations provide multi-angle information for feature extraction, enabling the model to capture significant features in different directions in the image.

[0038] S22: The feature images at multiple angles are fused by an addition operation, and the fusion results are nonlinearly transformed by an activation function to generate a fused feature image.

[0039] Specifically, the feature image extracted at each rotation angle retains the significant features of the corresponding direction. When these feature images are fused pixel by pixel through the addition operation, the features in multiple directions are superimposed to obtain a comprehensive feature representation. In order to further enhance the nonlinear expression ability of the features, the ReLU activation function is used to transform the fused feature image pixel by pixel, setting all negative values ​​to zero and keeping the positive values ​​unchanged. This can not only improve the convergence speed of the network, but also effectively reduce the gradient vanishing problem, and finally generate a fused feature image containing multi-directional information.

[0040] In one embodiment, if Figure 3 As shown, in step S21, the parameters of the convolution kernel are rearranged according to a predetermined rotation rule, specifically including: S211: Obtain the parameter matrix of the convolution kernel.

[0041] Specifically, the parameter matrix of the convolution kernel is usually stored in the form of a two-dimensional array, where each element represents the weight value of the convolution kernel at that position. The size of the convolution kernel can be fixed, such as 3×3 or 5×5. When obtaining the convolution kernel parameters, the parameter matrix of the convolution kernel can be directly accessed through the array operation method in the programming language and loaded into the memory to provide basic data support for subsequent rotation operations.

[0042] S212: In the horizontal rotation, the row index of the parameter matrix of the convolution kernel is reversely arranged to generate a horizontally rotated convolution kernel.

[0043] Specifically, horizontal rotation is achieved by inverting the index of each row in the matrix. For example, if the convolution kernel parameter matrix is , the order of each row after reversal is , this operation changes the parameter arrangement direction of the convolution kernel, but retains the overall distribution of the weight values.

[0044] S213: In the vertical rotation, the column indexes of the parameter matrix of the convolution kernel are reversely arranged to generate a vertically rotated convolution kernel.

[0045] Specifically, for the convolution kernel parameter matrix, vertical rotation is achieved by inverting the index of each column in the matrix, for example, , after reversing the column indices, we get , this operation changes the distribution of the matrix in the vertical direction, making the convolution kernel adapt to the extraction of different feature directions.

[0046] S214: In other rotations, the row and column indexes of the parameter matrix of the convolution kernel are adjusted according to the rotation angle to achieve parameter position transformation corresponding to the rotation angle.

[0047] Specifically, when processing non-horizontal and vertical rotations, rotation at any angle can be achieved through comprehensive adjustment of row and column indices. For example, for a 45-degree rotation, the elements in the matrix can be rearranged based on the diagonal. In the specific implementation, the target position can be calculated through the mapping function of the row and column indices, and each parameter of the convolution kernel can be moved to the new position to capture the features in that direction.

[0048] In one embodiment, if Figure 4 As shown, in step S30, multi-scale feature processing is performed on the fused feature image based on a lightweight convolutional network architecture to generate a target candidate region, specifically including: S31: Perform dimensionality reduction processing on the fused feature image through the pooling layer to reduce the feature dimension.

[0049] Specifically, the pooling layer divides the region on the fused feature image through a sliding window, and extracts the maximum value in each window as the feature value of the region. For example, for a 3×3 sliding window, the 9 pixel values ​​in the window are compared and the maximum value is taken. This operation not only reduces the resolution of the feature map, but also retains the key local features. In addition, the dimensionality reduction processing method of maximum value filtering can also enhance the edges and salient areas in the image, providing high-quality input feature maps for subsequent multi-scale reconstruction.

[0050] S32: Use an upsampling module to perform multi-scale reconstruction on the fusion feature image after dimension reduction to generate a multi-scale feature image.

[0051] Specifically, the fused feature image after dimensionality reduction is restored to feature maps of various resolutions through an upsampling module. For example, bilinear interpolation can be used to expand the low-resolution feature image to high resolution pixel by pixel, or the feature map can be reconstructed in the form of layer-by-layer upsampling through deconvolution operations. The multi-scale feature maps generated by each layer can express information at different levels. For example, the high-resolution layer captures the detailed features of small targets, while the low-resolution layer retains the global characteristics of large targets. These multi-scale feature maps provide comprehensive support for the extraction of target candidate areas.

[0052] S33: Extracting target candidate regions for target detection tasks based on the multi-scale feature image.

[0053] Specifically, in the multi-scale feature image, a sliding window is used to scan each layer of the feature map, and candidate region boxes are generated based on the feature distribution in the window. The initial position and size of these candidate region boxes are calculated through a regression function, and then the confidence scores are combined to filter out the areas that may contain the target. For example, the candidate boxes for small targets are extracted from high-resolution feature maps, while the candidate boxes for large targets are generated from low-resolution feature maps. All candidate boxes are finally integrated into target candidate regions for target detection tasks.

[0054] In one embodiment, if Figure 5 As shown, in step S31, the fusion feature image is subjected to dimensionality reduction processing through a pooling layer, which specifically includes: S311: Filter by maximum value The fusion feature image is reduced in dimension, where I out is the fused feature image after 3×3 maximum filtering, i and j are the offset parameters in the convolution operation, x+i and y+j are other pixels in the 3×3 area around the current pixel position after offset, and I (x+i, y+j) is the value of all adjacent pixels in the area.

[0055] Specifically, the maximum filter is calculated in the neighborhood of each pixel position (x, y) in the feature map, using a 3×3 sliding window centered on the pixel (x+i, y+j) and passing the max i,j∈（-1,1）I (x+i, y+j) assigns the maximum pixel value in the window to the output feature map at that position, where i and j are -1, 0, and 1, respectively, to select the 3×3 neighborhood around the current pixel. The sliding window starts from the upper left corner and gradually moves in rows and columns to cover the entire input feature map. At each slide, the pixel value in the corresponding window is used to find the maximum value through this formula. For example, for the pixel values ​​{2, 5, 1, 3, 9, 4, 6, 7, 8} in a specific window, the maximum value is 9, then in the output feature map, the pixel value at the current position is set to 9, thereby retaining the significant features of each local area, and effectively reducing the resolution of the feature map, providing dimensionality reduction results for subsequent feature processing.

[0056] S312: performing data rearrangement operation on the fused feature image after maximum value filtering based on direct memory access technology.

[0057] Specifically, after completing the maximum value filtering, DMA, or direct memory access technology, is used to directly copy the feature data from the input buffer to the output buffer. DMA can skip the traditional CPU scheduling process and efficiently complete data transmission through the hardware controller, ensuring that data will not compete in a multi-threaded environment, thereby greatly improving data processing speed and resource utilization.

[0058] In one embodiment, if Figure 6 As shown, in step S40, the position and classification of the target candidate area are optimized, which specifically includes: S41: Use the position optimization function to perform regression optimization on the detection box of the target candidate area.

[0059] Specifically, the center point coordinates and width and height parameters of the detection frame are adjusted through the regression optimization function. In the specific optimization process, the deviation between the detection frame and the true frame is first calculated according to the initial position of the detection frame, and then the parameters of the detection frame are gradient regressed according to the deviation value. For example, if the center point position of the detection frame is (50, 60) and the true frame is (55, 65), the center point of the detection frame is adjusted to be close to the true value through regression calculation, thereby optimizing the positioning accuracy of the detection frame.

[0060] S42: Use a classification optimization function to optimize the category of the target candidate area.

[0061] Specifically, the classification probability is calculated for the feature distribution of each candidate box. For example, the error between the predicted probability and the true category of each category is calculated through the cross entropy loss function, and then the network parameters are adjusted through the back propagation algorithm to optimize the classification result. For example, if the initial predicted probability distribution of a candidate box is [0.1, 0.2, 0.7], and the corresponding true category is the third category, the probability distribution after optimization is adjusted to [0.05, 0.1, 0.85], thereby improving the accuracy of classification.

[0062] In one embodiment, if Figure 7 As shown, in step S41, the detection frame of the target candidate area is regressively optimized using the position optimization function, which specifically includes: S411: Based on the detection frame information of the target candidate area, obtain the center point parameter data of the corresponding detection frame.

[0063] Specifically, the boundary parameters of each candidate box (such as the coordinates of the upper left corner and the lower right corner) are calculated and converted into the coordinate values ​​of the center point. For example, for a detection box, the coordinates of its upper left corner are (20, 30), the coordinates of its lower right corner are (60, 90), and the center point is (40, 60).

[0064] S412: Perform regression calculation on the center point parameter data of the detection frame using a position optimization function to obtain optimized detection frame parameters.

[0065] Specifically, according to the positional relationship between the center point offset of the detection frame and the true frame, the gradient descent algorithm is used to update the center point and boundary parameters of the detection frame. For example, if the center point of the true frame is (42, 63) and the center point of the current detection frame is (40, 60), the update amount of the center point calculated by the position optimization function is (2, 3), thereby adjusting the detection frame parameters to be close to the true frame and improving the position accuracy of the detection frame.

[0066] In one embodiment, if Figure 8 As shown, in step S42, the classification optimization function is used to optimize the category of the target candidate area, which specifically includes: S421: Acquire feature information in the target candidate area, and calculate the category probability of the candidate box through a classification optimization function.

[0067] Specifically, the category probability distribution is predicted for the feature information in the target candidate area. For example, the category score of the candidate box is converted into a probability distribution using the Softmax function. For a certain candidate box, the category score is [2.5, 1.8, 0.9], and the category probability calculated by the Softmax function is [0.65, 0.25, 0.10], thereby determining that the most likely target category of the candidate box is the first category.

[0068] S422: Generate a confidence score for the candidate box according to the category probability of the candidate box to improve the classification accuracy of the target detection.

[0069] Specifically, the classification probability is combined with the matching degree of the detection box to generate the confidence score of the candidate box. For example, if the category probability of a candidate box is 0.85 and the IoU matching degree is 0.7, the confidence score can be calculated as 0.595 using the formula confidence = category probability × IoU. The higher the confidence, the more likely the candidate box is a valid target.

[0070] In one embodiment, if Fig. 9 As shown, in step S40, the target detection result is generated by combining non-maximum suppression, including: S43: Sort the target candidate regions according to the confidence scores of the candidate boxes.

[0071] Specifically, all candidate boxes are sorted from high to low according to their confidence scores. For example, if the confidence scores of the candidate boxes are [0.8, 0.6, 0.4], they are arranged as the first, second, and third candidate boxes, respectively, to ensure that targets with high confidence are processed and retained first.

[0072] S44: Calculate the overlap between the candidate box corresponding to the current highest confidence score and other candidate boxes in turn, and remove the candidate areas whose overlap exceeds a preset selection threshold until all candidate areas are processed to obtain the target detection result.

[0073] Specifically, the intersection over union (IoU) is used as the calculation standard for the overlap of candidate boxes. For example, the IoUs of the current highest confidence candidate box and other candidate boxes are [0.2, 0.7, 0.85] respectively. If the preset threshold is 0.5, the candidate boxes with IoU values ​​of 0.7 and 0.85 are removed, and only non-overlapping candidate boxes are retained. This process is repeated until all candidate boxes are processed to ensure that the final detection result is free of redundancy.

[0074] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A rotational convolutional neural network optimization method based on low-power Internet of Things monitoring, characterized in that: The rotation convolutional neural network optimization method based on low-power Internet of Things monitoring includes: Acquire input image data, preprocess the input image data, and generate standard input data; Extracting features from the standard input data based on a rotational convolution unit, and fusing multi-angle features extracted after the convolution kernel is rotated at multiple angles to generate a fused feature image; Performing multi-scale feature processing on the fused feature image based on a lightweight convolutional network architecture to generate a target candidate region; The target candidate region is optimized in position and classification, and a target detection result is generated in combination with non-maximum suppression.

2. According to claim 1, the rotational convolutional neural network optimization method based on low-power Internet of Things monitoring is characterized in that: The step of extracting features from the standard input data based on the rotational convolution unit and fusing the multi-angle features extracted after the convolution kernel is rotated at multiple angles to generate a fused feature image specifically includes: By rearranging the parameters of the convolution kernel according to a predetermined rotation rule, multi-angle feature extraction is performed on the standard input data to obtain feature images corresponding to the multi-angles; The feature images at multiple angles are fused through an addition operation, and the fusion results are nonlinearly transformed through an activation function to generate the fused feature image.

3. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 2 is characterized in that: The step of rearranging the parameters of the convolution kernel according to a predetermined rotation rule specifically includes: Obtaining a parameter matrix of the convolution kernel; In the horizontal rotation, the row index of the parameter matrix of the convolution kernel is reversely arranged to generate a horizontally rotated convolution kernel; In the vertical rotation, the column index of the parameter matrix of the convolution kernel is reversely arranged to generate a vertically rotated convolution kernel; In other rotations, the row and column indexes of the parameter matrix of the convolution kernel are adjusted according to the rotation angle to achieve parameter position transformation corresponding to the rotation angle.

4. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 1 is characterized in that: The performing multi-scale feature processing on the fused feature image based on the lightweight convolutional network architecture to generate a target candidate region specifically includes: Performing dimensionality reduction processing on the fused feature image through a pooling layer to reduce feature dimensions; Using an upsampling module to perform multi-scale reconstruction on the fused feature image after dimension reduction to generate a multi-scale feature image; Based on the multi-scale feature image, the target candidate region for the target detection task is extracted.

5. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 4 is characterized in that: The dimensionality reduction processing of the fused feature image by the pooling layer specifically includes: Filter by maximum value The fusion feature image is reduced in dimension, wherein the I out is the fused feature image after 3×3 maximum filtering, i and j are the offset parameters in the convolution operation, x+i and y+j are other pixels in the 3×3 area around the current pixel position after offset, and I (x+i, y+j) is the value of all adjacent pixels in the area; A data rearrangement operation is performed on the fused feature image after the maximum value filtering based on direct memory access technology.

6. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 1 is characterized in that: The optimizing the position and classification of the target candidate area specifically includes: Using a position optimization function to perform regression optimization on the detection frame of the target candidate area; A classification optimization function is used to optimize the category of the target candidate region.

7. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 6 is characterized in that: The using the position optimization function to perform regression optimization on the detection frame of the target candidate area specifically includes: Based on the detection frame information of the target candidate area, obtaining the center point parameter data of the corresponding detection frame; The center point parameter data of the detection frame is regressed using the position optimization function to obtain optimized detection frame parameters.

8. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 6 is characterized in that: The optimizing the category of the target candidate region by using the classification optimization function specifically includes: Acquire feature information in the target candidate area, and calculate the category probability of the candidate box through a classification optimization function; A confidence score of the candidate box is generated according to the category probability of the candidate box to improve the classification accuracy of the target detection.

9. The rotational convolutional neural network optimization method based on low-power Internet of Things monitoring according to claim 8 is characterized in that: The generating target detection result by combining non-maximum suppression specifically includes: Sorting the target candidate regions according to the confidence scores of the candidate boxes; The overlap between the candidate box corresponding to the current highest confidence score and other candidate boxes is calculated in sequence, and the candidate areas whose overlap exceeds a preset selection threshold are removed until all the candidate areas are processed to obtain the target detection result.

Images