Student pointer type ammeter detection and reading method based on rotating small target detection
By building an improved rotation object detection model, the problem of low reading efficiency with large scale differences and high density in student pointer meter detection is solved, and high-precision detection and reading accuracy are achieved.
Patent Information
- Application Number
- CN202411236548.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art has problems of low reading efficiency and insufficient accuracy in student pointer meter detection, especially when facing scales with large scale differences and high density, the rotary frame detection method is still insufficient.
Using a method based on rotational small object detection, the improved rotational object detection model is built, including enhanced expansion residual blocks that enhance the effective receptive field, path aggregation network feature fusion module and lightweight detection head network to capture different scales and any directions. Features, improve detection accuracy.
The detection accuracy of targets of different sizes is significantly improved, feature fusion is optimized, detection performance and accuracy is improved, deep and shallow features are effectively integrated, and the accuracy and reliability of the final detection results are improved.
Smart Images

Figure CN119091426A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision technology, and in particular to a method for detecting and reading a student pointer meter based on rotating target detection. Background Art
[0002] With the advent of the era of artificial intelligence, machine vision technology and deep learning technology have been widely used in various fields. At present, in the scoring of physics, chemistry and biology experiments in middle schools, in addition to relying on traditional invigilators, practical and effective technical means are needed to objectively and real-timely score experimental operations. Pointer meters are indispensable devices in physical electromagnetic experiments. Currently, they mainly rely on manual readings by invigilators. This method is prone to negligence and erroneous readings, and requires a lot of human resources to ensure the accuracy of scoring, which is inefficient.
[0003] In recent years, with the significant progress of deep learning technology, many schemes have tried to use detection models to capture complete meter images from outdoor scenes. Subsequently, the meter pointer and scale are separated by image segmentation methods for accurate reading. However, this scheme is easily affected by lighting and shadows, resulting in limited segmentation effect and slow reading speed. In order to reduce reading time, some other schemes use target detection methods to directly read the meter. However, due to the large scale differences of the meter pointer, different scale density and angle, horizontal box detection is prone to missed detection due to excessive IoU calculation, and there is also a lot of redundancy. In order to more accurately adapt the pointer scale, the rotation box detection method came into being, avoiding the problem of missed detection caused by excessive IoU calculation. However, the current mainstream rotation detection method still has some shortcomings when facing scales with large scale differences and small sizes, which results in low efficiency in judging the meter readings of students in the examination room.
[0004] Therefore, in the related art, there is an urgent need for a method that can improve the detection and reading accuracy of students' pointer-type electric meters. Summary of the invention
[0005] Based on this, it is necessary to provide a student pointer meter detection and reading method based on rotating small target detection, which can improve the accuracy of student pointer meter detection and reading, in order to address the above technical problems.
[0006] In a first aspect, the present application provides a method for detecting and reading a student pointer meter based on rotating small target detection. The method comprises:
[0007] Obtain image data of pointer-type meters in students' electrical experiments, and obtain training data sets after preprocessing;
[0008] Build a rotation target detection model, which includes an improved enhanced dilated residual block with enhanced effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block with enhanced effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction;
[0009] Training the rotating target detection model based on the training data set;
[0010] Input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results;
[0011] A student pointer meter reading is determined based on the target position and category detection results.
[0012] Optionally, in one embodiment of the present application, the preprocessing includes:
[0013] Annotating the image data, marking pointers, scales and surface areas;
[0014] Convert the format of the annotated image data.
[0015] Optionally, in one embodiment of the present application, the improved enhanced dilated residual block for enhancing the effective receptive field includes a network module with a multi-branch residual structure, which is used to capture detail features, medium-scale features and global structural information.
[0016] Optionally, in one embodiment of the present application, the path aggregation network feature fusion module includes a feature alignment module, a multi-scale feature aggregation and a cross-layer feature fusion module, wherein the feature alignment module realizes direct splicing of cross-level feature maps by adjusting the number of channels, downsampling and upsampling operations, the multi-scale feature aggregation extracts scale sequence features by horizontal stacking and 3D convolution of feature maps, and the cross-layer feature fusion module enhances the feature fusion effect through weighted feature fusion and position attention mechanism.
[0017] Optionally, in one embodiment of the present application, determining the student pointer meter reading based on the target position and category detection result includes:
[0018] Determine coordinate information based on the target position and category detection result, and determine the midpoint of the scale line and the pointer line based on the coordinate information;
[0019] identifying the center of the dial based on the equation of the straight line corresponding to the midpoint;
[0020] The student pointer meter reading is determined based on the center of the dial and the midpoints of each scale line.
[0021] Optionally, in one embodiment of the present application, the method further includes:
[0022] The dial image is orthogonalized by using perspective transformation based on the target position and category detection results.
[0023] In the second aspect, the present application also provides a student pointer meter detection and reading device based on rotating small target detection. The device comprises:
[0024] The data acquisition module is used to obtain the image data of the pointer meter in the student's electrical experiment and obtain the training data set after preprocessing;
[0025] A rotation target detection model building module, used to build a rotation target detection model, the target detection model includes an improved enhanced dilated residual block for enhancing the effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block for enhancing the effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction;
[0026] A model training module, used for training the rotating object detection model based on the training data set;
[0027] The rotating target detection module is used to input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results;
[0028] The student pointer type electric meter reading determination module is used to determine the student pointer type electric meter reading based on the target position and category detection results.
[0029] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the methods described in the above embodiments.
[0030] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in each of the above embodiments are implemented.
[0031] The above-mentioned student pointer meter detection and reading method based on rotating small target detection, first, obtains the image data of the pointer meter in the student electrical experiment, and obtains the training data set after preprocessing; then, builds a rotating target detection model, the target detection model includes an improved enhanced dilated residual block with enhanced effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block with enhanced effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction; then, the rotating target detection model is trained based on the training data set; then, the image data of the meter to be detected is input into the trained rotating target detection model to obtain the target position and category detection results; finally, the student pointer meter reading is determined based on the target position and category detection results. That is to say, the present application proposes a method for detecting small rotating targets based on a multi-scale residual structure and an optimized feature fusion mechanism, designs an enhanced multi-scale residual structure, and expands the effective receptive field of the deep layer by introducing a multi-scale dilated convolution module in the backbone network, thereby significantly improving the detection accuracy of targets of different sizes. At the same time, a structure in which three layers of features are aggregated first and then distributed is proposed, which solves the problem of cross-layer information fusion in the neck network. The structure achieves efficient information interaction, reduces feature loss, and improves the accuracy of feature fusion by optimizing the aggregation and distribution of features. This improvement ensures that features of different scales and levels can interact effectively, thereby improving the detection performance. In addition, in terms of feature fusion in the output layer, a weighted feature fusion mechanism is introduced to solve the shortcomings of existing models in feature screening. By designing a weighted structure, the features are dynamically weighted, and more weights are given to important features, thereby improving the detection effect. This mechanism ensures that deep and shallow features can be more effectively fused, optimizing the accuracy and reliability of the final detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is an application environment diagram of a method for detecting and reading a student pointer type electric meter based on detecting a rotating small target in one embodiment;
[0033] Figure 2 A schematic diagram of a flow chart of a method for detecting and reading a student pointer type electric meter based on detecting a rotating small target in an embodiment;
[0034] Figure 3 is a schematic diagram of a rotating target detection model in one embodiment;
[0035] Figure 4 A schematic diagram of an improved enhanced dilated residual block for enhancing an effective receptive field in one embodiment;
[0036] Figure 5 is a schematic diagram of the structure of a feature alignment module in one embodiment;
[0037] Figure 6 is a schematic diagram of the structure of a cross-layer feature fusion module in one embodiment;
[0038] Figure 7 A schematic diagram of the structure of a lightweight detection head network in one embodiment;
[0039] Figure 8 It is a structural block diagram of a student pointer type electric meter detection and reading device based on rotating small target detection in one embodiment;
[0040] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0042] The student pointer meter detection and reading method based on rotating small target detection provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.
[0043] In one embodiment, Figure 2 As shown, a method for detecting and reading a student pointer meter based on rotating small target detection is provided. Figure 1 The server in the example is used to illustrate the following steps:
[0044] S201: Obtain image data of a pointer-type electric meter in a student electrical experiment, and obtain a training data set after preprocessing.
[0045] In the embodiment of the present application, first, a data set is prepared, the entire process of the student electrical experiment is recorded at an oblique viewing angle, the existing video is framed, and image data of different positions of the student pointer meter is obtained, and then a clear dial image is taken with a mobile device to expand the data set. The data set is preprocessed, including steps such as annotation and format conversion, to obtain a training data set.
[0046] Specifically, in one embodiment of the present application, the preprocessing includes:
[0047] S301: Annotate the image data, annotate the pointer, scale and surface area.
[0048] S303: Convert the format of the annotated image data.
[0049] In one embodiment of the present application, the dataset is annotated by roLabelImgV2 software, and the dial annotations are divided into three categories: pointers, scales, and surface areas. The dataset annotated by LabelImg software can generate a json type annotation file, which needs to be converted into a format that meets the training requirements of the YOLOv8 network before it is officially put into the network model for training. Specifically, the original image information, polygonal bounding box coordinate information, annotated target category and other information can be obtained by parsing the json file, and a target detection label file with the same name as the json annotation file but of type txt can be generated.
[0050] S203: Build a rotation target detection model, which includes an improved enhanced dilated residual block for enhancing the effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block for enhancing the effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction.
[0051] In the present application embodiment, Figure 3 As shown in the figure, the initial rotation target detection model is built based on the YOLOv8 network model, which is mainly composed of a backbone, a neck and a detection head network, including an improved enhanced dilated residual block for enhancing the effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network. Among them, the improved enhanced dilated residual block for enhancing the effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction.
[0052] Specifically, in one embodiment of the present application, the improved enhanced dilated residual block for enhancing the effective receptive field includes a network module with a multi-branch residual structure, which is used to capture detail features, medium-scale features and global structural information.
[0053] In one embodiment of the present application, the backbone network adopts a stacking operation similar to ResNet to stack a large number of bottleneck structures, but as the number of layers increases, the utility of the effective receptive field decreases marginally. Generally speaking, as the semantic representation of features is enhanced, deeper stages often require larger receptive fields. The receptive field calculation formula is as follows.
[0054]
[0055] Among them, l k-1 is the receptive field size corresponding to the k-1th layer, f k is the convolution kernel size of the kth layer, or the pooling size of the pooling layer. In order to enhance the backbone network's ability to extract multi-scale features, such as Figure 4 As shown in the figure, the bottleneck is replaced by a network module with a multi-branch residual structure in the last two C2f modules of the backbone network. Specifically, the input feature is first reduced to half of the original number of channels through a 3x3 convolution, and two parallel dilated convolutions with different hole rates d=3 and d=5 are introduced into the original structure to form a three-branch structure, and convolution kernels of different sizes are used to capture features of different scales and complexities. The 3x3 convolution kernel can well capture the details and high-frequency features in the image, which helps to maintain image quality. Compared with the standard 3x3 convolution, the dilated convolution with d=3 can cover a larger area, which helps to capture medium-scale features and contextual information. With a larger receptive field, the dilated convolution with d=5 can capture long-range dependencies in the image, which helps to understand the global structure and background information in the image. Finally, the concatenated feature map is converted to the same number of channels as the input channel using a 1x1 convolution. This design enables the network to learn and represent features at different levels to obtain feature maps with different receptive field sizes, thereby capturing features of different scales. In order to enhance the model's multi-scale perception ability of the target, thereby improving the target detection performance.
[0056] The structure based on path aggregation has the problem that adjacent layers cannot be directly fused. If features are to be transferred between the highest layer and the lowest layer, they can only be transferred indirectly by iterative upsampling or downsampling convolution operations in the middle layer, which inevitably causes some feature information loss during the fusion process. Therefore, based on the path aggregation network, this application combines cross-level multi-scale features with high-level features to construct a cross-level multi-scale feature fusion structure.
[0057] Specifically, in one embodiment of the present application, the path aggregation network feature fusion module includes a feature alignment module, a multi-scale feature aggregation and a cross-layer feature fusion module, wherein the feature alignment module realizes direct splicing of cross-level feature maps by adjusting the number of channels, downsampling and upsampling operations, the multi-scale feature aggregation extracts scale sequence features by horizontal stacking and 3D convolution of feature maps, and the cross-layer feature fusion module enhances the feature fusion effect by weighted feature fusion and position attention mechanism.
[0058] In one embodiment of the present application, a feature alignment module FAM (Feature Align Moudle) is used to perform cross-layer splicing to replace indirect splicing, and its structure is as follows: Figure 5 As shown in the figure. By adjusting the number of channels, downsampling and upsampling operations, the information provided by feature maps of different sizes can be effectively utilized to improve the recognition performance of densely overlapping small targets. Specifically, the number of channels of large, medium and small feature maps is first adjusted to align them with the feature sizes of deeper layers. For large-sized feature maps, downsampling is performed through a parallel structure of maximum pooling and average pooling to retain high-resolution features and detail information; for small-sized feature maps, upsampling is performed using nearest neighbor interpolation to maintain local features of low-resolution images and prevent the loss of small target feature information. Finally, the processed large, medium and small-sized feature maps are convolved and spliced together by channel. The output feature map has the same resolution as the S4 feature map, but the number of channels is three times that of the S4 feature map. By introducing a cross-level feature alignment module, the model is able to more directly fuse the information provided by feature maps of different depths.
[0059] Multi-scale feature aggregation is a structure in which three layers of features are aggregated first and then distributed to achieve multi-layer feature information interaction. The specific structural modules are: Figure 3 Scale-fusion in the module has a similar structure to the feature alignment module. Feature maps of different scales are stacked horizontally and their scale sequence features are extracted using 3D convolution. Since the high-resolution feature map level S3 contains information that is crucial for small object detection, the S3 level is used as the alignment reference. Specifically, 1x1 convolution is used to adjust the number of channels of the S4 and S5 feature layers and the nearest neighbor interpolation method is used to adjust the spatial size. Next, each feature layer is converted from a 3D tensor (height, width, channel) to a 4D tensor (depth, height, width, channel). Then, the 4D feature maps are spliced along the depth dimension to form a 3D feature map for subsequent convolution. Finally, 3D convolution, 3D batch normalization, and SiLU activation function are used to complete the scale sequence feature extraction.
[0060] In order to better distinguish the impact of different feature layers on the neck output results, a weighted structure is used to improve the feature fusion mechanism. Figure 6 As shown. First, the input feature map is convolved for feature extraction, then added for feature fusion, and then the obtained mixed features are globally averaged and pooled to obtain a feature map with a size of 1 and a number of channels C. The feature map is then passed through a one-dimensional convolution and a sigmoid function to obtain the weight of each channel, and the output value is between 0 and 1. In order to perform a weighted average on the input features of the position attention, the output value is used as the feature Figure 1 The weight is 1 minus the output value as the feature Figure 2 The weights of are used to achieve a soft-selected weighted average effect. The two weights are multiplied by the corresponding input features and then added together to complete the initial feature fusion. The calculation method of this part is shown in the following formula.
[0061] f out =α·f 1 +(1-α)·f 2
[0062] Among them, f 1 、f 2 is the input feature, and α is the weight map calculated by one-dimensional convolution and Sigmoid function.
[0063] The formula for the Sigmoid function is:
[0064]
[0065] Here, x represents the value of the matrix.
[0066] Next, the average values of the reduced feature map in height and width are calculated to obtain the average feature maps of rows and columns, which are used as references for subsequent attention weight calculations. Then, the two average feature maps are concatenated and converted into feature maps with the same number of channels as the original input feature map through convolution, ReLU activation function and batch normalization operations to achieve information fusion. Next, the fused feature map is divided into row and column parts, and the attention weights of rows and columns are obtained through two convolution layers respectively. These attention weights are mapped by the Sigmoid function and limited between 0 and 1, indicating the importance of each position. Finally, the attention weights of rows and columns are broadcasted on the corresponding dimensions to obtain the final attention matrix. The input feature map is multiplied by the attention matrix to obtain the output feature map after position attention adjustment. In general, this structure is a simple weighting and enhancement of the two features of the output layer to improve the detection effect.
[0067] In the present application embodiment, Figure 7The figure shows the structure of the lightweight detection head network, which adds the prediction of the target rotation angle. First, the shared convolution is defined. The structure consists of standard convolutions with group normalization. Then, the number of parameters is reduced through shared convolution. The Scale layer dynamically adjusts the scale of the feature map through a learnable scaling factor. The Scale layer is used for feature scaling, and finally the angle prediction and classification results are output. The predicted angle value is used to calculate the loss during model training and is used to decode the rotation bounding box during inference, thereby realizing target detection in any direction.
[0068] S205: Training the rotating target detection model based on the training data set.
[0069] In the embodiment of the present application, the training data set is divided into a training set and a test set in a ratio of 8:2. The rotation target detection model built based on the YOLOv8 network model is trained until convergence to obtain the optimal model. The device used in this application is NVIDIA A100, and the entire model is implemented in PyTorch. The initial learning rate is 0.01, the optimization strategy uses the SGD function, the weight decay is set to 0.0005, and the batch size is 16.
[0070] S207: Input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results.
[0071] In the embodiment of the present application, after the rotating target detection model is trained, the electric meter image data to be detected is input into the rotating target detection model, and a set of result sets describing the position and category are obtained through model reasoning. Among them, the pointer category is 0, the scale category is 1, the surface category is 2, and the coordinates of each category target are (x1y1, x2y2, x3y3, x4y4).
[0072] S209: Determine the student pointer meter reading based on the target position and category detection results.
[0073] In the embodiment of the present application, further calculations are performed based on the obtained target position and category detection results to determine the student pointer meter reading.
[0074] Specifically, in one embodiment of the present application, determining the student pointer meter reading based on the target position and category detection result includes:
[0075] S401: Determine coordinate information based on the target position and category detection result, and determine the midpoint of the scale line and the pointer line based on the coordinate information.
[0076] S403: Identify the center of the dial based on the straight line equation corresponding to the midpoint.
[0077] S405: Determine the student pointer meter reading based on the center of the dial and the midpoints of each scale line.
[0078] In one embodiment of the present application, the vertex coordinates of the scale lines and the pointer line are extracted from the detection results, and the midpoint of each line is calculated. Then, the linear equations of these midpoints are calculated by geometric methods. The linear equations are calculated by two-point formula to calculate the slope k and the intercept b. According to these linear equations, the intersection sets between the scale lines and the pointer lines are solved respectively. In order to accurately identify the center position of the dial, the K-means clustering method is used to cluster the intersections, and the cluster center is the center of the dial. Based on the center of the dial and the midpoint of each scale line, the angle of the pointer relative to the scale line is calculated, and the vector dot product formula is used to calculate the angle between the two vectors as the decimal part of the scale, and the total angle of the dial and the pointer position ratio are used as the integer part, and finally the specific reading pointed by the pointer is calculated.
[0079] In one embodiment of the present application, the method further includes:
[0080] The dial image is orthogonalized by using perspective transformation based on the target position and category detection results.
[0081] In one embodiment of the present application, considering the balance between target detection speed and accuracy, the detection accuracy is further improved, and the dial image is orthogonalized through perspective transformation based on the detection result. Specifically, the perspective transformation process uses a transformation matrix M to transform the source point coordinates to the target point coordinates so that the surface image is parallel to the paper surface.
[0082] In the above-mentioned student pointer meter detection and reading method based on rotating small target detection, first, the image data of the pointer meter in the student electrical experiment is obtained, and a training data set is obtained after preprocessing; then, a rotating target detection model is built, and the target detection model includes an improved enhanced dilated residual block with enhanced effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block with enhanced effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction; then, the rotating target detection model is trained based on the training data set; then, the image data of the meter to be detected is input into the trained rotating target detection model to obtain the target position and category detection results; finally, the student pointer meter reading is determined based on the target position and category detection results. That is to say, the present application proposes a method for detecting small rotating targets based on a multi-scale residual structure and an optimized feature fusion mechanism, designs an enhanced multi-scale residual structure, and expands the effective receptive field of the deep layer by introducing a multi-scale dilated convolution module in the backbone network, thereby significantly improving the detection accuracy of targets of different sizes. At the same time, a structure in which three layers of features are aggregated first and then distributed is proposed, which solves the problem of cross-layer information fusion in the neck network. The structure achieves efficient information interaction, reduces feature loss, and improves the accuracy of feature fusion by optimizing the aggregation and distribution of features. This improvement ensures that features of different scales and levels can interact effectively, thereby improving the detection performance. In addition, in terms of feature fusion in the output layer, a weighted feature fusion mechanism is introduced to solve the shortcomings of existing models in feature screening. By designing a weighted structure, the features are dynamically weighted, and more weights are given to important features, thereby improving the detection effect. This mechanism ensures that deep and shallow features can be more effectively fused, optimizing the accuracy and reliability of the final detection results.
[0083] The following is a specific implementation example to illustrate the specific implementation steps of the student pointer meter detection and reading method based on rotating small target detection of the present application. First, S501, obtain the image data of the pointer meter in the student electrical experiment, and obtain the training data set after preprocessing. Specifically, S503-S505, annotate the image data, annotate the pointer, scale and surface area; and convert the format of the annotated image data.
[0084] Afterwards, S507, a rotation target detection model is constructed, wherein the target detection model includes an improved enhanced dilated residual block for enhancing effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block for enhancing effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize arbitrary direction target detection. Among them, S509, the improved enhanced dilated residual block for enhancing effective receptive field includes a network module with a multi-branch residual structure, which is used to capture detail features, medium-scale features and global structural information. S511, the path aggregation network feature fusion module includes a feature alignment module, a multi-scale feature aggregation and a cross-layer feature fusion module, wherein the feature alignment module realizes direct splicing of cross-level feature maps by adjusting the number of channels, downsampling and upsampling operations, the multi-scale feature aggregation extracts scale sequence features by horizontal stacking and 3D convolution in feature maps, and the cross-layer feature fusion module enhances the feature fusion effect by weighted feature fusion and position attention mechanism.
[0085] Then, in step S513, the rotating target detection model is trained based on the training data set. In step S515, the image data of the electric meter to be detected is input into the trained rotating target detection model to obtain the target position and category detection results. In step S517, the dial image is orthogonalized using perspective transformation based on the target position and category detection results.
[0086] Finally, S519, the student pointer meter reading is determined based on the target position and category detection results. Specifically, S521-S525, coordinate information is determined based on the target position and category detection results, the midpoint of the scale line and the pointer line is determined based on the coordinate information; the center of the dial is identified based on the straight line equation corresponding to the midpoint; the student pointer meter reading is determined based on the center of the dial and the midpoint of each scale line.
[0087] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0088] Based on the same inventive concept, the embodiment of the present application also provides a student pointer meter detection and reading device based on rotating small target detection for realizing the student pointer meter detection and reading method based on rotating small target detection involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the student pointer meter detection and reading device based on rotating small target detection provided below can refer to the limitations of the student pointer meter detection and reading method based on rotating small target detection above, and will not be repeated here.
[0089] In one embodiment, Figure 8 As shown, a student pointer meter detection and reading device 800 based on rotating small target detection is provided, comprising: a data acquisition module 801, a rotating target detection model building module 803, a model training module 805, a rotating target detection module 807 and a student pointer meter reading determination module 809, wherein:
[0090] The data acquisition module 801 is used to acquire the image data of the pointer-type electric meter in the student electrical experiment and obtain the training data set after preprocessing.
[0091] The rotating target detection model building module 803 is used to build a rotating target detection model, and the target detection model includes an improved enhanced dilated residual block with enhanced effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block with enhanced effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction.
[0092] The model training module 805 is used to train the rotating target detection model based on the training data set.
[0093] The rotating target detection module 807 is used to input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results.
[0094] The student pointer meter reading determination module 809 is used to determine the student pointer meter reading based on the target position and category detection results.
[0095] In one embodiment of the present application, the data acquisition module is further used to:
[0096] Annotating the image data, marking pointers, scales and surface areas;
[0097] Convert the format of the annotated image data.
[0098] In one embodiment of the present application, the improved enhanced dilated residual block for enhancing the effective receptive field includes a network module with a multi-branch residual structure, which is used to capture detail features, medium-scale features and global structural information.
[0099] In one embodiment of the present application, the path aggregation network feature fusion module includes a feature alignment module, a multi-scale feature aggregation and a cross-layer feature fusion module, wherein the feature alignment module realizes direct splicing of cross-level feature maps by adjusting the number of channels, downsampling and upsampling operations, the multi-scale feature aggregation extracts scale sequence features by horizontal stacking and 3D convolution of feature maps, and the cross-layer feature fusion module enhances the feature fusion effect by weighted feature fusion and position attention mechanism.
[0100] In one embodiment of the present application, the student pointer meter reading determination module is also used for:
[0101] Determine coordinate information based on the target position and category detection result, and determine the midpoint of the scale line and the pointer line based on the coordinate information;
[0102] identifying the center of the dial based on the equation of the straight line corresponding to the midpoint;
[0103] The student pointer meter reading is determined based on the center of the dial and the midpoints of each scale line.
[0104] In one embodiment of the present application, the student pointer meter reading determination module is also used for:
[0105] The dial image is orthogonalized by using perspective transformation based on the target position and category detection results.
[0106] Each module in the above-mentioned student pointer meter detection and reading device based on rotating small target detection can be fully or partially implemented by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0107] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a student pointer meter detection and reading method based on rotating small target detection is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0108] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0109] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0111] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0115] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for detecting and reading student pointer type electric meters based on rotating small target detection, It is characterized in that The method comprises: Obtain image data of pointer-type meters in students' electrical experiments, and obtain training data sets after preprocessing; Build a rotation target detection model, which includes an improved enhanced dilated residual block with enhanced effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block with enhanced effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction; Training the rotating target detection model based on the training data set; Input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results; A student pointer meter reading is determined based on the target position and category detection results.
2. The method for detecting and reading a student pointer type electric meter based on detecting a rotating small target according to claim 1, It is characterized in that The pre-processing comprises: Annotating the image data, marking pointers, scales and surface areas; Convert the format of the annotated image data.
3. The method for detecting and reading a student pointer type electric meter based on detecting a rotating small target according to claim 1, It is characterized in that The improved enhanced dilated residual block for enhancing the effective receptive field includes a network module with a multi-branch residual structure, which is used to capture detail features, medium-scale features and global structural information.
4. The method for detecting and reading a student pointer type electric meter based on detecting a rotating small target according to claim 1, It is characterized in that The path aggregation network feature fusion module includes a feature alignment module, a multi-scale feature aggregation and a cross-layer feature fusion module, wherein the feature alignment module realizes direct splicing of cross-level feature maps by adjusting the number of channels, downsampling and upsampling operations, the multi-scale feature aggregation extracts scale sequence features by horizontal stacking and 3D convolution of feature maps, and the cross-layer feature fusion module enhances the feature fusion effect by weighted feature fusion and position attention mechanism.
5. The method for detecting and reading a student pointer type electric meter based on detecting a rotating small target according to claim 1, It is characterized in that Determining the student pointer meter reading based on the target position and category detection result includes: Determine coordinate information based on the target position and category detection result, and determine the midpoint of the scale line and the pointer line based on the coordinate information; identifying the center of the dial based on the equation of the straight line corresponding to the midpoint; The student pointer meter reading is determined based on the center of the dial and the midpoints of each scale line.
6. The method for detecting and reading a student pointer type electric meter based on detecting a rotating small target according to claim 1, It is characterized in that The method further comprises: The dial image is orthogonalized by using perspective transformation based on the target position and category detection results.
7. A student pointer meter detection and reading device based on rotating small target detection, It is characterized in that The device comprises: The data acquisition module is used to obtain the image data of the pointer meter in the student's electrical experiment and obtain the training data set after preprocessing; A rotation target detection model building module, used to build a rotation target detection model, the target detection model includes an improved enhanced dilated residual block for enhancing the effective receptive field, a path aggregation network feature fusion module and a lightweight detection head network, wherein the improved enhanced dilated residual block for enhancing the effective receptive field is used to capture features of different scales, the path aggregation network feature fusion module is used to realize multi-scale target detection, and the lightweight detection head network is used to realize target detection in any direction; A model training module, used for training the rotating object detection model based on the training data set; The rotating target detection module is used to input the image data of the electric meter to be detected into the trained rotating target detection model to obtain the target position and category detection results; The student pointer type electric meter reading determination module is used to determine the student pointer type electric meter reading based on the target position and category detection results.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Water meter value reading method based on target detection and segmentation identification
CN113569835A
Recognition method, device and equipment for indication of pointer type instrument and storage medium
CN116844058A
Target detection method and system fusing visual expansion mechanism and path aggregation
CN117994531A
Remote sensing image target detection method
CN118314434A
Reading identification method for pointer type instrument
CN118351521A
Cited By
SAR image target detection method and device based on data augmentation
CN120563796A
A SAR image target detection method and device based on data augmentation
CN120563796B
