Complex multi-feature curve image automatic identification method and device
By using a convolutional neural network classification model in curve image recognition, including an inverted linear bottleneck convolution module, the problem of complex curve image recognition is solved and efficient and accurate recognition effect is achieved.
Patent Information
- Application Number
- CN202311415965.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-27
- Publication Date
- 2025-04-29
AI Technical Summary
The prior art is difficult to effectively identify curved images of complex feature variations, especially in the face of diversity, noise and large-scale data.
A target convolutional neural network classification model is adopted, including the first convolutional layer, a plurality of second convolutional layers and a third layer, and the third layer includes the third convolutional layer, an average pooling layer and a fully connected layer. This model analyzes the weight relationship between features through the inverted linear bottleneck convolution module of deep separable convolution, improving recognition efficiency and accuracy.
Automatic recognition of complex multi-feature curve images is realized, recognition efficiency and accuracy are improved, and can be better applied to actual production scenarios.
Smart Images
Figure CN119904663A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a method and device for automatically recognizing complex multi-feature curve images. Background Art
[0002] At present, we can manually classify the curve images with complex feature changes as the original data set for automatic recognition of curve image types, laying a solid foundation for subsequent work. Identifying data of curve images with complex features can facilitate data processing in subsequent work and has very important application value.
[0003] For curve data containing different types of curve images, their feature changes are diverse. For example, there are an uncertain number of curves in the image, and the curves themselves have the difference between smoothness and fluctuation. It is necessary to find a recognition method based on the characteristics of the curves in the image. Extracting the required target part from the detected image and filtering out the interference factors to obtain the target image is the main research goal of image recognition technology. Automatic recognition of curve images usually relies on complex algorithms and technologies, such as computer vision, image processing, pattern recognition and machine learning. These methods combine mathematical modeling, statistical analysis and data processing technology to analyze the features in the curve image and perform pattern matching and classification. However, these methods do not have a unified automatic processing framework. When facing data patterns in specific scenarios, it is difficult to find a suitable recognition method for processing. The corresponding automatic curve recognition method can be gradually improved according to different scenarios to complete the corresponding curve recognition task.
[0004] Automatic recognition of curve images with complex features is a challenging task. The difficulty is mainly due to the changeable trend of the curve image itself, the presence of noise and distortion in the image, and the impact of data size. Since curve images can have various shapes, curvatures and complexities, some curves may be smooth, while others may have sharp turning points or irregular shapes. In some cases, curves may cross or overlap, making it difficult to identify the boundaries and shapes of each curve, which also makes the recognition task of curve images more difficult. In addition, curve images are usually affected by noise and distortion during acquisition, transmission or storage, which may cause subtle fluctuations or discontinuities on the curve, making it more difficult to accurately identify the shape of the curve. Curve graphics may contain a large number of data points. Processing large-scale data sets requires efficient algorithms and computing resources, and may increase the complexity of recognition. For curve images with different characteristics in many scenes, such as fluctuations, faster rising trends of curves, slower rising trends of curves, whether there are obvious inflection points during the rising process, and the position of the inflection points in the curve, they need to be identified as corresponding categories of curve types. However, due to the complexity and diversity of curve images, automatically recognizing all curve images with full accuracy remains a challenging problem.
[0005] To address the above problems, no effective solution has been proposed yet. Summary of the invention
[0006] The embodiments of this specification provide a method and device for automatically recognizing complex multi-feature curve images to solve the problem of difficulty in recognizing complex curves in the prior art.
[0007] The embodiment of this specification provides a complex multi-feature curve image automatic recognition method, including:
[0008] Acquire a characteristic curve image of a target to be identified;
[0009] Inputting the target characteristic curve image into a target convolutional neural network classification model to obtain a target category corresponding to the target characteristic curve;
[0010] Among them, the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result.
[0011] In one embodiment, the inverted linear bottleneck convolution module includes a fourth convolution layer, a fifth convolution layer and a sixth convolution layer; the fourth convolution layer is used to perform point-by-point convolution on the input feature matrix and adapt the dimension of the output channel according to the expansion multiple; the fifth convolution layer is used to perform channel-by-channel convolution on the feature matrix output by the fourth convolution layer to perform feature extraction; the sixth convolution layer is used to perform point-by-point convolution on the input feature matrix and restore the original channel dimension.
[0012] In one embodiment, the inverted linear bottleneck convolution module also includes a No-Local attention operation layer, and the No-Local attention operation layer is used to perform global feature extraction on the feature matrix output by the fifth convolution layer.
[0013] In one embodiment, the inverted linear bottleneck convolution module also includes an attention feature fusion layer, and the attention feature fusion layer is used to perform attention feature fusion on the feature matrix output by the sixth convolution layer.
[0014] In one embodiment, the inverted linear bottleneck convolution module also includes an Early Stopping Regularization layer, which is used to monitor the performance on the validation set during training to avoid overfitting, and is also used to evaluate the performance of the convolutional neural network classification model after each iteration process. If the performance does not improve, the model training is stopped.
[0015] In one embodiment, the target convolutional neural network classification model is a classification model based on the EfficientNet convolutional neural network.
[0016] In one embodiment, the target convolutional neural network classification model includes an improved EfficientNet-B0 convolutional neural network classification model; the EfficientNet-B0 convolutional neural network classification model includes 7 fifth convolutional layers, and the improved EfficientNet-B0 convolutional neural network classification model is achieved by adding a fifth convolutional layer between the second fifth convolutional layer and the third fifth convolutional layer of the EfficientNet-B0 convolutional neural network, and adding a fifth convolutional layer between the sixth fifth convolutional layer and the seventh fifth convolutional layer of the EfficientNet-B0 convolutional neural network.
[0017] In one embodiment, the target characteristic curve image includes one of the following: a thread make-up torque curve of an oil pipe gas seal, an electroencephalogram signal curve image, and an electrocardiogram signal curve image.
[0018] The embodiment of this specification also provides a complex multi-feature curve image automatic recognition device, including:
[0019] An acquisition module, used for acquiring a characteristic curve image of a target to be identified;
[0020] A classification module, used for inputting the target characteristic curve image into a target convolutional neural network classification model to obtain a target category corresponding to the target characteristic curve;
[0021] Among them, the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result.
[0022] The embodiments of the present specification also provide a computer device, including a processor and a memory for storing processor executable instructions, wherein when the processor executes the instructions, the steps of the complex multi-feature curve image automatic recognition method described in any of the above embodiments are implemented.
[0023] The embodiments of the present specification also provide a computer-readable storage medium on which computer instructions are stored. When the instructions are executed, the steps of the method for automatically identifying complex multi-feature curve images described in any of the above embodiments are implemented.
[0024] In an embodiment of the present specification, a complex multi-feature curve image automatic recognition method is provided, which can obtain a target feature curve image to be recognized, input the target feature curve image into a target convolutional neural network classification model, and obtain the target category corresponding to the target feature curve, wherein the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer, and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, and the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result. In this embodiment, by inputting the target feature curve image into a trained target convolutional neural network classification model, the automatic classification of the target feature curve can be achieved, and the recognition efficiency of the feature curve image can be improved. Further, since the convolutional neural network classification model includes an inverted linear bottleneck convolution module, the extracted features can be weighted analyzed, which can improve the recognition effect and accuracy of the feature curve image, and has good generalization, and can be well applied to actual production scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The drawings described herein are used to provide a further understanding of this specification, constitute a part of this specification, and do not constitute a limitation of this specification. In the drawings:
[0026] Figure 1 A flow chart of a complex multi-feature curve image automatic recognition method according to an embodiment of the present specification is shown;
[0027] Figure 2 A schematic diagram showing the structure of the MBConv module in an embodiment of this specification is shown;
[0028] Figure 3 A schematic diagram of the structure of the No-Local attention module in an embodiment of this specification is shown;
[0029] Figure 4 A schematic diagram of the structure of an AFF feature fusion module in an embodiment of this specification is shown;
[0030] Figure 5 A schematic diagram of a complex multi-feature curve image automatic recognition device in an embodiment of this specification is shown;
[0031] Figure 6 A schematic diagram of a computer device in an embodiment of the present specification is shown. DETAILED DESCRIPTION
[0032] The principles and spirit of this specification will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement this specification, and are not intended to limit the scope of this specification in any way. On the contrary, these embodiments are provided to make this specification more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.
[0033] Those skilled in the art will appreciate that the embodiments of this specification may be implemented as a system, device, method, or computer program product. Therefore, this specification may be implemented in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.
[0034] The embodiments of this specification provide a method for automatically recognizing complex multi-feature curve images. Figure 1A flow chart of a complex multi-feature curve image automatic recognition method in an embodiment of the present specification is shown. Although the present specification provides method operation steps or device structures as shown in the following embodiments or drawings, more or fewer operation steps or module units may be included in the method or device based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure described in the embodiments of the present specification and shown in the drawings. When the method or module structure is applied to an actual device or terminal product, it can be connected according to the method or module structure shown in the embodiments or drawings for sequential execution or parallel execution (for example, a parallel processor or a multi-threaded processing environment, or even a distributed processing environment).
[0035] Specifically, Figure 1 As shown, a complex multi-feature curve image automatic recognition method provided by an embodiment of this specification may include the following steps:
[0036] Step S101, obtaining a target characteristic curve image to be identified.
[0037] The image recognition method in this embodiment can be applied to a server. The server can obtain a target characteristic curve image to be recognized. The target characteristic curve image can be a curve image with complex and diverse characteristics. The target characteristic curve image can include one of the following: a torque curve of a threaded oil pipe gas seal, an electroencephalogram signal curve image, an EGG electrocardiogram signal curve image, etc.
[0038] Step S102: input the target feature curve image into a target convolutional neural network classification model to obtain a target category corresponding to the target feature curve.
[0039] The server can input the obtained target feature curve image into the target convolutional neural network classification model to obtain the target category corresponding to the target feature curve.
[0040] In some embodiments of the present specification, the target characteristic curve image includes one of the following: a thread-makeup torque curve of an oil pipe gas seal, an electroencephalogram signal curve image, and an electrocardiogram signal curve image.
[0041] For example, when the target characteristic curve image is a thread tightening torque curve of a gas seal of an oil pipe, the target category can be one of the following: qualified curve, fluctuation curve, and no torque inflection point curve. When the target characteristic curve image is an EEG signal curve image, the target category can be one of the following: normal EEG signal and abnormal EEG signal. When the target characteristic curve image is an EEG signal curve image, the target category can be one of the following: normal EEG, myocardial ischemia EEG, and coronary heart disease EEG.
[0042] The target convolutional neural network classification model may include a first convolutional layer, a plurality of second convolutional layers, and a third layer. The third layer may include a third convolutional layer, an average pooling layer, and a fully connected layer. The first convolutional layer is used to perform preliminary feature extraction on the target feature curve image. The second convolutional layer may include an inverted linear bottleneck convolution module of a depthwise separable convolution for analyzing the weight relationship between features. The third layer is used to output a classification result.
[0043] The first convolution layer can be a conventional convolution layer, which is used to perform preliminary feature extraction on the feature curve image. Each of the multiple second convolution layers can include an inverted linear bottleneck convolution (Mobile Inverted Bottleneck Convolution, MBConv) module. The MBConv module can help the classification model better analyze the weight relationship between features. The third convolution layer can be a 1×1 convolution layer. The average pooling layer is used to average the feature points in the neighborhood. The fully connected layer is used to map the learned distributed feature representation to the sample label space, playing the role of a classifier.
[0044] In the above embodiment, by inputting the target feature curve image into the trained target convolutional neural network classification model, the target feature curve can be automatically classified, and the recognition efficiency of the feature curve image can be improved. Furthermore, since the convolution neural network classification model includes an inverted linear bottleneck convolution module, the extracted features can be weighted and analyzed, which can improve the recognition effect and accuracy of the feature curve image, and has good generalization, and can be well applied in actual production scenarios.
[0045] In some embodiments of the present specification, the inverted linear bottleneck convolution module may include a fourth convolution layer, a fifth convolution layer and a sixth convolution layer. The fourth convolution layer is used to perform point-by-point convolution on the input feature matrix and resize the dimension of the output channel according to the expansion multiple. The fifth convolution layer is used to perform channel-by-channel convolution on the feature matrix output by the fourth convolution layer to extract features. The sixth convolution layer is used to perform point-by-point convolution on the input feature matrix and restore the original channel dimension.
[0046] Specifically, the MBConv module first performs 1×1 point-by-point convolution on the input feature matrix through the fourth convolution layer and changes the output channel dimension according to the expansion multiple (when the expansion multiple is 1, there is no 1×1 convolution in this layer). Then, the fifth convolution layer uses k×k channel-by-channel convolution (Depthwise Convolution) to extract features, and then ends with the sixth convolution layer (1×1 point-by-point convolution) to restore the original channel dimension, and finally performs connection inactivation. The above method can better extract features and analyze the weight relationship between features.
[0047] In some embodiments of the present specification, the inverted linear bottleneck convolution module may also include a No-Local attention operation layer, which is used to perform global feature extraction on the feature matrix output by the fifth convolution layer. After feature extraction through the fifth convolution layer, the No-Local attention operation layer directly captures long-range dependencies by calculating the interaction between any two positions without being limited to adjacent points. This is equivalent to constructing a convolution kernel as large as the size of the feature map, so that more information can be maintained, and ultimately a relatively large search range can be achieved. The corresponding weighted operation is used to increase the receptive field of the convolution process, rather than being limited to a local area. By using the No-Local attention operation layer in the MBConv module, global features can be extracted and the accuracy of automatic recognition of complex multi-feature curve images can be improved.
[0048] In some embodiments of the present specification, the inverted linear bottleneck convolution module may further include an attention feature fusion layer, and the attention feature fusion layer is used to perform attention feature fusion on the feature matrix output by the sixth convolution layer.
[0049] In this embodiment, the purpose of feature fusion is to merge the features extracted from the image into a feature that is more discriminative than the input feature. In order to improve the network's ability to utilize the extracted features, an AFF (Attentional Feature Fusion) layer can be introduced in the MBConv module to help the model better fuse the curve image features extracted by the convolutional attention module. For example, for the input features X and Y, the initial fusion is first performed using the element-by-element summation method, and the obtained fusion features are passed into two different branches to extract the attention weights. One branch uses global average pooling plus point-by-point convolution to extract the global attention of the feature, and the other branch directly uses point-by-point convolution to extract the local attention of the feature. Then the attention weights obtained by the two branches are summed element by element, and then a Sigmoid operation is performed to obtain the fusion weight. Finally, the input features are weighted averaged based on the fusion weight to obtain the final fusion feature. By adopting the attention feature fusion mechanism in the MBConv module, the semantically and scale-inconsistent features can be better fused, and the curve recognition effect and accuracy can be further improved.
[0050] like Figure 2As shown, in some embodiments of the present specification, the inverted linear bottleneck convolution module may also include an Early Stopping regularization layer, which is used to monitor the performance on the validation set during training to avoid overfitting, and is also used to evaluate the performance of the convolutional neural network classification model after each iteration process. If the performance does not improve, the model training is stopped.
[0051] In this embodiment, the generalization of the model is improved by using the Early Stopping Regularization technique. It avoids overfitting by monitoring the performance on the validation set during the training process. The performance of the model is evaluated after each iteration. If the performance does not improve, the training is stopped. This strategy improves the classification effect of the fuzzy area between different categories, thereby improving the generalization performance. The specific process of the Early Stopping Regularization technique is as follows: (1) Dataset division: the original data set is divided into a training set and a validation set. The training set is used for parameter update and fitting of the model, and the validation set is used to evaluate the performance of the model; (2) Model training: at the end of each training cycle (epoch), the performance indicators of the model on the validation set, such as accuracy, loss value, etc., are calculated; (3) Monitoring validation performance: tracking the performance indicators on the validation set and observing their changing trends. Usually, a parameter called "patience" is set to define the number of training cycles in which the validation performance does not improve; (4) Determine the stopping condition: if the validation performance does not improve within a continuous "patience" training cycle, that is, the performance begins to decline, indicating that the model begins to overfit the training data, then it can be determined to stop the training and select the model parameters with the best validation performance. The stopping criteria of the Early stopping regularization technique are usually based on the changes in performance indicators on the validation set. When the performance on the validation set no longer improves or begins to decline, the conditions for stopping training can be triggered. Common stopping criteria include: changes in loss function, changes in accuracy, Early stopping patience, and threshold settings. The choice of stopping criteria depends on the characteristics of the specific problem and data set. Different tasks may require different stopping criteria. By reasonably selecting the stopping criteria, the model can be prevented from overfitting and its generalization ability can be ensured. Early stopping can prevent overfitting, save time and resources, simplify model selection, and improve model robustness. It is of great significance. It effectively improves the generalization ability of the model of the present invention and improves the prediction accuracy of the model on unseen data. In the above manner, the curve recognition effect can be improved, the recognition accuracy and generalization can be improved, and it can be better applied to actual production scenarios.
[0052] From the perspective of real-time and lightweight industrial applications, in some embodiments of this specification, the target convolutional neural network classification model can be a classification model based on the EfficientNet convolutional neural network. By adopting the EfficientNet convolutional neural network, characteristic curve recognition can be performed simply but efficiently.
[0053] In some embodiments of the present specification, the target convolutional neural network classification model may include an improved EfficientNet-B0 convolutional neural network classification model. The EfficientNet-B0 convolutional neural network classification model may include 7 second convolutional layers. The improved EfficientNet-B0 convolutional neural network classification model is obtained by adding a second convolutional layer between the second second convolutional layer and the third second convolutional layer of the EfficientNet-B0 convolutional neural network, and adding a second convolutional layer between the sixth second convolutional layer and the seventh second convolutional layer of the EfficientNet-B0 convolutional neural network. In this embodiment, the target convolutional neural network classification model is obtained by improving the EfficientNet-B0 convolutional neural network classification model. By adding two convolutional layers to the EfficientNet-B0 convolutional neural network, the efficiency of feature curve recognition can be further improved.
[0054] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For details, please refer to the description of the above-mentioned related processing related embodiments, and no further description is given here.
[0055] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0056] The above method is described below in conjunction with a specific embodiment. However, it should be noted that the specific embodiment is only for better illustrating the present specification and does not constitute an improper limitation on the present specification.
[0057] In this specific embodiment, a method for automatic recognition of complex multi-feature curve images is provided. In this embodiment, according to the distribution law of pixels in the curve area of the curve image, from the perspective of real-time and lightweight of industrial applications, combined with the EfficientNet-B0 neural network classification model, curve data can be automatically recognized from curve images with complex feature changes, and the accuracy and efficiency of curve image data recognition can be improved, laying a solid foundation for subsequent work. The method is universal in application scenarios, and the extracted curve data can be classified into different types according to whether it fluctuates, steepness, etc. through the rising, falling, and fluctuating trends of the curve, which can be subdivided into qualified curves, fluctuating curves, and curves without torque inflection points. It performs well in batch recognition tasks with curve images containing complex backgrounds, interference or low resolution. After making relevant data sets, it can also be used in recognition tasks in the fields of oil well spiral buckling torque curves, EEG signals, EGG electrocardiogram signals, etc.
[0058] In this specific embodiment, the complex multi-feature curve image automatic recognition method can be implemented through the following steps.
[0059] Step 1: For curve image classification, the improved EfficientNet-B0 network model with 11 layers of convolution is used. First, the curve image is subjected to preliminary feature extraction through 3×3 convolution, and then a series of inverted linear bottleneck convolution modules (Mobile Inverted Bottleneck, MBConv) with depthwise separable convolution are used to help the model better analyze the weight relationship between features. Finally, a 1×1 convolution output layer is used to output the curve classification recognition result.
[0060] Step 2: Based on the model in step 1, optimize the convolution structure of the EfficientNet-B0 network model by adding the MBConv convolution module, adjusting the convolution kernel size, and the convolution step size.
[0061] Step 3: Based on step 2, a Non-Local attention mechanism is introduced to all MBConv modules to make the network pay more attention to the global information of the curve.
[0062] Step 4: Based on step 3, attention feature fusion (AFF) is introduced to all MBConv modules to merge the features extracted from the image into a feature that is more discriminative than the input feature.
[0063] Step 5: Based on step 4, early stopping regularization technology is introduced to all MBConv modules to improve the classification effect of fuzzy areas of different curve categories, thereby improving the generalization performance.
[0064] In this embodiment, the complex multi-feature curve image automatic recognition method can be used to train and test data by sending a data set made of curve graphics with complex features into the optimized EfficientNet-B0 classification network model, and finally used to identify the types of other curves. It can be subdivided into the following contents.
[0065] The specific details of the EfficientNet-B0 convolutional structure optimization are as follows.
[0066] In order to enrich the feature content extracted by the network model, based on the EfficientNet-B0 network model (as shown in Table 1, which shows the framework of the EfficientNet-B0 network model before improvement), a convolution layer is added between the original third and fourth layers of the training network. The difference between the added convolution layer and the third layer is that the convolution kernel size becomes 5×5 and the step size becomes 1, and the other parts are the same as the third layer; in addition, a convolution layer is added between the original seventh and eighth layers of the training network. The difference between the added convolution layer and the seventh layer is that the convolution kernel size becomes 3×3 and the step size becomes 1, and the other parts are the same as the seventh layer. After adding the convolution layer, EfficientNet has a total of 11 convolution layers. The adjusted EfficientNet model framework is shown in Table 2, where Operator represents the operation in the corresponding Stage, Resolution corresponds to the height and width of the output feature map of each Stage, Channels corresponds to the number of output feature channels of the Stage, and Layers indicates how many times the Operator in the Stage is repeated. Stage1 is a regular convolution layer. Please refer to Figure 2 , which shows the structural diagram of the MBConv module in this embodiment. Stage 2 to Stage 10 are composed of 21 Figure 2 The MBConv modules shown are stacked. Figure 2 As shown in , the MBConv module is an inverted residual structure that combines deep separable convolution and attention mechanism, which can be divided into two parts: the main branch and the residual branch. Figure 2 As shown, in the main branch, the input feature matrix is first convolved point by point with a step size of s1( Figure 2Conv 1×1, s1) in the stage, and changes the output channel dimension according to the expansion multiple (when the expansion multiple is 1, there is no 1×1 convolution layer). Then use k×k channel-by-channel convolution (Depthwise Convolution) for feature extraction (step size is s1 / s2), and then end with 1×1 point-by-point convolution (step size is s1) to restore the original channel dimension, and finally perform connection inactivation. Stage11 consists of a 1×1 convolution layer, an average pooling layer, and a fully connected layer. The convolution layers in Stage1 and Stage11 both contain a batch normalization operation (BatchNormalization, BN), and use the Swish activation function. In the residual branch, a 1×1 convolution kernel is first used to reduce the number of feature channels, thereby reducing the complexity of the model. Then, a depth-wise separable convolution is used to perform convolution operations on the features in the spatial and channel directions to extract feature representations with richer semantic information. Then, a 1×1 convolution kernel is used to restore the number of feature channels so that it is the same as the number of channels of the input features. Finally, the features after channel number restoration are added to the input features at the element level to form a residual branch.
[0067] Table 1
[0068]
[0069] Table 2
[0070]
[0071] The specific details of the introduction of the No-Local attention mechanism are as follows.
[0072] Since the pixel area of the curve part in the curve image occupies a small part of the whole image, and most of the other image pixel content is blank or image border part, the traditional neural network convolution operation filtering method is mostly 3×3 or 5×5 convolution kernel, and then move across the whole image. This operation processes the 3×3 local information of the image, which may cause most of the processed area to be blank or other irrelevant areas. In addition, the recognition task in the curve image not only focuses on the detailed information of each part of the curve, but also needs to pay attention to the global information of the curve, such as the overall trend of the curve and the range information of the curve part in the image. Please refer to Figure 3 , which shows a schematic diagram of the structure of the No-Local attention module in this embodiment. The specific operation process of the No-Local attention operation is as follows Figure 3As shown in the figure, by calculating the interaction between any two positions, long-range dependencies can be captured directly without being limited to adjacent points. This is equivalent to constructing a convolution kernel as large as the feature map size, which can maintain more information and ultimately achieve a larger search range. The corresponding weighted operations are used to increase the receptive field of the convolution process, rather than being limited to a local area. Figure 3 In , X is the input feature, Z is the output feature. T represents the number of feature map channels, H represents the height of the feature map, and W represents the width of the feature map. Figure 3 As shown in the figure, firstly, the input image is linearly mapped, and then the features of θ, φ, and g are obtained; the three feature dimensions mentioned above are combined, and then the matrix dot multiplication operation is performed on θ and φ to obtain something similar to the covariance matrix; then the Softmax operation is performed on the autocorrelation feature to obtain a weight of 0 to 1, which is the required attention coefficient. Finally, these coefficients are multiplied back to the feature matrix g, and then the number of channels is expanded (1×1 convolution), and the residual operation is performed with the original input image to obtain the final output result z.
[0073] The specific calculation process of No-Local operation is as follows:
[0074]
[0075] In the above formula, y is the output vector, x is the input feature map, i represents the current output position, such as space, time, or spatiotemporal index, and its response is enumerated and calculated for j. j represents the global response. The f function calculates the similarity between i and j. The g function represents the position of the calculated feature map in j. The final y is obtained after normalization by the response factor C(x).
[0076] The calculation formula of the f function is as follows:
[0077]
[0078]
[0079] The calculation formula of the g function is as follows:
[0080] g(x j )=W g x j (4)
[0081] Where W g Represents the weight matrix that needs to be learned.
[0082] like Figure 3As shown in the figure, by introducing the No-Local attention operation after the k×k channel-by-channel convolution in the MBConv module in the EfficientNet-B0 network model, the model can obtain more global information in the curve image, providing richer global features for subsequent classification.
[0083] The specific details of using the AFF feature fusion method are as follows.
[0084] The purpose of feature fusion is to merge the features extracted from the image into a feature that is more discriminative than the input feature. In most tasks, simple linear operations are usually used to achieve the fusion operation of different features. The more classic image fusion methods are concatenation and element-by-element summation. The idea of the concatenation fusion method is to directly connect the two features. If the dimensions of the two input features F1 and F2 are x and y respectively, the dimension of the output feature is x+y; the element-by-element summation method is to directly add the corresponding elements of the input features. The premise of this operation is that the dimensions of the two features are the same. Assuming that the two input features are F1 and F2, the expression of the output feature Z is Z=F1+iF2, where i is an imaginary unit. Although the feature fusion methods of concatenation and summation are simple and effective, they are not necessarily the best choice. In order to improve the network's ability to utilize the extracted features, AFF feature fusion is introduced in the EfficientNet-B0 network to help the model better fuse the curve image features extracted by the convolutional attention module.
[0085] Please refer to Figure 4 , shows a schematic diagram of the structure of the AFF feature fusion module in this embodiment. Figure 4 As shown in the figure, for the input features X and Y, the initial fusion is first performed by element-by-element summation, and the obtained fused features are passed to two different branches to extract the attention weights. One branch uses global average pooling (GlobalAvgPooling) plus point-wise convolution (Point-wise Conv) to extract the global attention of the features, and the other branch directly uses point-wise convolution to extract the local attention of the features. Then the attention weights obtained from the two branches are summed element-by-element, and then a Sigmoid operation is performed to obtain the fusion weights. Finally, the input features are weighted averaged based on the fusion weights to obtain the final fusion features. For given input features X and Y, AFF is calculated as follows:
[0086]
[0087] In the formula, represents the initial fusion operation, Represents the obtained fusion weight, whose value is a real number between 0 and 1, which allows the network to perform a weighted average between X and Y. Represents the cross product.
[0088] The specific method of introducing the Early stopping regularization technology is as follows.
[0089] The generalization of the model is improved by using the Early Stopping regularization technique, which avoids overfitting by monitoring the performance on the validation set during training, evaluates the performance of the model after each iteration, and stops training if the performance does not improve. This strategy improves the generalization performance by improving the classification effect in the fuzzy areas between different categories.
[0090] The specific process of Early Stopping Regularization is as follows: (1) Dataset division: divide the original data set into training set and validation set. The training set is used to update and fit the model parameters, and the validation set is used to evaluate the model performance. (2) Model training: at the end of each training cycle (epoch), calculate the performance indicators of the model on the validation set, such as accuracy, loss value, etc. (3) Monitor validation performance: track the performance indicators on the validation set and observe their changing trends. Usually, a parameter called "patience" is set to define the number of training cycles in which the validation performance has not improved. (4) Determine the stopping condition: if the validation performance has not improved within a continuous "patience" training cycle, that is, the performance begins to decline, indicating that the model begins to overfit the training data, then it can be determined to stop training and select the model parameters with the best validation performance.
[0091] The stopping criteria of the Early Stopping regularization technique are usually based on the change of performance indicators on the validation set. When the performance on the validation set no longer improves or begins to decline, the conditions for stopping training can be triggered. Common stopping criteria include: change in loss function, change in accuracy, patience of Early Stopping, and setting of thresholds. The choice of stopping criteria depends on the characteristics of the specific problem and dataset. Different tasks may require different stopping criteria. By reasonably selecting the stopping criteria, we can prevent the model from overfitting and ensure its generalization ability.
[0092] As a regularization technique, early stopping is of great significance in preventing overfitting, saving time and resources, simplifying model selection, and improving model robustness. It effectively improves the generalization ability of the model of the present invention and improves the prediction accuracy of the model on unseen data.
[0093] The characteristic curve recognition method in this embodiment can achieve high accuracy and good generalization for curve recognition with complex and diverse features, and can be well used in actual production scenarios.
[0094] Based on the same inventive concept, the embodiments of this specification also provide a complex multi-feature curve image automatic recognition device, as described in the following embodiments. Since the principle of solving the problem by the complex multi-feature curve image automatic recognition device is similar to that of the complex multi-feature curve image automatic recognition method, the implementation of the complex multi-feature curve image automatic recognition device can refer to the implementation of the complex multi-feature curve image automatic recognition method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable. Figure 5 is a structural block diagram of a complex multi-feature curve image automatic recognition device according to an embodiment of this specification, such as Figure 5 As shown, it includes: an acquisition module 501 and a classification module 502, and the structure is described below.
[0095] The acquisition module 501 is used to acquire the target characteristic curve image to be identified.
[0096] The classification module 502 is used to input the target feature curve image into the target convolutional neural network classification model to obtain the target category corresponding to the target feature curve.
[0097] Among them, the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result.
[0098] In some embodiments of the present specification, the inverted linear bottleneck convolution module may include a fourth convolution layer, a fifth convolution layer and a sixth convolution layer; the fourth convolution layer is used to perform point-by-point convolution on the input feature matrix and adapt the dimension of the output channel according to the expansion multiple; the fifth convolution layer is used to perform channel-by-channel convolution on the feature matrix output by the fourth convolution layer to perform feature extraction; the sixth convolution layer is used to perform point-by-point convolution on the input feature matrix and restore the original channel dimension.
[0099] In some embodiments of the present specification, the inverted linear bottleneck convolution module may further include a No-Local attention operation layer, and the No-Local attention operation layer is used to perform global feature extraction on the feature matrix output by the fifth convolution layer.
[0100] In some embodiments of the present specification, the inverted linear bottleneck convolution module may further include an attention feature fusion layer, and the attention feature fusion layer is used to perform attention feature fusion on the feature matrix output by the sixth convolution layer.
[0101] In some embodiments of the present specification, the inverted linear bottleneck convolution module may also include an Earlystopping regularization layer, which is used to monitor the performance on the validation set during training to avoid overfitting, and is also used to evaluate the performance of the convolutional neural network classification model after each iteration process. If the performance does not improve, the model training is stopped.
[0102] In some embodiments of the present specification, the target convolutional neural network classification model may be a classification model based on an EfficientNet convolutional neural network.
[0103] In some embodiments of the present specification, the target convolutional neural network may include an improved EfficientNet-B0 convolutional neural network classification model; the EfficientNet-B0 convolutional neural network classification model may include 7 second convolutional layers, and the improved EfficientNet-B0 convolutional neural network classification model is achieved by adding a second convolutional layer between the second second convolutional layer and the third second convolutional layer of the EfficientNet-B0 convolutional neural network, and adding a second convolutional layer between the sixth second convolutional layer and the seventh second convolutional layer of the EfficientNet-B0 convolutional neural network.
[0104] In some embodiments of the present specification, the target characteristic curve image includes one of the following: a thread-makeup torque curve of an oil pipe gas seal, an electroencephalogram signal curve image, and an electrocardiogram signal curve image.
[0105] From the above description, it can be seen that the embodiments of this specification achieve the following technical effects: by inputting the target feature curve image into the trained target convolutional neural network classification model, the target feature curve can be automatically classified, and the recognition efficiency of the feature curve image can be improved. Furthermore, since the convolution neural network classification model includes an inverted linear bottleneck convolution module, the extracted features can be weighted and analyzed, which can improve the recognition effect and accuracy of the feature curve image, and has good generalization, and can be better applied to actual production scenarios.
[0106] This specification also provides a computer device, which can be found in Figure 6 The computer device structure diagram of the complex multi-feature curve image automatic recognition method provided by the embodiment of this specification is shown, and the computer device may specifically include an input device 61, a processor 62, and a memory 63. Among them, the memory 63 is used to store processor executable instructions. When the processor 62 executes the instructions, the steps of the complex multi-feature curve image automatic recognition method described in any of the above embodiments are implemented.
[0107] In this embodiment, the input device may specifically be one of the main devices for information exchange between the user and the computer system. The input device may include a keyboard, a mouse, a camera, a scanner, a light pen, a handwriting input board, a voice input device, etc.; the input device is used to input the original data and the program for processing these numbers into the computer. The input device can also obtain and receive data transmitted from other modules, units, and devices. The processor can be implemented in any appropriate manner. For example, the processor can take the form of a computer-readable medium, a logic gate, a switch, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable logic controller, and an embedded microcontroller, etc., such as a microprocessor or a processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor. The memory may specifically be a memory device used to store information in modern information technology. The memory may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function without a physical form is also called a memory, such as a RAM, a FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, a TF card, etc.
[0108] In this embodiment, the functions and effects specifically realized by the computer device can be explained in comparison with other embodiments and will not be described in detail here.
[0109] In an embodiment of the present specification, a computer storage medium based on a method for automatically identifying complex multi-feature curve images is also provided, wherein the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the steps of the method for automatically identifying complex multi-feature curve images described in any of the above embodiments are implemented.
[0110] In this embodiment, the storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD) or a memory card. The memory may be used to store computer program instructions. The network communication unit may be an interface for network connection communication set in accordance with the standard specified by the communication protocol.
[0111] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other embodiments and will not be repeated here.
[0112] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of this specification can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of this specification are not limited to any specific combination of hardware and software.
[0113] It should be understood that the above description is for illustration and not for limitation. Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this specification should not be determined with reference to the above description, but should be determined with reference to the preceding claims and the full scope of equivalents to which these claims belong.
[0114] The above description is only the preferred embodiment of this specification and is not intended to limit this specification. For those skilled in the art, the embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.
Claims
1. A complex multi-feature curve image automatic recognition method, characterized in that: include: Acquire a characteristic curve image of a target to be identified; Inputting the target characteristic curve image into a target convolutional neural network classification model to obtain a target category corresponding to the target characteristic curve; Among them, the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result.
2. The complex multi-feature curve image automatic recognition method according to claim 1, characterized in that: The inverted linear bottleneck convolution module includes a fourth convolution layer, a fifth convolution layer and a sixth convolution layer; the fourth convolution layer is used to perform point-by-point convolution on the input feature matrix and adapt the dimension of the output channel according to the expansion multiple; the fifth convolution layer is used to perform channel-by-channel convolution on the feature matrix output by the fourth convolution layer to extract features; the sixth convolution layer is used to perform point-by-point convolution on the input feature matrix and restore the original channel dimension.
3. The complex multi-feature curve image automatic recognition method according to claim 2, characterized in that: The inverted linear bottleneck convolution module also includes a No-Local attention operation layer, which is used to perform global feature extraction on the feature matrix output by the fifth convolution layer.
4. The complex multi-feature curve image automatic recognition method according to claim 2, characterized in that: The inverted linear bottleneck convolution module also includes an attention feature fusion layer, which is used to perform attention feature fusion on the feature matrix output by the sixth convolution layer.
5. The complex multi-feature curve image automatic recognition method according to claim 4, characterized in that: The inverted linear bottleneck convolution module also includes an Early Stopping regularization layer, which is used to monitor the performance on the validation set during training to avoid overfitting, and is also used to evaluate the performance of the convolutional neural network classification model after each iteration process. If the performance does not improve, the model training is stopped.
6. The complex multi-feature curve image automatic recognition method according to claim 1, characterized in that: The target convolutional neural network classification model is a classification model based on the EfficientNet convolutional neural network.
7. The complex multi-feature curve image automatic recognition method according to claim 1, characterized in that: The target convolutional neural network classification model includes an improved EfficientNet-B0 convolutional neural network classification model; the EfficientNet-B0 convolutional neural network classification model includes 7 second convolutional layers, and the improved EfficientNet-B0 convolutional neural network classification model is achieved by adding a second convolutional layer between the second second convolutional layer and the third second convolutional layer of the EfficientNet-B0 convolutional neural network, and adding a second convolutional layer between the sixth second convolutional layer and the seventh second convolutional layer of the EfficientNet-B0 convolutional neural network.
8. The complex multi-feature curve image automatic recognition method according to claim 1, characterized in that: The target characteristic curve image includes one of the following: a thread-makeup torque curve of an oil pipe gas seal, an electroencephalogram signal curve image, and an electrocardiogram signal curve image.
9. A complex multi-feature curve image automatic recognition device, characterized in that: include: An acquisition module, used for acquiring a characteristic curve image of a target to be identified; A classification module, used to input the target feature curve image into a target convolutional neural network classification model to obtain a target category corresponding to the target feature curve; Among them, the target convolutional neural network classification model includes a first convolutional layer, multiple second convolutional layers, and a third layer, and the third layer includes a third convolutional layer, an average pooling layer and a fully connected layer; the first convolutional layer is used to perform preliminary feature extraction on the target feature curve image, the second convolutional layer includes an inverted linear bottleneck convolution module of a depth-separable convolution, which is used to analyze the weight relationship between features, and the third layer is used to output the classification result.
10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.