Lightweight rule connected domain recognition algorithm

Abstract

The embodiment of the invention provides a lightweight rule connected domain recognition algorithm, which comprises the following steps: acquiring an input image, completing a connected domain positioning task by adopting a target detection algorithm, and judging the position of a rule connected domain; if it is identified that the input image contains the regular connected domain, segmenting the regular connected domain in the input RGB image according to the prediction box, and performing subsequent color identification; and performing feature extraction by adopting depth separable convolution, and outputting the feature. High-dimensional semantic features of connected domains are extracted through a deep neural network, feature representation of regular connected domains and feature representation of irregular connected domains are distinguished, the problems that feature parameters are sensitive, irregular domains are mistakenly recognized, and manual design feature robustness is poor due to traditional manual design features are solved, and meanwhile, through lightweight network structure design, the design efficiency is improved. The parameter quantity and the floating point operand of the algorithm model are reduced, the model reasoning time delay is reduced, end-side deployment and operation of the algorithm are ensured, and the problem of rule connected domain recognition is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and pattern recognition, and in particular to a lightweight regular connected domain recognition algorithm. Background Art

[0002] The Chinese patent with the patent number CN115937238A discloses a method, device, computing device and storage medium for identifying paint types. It proposes a recognition algorithm based on threshold segmentation to address the problems existing in the identification of connected domains. The algorithm sets the H, S, and V empirical threshold ranges of connected domains of different colors through a large number of image samples collected on site, and segments the connected domains according to the thresholds, and then comprehensively judges the type according to the number of pixels in the connected domain and the distance between different connected domains. For some images of irregular connected domains, the algorithm has achieved accurate recognition results. However, the algorithm also has some problems. For example: ① Since it is necessary to filter out some irregular connected domains with lighter colors through color thresholds, the H, S, and V threshold ranges are set to be relatively narrow to ensure accurate recognition results, which makes the threshold setting process more cumbersome and sensitive to changes in the color of the connected domain, making it difficult to adjust to the optimal value; ② For some finer irregular connected domains, the connected domain area obtained by the threshold segmentation method is often large, which is prone to misidentification; ③ For some irregular connected domains, it is difficult to set universal features for them, resulting in the inability to ensure recognition accuracy. Summary of the invention

[0003] In view of the above-mentioned problems existing in the prior art, an embodiment of the present invention provides a lightweight regular connected domain identification algorithm, which extracts high-dimensional semantic features of the connected domain through a deep neural network, distinguishes the feature representations of regular connected domains and irregular connected domains, and avoids the problems of feature parameter sensitivity, irregular domain misidentification, and poor robustness of manually designed features caused by traditional manually designed features. At the same time, through the lightweight network structure design, the parameter amount and floating-point calculation amount of the algorithm model are reduced, the model inference delay is reduced, the algorithm terminal deployment and operation are guaranteed, and the regular connected domain identification problem is solved.

[0004] The embodiment of the present invention provides a lightweight regular connected domain identification algorithm, including:

[0005] Get the input image, use the target detection algorithm to complete the connected domain positioning task, and determine the position of the regular connected domain;

[0006] If the input image is identified to contain a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition;

[0007] Depthwise separable convolution is used for feature extraction and output.

[0008] In some embodiments of the present invention, when selecting the target detection algorithm, it is determined based on three aspects: model reasoning speed, model accuracy and deployment difficulty.

[0009] In some embodiments of the present invention, the target detection algorithm is used to complete the connected domain positioning task, and the position of the regular connected domain is determined, including:

[0010] Design a regular connected domain recognition algorithm model based on YOLOv5, and identify regular connected domains through the designed YOLOv5 model;

[0011] If the input image is identified to contain a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition.

[0012] In some embodiments of the present invention, the use of a target detection algorithm to complete the connected domain positioning task and determine the position of the regular connected domain also includes:

[0013] If it is recognized that the input image does not contain a regular connected domain, the recognition is terminated.

[0014] In some embodiments of the present invention, when performing color recognition, the method includes:

[0015] Set an empirical threshold range for the H, S, and V channels of each color connected domain, and segment the images of the three channels according to the empirical threshold range;

[0016] Take the intersection of the connected domain segmentation results of the same color to obtain the number of pixels in the connected domain area of ​​the same color;

[0017] If the number of pixels meets the set value, the corresponding color result is output.

[0018] In some embodiments of the present invention, the depthwise separable convolution includes depthwise convolution and pointwise convolution, and the depthwise convolution groups the input feature map according to feature channels when performing the convolution operation.

[0019] In some embodiments of the present invention, the feature extraction using depthwise separable convolution includes:

[0020] The feature extraction module of the YOLOv5 model is introduced into the depth-separable convolution. Specifically, YOLOv5 Model The Mobile-C3 module is used to improve the C3 module , use the IBN module to replace the BottleNeck module in the original C3 module.

[0021] In some embodiments of the present invention, the IBN module adopts an inverted bottleneck structure.

[0022] In some embodiments of the present invention, when performing feature extraction, the method specifically includes:

[0023] Mix the information of the input feature map and adjust the feature map dimension through 1×1 point-by-point convolution;

[0024] Use 3×3 depth convolution to extract features for each channel;

[0025] Use 1×1 point-wise convolution to reduce the feature map to the output dimension of the BottleNeck module.

[0026] Compared with the prior art, the beneficial effect of the lightweight regular connected domain identification algorithm provided by the embodiment of the present invention is that it extracts high-dimensional semantic features of the connected domain through a deep neural network, distinguishes the feature representations of regular connected domains and irregular connected domains, and avoids the problems of feature parameter sensitivity, irregular domain misidentification, and poor robustness of manually designed features caused by traditional manually designed features. At the same time, through the lightweight network structure design, the parameter amount and floating-point calculation amount of the algorithm model are reduced, the model inference delay is reduced, the algorithm terminal deployment and operation are guaranteed, the regular connected domain identification problem is solved, and it is suitable for industrial field deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of standard convolution in depth convolution and point-by-point convolution in the lightweight regular connected domain recognition algorithm provided in an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of deep convolution and point-by-point convolution in a lightweight regular connected domain recognition algorithm provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of an improved Mobile-C3 module in a lightweight regular connected domain identification algorithm provided in an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of an IBN module in a lightweight regular connected domain identification algorithm provided in an embodiment of the present invention;

[0031] Figure 5 Schematic diagram of a paint data set in a lightweight regular connected domain recognition algorithm provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0034] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0035] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0036] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0037] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to determine the true intent based on the user's historical operations and to avoid unnecessary or redundant details that make the present application unclear. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but are merely used as the basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0038] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0039] The embodiment of the present invention provides a lightweight regular connected domain identification algorithm, such as Figures 1 to 5 As shown, the method includes:

[0040] Get the input image, use the target detection algorithm to complete the connected domain positioning task, and determine the position of the regular connected domain;

[0041] If the input image is identified to contain a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition;

[0042] Depthwise separable convolution is used for feature extraction and output.

[0043] To facilitate understanding of the above technical solution, a detailed description is given below:

[0044] (1) Overall process of the rule-connected domain identification algorithm

[0045] The methods in the prior art are difficult to ensure the accuracy of connected domain recognition. The main reason is that it is difficult to manually design stable and reliable connected domain features. The deep learning algorithm extracts deep semantic features in an "end-to-end" manner, which to a certain extent avoids the problems caused by manually designed features and has been widely used in the industrial field in recent years. In view of the advantages of deep learning algorithms, this technical solution designs a regular connected domain recognition algorithm based on deep learning. In the recognition process, in order to judge the position of the regular connected domain, the target detection algorithm is used to complete the connected domain positioning task. In actual tasks, the selection of target detection algorithms is mainly considered from three aspects: model reasoning speed, model accuracy and deployment difficulty. Because the YOLOv5 model is lightweight, highly accurate, easy to deploy and has extensive related research, it has been widely used in the industrial field. This technical solution designs a regular connected domain recognition algorithm model based on YOLOv5.

[0046] The recognition process of the designed regular connected domain recognition algorithm model mainly includes two parts: YOLOv5 connected domain recognition and color recognition. The YOLOv5 model is used to identify regular connected domains. If the model recognizes that the input image does not contain a regular connected domain, the recognition ends; if the model recognizes that the input image contains a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition.

[0047] For the color recognition process, refer to the recognition algorithm based on threshold segmentation in the previous patent. First, set an empirical threshold range for the H, S, and V channels of each color connected domain, segment the images of the three channels according to the empirical threshold range, and take the intersection of the connected domain segmentation results of the same color to obtain the number of pixels in the connected domain area of ​​the same color. If the number of pixels meets the set value, the corresponding color result is output. It is worth noting that since the prediction box has been identified as containing a regular connected domain, the threshold range of H, S, and V can be set relatively wide to ensure that the connected domain color is not missed.

[0048] (2) Lightweight recognition algorithm module

[0049] In order to ensure that this algorithm can be easily deployed on the terminal side and that the inference latency meets the requirements, this technical solution algorithm introduces a lightweight feature extraction module to obtain a lightweight regular connected domain recognition algorithm model. This model maintains a high recognition accuracy and meets the inference latency requirements of actual deployment.

[0050] The Backbone module and Neck module of the YOLOv5 model widely use the C3 module for feature extraction. In the C3 module, standard convolution operations are generally used. Standard convolution operations are computationally intensive. In this recognition scenario, depthwise separable convolution can be used to replace them. Depthwise separable convolution consists of two parts: depthwise convolution and pointwise convolution. Depthwise convolution is different from standard convolution, such as Figure 1 As shown in Figure 1, the depthwise convolution processes the input feature maps by groups according to feature channels during the convolution operation. Under the same number of input and output channels, compared with the standard convolution, the use of depthwise separable convolution can effectively reduce the number of parameters and floating-point operations of the convolution layer.

[0051] The algorithm of this technical solution introduces deep separable convolution into the feature extraction module of YOLOv5. Specifically, the C3 module of YOLOv5 is improved to the Mobile-C3 module, such as Figure 2 As shown, all BottleNeck modules in the original C3 module are replaced with lighter IBN (Inverted BottleNeck) modules, and other structures remain unchanged.

[0052] The IBN module is designed with reference to the MobileNet-V2 network structure and uses an inverted bottleneck structure, such as Figure 3 As shown. Specifically, for the input feature map, first use 1×1 point-by-point convolution to mix the information of the input feature map and adjust the feature map dimension. For the setting of the number of output channels of the 1×1 convolution layer, this algorithm uses twice the number of output channels of the 1×1 convolution layer of the BottleNeck module in the original C3 module. Then use 3×3 depth convolution to extract the features of each channel, and finally use 1×1 point-by-point convolution to reduce the dimension of the feature map to the output dimension of the BottleNeck module in the original C3 module. For the setting of residual connection, it is the same as the YOLOv5 model. When the Backbone module uses Mobile-C3, shortcut=True, and residual connection is used to add the input feature map to the output feature map; when the Neck module uses Mobile-C3, residual connection is not used.

[0053] When using the algorithm of this technical solution for specific implementation, it is necessary to first build a training data set. Taking a specific scenario that uses a regular connected domain recognition algorithm - oil pipe paint recognition as an example, this data set is based on 40,982 RGB oil pipe images with a resolution of 640×640 collected on-site at the oil pipe maintenance plant to create an oil pipe paint data set, in which normal paint is a regular connected domain, and sticky paint and hand-painted paint are irregular connected domains. In the process of data set creation, 917 images were selected from the collected images as the paint recognition data set. Some images in the data set are entered Figure 5As shown in Figure 1, images containing sticky paint are a type of sample that are not labeled during the actual training process, but are instead classified as belonging to the YouTube category.

[0054] 70% (640 images) of the dataset is used as the training set, and the remaining 20% ​​(277 images) is used as the test set, and the proportion of different types of images in the training set and the test set is guaranteed to be equal.

[0055] Subsequently, the training set is input into the lightweight oil pipe paint recognition model for training, because the model will eventually be deployed on the CPU side. The experimental code is written using Python3.8 and Pytorch1.13.1, and the 2022.3.0 version of the OpenVino development kit is used. During the model training process, for the data input end, some common methods are used to enhance the data, such as: ① Mosaic data enhancement, four training samples are randomly cropped and then spliced ​​into an image as training data; ② Random horizontal / vertical flipping, flipping the image horizontally or vertically at a certain ratio; ③ HSV enhancement, increasing the diversity of the data by adjusting the hue, saturation, and brightness of the image. The training process sets the number of samples in a batch (batch-size) to 8, the number of iterations (epoch) to 300 rounds, and the parameters are optimized using stochastic gradient descent. The initial learning rate is set to 0.01, and the cosine annealing method is used to dynamically adjust the learning rate during training.

[0056] After the training is completed, the model optimizer of the OpenVino platform is used to optimize the trained network model, compress some structures in the network, and then convert them into an intermediate representation file in IR format. Finally, the inference engine is responsible for the forward reasoning of the model. In the testing phase, the model achieved 98.5mAP and 38.61fps on an Intel(R) Core(TM) i5-6500 CPU@3.19GHz and 16G memory platform, meeting the accuracy requirements of actual scenarios and satisfying the inference latency requirements. This specific implementation case verifies the effectiveness of the lightweight rule-connected domain recognition algorithm.

[0057] It can be seen from the above technical scheme that the beneficial effect of the lightweight regular connected domain identification algorithm provided by the above embodiment of the present invention is that it extracts high-dimensional semantic features of the connected domain through a deep neural network, distinguishes the feature representations of regular connected domains and irregular connected domains, and avoids the problems of feature parameter sensitivity, irregular domain misidentification, and poor robustness of manually designed features caused by traditional manually designed features. At the same time, through the lightweight network structure design, the parameter amount and floating-point calculation amount of the algorithm model are reduced, the model inference delay is reduced, the algorithm terminal deployment and operation are guaranteed, the regular connected domain identification problem is solved, and it is suitable for industrial site deployment.

[0058] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A lightweight regular connected domain identification algorithm, characterized in that: include: Get the input image, use the target detection algorithm to complete the connected domain positioning task, and determine the position of the regular connected domain; If the input image is identified to contain a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition; Depthwise separable convolution is used for feature extraction and output.

2. The lightweight regular connected domain identification algorithm according to claim 1 is characterized in that: When selecting the target detection algorithm, it is determined based on three aspects: model reasoning speed, model accuracy and deployment difficulty.

3. The lightweight regular connected domain identification algorithm according to claim 2 is characterized in that: The target detection algorithm is used to complete the connected domain positioning task and determine the position of the regular connected domain, including: Design a regular connected domain recognition algorithm model based on YOLOv5, and identify regular connected domains through the designed YOLOv5 model; If it is recognized that the input image contains a regular connected domain, the regular connected domain in the input RGB image is segmented according to the prediction box for subsequent color recognition.

4. The lightweight regular connected domain identification algorithm according to claim 3 is characterized in that: The target detection algorithm is used to complete the connected domain positioning task and determine the position of the regular connected domain, and further includes: If it is recognized that the input image does not contain a regular connected domain, the recognition is terminated.

5. The lightweight regular connected domain identification algorithm according to claim 4 is characterized in that: When performing color recognition, the method includes: Set an empirical threshold range for the H, S, and V channels of each color connected domain, and segment the images of the three channels according to the empirical threshold range; Take the intersection of the connected domain segmentation results of the same color to obtain the number of pixels in the connected domain area of ​​the same color; If the number of pixels meets the set value, the corresponding color result is output.

6. The lightweight regular connected domain identification algorithm according to claim 5, characterized in that: The depthwise separable convolution includes depthwise convolution and pointwise convolution. The depthwise convolution groups the input feature map according to feature channels when performing the convolution operation.

7. The lightweight regular connected domain identification algorithm according to claim 6, characterized in that: The feature extraction using depth-wise separable convolution includes: The feature extraction module of the YOLOv5 model is introduced into the depth-separable convolution. Specifically, YOLOv5 model Improve the C3 module with the Mobile-C3 module , use the IBN module to replace the BottleNeck module in the original C3 module.

8. The lightweight regular connected domain identification algorithm according to claim 7 is characterized in that: The IBN module adopts an inverted bottleneck structure.

9. The lightweight regular connected domain identification algorithm according to claim 8, characterized in that: When performing feature extraction, the method specifically includes: Mix the information of the input feature map and adjust the feature map dimension through 1×1 point-by-point convolution; Use 3×3 depth convolution to extract features of each channel; Use 1×1 point-wise convolution to reduce the feature map to the output dimension of the BottleNeck module.

Images