Adaptive industrial surface defect segmentation network establishment method and application thereof

Abstract

The application discloses an adaptive industrial surface defect segmentation network establishment method and application thereof, belongs to the field of defect segmentation, and comprises the following steps: based on neural architecture search, combining prior knowledge of an existing manually designed defect detection architecture, designing a potential segmentation network search space which is simple but suitable for an industrial scene, and completing network-level search space design; summarizing common operations in the industrial detection architecture, additionally adding channel attention operations and spatial attention operations as candidate operations, constructing basic units in the network, and completing unit-level search space design; since the overall network architecture is relatively fixed, the final search range is simplified to the structure of the unit instead of the entire network, so that the architecture construction with good performance is completed in a shorter time under the condition of limited time resources, and the network design cost is greatly saved. On this basis, the search strategy is further optimized, and the search efficiency can be further improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of defect segmentation, and more particularly relates to an adaptive industrial surface defect segmentation network establishment method and application thereof. BACKGROUND

[0002] Industrial defect segmentation, which represents locating and marking the defect region in the image, plays a crucial role in the quality control of industrial production. In recent years, the rapid development of convolutional neural network (CNN) has made it widely used in defect segmentation. However, there are various types of industrial surface defects, and the characteristics of different types of defects differ greatly. There is no single CNN architecture that can achieve the best performance on different detection tasks. For a specific detection task, there is no unified design paradigm for designing a good architecture. The design of the network relies on a large amount of expert knowledge of defect features and requires researchers to combine practical experience and conduct multiple experiments, which consumes a lot of manpower, time and computing cost. Therefore, how to find a way to design a good architecture has become the key to efficiently solving industrial defect segmentation.

[0003] Image segmentation in natural scenes is used to segment specific objects from images. For natural image segmentation, neural architecture search technology has been widely used. Neural architecture search attempts to generate adaptive data-driven networks through automatic search to replace the manual design process. The principle of neural architecture search is to search for the best architecture in a given set of candidate neural architectures (called search space) using a specific strategy.

[0004] The search space is crucial for neural architecture automatic search. Designing a suitable search space for surface defect detection in industrial scenarios is a challenging problem: first, searching the entire architecture can cover a larger search space, but the combinatorial explosion problem caused by the complex search space greatly increases the search difficulty. In the resource-limited industrial scenario, it cannot always guarantee the best performance, and it generates more search cost. Searching for specific building blocks of architecture simplifies the search space, but its expressiveness is not enough. Architectures with better performance may not be included in it. Second, the appearance, shape and size of surface defects of industrial products can be extremely diverse. Using only single-level features of neural networks may cause feature information loss, combined with the influence of environmental background and complex noise in industrial scenarios, the detection network needs to have more sensitive defect perception ability compared to the segmentation network in natural scenarios.

[0005] In general, although existing neural architecture search techniques can effectively search for better architectures in natural image segmentation, they are not suitable for industrial scenarios. Therefore, exploring suitable neural architecture search methods to adaptively generate defect detection networks that match the detection task is crucial for cost reduction and efficiency improvement in industrial detection and promoting the automation detection process. SUMMARY

[0006] To overcome the defects of the prior art and improve the demand, the present application provides an adaptive industrial surface defect segmentation network establishment method and its application, which aims to design a unit-level search space and a network-level search space in view of the characteristics of various industrial surface defects, effectively reducing the search range, and completing the construction of a system with good performance in a shorter time under the condition of limited time resources.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, an adaptive industrial surface defect segmentation network establishment method based on neural architecture search technology is provided, comprising:

[0008] (S1) establishing an initial super network; the initial super network comprises a plurality of basic units stacked in sequence;

[0009] The basic unit is a directed acyclic graph, wherein the nodes represent feature maps, and the edges represent operations. Each edge ending with an intermediate node is a to-be-searched edge, which is composed of a plurality of candidate operations. The operation represented by the to-be-searched edge is: performing each candidate operation on the feature map corresponding to the starting point, and then performing weighted summation on the operation results according to the first weight corresponding to the candidate operation, as the feature map corresponding to the ending point. The input of the basic unit is connected to the output of each intermediate node as a residual channel, and the output of the basic unit is the connection of the residual channel; the basic unit is divided into normal units and dimension reduction units, the output size of the normal unit is unchanged relative to the input size, and the output size of the dimension reduction unit is reduced by half relative to the input size, and the normal units and the dimension reduction units are alternately stacked;

[0010] The initial super network further comprises: an initial convolution operation layer arranged before the plurality of basic units, used for performing an initial dimension reduction operation on the to-be-segmented image to obtain an initial feature map; and a feature fusion structure arranged after the plurality of basic units, used for restoring the feature images output by each normal unit and the last dimension reduction unit to be consistent with the size of the to-be-segmented image, and then performing weighted fusion according to the second weight of each level of feature image to obtain a defect segmentation result;

[0011] (S2) taking the first weight as an architecture weight, taking the remaining to-be-optimized parameters in the network as network weights, training the initial super network by using a training data set of a target industrial surface defect detection task, to optimize the architecture weight and the network weight, and adjusting the architecture of the basic unit according to the optimized architecture weight, so that each to-be-searched edge only retains the candidate operation with the highest first weight, to obtain a target sub network;

[0012] (S3) training the target sub network by using the training data set, to obtain an industrial surface defect segmentation network for outputting a defect segmentation map of an industrial surface image.

[0013] Further, for any intermediate node O mi , the corresponding to-be-searched edge includes an edge between the intermediate node O mi and all the forward nodes of the intermediate node O mi in the same basic unit.

[0014] Further, when the current basic unit is preceded by other basic units, the corresponding to-be-searched edge further includes an edge between the output node of the first K basic units and the intermediate node O mi in the current basic unit. mi

[0015] wherein the value of K is 1 or 2.

[0016] Further, in step (S2), the initial super network is trained by using the training data set of the target industrial surface defect detection task, to optimize the architecture weight and the network weight, and the optimization is completed by using a progressive search strategy.

[0017] The progressive search strategy includes:

[0018] (S21) dividing the training data set into a network training data set and an architecture training data set;

[0019] (S22) training the current initial super network to perform nested optimization on the architecture weight and the network weight, until a preset number of iterations I in each stage is reached; when optimizing the architecture weight, the current network weight is fixed and the architecture training data set is used for training; when optimizing the network weight, the current architecture weight is fixed and the network training data set is used for training;

[0020] (S23) if there is only one candidate operation in each to-be-searched edge of each basic unit, directly entering step (S24); otherwise, according to the current architecture weight, removing part of the candidate operations with the lowest first weight in each to-be-searched edge of each basic unit, and then entering step (S24);

[0021] ​(S24) If the preset number of training stages K is not reached, go to step (S22) to start the next stage of search; otherwise, the search is ended.

[0022] Further, in step (S22), in the first I1 rounds of iterations of the current stage, only the network weights are optimized, and the architecture weights are not optimized; in the last I2 rounds of iterations, the network weights and the architecture weights are alternately optimized in each round of iteration.

[0023] Wherein, I1+I2=I.

[0024] Further, in step (S2) and step (S3), the training loss function is:

[0025] Loss=Loss1+Loss2;

[0026] Wherein, Loss represents the total loss; Loss1 represents the segmentation loss, which is used to represent the difference between the defect segmentation result and the data label; Loss2 represents the deep supervision loss, which is used to represent the difference between the feature image at each level and the data label.

[0027] Further, step (S2) further comprises: after adjusting the architecture of the basic unit according to the optimized architecture weight, so that each edge of each basic unit only contains the candidate operation with the highest architecture weight, using the first weight of the candidate operation of each edge as the weight of the edge, for each intermediate node, only keeping the two edges with the largest weights between the intermediate node and its forward nodes, and masking the remaining edges.

[0028] Further, the candidate operations include: no operation, skip connection, 3*3 separable convolution, 5*5 separable convolution, 7*7 separable convolution, 3*3 separable atrous convolution, 5*5 separable atrous convolution, 3*3 maximum pooling operation, 3*3 average pooling operation, channel attention operation and spatial attention operation.

[0029] Further, in the initial super network, the plurality of basic units stacked in turn are specifically 4 normal units and 4 dimension reduction units stacked alternately.

[0030] According to another aspect of the present application, an industrial surface defect segmentation method is provided, comprising:

[0031] Input the industrial surface image to be segmented into the industrial surface defect segmentation network; the industrial surface defect segmentation network is established by the adaptive industrial surface defect segmentation network establishment method based on the neural architecture search technology provided by the present application;

[0032] Obtain the defect segmentation result from the output of the industrial surface defect segmentation network.

[0033] According to still another aspect of the present application, a computer readable storage medium is provided, comprising a stored computer program; the computer program, when executed by a processor, controls the device where the computer readable storage medium is located to perform the neural architecture search technology-based adaptive industrial surface defect segmentation network establishment method provided by the present application, and / or the industrial surface defect segmentation method provided by the present application.

[0034] In general, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0035] (1) The present application is based on neural architecture search technology, combines the prior knowledge of manually designed defect detection architecture, designs a simplified but suitable potential segmentation network search space for industrial scenarios, and designs the structure of the basic unit in the segmentation network search space based on the common operations in the neural network under the industrial scenario. Thus, the unit-level search space and the network-level search space are designed. In the subsequent search process, the network is trained by means of the data set of the industrial surface defect detection task, and the basic unit is adjusted by means of the training result. The basic unit with the optimal structure is finally determined, and the structure of the network is also determined. Through this search method, the search range is simplified to the structure of the architecture unit rather than the entire network. Based on the determined network obtained by searching, the network is further retrained by means of the data set of the industrial surface defect detection task, so that the finally established network adapts to the specific industrial surface defect detection task and has good segmentation performance. In general, compared with the existing neural architecture search method which searches for the optimal architecture from a large number of unknown network architectures, the present application can effectively reduce the search range and ensure the optimal network performance. Under the condition of limited time resources, the architecture construction with good performance can be completed in a shorter time.

[0036] (2) In the preferred scheme of the present application, the basic unit is designed to contain residual connection and contain multiple intermediate nodes. At the same time, the intermediate nodes of the basic unit will receive the feature mapping of the forward nodes in the same unit, and also receive the output of the previous 1-2 basic units. Thus, the receptive field of the basic unit can be expanded and the performance of the network can be improved without significantly increasing the complexity of the network.

[0037] (3) In the preferred scheme of the present application, the search of the basic unit of the optimal structure is completed by using a progressive search strategy containing a deep supervision mechanism, specifically, the whole process is divided into multiple stages of training, after the training of each stage is completed, pruning of the candidate operations is performed based on the weights of the candidate operations in each to-be-searched edge in the basic unit, only the candidate operations with higher weights are retained, and in the whole search process, the difference between the feature images at each level and the data label is additionally added to the constraint of the training process, which can promote the search process to converge better and faster, and ensure the relevance of the search and evaluation stages of the network, effectively improving the efficiency and accuracy of the search.

[0038] (4) In the preferred scheme of the present application, a deep supervision loss is introduced into the loss function of the training of the architecture weight and the network weight, so that the difference between the feature images at each level output by the network and the data label is minimized, thereby accelerating the convergence of the network and improving the performance of the network.

[0039] (5) In the preferred scheme of the present application, in the case of determining the basic unit structure by pruning the candidate operations, the number of forward information received by each intermediate node is further reduced, and only two strongest operations from different nodes collected from all previous nodes are retained, thereby further simplifying the network structure without affecting the network performance.

[0040] (6) In the preferred scheme of the present application, the determined candidate operation set includes not only common operations such as separable convolution, separable dilated convolution and pooling operation, but also channel attention operation and spatial attention operation, thereby improving the detection capability of irregular and diversified defects and enhancing the ability to automatically focus on key defects in complex environments.

[0041] (7) In the preferred scheme of the present application, the designed network specifically includes 4 normal units and 4 dimension reduction units stacked alternately, and practice shows that the network architecture can achieve the best balance between complexity and performance, and the search of the optimal basic unit based on the architecture can effectively guarantee the performance of the network. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The industrial surface defect segmentation network structure schematic diagram provided for the embodiment of the present application;

[0043] Figure 2 The basic unit structure schematic diagram provided for the embodiment of the present application;

[0044] Figure 3 A progressive search strategy schematic diagram provided for an embodiment of the present application;

[0045] Figure 4 A network architecture automatic generation schematic diagram provided for an embodiment of the present application;

[0046] Figure 5 A comparison schematic diagram of an adaptive industrial surface defect segmentation network establishment method provided for an embodiment of the present application and an existing establishment method. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0048] In the present application, the terms "first", "second", etc. (if any) in the present application and the drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0049] In view of the technical problems that manual customization of defect detection architecture is time-consuming and laborious, and existing neural architecture search technology cannot be applied to industrial surface defect segmentation application scenarios with a large number of defect types, the present application provides an adaptive industrial surface defect segmentation network establishment method and its application by fusing the prior knowledge of the manually designed defect detection architecture. The overall idea is as follows: based on the prior knowledge of the existing excellent manually designed defect detection architecture, a potential segmentation network search space suitable for industrial scenarios is designed, the network-level search space design is completed; and the commonly used operations in the neural network under the industrial scenario are summarized, and channel attention operation and spatial attention operation are additionally added as candidate operations to construct the basic unit in the network, and the unit-level search space design is completed; since the overall architecture of the network is relatively fixed, the final search range will be simplified to the structure of the unit, rather than the entire network, so that the architecture construction with good performance can be completed in a shorter time under the condition of limited time resources, and the manpower, time and calculation cost are greatly saved. On this basis, the search strategy is further optimized, which can further improve the search efficiency.

[0050] Based on the above idea, the present application mainly includes three stages of search space definition, network structure search and network retraining. First, the search space is defined, which defines the architecture set that can be represented in principle. In the present application, the search space of the detection network is inspired by the hand-designed detection network, which is defined as a repeated stack of multiple expressive basic units, where the unit is composed of a given candidate operation (common operations in neural networks in industrial scenarios, such as convolution, pooling, etc., and channel attention operation and spatial attention operation additionally added). Correspondingly, the search range is simplified to the structure of the architecture unit rather than the entire network. Next, the network structure is searched. A progressive search strategy based on the importance of the candidate space with a deep supervision mechanism is used to gradually explore the search space. During the search process, the progressive search will gradually delete those unimportant candidate operations according to the contribution degree of the candidate operation to the performance of the architecture, so as to obtain a good basic unit. Then, the optimal unit learned is used to replace the original super network to obtain the final determined network structure. On this basis, the weight parameters of the network will be completely trained from scratch on this determined network structure to ensure the complete convergence of the generated network. The final network model can realize end-to-end segmentation prediction map output and be used to evaluate the performance of surface defect detection.

[0051] The following is an example.

[0052] Example 1

[0053] A method for establishing an adaptive industrial surface defect segmentation network based on neural architecture search technology, comprising:

[0054] (S1) Establish an initial super network; the network structure is as shown in Figure 1 .

[0055] As shown in Figure 1 , the initial super network includes a plurality of basic units stacked in sequence, and the basic units are divided into normal units and dimension reduction units; the output of the normal unit is unchanged relative to the input size, and the output of the dimension reduction unit is reduced by half relative to the input size, and the normal units and the dimension reduction units are alternately stacked;

[0056] The initial super network further includes an initial convolution operation layer arranged before the plurality of basic units, for performing an initial dimension reduction operation on the image to be segmented to obtain an initial feature map; as shown in Figure 1 , in the present embodiment, the initial convolution operation is specifically composed of two convolution layers stacked;

[0057] and a feature fusion structure arranged behind the plurality of basic units, configured to recover feature images output by each normal unit and the last dimension reduction unit to the size of the image to be segmented, and then perform weighted fusion according to the second weights of the feature images of each level to obtain a defect segmentation result; in the feature fusion structure, the size recovery of the feature images is realized by upsampling the feature images; the feature images of each level have different scales, and the second weights reflect the importance of the feature images of different scales, the feature fusion structure realizes multi-scale information fusion according to the importance of features, and can enhance the automatic adaptability to multi-scale defects and improve the detection accuracy.

[0058] In this embodiment, the overall architecture design of the network combines the prior knowledge of the manual defect detection architecture, is similar to the architecture of the existing defect segmentation network, and the overall architecture of the network remains unchanged in the later search process, the second weights of the feature images of each level, and the parameters in the basic unit are dynamically updated in the later search process.

[0059] The structure of the basic unit is shown in Figure 2 Each basic unit is a directed acyclic graph, wherein the nodes represent feature maps, and the edges represent operations; the nodes in the directed acyclic graph corresponding to each basic unit can be divided into input nodes, intermediate nodes and output nodes, and each edge ending at an intermediate node is a to-be-searched edge, which is composed of a plurality of candidate operations; the operation represented by the to-be-searched edge is: performing each candidate operation on the feature map corresponding to the starting point, and then performing weighted summation on the operation results according to the first weights corresponding to the candidate operations to obtain the feature map corresponding to the ending point; the output of the basic unit is obtained by connecting the output of each intermediate node to the input of the basic unit as a residual channel;

[0060] In this embodiment, the determination of the candidate operations also combines the prior knowledge of the design of the existing defect segmentation network, and the selected candidate operations include common operations in neural networks in industrial scenarios, specifically including: no operation, skip connection, 3*3 separable convolution, 5*5 separable convolution, 7*7 separable convolution, 3*3 separable atrous convolution, 5*5 separable atrous convolution, 3*3 max pooling operation and 3*3 average pooling operation; in addition, in order to improve the detection ability of the network to irregular and diversified defects, and to enhance the ability to automatically focus on key defects in complex environments, the determined candidate operations in this embodiment also include: channel attention operation and spatial attention operation.

[0061] In each to-be-searched edge, each candidate operation is assigned a weight representing importance, i.e., the first weight, which can be updated in backpropagation, and the internal parameters of each candidate operation will also be dynamically updated in the subsequent search and training process.

[0062] In practical applications, the intermediate nodes in the basic unit can be numbered in sequence, and the nodes with smaller numbers are forward nodes; as Figure 2 shown, in this embodiment, the basic unit specifically includes 4 intermediate nodes, namely Figure 2 nodes 1, node 2, node 3, and node 4 in

[0063]

[0064] Among them, i and j respectively represent two different nodes, i < j means that i is a forward node of j, and the data stream needs to be transmitted from i to j, x i represents the feature map corresponding to node i, O represents the set of candidate operations, o and o' represent candidate operations in the set of candidate operations, and respectively represent the weights of the candidate operations; represents the feature map corresponding to node j after the data is transmitted from node i to node j.

[0065] As Figure 2 shown, in this embodiment, in addition to being connected to the input node through the edge to be searched, the intermediate nodes in the basic unit also receive the feature maps of all forward nodes and the outputs of the previous two units through the edge to be searched; for example, node 2 is connected to the input node and node 1 through the edge to be searched, and in addition, it is also connected to the outputs (input 2) of the previous two basic units through the edge to be searched; by receiving the outputs of the previous basic units, more forward information can be received, expanding the receptive field; in some other embodiments of the present invention, it is also possible to only receive the output of the previous basic unit. It should be noted that receiving the outputs of more (more than 2) previous basic units will significantly increase the complexity of the network, while the effect of expanding the receptive field is not obvious.

[0066] Based on Figure 1 the network architecture shown, this embodiment further includes:

[0067] (S2) Using the first weight as the architecture weight and the remaining parameters to be optimized in the network as the network weight, training the initial super network using the training dataset of the target industrial surface defect detection task to optimize the architecture weight and the network weight, and adjusting the architecture of the basic unit according to the optimized architecture weight, so that each edge to be searched only retains the candidate operation with the highest first weight, obtaining the target sub-network;

[0068] In this embodiment, the training dataset of the target industrial surface defect detection task is used for training to optimize the architecture weight and the network weight, and the unit architecture is adjusted to realize network architecture search. The search starts from a relatively unrestricted large super network, and the search process is regarded as a nested optimization problem of the architecture weight and the network weight:

[0069]

[0070]

[0071] The above formula represents that the architecture weight α is obtained by minimizing , and the network weight is represented as , which is the best network weight under a given α. It is easy to understand that the network weight specifically includes the second weight and the internal parameters of each candidate operation of each edge in each basic unit.

[0072] In the search process of this embodiment, the overall network architecture remains unchanged, but the structure of the edges to be searched in the basic unit changes. Therefore, in this embodiment, the search range is simplified to the structure of the basic unit rather than the entire network, thereby realizing a small and precise search space, which can cope with the problem that defect samples are relatively scarce in industrial scenarios and are easily disturbed by complex surface defects.

[0073] In order to effectively improve the search efficiency of the optimal architecture, this embodiment proposes a progressive search strategy. The strategy divides the entire search process into K=4 stages, each of which trains the network, and the number of iteration rounds of network training in each stage is set to I=70. After the training of the current stage is completed, the candidate operations of the edges to be searched in the basic unit are pruned based on the optimized architecture parameters, and then the next stage of search is performed. Specifically, as shown in Figure 3 In step (S2) of this embodiment, the initial super network is trained using the training dataset of the target industrial surface defect detection task to optimize the architecture weight and the network weight, which is completed through the progressive search strategy. The progressive search strategy includes:

[0074] (S21) dividing the training dataset into a network training dataset and an architecture training dataset;

[0075] (S22) training the current initial super network to perform nested optimization of the architecture weight and the network weight until the preset number of iteration rounds I of each stage is reached. When optimizing the architecture weight, the current network weight is fixed and unchanged, and the architecture training dataset is used for training. When optimizing the network weight, the current architecture weight is fixed and unchanged, and the network training dataset is used for training;

[0076] To avoid the instability of the network at the beginning of the training, which leads to poor search performance, only the network weight w is updated in the first few iterations of each stage; Specifically, in the first I1 iterations of the current stage, only the network weight w is optimized, and the architecture weight is not optimized; In the last I2 iterations, the network weight and the architecture weight are alternately optimized in each iteration;

[0077] Optionally, in the embodiment, I1 = 20, I2 = 50;

[0078] (S23) If there is only one candidate operation in each edge of each basic unit, go directly to step (S24); Otherwise, according to the current architecture weight, remove the candidate operation with the lowest first weight in each edge of each basic unit, and go to step (S24);

[0079] In the embodiment, after the end of each stage of training, the candidate operations contained in each edge to be searched are pruned according to the first weight, and only the Top k candidate operations with strong importance (higher weight) are retained; Top k The number of candidate operations retained in the kth stage is represented by Top1 = 7, Top2 = 4, Top3 = 2, and Top4 = 1. Based on this progressive search strategy, the search space of the super network constructed based on the original candidate operation set O is gradually reduced to only one unique candidate operation in each edge of each unit.

[0080] It should be noted that in the search process, the number of stages, the number of training rounds in each stage, and the number of training rounds in which only the network parameters are updated in each stage can be adjusted according to actual application conditions.

[0081] (S24) If the number of preset training stages K is not reached, go to step (S22) to start the next stage of search; Otherwise, the search is ended.

[0082] To simplify the network architecture without affecting the network performance, the embodiment further includes the following steps after step (S24):

[0083] Taking the first weight of the candidate operation of each edge as the weight of the edge, for each intermediate node, only the two edges with the largest weight between the intermediate node and its forward node are retained, and the remaining edges are masked.

[0084] Through this operation, for each intermediate node in the basic unit, only two strongest operations from different nodes collected from all previous nodes are retained, and the basic unit structure will be further simplified.

[0085] Through the above training manner, the optimal basic unit structure can be determined, and the search of the unit-level search space is completed.

[0086] In order to further reduce the difficulty of network architecture search and improve the search efficiency, as shown in the following formula, a deep supervision mechanism is introduced in the training process to minimize the difference between the feature images of different scales and the data labels (i.e. the labeled defect regions), and the mechanism can be realized by adding the following deep supervision loss Loss2 in the loss function: Figure 3

[0087]

[0088] wherein, Loss (i) represents the loss of the i-th level feature image relative to the data label; the introduction of the deep supervision mechanism can promote the better and faster convergence of the search process, and ensure the relevance of the network search and evaluation stages;

[0089] In addition, the segmentation loss Loss1 for minimizing the difference between the defect segmentation result and the data label is also designed in the loss function, and finally the overall loss function is as follows:

[0090] Loss = Loss1 + Loss2.

[0091] Through the above training manner, the architecture weight and the network weight will be effectively optimized, wherein the architecture weight determines the network structure, and the network weight determines the performance of the network under the network structure.

[0092] After determining the optimal structure of the basic unit in the network architecture through step (S2), the embodiment further comprises:

[0093] (S3) training the target sub-network using the training data set to obtain an industrial surface defect segmentation network for segmenting out defects in the industrial surface image;

[0094] It is easy to understand that, since the structure of the basic unit has been determined, only the network weight is optimized in the training process of step (S3).

[0095] In step (S3) of the embodiment, the loss function of network training is the same as that of step (S2), and the specific description can be referred to the above description of step (S2), which will not be repeated here.

[0096] The overall process of establishing the above industrial surface defect segmentation network can be described by Figure 4 The comparison between the method and the existing method is as follows: Figure 5 ​The embodiment can adaptively generate appropriate architecture under data driving, and finally generate a defect segmentation model which is more lightweight and has stronger performance than the existing manually customized defect detection architecture in the industrial scene.

[0097] In practice, the method provided by the embodiment can search for an architecture that matches the industrial scene detection task better, and the absolute improvement of IoU is more than 3%. On the benchmark of the most advanced manually designed defect detection architecture at the present stage, the method provided by the embodiment realizes the highest precision of defect detection by searching with only ~10% of the parameter amount (1M-2M) and ~5% of the floating point calculation amount.

[0098] Embodiment 2

[0099] An industrial surface defect segmentation method comprises:

[0100] An industrial surface image to be segmented is input into an industrial surface defect segmentation network; the industrial surface defect segmentation network is established by the adaptive industrial surface defect segmentation network establishment method based on the neural architecture search technology provided by the above-mentioned embodiment 1.

[0101] A defect segmentation result is obtained from the output of the industrial surface defect segmentation network.

[0102] Since the industrial surface defect segmentation network established by the embodiment 1 has a lighter weight and better performance, based on the industrial surface defect segmentation network, the embodiment can more efficiently and accurately complete the industrial surface defect segmentation.

[0103] Embodiment 3

[0104] A computer readable storage medium comprises a stored computer program; when the computer program is executed by a processor, the computer readable storage medium controls the device where the computer readable storage medium is located to execute the adaptive industrial surface defect segmentation network establishment method based on the neural architecture search technology provided by the above-mentioned embodiment 1, and / or the industrial surface defect segmentation method provided by the above-mentioned embodiment 2.

[0105] Those skilled in the art will readily understand that the above description is only of the preferred embodiments of the present application and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for establishing an adaptive industrial surface defect segmentation network based on a neural architecture search technology, characterized in that, Comprise: (S1) Establish an initial super network; The initial super network comprises a plurality of basic units stacked in turn; The basic unit is a directed acyclic graph, wherein the nodes represent feature maps, the edges represent operations, and each edge ending with an intermediate node is a to-be-searched edge composed of a plurality of candidate operations, and the operation represented by the to-be-searched edge is: performing each candidate operation on the feature map corresponding to the starting point, respectively, and then performing weighted summation on the operation results according to the first weight corresponding to the candidate operation to obtain the feature map corresponding to the ending point; The input of the basic unit is connected to the output of each intermediate node as a residual channel, and the output of the basic unit is obtained after the input is connected to the output of each intermediate node; the basic unit is divided into a normal unit and a dimension reduction unit, the output of the normal unit is unchanged relative to the input size, and the output of the dimension reduction unit is reduced by half relative to the input size, and the normal unit and the dimension reduction unit are alternately stacked; The initial super network further comprises: a convolution operation layer arranged before the plurality of basic units, used for performing an initial dimension reduction operation on the to-be-segmented image to obtain an initial feature map; and a feature fusion structure arranged after the plurality of basic units, used for restoring the feature images output by each normal unit and the last dimension reduction unit to be consistent with the size of the to-be-segmented image, then performing weighted fusion on the feature images of each level according to the second weight to obtain a defect segmentation result; (S2) Use the first weight as the architecture weight, use the remaining to-be-optimized parameters in the network as the network weight, train the initial super network using a training data set of a target industrial surface defect detection task to optimize the architecture weight and the network weight, and adjust the architecture of the basic unit according to the optimized architecture weight, so that each to-be-searched edge only retains the candidate operation with the highest first weight, to obtain a target sub-network; (S3) Train the target sub-network using the training data set to obtain an industrial surface defect segmentation network for outputting a defect segmentation map of an industrial surface image.

2. The method of claim 1, wherein the method further comprises: For any one intermediate node O in the basic unit mi The corresponding edge to be searched includes the edge between the intermediate node O mi and all the forward nodes of the intermediate node O mi in the same basic unit. And, when the current basic unit is preceded by other basic units, the intermediate node O mi The corresponding edge to be searched further includes edges between the output nodes of the first K basic units and the intermediate node O mi of the current basic unit. Wherein, the value of K is 1 or 2.

3. The method of claim 1 or 2, wherein the method further comprises: In step (S2), the initial super network is trained using a training data set of a target industrial surface defect detection task to optimize the architecture weight and the network weight, and the progressive search strategy is completed; The progressive search strategy comprises: (S21) Divide the training data set into a network training data set and an architecture training data set; (S22) Train the current initial super network to perform nested optimization on the architecture weight and the network weight until a preset number of iterations I in each stage is reached; when optimizing the architecture weight, the current network weight is fixed and the architecture training data set is used for training; when optimizing the network weight, the current architecture weight is fixed and the network training data set is used for training; (S23) If there is only one candidate operation in each to-be-searched edge of each basic unit, directly enter step (S24); otherwise, according to the current architecture weight, remove the part of the candidate operations with the lowest first weight in each to-be-searched edge of each basic unit, and then enter step (S24); (S24) If the preset number of training stages K is not reached, go to step (S22) to start the search of the next stage; otherwise, the search is ended.

4. The method of claim 3, wherein the neural architecture search technique-based adaptive industrial surface defect segmentation network is established by, In the step (S22), in the first I1 rounds of iterations of the current stage, only the network weights are optimized, and the architecture weights are not optimized. In the latter I2 rounds of iterations, the network weights and the architecture weights are alternately optimized in each round of iteration. Wherein, I1+I2=I.

5. The method of claim 1 or 2, wherein the method further comprises: In the step (S2) and the step (S3), the training loss function is: Loss=Loss1+Loss2; Wherein, Loss represents the total loss; Loss1 represents the segmentation loss, which is used to represent the difference between the defect segmentation result and the data label; Loss2 represents the deep supervision loss, which is used to represent the difference between the feature image at each level and the data label.

6. The method of claim 1 or 2, wherein the method further comprises: The step (S2) further comprises: after adjusting the architecture of the basic unit according to the optimized architecture weight, so that each edge of each basic unit only contains the candidate operation with the highest architecture weight, taking the first weight of the candidate operation of each edge as the weight of the edge, for each intermediate node, only keeping the two edges with the largest weights between the intermediate node and its forward nodes, and masking the remaining edges.

7. The method of claim 1 or 2, wherein the method further comprises: The candidate operations include: no operation, skip connection, 3*3 separable convolution, 5*5 separable convolution, 7*7 separable convolution, 3*3 separable atrous convolution, 5*5 separable atrous convolution, 3*3 max pooling operation, 3*3 average pooling operation, channel attention operation and spatial attention operation.

8. The method of claim 1 or 2, wherein the method further comprises: In the initial super network, the plurality of basic units stacked in turn are specifically 4 normal units and 4 dimension reduction units stacked alternately.

9. An industrial surface defect segmentation method, characterized by, Comprise: input the industrial surface image to be segmented into the industrial surface defect segmentation network; the industrial surface defect segmentation network is established by the neural architecture search technology-based adaptive industrial surface defect segmentation network establishment method in any one of claims 1-8; obtain the defect segmentation result from the output of the industrial surface defect segmentation network.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium; when the computer program is executed by the processor, the device where the computer readable storage medium is located executes the neural architecture search technology-based adaptive industrial surface defect segmentation network establishment method in any one of claims 1-8, and / or the industrial surface defect segmentation method in claim 9. The computer program is stored in the computer readable storage medium; when the computer program is executed by the processor, the device where the computer readable storage medium is located executes the neural architecture search technology-based adaptive industrial surface defect segmentation network establishment method in any one of claims 1-8, and / or the industrial surface defect segmentation method in claim 9.

Images