Forming mesh defect detection method and device based on improved convolutional neural network

Abstract

The present invention discloses a forming net defect detection method and device based on an improved convolutional neural network, the method comprising: setting a lighting system to irradiate a set light on the forming net; collecting image data with defects, preprocessing the data set, and obtaining an image sample set; constructing a defect recognition and positioning model, the defect recognition and positioning model comprising a detection module, a feature module, and a boundary prediction module; inputting the image sample set into the defect recognition and positioning model for training, and obtaining a trained defect recognition and positioning model; the detection module adopts a deep separable convolution, including deep convolution and point-by-point convolution, and the deep convolution and point-by-point convolution are relatively independently set; inputting the image to be identified into the trained defect recognition and positioning model, and outputting a defect positioning result. The present invention can identify and locate the defects of the forming net through prediction, thereby improving the robustness of the defect recognition and positioning model and the efficiency of forming net defect detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a forming mesh defect detection method and device based on an improved convolutional neural network. Background Art

[0002] In the modern manufacturing process, the quality of the forming mesh has a vital impact on the product quality. However, the defect detection of the forming mesh currently relies mainly on manual work, which is not only inefficient but also has poor accuracy. Therefore, it is particularly important to develop an efficient and accurate forming mesh defect detection technology. Summary of the invention

[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a forming net defect detection method and device based on an improved convolutional neural network. The improved convolutional neural network is used to extract the longitude and latitude features of the forming net for detection, and the defects of the forming net can be identified and located through prediction, thereby improving the robustness of the defect recognition and positioning model and the efficiency of forming net defect detection.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a forming mesh defect detection method based on an improved convolutional neural network, comprising the following steps:

[0006] Set up the lighting system to shine the set light on the forming wire;

[0007] Collect image data with defects, preprocess the data set, and obtain an image sample set;

[0008] A defect recognition and positioning model is constructed based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain a trained defect recognition and positioning model. The detection module adopts a depth separable convolution, including a depth convolution and a point-by-point convolution. The depth convolution and the point-by-point convolution are set relatively independently.

[0009] The image to be identified is input into the trained defect identification and positioning model, and the defect positioning result is output.

[0010] As a preferred technical solution, the depthwise convolution includes multiple separate convolution kernels, and only one convolution kernel is set for each input channel; the point-by-point convolution includes a point-by-point convolution kernel.

[0011] As a preferred technical solution, the data set is input into the detection module, and each channel in the image data is processed separately using the convolution kernel in the deep convolution to generate the same number of output channels as the input channels;

[0012] The point-by-point convolution kernel is used to linearly combine the output of the depth convolution to obtain the longitude and latitude features of the forming mesh.

[0013] As a preferred technical solution, the enhanced detected warp and weft characteristics of the forming mesh include:

[0014] Upsampling the detected longitude and latitude features of the forming mesh to obtain a deep feature map; fusing the deep feature map with a shallower feature map of the previous layer;

[0015] The detected longitude and latitude features of the forming net are convolved and downsampled to obtain a shallow feature map; the shallow feature map is fused with a deeper feature map of the next layer.

[0016] As a preferred technical solution, it also includes an inactivation step and a batch standardization step;

[0017] The deactivation step includes: randomly selecting a portion of nodes of the convolutional neural network and setting the outputs of these nodes to zero;

[0018] The batch standardization step includes: standardizing each batch of defect location result data output to have a zero mean and a unit variance;

[0019] After the defect location result is output, one or all of the above steps are randomly executed.

[0020] As a preferred technical solution, the lighting system adopts RGB surface light source.

[0021] As a preferred technical solution, the bottom surface of the forming mesh is illuminated, and the light emission angle, intensity, quantity and frequency are set.

[0022] As a preferred technical solution, the pretreatment includes:

[0023] Rotate, translate, scale, flip, and adjust brightness and contrast of defective image data;

[0024] Normalize the image data with defects;

[0025] Mark defects in image data.

[0026] In a second aspect, the present invention further provides a forming net defect detection device based on an improved convolutional neural network, which is applied to the forming net defect detection method based on an improved convolutional neural network, and includes an environment setting module, a data processing module, a model building and training module, and a detection module;

[0027] An environment setting module is used to set the lighting system to irradiate the set light on the forming net;

[0028] A data processing module is used to collect image data with defects, pre-process the data set, and obtain an image sample set;

[0029] The model building and training module is used to build a defect recognition and positioning model. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts a depth separable convolution, including a depth convolution and a point-by-point convolution. The depth convolution and the point-by-point convolution are set relatively independently.

[0030] The detection module is used to input the image to be identified into the trained defect recognition and positioning model and output the defect positioning result.

[0031] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0032] at least one processor; and,

[0033] a memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can perform the forming mesh defect detection method based on the improved convolutional neural network.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] The present invention adopts the improved YOLOv8N model as the defect recognition and positioning model, and decomposes the depthwise convolution and pointwise convolution in the depthwise separable convolution into two independent steps, which can greatly reduce the number of parameters and computational complexity of the model while maintaining good real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 It is a flow chart of a forming mesh defect detection method based on an improved convolutional neural network according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the traditional YOLOv8 N model structure;

[0040] Figure 3 A schematic diagram of the depthwise separable convolution structure in the improved YOLOv8 N model according to an embodiment of the present invention;

[0041] Figure 4 Schematic diagrams of samples of three types of forming meshes of the present invention;

[0042] Figure 5 A schematic diagram of a defect detection sample according to an embodiment of the present invention;

[0043] Figure 6 A partial schematic diagram of a defect detection sample structure according to an embodiment of the present invention;

[0044] Figure 7 It is a schematic diagram of a detection frame of various defective forming meshes according to an embodiment of the present invention;

[0045] Figure 8 It is a structural schematic diagram of a forming mesh defect detection device based on an improved convolutional neural network according to an embodiment of the present invention;

[0046] Fig. 9 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0048] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0049] See also Figure 1 This embodiment provides a forming mesh defect detection method based on an improved convolutional neural network, comprising the following steps:

[0050] S1. Set up the lighting system and irradiate the set light onto the forming net.

[0051] In this embodiment, in order to improve the detection accuracy, a suitable lighting system should be designed to obtain high-quality image data. A good lighting system design can achieve good imaging effects, thereby improving the resolution of the entire system and simplifying the calculation of the software, while inappropriate lighting will cause many problems. Therefore, the design of the lighting system is an important research content. Currently, the commonly used artificial light sources mainly include incandescent lamps, halogen lamps, xenon lamps, fluorescent lamps and LED lamps. Among them, LED lamps can emit approximately monochromatic light. Compared with other light sources, they have the advantages of high luminous intensity, stable operation, long service life, free shape, and various colors, and are more suitable for use as artificial light sources in machine vision systems. In addition to the shape design of the light source, another aspect of obtaining a stable image input is to select the color of the light source. Even for light sources of the same shape, the images obtained due to different colors will be very different. Therefore, multiple experiments are required to select a suitable lighting system.

[0052] In particular, this embodiment uses an RGB surface light source (R:105 G:0 B:0) for bottom lighting.

[0053] S2. Collect image data with defects, preprocess the data set, and obtain an image sample set.

[0054] In the actual production process of the factory, the defective forming mesh fabric is retained, and then the data is photographed on the built optical platform to collect sample data sets. If the existing data is too little, it can be expanded through data enhancement. The details are as follows:

[0055] First, the diversity of training data can be increased by rotating, translating, scaling, flipping, adjusting brightness or contrast, etc. In this way, a large number of new and different training samples can be generated even with a limited number of original images.

[0056] Secondly, since the acquired images have uneven positions and sizes, the image data with defects needs to be normalized. Then, for the subsequent convolutional neural network learning and training, this embodiment marks the defects in the image data, thereby improving the generalization ability of the sample.

[0057] In particular, this example uses labelImg to create a dataset in YOLO annotation format. To install labelImg, follow these steps:

[0058] (1) Select the YOLO annotation format in the text format;

[0059] (2) Select the path of the sample image and open it;

[0060] (3) Select the rectangular annotation box;

[0061] (4) Mark network problems and add tags;

[0062] (5) Save the results;

[0063] (6) Select the next image and repeat steps (3) to (6).

[0064] Finally, considering that the defect images of the forming mesh surface are mainly composed of low-level features such as contours and textures, when detecting these defects, the network should focus on extracting low-level semantic features of the image. These features are usually highlighted in the shallow feature maps of the convolutional neural network.

[0065] S3. Construct a defect recognition and positioning model based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts depthwise separable convolution, including depthwise convolution and pointwise convolution, and the depthwise convolution and pointwise convolution are set relatively independently.

[0066] In this embodiment, the YOLOv8 N model is used. As the fastest and smallest model, this model can meet the requirements of reducing computational complexity and memory requirements. However, the forming net detected by this embodiment is multidimensional, and huge computational overhead is still required in terms of the amount of computation during detection. Therefore, this embodiment improves the traditional YOLOv8 N model to further reduce the number of model parameters and computational complexity, while maintaining good real-time performance, such as Figure 2 and Figure 3 shown.

[0067] Specifically, this embodiment proposes an improved YOLOv8 N model, including a backbone network Backbone, a neck Neck and a detection head Head.

[0068] like Figure 3As shown, for the backbone network Backbone, this embodiment uses it as a detection module, which is mainly responsible for extracting useful features from the input image. The convolution layer of this embodiment adopts depthwise separable convolution (Conv in the figure), which decomposes the traditional convolution operation into two independent steps: depthwise convolution and pointwise convolution. Depthwise convolution: In this step, each input channel is processed by a separate convolution kernel to generate output channels with the same number of input channels. This means that each convolution kernel is only responsible for processing one input channel, instead of processing all input channels at the same time like standard convolution. This can significantly reduce the number of parameters of the convolution kernel. Pointwise convolution: After the depth convolution, a 1x1 convolution kernel (also called a pointwise convolution kernel) is used to linearly combine the output of the depth convolution to generate a new feature map. The purpose of this step is to increase the expressive power and nonlinearity of the network. Through these two steps, the depthwise separable convolution can greatly reduce the number of parameters and computational complexity of the model while maintaining good performance. For the neck, this embodiment uses it as a feature module to further process and enhance the features extracted by the backbone network, such as multi-scale feature fusion through feature pyramid network (FPN) or path aggregation network (PANet). In order to better detect objects of different sizes, this implementation adopts two paths for feature enhancement processing, as follows:

[0069] The first path: From top to bottom, the deep feature map is upsampled to increase its spatial resolution. The upsampled deep feature map is fused with the shallower feature map of the previous layer. This fusion is usually achieved by element-by-element addition or channel concatenation. In this way, the shallower feature map (containing more spatial details) is injected with semantic information from the deep layer, which helps the model recognize objects at a finer scale.

[0070] The second path: From bottom to top, the detected latitude and longitude features of the forming net are convoluted and downsampled to obtain a shallow feature map; the shallow feature map is fused with the deeper feature map of the next layer. Among them, the convolution and downsampling are implemented through convolution or pooling operations with a step size of 2, thereby reducing the feature space size. After downsampling, the shallow feature map will be fused with the deeper feature map. In this way, the deep feature map obtains the detailed information from the shallow layer.

[0071] For the detection head, this part is responsible for the final category prediction and bounding box regression based on the fused feature map. In YOLOv8, the head part outputs the category probability, confidence and coordinate information of each detection box.

[0072] The samples in the data set are input into the constructed defect recognition and positioning model for training, and the trained defect recognition and positioning model is obtained. Then, as shown in step S4, the model performance is verified and the defects of the forming mesh are detected.

[0073] S4. Input the image to be identified into the trained defect identification and positioning model, and output the defect positioning result.

[0074] In deep learning, overfitting is a common problem, which refers to the model performing well on the training data but poorly generalizing on unseen data. To prevent overfitting, regularization techniques can be used, of which Dropout and Batch Normalization are two commonly used methods. Dropout is a regularization technique that randomly sets some nodes of the neural network to 0 during training, which means that these nodes will not contribute anything during the forward propagation process. Doing so prevents the model from becoming overly dependent on noise or insignificant details in the training data, thereby enhancing the generalization ability of the model. Batch Normalization is another effective regularization technique that can not only reduce overfitting but also speed up the training process. The main idea of ​​Batch Normalization is to standardize the input of each batch of data so that it has zero mean and unit variance.

[0075] In this implementation, these two steps or one of them may also be implemented to prevent the model from being overly dependent on noise or insignificant details in the training data, thereby enhancing the generalization ability of the model.

[0076] Specifically, the steps to implement Dropout include:

[0077] In each training batch, a portion of the nodes in the network are randomly selected and their outputs are set to 0;

[0078] During training, only nodes that have not been dropped participate in forward and backward propagation.

[0079] It should be noted that when testing or evaluating the model, Dropout is not used, but the output of the network needs to be scaled to compensate for the reduction of nodes caused by Dropout during training. Usually, this scaling factor is equal to the probability of Dropout.

[0080] The steps to implement Batch Normalization include:

[0081] Normalize the input data of each batch by subtracting the batch mean and dividing it by its standard deviation;

[0082] Learnable parameters are introduced, including scaling factors and offsets.

[0083] It should also be noted that Dropout and Batch Normalization are usually provided as layers or modules, and can be added directly after the depthwise separable convolution when building a defect recognition and positioning model.

[0084] In addition, to clearly understand the actual structure of the forming fabric, please refer to Figure 4 . Figure 4 Parts a) and b) show the surface and bottom of the forming fabric respectively, and the lines therein include the connecting warp, the face weft and the bottom weft. Figure 4 Parts c) and d) show the weft and warp sections of the 1.5-layer forming fabric, respectively; then, Figure 4 Parts e) and f) show the weft and warp sections of the two-layer forming fabric, respectively; finally, Figure 4 Parts g) and h) show the weft and warp cross sections of the 3-layer forming fabric, respectively.

[0085] Further, in order to clearly understand the missing local structure of the forming fabric, please refer to Figure 6 . Figure 6 Part a) shows that the bottom weft of the three-layer forming mesh is missing, where holes appear in the horizontal direction that are different from other areas. The actual structure is shown in the lower middle part of the figure. The surface weft and connecting warp of the forming mesh are intact, but the weft on the bottom surface is missing, resulting in a missing bottom weft of the three-layer mesh. Figure 6 Part b) shows that the two-layer forming mesh has missing surface wefts, where the two-layer forming mesh also has holes in the horizontal direction that are different from other areas, but due to the reduction in the number of layers of the forming mesh, the display is not obvious; in the lower half of part b), it can be seen that the connecting warp and the bottom weft are not missing, but a section of the surface weft is broken. Other types of missing or defects are manifested as the loss of one or all features of the surface weft, connecting warp or bottom weft, and the appearance of covering blocks on the forming mesh due to stains or other adhesive attachments.

[0086] This embodiment will Figure 5 The sample input is trained into the defect recognition and positioning model, and the detection frame can be located and obtained through prediction. The corresponding defect type can be known from the detection frame. In this implementation, the following defects can be detected: 1.5-layer net bottom weft missing (1.5dw), 1.5-layer net warp missing (1.5lj), 2-layer net bottom weft missing (2dw), 2-layer net warp missing (2lj), 2-layer net surface weft missing (2mw), 3-layer net bottom weft missing (3dw) and stains (wuzi). Figure 7 As shown in the figure, the output defect location result shows the detection box of the corresponding defect position, and the category probability and confidence are displayed on the detection box. Figure 7"1.5dw0.74" is displayed in the figure, which means the forming fabric is 1.5 layers, the current defect is the missing bottom weft, and the confidence level is 0.74.

[0087] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0088] Based on the same idea as the forming mesh defect detection method based on the improved convolutional neural network in the above-mentioned embodiment, the present invention also provides a forming mesh defect detection device based on the improved convolutional neural network, which can be used to execute the above-mentioned forming mesh defect detection method based on the improved convolutional neural network. For the convenience of explanation, the structural schematic diagram of the embodiment of the forming mesh defect detection device based on the improved convolutional neural network only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0089] See also Figure 8 In another embodiment of the present application, a forming mesh defect detection device 10 based on an improved convolutional neural network is provided, the system comprising an environment setting module 11, a data processing module 12, a model building and training module 13 and a detection module 14;

[0090] An environment setting module 11 is used to set a lighting system to irradiate a set light onto the forming net;

[0091] The data processing module 12 is used to collect image data with defects, pre-process the data set, and obtain an image sample set;

[0092] The model building and training module 13 is used to build a defect recognition and positioning model based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts a depth separable convolution, including a depth convolution and a point-by-point convolution. The depth convolution and the point-by-point convolution are relatively independently set. The depth convolution includes multiple separate convolution kernels. Each input channel is only set with one convolution kernel. The point-by-point convolution includes a point-by-point convolution kernel. After the point-by-point convolution, an h-swish activation function is set.

[0093] The detection module 14 is used to input the image to be identified into the trained defect identification and positioning model and output the defect positioning result; after completing the output of the defect positioning result, randomly perform one or all of the following steps:

[0094] Inactivation step: randomly select a portion of the nodes in the convolutional neural network and set the output of these nodes to zero;

[0095] Batch normalization step: Each batch of defect location result data output is standardized to have zero mean and unit variance.

[0096] It should be noted that the forming net defect detection device based on the improved convolutional neural network of the present invention corresponds one to one with the forming net defect detection method based on the improved convolutional neural network of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the forming net defect detection method based on the improved convolutional neural network are applicable to the embodiment of the forming net defect detection method based on the improved convolutional neural network. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0097] In addition, in the implementation of the forming net defect detection device based on the improved convolutional neural network in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the forming net defect detection device based on the improved convolutional neural network is divided into different program modules to complete all or part of the functions described above.

[0098] See also Fig. 9 In one embodiment, an electronic device for implementing a forming net defect detection method based on an improved convolutional neural network is provided. The electronic device 20 may include a first processor 21, a first memory 22 and a bus, and may also include a computer program stored in the first memory 22 and executable on the first processor 21, such as a forming net defect detection program 23 based on an improved convolutional neural network.

[0099] The first memory 22 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 22 can be an internal storage unit of the electronic device 20, such as a mobile hard disk of the electronic device 20. In other embodiments, the first memory 22 can also be an external storage device of the electronic device 20, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 20. Further, the first memory 22 can also include both an internal storage unit of the electronic device 20 and an external storage device. The first memory 22 can not only be used to store application software and various types of data installed in the electronic device 20, such as the code of the forming network defect detection program 23 based on the improved convolutional neural network, but also can be used to temporarily store data that has been output or is to be output.

[0100] In some embodiments, the first processor 21 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The first processor 21 is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 20 by running or executing programs or modules stored in the first memory 22, and calling data stored in the first memory 22.

[0101] Fig. 9 Only an electronic device with components is shown, and those skilled in the art will understand that Fig. 9 The structure shown does not constitute a limitation on the electronic device 20 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0102] The forming mesh defect detection program 23 based on the improved convolutional neural network stored in the first memory 22 of the electronic device 20 is a combination of multiple instructions. When running in the first processor 21, it can achieve:

[0103] Set up the lighting system to shine the set light on the forming wire;

[0104] Collect image data with defects, preprocess the data set, and obtain an image sample set;

[0105] A defect recognition and positioning model is constructed based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts a depth-separable convolution, including a depth-wise convolution and a point-by-point convolution. The depth-wise convolution and the point-by-point convolution are relatively independently set. The depth-wise convolution includes a plurality of separate convolution kernels. Each input channel is only provided with one convolution kernel. The point-by-point convolution includes a point-by-point convolution kernel. After the point-by-point convolution, an h-swish activation function is set.

[0106] Input the image to be identified into the trained defect recognition and positioning model, and output the defect positioning result; after the defect positioning result is output, randomly execute one or all of the following steps:

[0107] Inactivation step: randomly select a portion of the nodes in the convolutional neural network and set the output of these nodes to zero;

[0108] Batch normalization step: Each batch of defect location result data output is standardized to have zero mean and unit variance.

[0109] Furthermore, if the module / unit integrated in the electronic device 20 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A forming mesh defect detection method based on an improved convolutional neural network, characterized in that: The steps include: Set up the lighting system to shine the set light on the forming wire; Collect image data with defects, preprocess the data set, and obtain an image sample set; A defect recognition and positioning model is constructed based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts a depth-separable convolution, including a depth-wise convolution and a point-by-point convolution. The depth-wise convolution and the point-by-point convolution are relatively independently set. The depth-wise convolution includes a plurality of separate convolution kernels. Each input channel is only provided with one convolution kernel. The point-by-point convolution includes a point-by-point convolution kernel. After the point-by-point convolution, an h-swish activation function is set. Input the image to be identified into the trained defect recognition and positioning model, and output the defect positioning result; after the defect positioning result is output, randomly execute one or all of the following steps: Inactivation step: randomly select a portion of the nodes in the convolutional neural network and set the output of these nodes to zero; Batch normalization step: Each batch of defect location result data output is normalized to have zero mean and unit variance; The defect recognition and positioning model includes a backbone network Backbone, a neck Neck and a detection head Head; the backbone network Backbone is used as a detection module to extract useful features from the input image. The convolution layer uses depthwise separable convolution to decompose the traditional convolution operation into two independent steps: depthwise convolution and pointwise convolution. Depthwise convolution: In this step, each input channel is processed by a separate convolution kernel to generate output channels with the same number of input channels, which means that each convolution kernel is only responsible for processing one input channel, rather than processing all input channels at the same time like standard convolution; pointwise convolution: After the depthwise convolution, a 1x1 convolution kernel is used to linearly combine the output of the depthwise convolution to generate a new feature map; For the neck, it is used as a feature module to further process and enhance the features extracted by the backbone network, including multi-scale feature fusion through a feature pyramid network or a path aggregation network; two paths are taken for feature enhancement processing, as follows: The first path: From top to bottom, the deep feature map is upsampled to increase its spatial resolution. The upsampled deep feature map is fused with the shallower feature map of the previous layer. This fusion is achieved by element-by-element addition or channel concatenation. The shallower feature map is injected with semantic information from the deep layer. The second path: from bottom to top, convolution processing and downsampling are performed on the detected longitude and latitude features of the forming net to obtain a shallow feature map; the shallow feature map is fused with the deeper feature map of the next layer; the convolution processing and downsampling are implemented by convolution or pooling operations with a step size of 2, and after downsampling, the shallow feature map will be fused with the deeper feature map; The detection head is responsible for final category prediction and bounding box regression based on the fused feature map. In the defect recognition and positioning model, the head outputs the category probability, confidence, and coordinate information of each detection box. The data set is input into the detection module, and each channel in the image data is processed separately using the convolution kernel in the deep convolution to generate the same number of output channels as the input channels; The point-by-point convolution kernel is used to linearly combine the output of the depth convolution to obtain the longitude and latitude characteristics of the forming mesh; The enhanced detected warp and weft features of the forming mesh include: Upsampling the detected longitude and latitude features of the forming mesh to obtain a deep feature map; fusing the deep feature map with a shallower feature map of the previous layer; The detected longitude and latitude features of the forming net are convolved and downsampled to obtain a shallow feature map; the shallow feature map is fused with a deeper feature map of the next layer.

2. The forming mesh defect detection method based on the improved convolutional neural network according to claim 1 is characterized in that: The lighting system adopts RGB surface light source, wherein the R value is 90-120.

3. The forming mesh defect detection method based on the improved convolutional neural network according to claim 1 is characterized in that: The preprocessing comprises: Rotate, translate, scale, flip, and adjust brightness and contrast of defective image data; Normalize the image data with defects; Mark defects in image data.

4. A forming mesh defect detection device based on an improved convolutional neural network, characterized in that: A forming mesh defect detection method based on an improved convolutional neural network applied to any one of claims 1 to 3, comprising an environment setting module, a data processing module, a model building and training module, and a detection module; An environment setting module is used to set the lighting system to irradiate the set light on the forming net; A data processing module is used to collect image data with defects, pre-process the data set, and obtain an image sample set; A model building and training module is used to build a defect recognition and positioning model based on YOLOv8N. The defect recognition and positioning model includes a detection module, a feature module and a boundary prediction module. The detection module is used to detect the longitude and latitude features of the forming mesh. The feature module is used to enhance the detected longitude and latitude features of the forming mesh. The boundary prediction module is used to define the detection frame and output the attribute information of each detection frame. The image sample set is input into the defect recognition and positioning model for training to obtain the trained defect recognition and positioning model. The detection module adopts a depth separable convolution, including a depth convolution and a point-by-point convolution. The depth convolution and the point-by-point convolution are relatively independently set. The depth convolution includes multiple separate convolution kernels. Each input channel is only set with one convolution kernel. The point-by-point convolution includes a point-by-point convolution kernel. After the point-by-point convolution, an h-swish activation function is set. The detection module is used to input the image to be identified into the trained defect recognition and positioning model and output the defect positioning result; after completing the output of the defect positioning result, randomly perform one or all of the following steps: Inactivation step: randomly select a portion of the nodes in the convolutional neural network and set the output of these nodes to zero; Batch normalization step: Each batch of defect location result data output is standardized to have zero mean and unit variance.

5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the forming mesh defect detection method based on the improved convolutional neural network as described in any one of claims 1 to 3.

Images