A flat material surface quality detection method and device based on a lightweight saliency detection model and a medium

Abstract

The application discloses a kind of based on lightweight saliency detection model's flat material surface quality detection method, equipment and medium, it is related to image recognition field, the detection method includes the following steps: S1, label flat material surface damage data set, divide into training set and test set;S2, construct lightweight saliency detection model;S3, construct loss function, using deep supervision strategy is to the output feature map of each stage of decoder generation prediction and calculates loss;S4, utilize training set training detection model;S5, inference obtains the saliency map of surface defect.The application is by constructing the lightweight backbone network with scale self-adaptive feature extraction module as core, model parameter quantity only has 2.29M, while maintaining lightweight architecture, reaches 62fps inference speed, can efficiently extract the key features of flat material surface defect, meet the demand of high-speed detection of industrial production line, ensure that real-time detection target is achieved under the premise of not sacrificing accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a flat material surface quality detection method and device based on a lightweight saliency detection model and a medium. BACKGROUND

[0002] With the rapid development of modern industrial manufacturing technology, flat materials, as an important basic industrial material, are widely used in key fields such as construction, automobiles, ships, aerospace, and home appliance manufacturing. The surface quality of flat materials directly affects the structural safety, service life, and appearance performance of products. For example, in the production of automobile steel plates, surface scratches or rust may cause defects in the subsequent coating process, and even cause structural corrosion. In the application of metal foil materials for precision electronic components, small deformations or cracks may cause serious problems such as circuit short circuits. Therefore, flat material surface quality detection is a key link in the industrial production process to ensure product qualification rate, and has important significance for improving production efficiency and reducing manufacturing cost.

[0003] In recent years, with the deepening of the concept of intelligent manufacturing and Industry 4.0, the traditional detection method relying on manual visual inspection has been difficult to meet the high-speed and high-precision detection requirements of modern production lines, and it is urgent to realize efficient recognition and positioning of surface defects through automation and intelligent technology. In the existing technology, computer vision-based detection technology has gradually emerged. Early methods mostly use traditional image processing algorithms such as threshold segmentation and edge detection, but such methods are sensitive to changes in light and surface texture interference, have poor generalization ability, and are difficult to adapt to complex industrial scenes. In recent years, with the development of deep learning technology, saliency detection algorithms based on convolutional neural networks (CNN) have been widely used in surface defect detection. The core idea is to automatically learn defect features through a deep learning model to realize segmentation and positioning of salient objects.

[0004] Most current saliency detection algorithms use VGG, Resnet, transformer, etc. as the backbone architecture, and typical schemes such as CPD, U2Net, etc. extract multi-level features through VGG / ResNet backbone networks, utilize feature pyramid fusion to integrate deep semantic information, and achieve high defect recall rate in laboratory environments. Meanwhile, PFSNet based on Transformer architecture further introduces a self-attention mechanism to further improve positioning accuracy in complex texture backgrounds.

[0005] However, these methods expose serious defects in actual industrial deployment: on the one hand, the backbone network such as VGG has a parameter amount of more than 130M, resulting in a bloated model volume, and its storage requirement exceeds the carrying capacity of embedded devices; on the other hand, industrial production lines have strict requirements on detection speed, for example, the running speed of a cold-rolled steel plate production line can reach hundreds of meters per minute, and the detection system is required to have a processing capacity of dozens of frames per second (fps), and the single-frame inference time of a complex structure such as ResNet usually exceeds 80ms, which cannot meet the real-time detection requirement, and is easy to cause detection delay or data backlog; in addition, although some models reduce the parameter amount through lightweight design, the excessive simplification leads to insufficient feature extraction capability, and the adaptability to defects of different scales and shapes is poor, and the detection accuracy is significantly reduced.

[0006] Therefore, there is an urgent need for a model that balances lightweight, real-time and detection accuracy to solve the above problems. SUMMARY

[0007] The present application overcomes the shortcomings of the prior art and provides a flat material surface quality detection method, device and medium based on a lightweight saliency detection model, which realizes accurate detection of the flat material surface quality by designing a lightweight saliency detection model that balances detection effect and real-time performance, thereby making up for the defect that the real-time performance of the existing method cannot meet the actual industrial detection requirement.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is as follows: in a first aspect, the present application provides a flat material surface quality detection method based on a lightweight saliency detection model, comprising the following steps:

[0009] S1, label the flat material surface damage data set and divide it into a training set and a test set;

[0010] S2, construct a lightweight saliency detection model, which is composed of a bottom-up encoder, a top-down decoder and a lateral connection between the encoder and the decoder;

[0011] S3, construct a loss function, and use a deep supervision strategy to generate a prediction and calculate a loss for the output feature map of each stage of the decoder;

[0012] S4, train the detection model using the training set, and use an Adam optimizer and a poly learning rate decay strategy;

[0013] S5, normalize the test set image, input the detection model, and infer to obtain a saliency map of the surface defect.

[0014] In a preferred embodiment of the present application, in the step of S2, the encoder is constructed with a scale adaptive feature extraction module (SAFE) as a basic unit, including five stages {E1, E2, E3, E4, E5}, and each stage stacks different numbers of SAFE modules; the depth separable convolution is used between stages to adjust the channel number and the feature map size; the SAFE module includes a multi-scale feature interaction module (MFI) and a dynamic selection module (DS).

[0015] In a preferred embodiment of the present application, the MFI module realizes feature interaction through the following steps:

[0016] S21, a plurality of depth separable convolutions with different expansion rates are used to process the input feature map to obtain N branch feature maps ; wherein C , H , W are the channel number, height and width respectively; R is a real number set; ;

[0017] S22, after 3x3 average pooling operation on , the residual connection is added to , and the element-wise cumulative output feature map of each branch is obtained to obtain a feature map , which is input to the DS module.

[0018] In a preferred embodiment of the present application, the DS module dynamically allocates feature weights through the following steps:

[0019] S211, the feature map is processed by 3x3 convolution, and is input to a dynamic measurement module (DMM), and the global context information in the channel dimension is extracted by global average pooling to embed a learnable vector ;

[0020] S212, the Z is input to a multi-layer perceptron, the dynamic attention weight of each branch is calculated, and the corresponding branch feature is multiplied by the channel and then pixel-wise accumulated, and the residual connection is combined to output the SAFE module result ; wherein FC ( Z ) represents full connection layer processing on the vector Z; ReLU represents nonlinear activation function processing on the output of FC ( Z ); FC represents the second full connection layer processing on the output of ReLU ; is a normalization processing on the output of the final full connection layer. is the result of the DS module after dynamic attention weight distribution and multi-branch feature fusion.

[0021] In a preferred embodiment of the application, the number of SAFE modules in each stage of the encoder is: one in each of stages E1 and E2, three in each of stages E3 and E5, and six in stage E4.

[0022] In a preferred embodiment of the application, in the step of S2, the decoder is constructed based on a multi-scale feature aggregation module MFA, which includes a feature fusion module and an MFA module; the feature fusion module integrates the encoder features, the pyramid pooling module output features and the up-sampling features, and processes them through a depth separable convolution.

[0023] In a preferred embodiment of the application, the MFA module realizes cross-scale aggregation through the following steps:

[0024] S2111, divide the input features into 4 parts by channel , , and use a depth separable convolution with an expansion rate to extract feature maps ;

[0025] S2112, establish element-wise addition between different branches through residual connection to generate decoder output feature maps .

[0026] In a preferred embodiment of the application, in the step of S3, the loss function adopts a deep supervision strategy, and the output feature maps of each stage of the decoder are generated through 1x1 convolution and sigmoid activation function , and the loss function is defined as:

[0027] ; wherein, is a standard cross-entropy loss function; is a labeled saliency image; is a weight coefficient.

[0028] In a second aspect, the embodiments of the application provide an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor;

[0029] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the flat surface quality detection method based on the lightweight saliency detection model.

[0030] In a third aspect, the embodiments of the present application provide a computer readable storage medium, which stores computer instructions for causing a processor to implement the flat material surface quality detection method based on the lightweight saliency detection model when executed.

[0031] The present application solves the defects in the background art and has the following advantages:

[0032] (1) The present application provides a flat material surface quality detection method, device and medium based on a lightweight saliency detection model. By constructing a lightweight backbone network with a scale-adaptive feature extraction module as the core, combining multi-scale feature interaction and dynamic selection mechanism, the model parameter quantity and computational complexity are significantly reduced. At the same time, the representation ability of different scale features is enhanced through a dynamic weight distribution strategy. The model parameter quantity is only 2.29M. While maintaining a lightweight architecture, it achieves an inference speed of 62fps. It can efficiently extract the key features of flat material surface defects, and then realize fast inference speed and good feature learning ability. It effectively solves the problem of insufficient real-time performance caused by the large parameter quantity of the deep model in the prior art, thereby meeting the demand for high-speed detection of industrial production lines and ensuring real-time detection on the premise of not sacrificing accuracy.

[0033] (2) The decoder architecture based on the multi-scale feature aggregation module innovatively designed in the present application extracts multi-scale information by splitting the features by channel and using convolution with different dilation rates, and establishes cross-branch residual connection, which strengthens the correlation between different scale feature maps. The decoder can better fuse the deep semantic information and shallow detail features from the encoder, improving the positioning accuracy of small defects and defects in complex backgrounds. It effectively solves the problem of information loss when the pyramid pooling module outputs features across scales, thereby enhancing the model's adaptability to diverse defect morphologies and ensuring stable output of high-quality saliency maps in various industrial scenarios.

[0034] (3) The present application uses a deep supervision loss function to perform multi-scale optimization on the output feature maps of each stage of the decoder. By imposing cross-entropy loss on the prediction results of different levels, it improves the transparency and gradient propagation efficiency of the hidden layer learning process, promotes the collaborative learning of multi-level features by the model, avoids the problem of insufficient deep feature learning caused by traditional single-scale loss functions, and further improves the detection accuracy and generalization ability of the model. The model can still maintain stable defect recognition performance in the face of complex situations such as light changes and surface texture interference. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only represent some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0036] Figure 1 is a flat material surface quality detection method flow chart based on a lightweight saliency detection model of embodiment 1 of the present application.

[0037] Figure 2 is a detection model overall network architecture diagram of embodiment 1 of the present application.

[0038] Figure 3 is a detection model backbone network parameter setting diagram of embodiment 1 of the present application.

[0039] Figure 4 is a detection model SAFE module design diagram of embodiment 1 of the present application.

[0040] Figure 5 is a detection model decoder design diagram of embodiment 1 of the present application.

[0041] Figure 6 is a detection model MFA module design diagram of embodiment 1 of the present application.

[0042] Figure 7 is a flat material physical object and a saliency map output by the detection model of embodiment 1 of the present application.

[0043] Figure 8 shows an electronic device structure schematic diagram that can be used to implement embodiment 1 of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the scope of protection of the present application.

[0045] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0046] Embodiment 1: as Figure 1As shown, a flat material surface quality detection method based on a lightweight saliency detection model includes the following steps:

[0047] S1, label the flat material surface damage data set, and divide it into training set and test set in the proportion of 4:1;

[0048] In this embodiment, in the data set labeling, the flat material surface images covering different types, i.e. including but not limited to scratches, rust or deformation, different lighting conditions and surface textures are collected to ensure sample diversity. A professional image labeling tool (such as LabelMe, VGG Image Annotator) is used to manually label the damage area in the image at the pixel level, i.e. to mark the defect contour pixel by pixel, to generate a saliency image with the same size as the original image, i.e. a label image, in which the defect area pixel value is set to 1, i.e. foreground, and the non-defect area is set to 0, i.e. background, forming a one-to-one corresponding data pair of the original image and the label image.

[0049] Further, the labeled data set is randomly divided into training set and test set in the proportion of 4:1. When dividing, it is necessary to ensure that the defect types and scale distribution of the training set and the test set are consistent (such as similar proportions of large-scale rust and tiny scratches), so as to avoid the decline of the model generalization ability due to unbalanced data distribution.

[0050] Further, the generated saliency image is directly used as the supervision signal of the loss function in S3 step to calculate the difference between the prediction result and the true label. The training set after division is used for model parameter learning in S4 step, and the test set is used for inference effect evaluation in S5 step, to ensure the model verification generalization ability on unseen data.

[0051] S2, construct a lightweight saliency detection model, the overall network architecture of which is as shown in Figure 2 E1-E5 respectively represent the five stages of the encoder constructed in this embodiment {E1, E2, E3, E4, E5}; D1-D5 respectively represent the five stages of the decoder constructed in this embodiment {D1, D2, D3, D4, D5}; PPM represents the pyramid pooling module constructed in this embodiment; represents the feature map output by each stage of the decoder, represents the feature map output by each stage of the encoder, wherein .

[0052] In this embodiment, the model is composed of a bottom-up encoder, a top-down decoder and a lateral connection between them. The specific construction process includes the following steps:

[0053] 2-a. Construction of the encoder, a scale adaptive feature extraction module SAFE is designed as a unit of the encoder; as shown in Figure 2As shown, the encoder includes five stages, denoted as {E1, E2, E3, E4, E5}, and each stage of the encoder is stacked by different number of SAFE modules.

[0054] Optionally, since the resolution of the shallow network feature map is high, in order to reduce the amount of calculation, only 1 SAFE module is used in E1 and E2 stages, 3 SAFE modules in E3 and E5 stages, and 6 SAFE modules in E4 stage.

[0055] As shown, the depth separable convolution is used between each stage to adjust the number of channels and the size of the feature map, Figure 3 The specific parameter settings in the backbone network are shown in Table 1, and the design of the SAFE module as the basic unit of the encoder is shown in Table 2. Figure 4 As shown, the SAFE module includes a multi-scale feature interaction module MFI and a dynamic selection module DS.

[0056] In Table 2, Figure 4 In Table 2, DDS represents a depth separable convolution; AP represents an average pooling; Conv 3×3 represents a 3×3 convolution; DMM represents a dynamic measurement sub-module; GAP represents a global average pooling; FC represents a fully connected layer; ReLU represents a ReLU activation function; Softmax represents a normalization processing; ⊕ represents a bitwise addition; and ⊗ represents a bitwise multiplication.

[0057] Further, in the MFI module, first, a plurality of depth separable convolutions (DDS) with different expansion rates are used to process the input feature map to obtain branch feature maps ; wherein C , H , W are the number of channels, height and width, respectively; is a real set; . After being processed by a 3×3 average pooling operation (AP), the feature interaction is realized by adding the residual connection to , so as to enhance the representation ability of each branch to the salient target; the output feature maps of each branch are element-wise accumulated to obtain a feature map , which is input to the DS module.

[0058] Further, the DS module is data-driven, and can adaptively perceive and measure the importance of different scale features according to the changes of the input feature map. Specifically, first, a 3×3 convolution is used to process the feature map , and then the processed feature map is input into a dynamic measurement sub-module DMM, which uses a global average pooling (GAP) operation to extract the global context information of in the channel dimension, and embeds the global context information into a learnable vector ; wherein, Z the i-th component of the vector c is represented as:

[0059] ; wherein, c represents the channel index, corresponding to the channel dimension of the feature map, ; and represent the pixel coordinates in the height direction and the width direction of the feature map, respectively; specifically, the feature map is processed by a global average pooling operation in the channel dimension, and the average value of all pixels c ( x , y ) in each channel Z is obtained to obtain the i-th component of the learnable vector c .

[0060] Next, the learnable vector Z is input into a simple multi-layer perceptron (MLP) for processing, which consists of two fully connected layers, a nonlinear ReLU, and a softmax; since the MLP is learnable, it can dynamically assign different attention weights to each scale feature, and the calculation formula of the dynamic attention weight i of the i-th branch is:

[0061] ;

[0062] wherein, ; FC ( Z ) represents the fully connected layer processing of the vector Z; ReLU represents the nonlinear activation function processing of the output of FC ( Z ); FC represents the second fully connected layer processing of the output of ReLU ; is the normalization processing of the output of the final fully connected layer;

[0063] After that, is multiplied by the corresponding branch feature by channel, and the multiplied result is integrated by pixel-wise addition to obtain the feature map :

[0064] ;

[0065] Finally, the original input feature map X is element-wise added to through residual connection to obtain the output of the final SAFE module​Y :

[0066] .

[0067] 2-b. Decoder construction, a decoder based on MFA module is designed as shown in Figure 5 . n The decoder D

[0068] In Figure 5 , DSConv k x k represents the deep separable k x k dilated convolution module constructed in this embodiment; Upsample represents the upsample module constructed in this embodiment; Conv 1x1 represents the 1x1 convolution module constructed in this embodiment; Add represents the bit-wise addition module constructed in this embodiment.

[0069] As can be seen from Figure 2 , unlike the classical encoder-decoder network structure, the output of the last stage (E5) of the encoder is passed through the pyramid pooling module PPM in this embodiment to further improve the learning of global features by the network, and the output features of the PPM are input into the decoder of each stage for feature fusion, so as to fully utilize the semantic information of the deep layers of the network.

[0070] As shown in Figure 5 , the feature fusion module fuses features from three directions, and uses a deep separable dilated convolution to integrate the fused results.

[0071] After the features from the three directions are fused, it is not enough to simply use a convolution layer for processing, especially the output features of the PPM module will undergo a relatively large scale span (maximum 16 times scale span) in the upsample process, and it is necessary to reasonably establish the connection across the scale to further process the fused features by constructing a multi-scale feature aggregation module (MFA).

[0072] The implementation of the multi-scale feature aggregation module (MFA) is shown in Figure 6 . In Figure 6 , Split represents the channel division module constructed in this embodiment; DDS represents the deep separable convolution constructed in this embodiment; Add represents the bit-wise addition module constructed in this embodiment; Concat represents the channel splicing module constructed in this embodiment; Conv represents the convolution module constructed in this embodiment.

[0073] As shown in Figure 6 , the output features of the feature fusion module are taken as the input of the MFA module, denoted as Fand divided into 4 parts by channel, and the 4 parts are respectively denoted as , wherein , the extraction of different scale information is performed by depth separable convolution with an expansion rate of , four different branches are formed, and a feature map is obtained, wherein .

[0074] Further, in order to realize the aggregation of features across scales, residual connections are established between different branches, and the residual connections are obtained by element-wise addition:

[0075] .

[0076] Further, the feature map output by each stage of the decoder is denoted as , the feature map output by each stage of the encoder is denoted as , wherein , and the following is obtained:

[0077] ;

[0078] wherein is the depth separable convolution of the i th stage k × k ; Up is an up-sampling operation; denotes a 1×1 convolution operation.

[0079] S3, constructing a loss function;

[0080] Specifically, deep supervision is used to improve the transparency of the hidden layer learning process. For the output feature map of the decoder at each stage, a 1×1 convolution and sigmoid activation function are used to generate multiple predictions, i.e. , wherein .

[0081] Further, a standard cross-entropy loss function is used, and the function definition is as follows:

[0082] ; wherein is a standard cross-entropy loss function; is a labeled saliency image; is a weight coefficient, which is set to 0.4 in order to balance the weight value of the loss.

[0083] S4, training the detection model using the training set;

[0084] Specifically, the data set obtained in the S1 step is used to train the detection model constructed in the S2 step, and the backbone network part in the model is initialized with a pre-model; an Adam optimizer is used for optimization, and the parameters are set to , , and the weight is attenuated by 10 -4 , and the batchsize is set to 20; the initial learning rate is set to 0.0005, and a poly learning rate decay strategy is used, and the learning rate at the nth iteration is:

[0085] ; wherein, is the initial learning rate; the power is set to 0.9, and the model is iteratively trained for 100 rounds, i.e. #epochs .

[0086] S5, reasoning and obtaining a detection result;

[0087] Specifically, after the test set image is normalized, it is input into the trained saliency detection model, and the final saliency map is obtained; as shown in Figure 7 , the left side is the original image of the surface defect of the flat material, and the right side is the saliency map output by the lightweight saliency detection model constructed in the embodiment; wherein, the parameter amount of the lightweight saliency detection model in the embodiment is only 2.29M, the hardware environment is shown in Table 1, and the inference speed comparison of the lightweight saliency detection model and the traditional model in the embodiment 1 is shown in Table 2.

[0088] Table 1: Hardware environment of the lightweight saliency detection model in embodiment 1

[0089]

[0090] Table 2: Inference speed comparison of the lightweight saliency detection model and the traditional model in embodiment 1

[0091]

[0092] As shown in Table 2, FPS is used as a quantitative indicator of inference speed, which represents the number of images that the model can process per second during inference in deep learning. The higher the FPS value, the faster the inference speed of the model. The application constructs a lightweight backbone network with a scale adaptive feature extraction module as the core, combines multi-scale feature interaction and dynamic selection mechanism, significantly reduces the model parameter quantity and computational complexity, and enhances the representation ability of different scale features through a dynamic weight distribution strategy. While maintaining a lightweight architecture, it achieves an inference speed of 62fps, efficiently extracts key features of flat surface defects, and achieves fast inference speed and good feature learning ability, effectively solving the problem of insufficient real-time performance caused by large parameter quantity in the prior art, thereby meeting the demand for high-speed detection of industrial production lines and ensuring real-time detection without sacrificing accuracy.

[0093] Specifically, the high speed of the model is mainly due to the fast forward calculation ability of the lightweight backbone network and the efficiency of the model structure. The lightweight design in the backbone network and the intelligent focusing of the scale adaptive module reduce a large number of unnecessary floating point operations. Through the encoder-decoder structure combined with horizontal connection, the model can directly perform target detection on the multi-scale feature map fused with high-level semantic information and low-level spatial details in one forward propagation, completely avoiding the huge computational overhead of multiple processing of input images required by traditional multi-scale detection methods (such as image pyramid), thereby achieving extremely high inference speed.

[0094] At the same time, the guarantee of detection accuracy depends on the powerful multi-scale feature fusion ability of the model and the improved feature quality of the scale adaptive module. The encoder extracts multi-level features, the decoder transmits strong semantic information through upsampling, and the horizontal connection accurately fuses these high-level semantic information with rich spatial detail features of the corresponding level of the encoder. The encoder extracts multi-level features, the decoder transmits strong semantic information through upsampling, and the horizontal connection accurately fuses these high-level semantic information with rich spatial detail features of the corresponding level of the encoder. The scale adaptive module embedded in the lightweight backbone further improves the effectiveness of the features, which can dynamically adapt to targets and complex backgrounds of different sizes and shapes, ensuring that even under the condition of limited parameters and calculations, the extracted features are rich and targeted enough to maintain high detection accuracy.

[0095] Embodiment 2 Figure 8An electronic device structure diagram that can be used to implement the embodiment 1 of the present application is shown. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0096] As shown in Figure 8 The electronic device 10 includes at least one processor 11, and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., connected to the at least one processor 11 in communication, where the memory stores a computer program executable by the at least one processor 11, and the computer program is executed by the at least one processor 11 to enable the at least one processor 11 to perform the method provided by the present application.

[0097] Further, the processor 11 can perform various appropriate actions and processes according to the computer program stored in the Read-Only Memory (ROM) 12 or loaded into the Random Access Memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0098] Various components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0099] Further, the processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Process (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, e.g., the method of resource management for a database.

[0100] In some specific embodiments, the method of resource management for a database can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the method of resource management for a database described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the method of resource management for a database by other any appropriate means, e.g., by means of firmware.

[0101] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a System on Chip (SOC), a Complex Programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0102] Computer programs used to implement the present methods can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flow diagrams and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0103] In the context of the present application, the computer readable storage medium stores computer instructions for causing a processor to implement the method of resource management of a database provided by the present application when executed. The computer readable storage medium can be a tangible medium which can contain or store the computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium can be a machine readable signal medium. More specific examples of the machine readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0104] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0105] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0106] Optionally, the computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS) services.

[0107] The above is according to the ideal embodiment of the present application, through the above description, for those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and can be realized in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0108] In addition, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments that those skilled in the art can understand.

Claims

1. A method for flat surface quality detection based on a lightweight saliency detection model, characterized in that, The method comprises the following steps: S1, marking a flat material surface damage data set, and dividing it into a training set and a test set; S2, constructing a lightweight saliency detection model, which is composed of a bottom-up encoder, a top-down decoder, and a lateral connection between the encoder and the decoder; S3, constructing a loss function, using a deep supervision strategy to generate a prediction and calculate a loss for the output feature map of each stage of the decoder; S4, training the detection model using the training set, using an Adam optimizer and a poly learning rate decay strategy; S5, normalizing the test set image, inputting it into the detection model, and obtaining a saliency map of the surface defect through inference; In the step S2, the encoder is constructed with a scale adaptive feature extraction module SAFE as the basic unit, containing five stages {E1, E2, E3, E4, E5}, and each stage stacks different numbers of SAFE modules; depth separable convolution is used between stages to adjust the channel number and feature map size; the SAFE module includes a multi-scale feature interaction module MFI and a dynamic selection module DS; The dynamic selection module DS dynamically allocates feature weights through the following steps: S211、output the feature map output by the multi-scale feature interaction module MFI A 3x3 convolution processing is performed, an input dynamic measurement quantum module DMM is used, global context information in the channel dimension is extracted through global average pooling, and a learnable vector is embedded ; S212、combine the results of the DS module and the SAFE module Z Input the multi-layer perceptron, calculate the dynamic attention weight of each branch , and multiply it with the corresponding branch feature by channel, then accumulate pixel by pixel, combine with the residual connection to output the SAFE module result ; wherein, FC ( Z ) represents full connection layer processing on vector Z; ReLU represents the output of FC ( Z ) to the nonlinear activation function processing; FC represents the output of ReLU to the second full connection layer processing; is to normalize the output of the final full connection layer; is the result of the DS module after dynamic attention weight distribution and multi-branch feature fusion.

2. The flat material surface quality detection method based on a lightweight saliency detection model according to claim 1, characterized in that: The multi-scale feature interaction module MFI realizes feature interaction through the following steps: S21, using a plurality of depth separable convolutions with different expansion rates, processing the input feature map to obtain a plurality of branch feature maps; wherein, , N , C , H , W , , wherein, n, h and w respectively represent the number of channels, height and width; R is a real set. S22, after 3x3 average pooling operation, residual connection and addition, element-wise accumulation of each branch output feature map to obtain a feature map, and input to the DS module.

3. The flat material surface quality detection method based on a lightweight saliency detection model according to claim 1, characterized in that: The number of SAFE modules in each stage of the encoder is: one in each of E1 and E2 stages, three in each of E3 and E5 stages, and six in E4 stage.

4. The flat material surface quality detection method based on a lightweight saliency detection model according to claim 1, characterized in that: In the step S2, the decoder is constructed based on a multi-scale feature aggregation module MFA, which includes a feature fusion module and an MFA module; the feature fusion module integrates the encoder features, the pyramid pooling module output features, and the up-sampling features, and processes them through depth separable convolution.

5. The flat material surface quality detection method based on a lightweight saliency detection model according to claim 4, characterized in that: The MFA module realizes cross-scale aggregation through the following steps: S2111, divide the input features into 4 parts by channel , , use the depth separable convolution with the expansion rate to extract the feature map ; S2112. Element-wise addition between different branches is established through residual connection to generate the decoder output feature map .

6. The flat material surface quality detection method based on a lightweight saliency detection model according to claim 1, characterized in that: In the step of the S3, the loss function adopts a deep supervision strategy, and a prediction is generated for the feature map output by each stage of the decoder through a 1x1 convolution and a sigmoid activation function , and the loss function is defined as: ; wherein, is the standard cross-entropy loss function; is the annotated saliency image; is the weight coefficient.

7. An electronic device, comprising: It comprises: At least one processor; and a memory connected in communication with the at least one processor; Wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the flat material surface quality detection method based on the lightweight saliency detection model of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to execute when the flat material surface quality detection method based on the lightweight saliency detection model of any one of claims 1-6 is implemented.

Images