An industrial image defocus evaluation method and device, electronic equipment and storage medium

Abstract

The application provides an industrial image defocus evaluation method, which can be applied to the technical field of computer vision and the technical field of industrial precision detection. The method comprises the following steps: acquiring an industrial image through a visual sensor, and dividing the industrial image into blocks to obtain a set of image blocks; inputting each image block in the set of image blocks into a trained image block defocus evaluation network respectively to obtain a defocus evaluation result of each image block, and taking the average of all defocus evaluation results of the image blocks as the defocus evaluation result of the industrial image. The application further provides an industrial image defocus evaluation device, equipment and a storage medium. The application can improve the precision of defocus evaluation while being convenient to deploy and apply.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and the technical field of industrial precision detection, and more particularly to an industrial image defocus evaluation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In recent years, computer vision technology has entered people's daily work and life. In the field of industrial precision detection, high-precision cameras used often have fixed focal length and shallow depth of field, which makes the range of clear imaging smaller, and thus defocus blur is easily generated. Therefore, it is of great significance to evaluate the defocus of industrial images. Since the degree of image defocus is positively correlated with image sharpness, and is robust to exposure conditions, defocus evaluation methods based on image sharpness have attracted much attention. Traditional defocus evaluation methods based on image sharpness usually rely on image feature extraction and quantitative analysis, mainly including two categories: defocus evaluation methods based on image gradient information and defocus evaluation methods based on image frequency domain analysis. Defocus evaluation methods based on image gradient information evaluate the degree of image defocus by calculating the gradient information of the pixels in the image. Commonly used gradient information extraction operators include Sobel operator, Scharr operator, Laplacian operator, etc. These operators perform convolution operation on the image to obtain the gradient information of the pixels in the image, complete the representation of image sharpness, and then obtain the defocus evaluation result. In addition, other features such as entropy and contrast can be combined to further improve the accuracy of image defocus evaluation. Defocus evaluation methods based on image frequency domain analysis convert the image from the spatial domain to the frequency domain and analyze the frequency components of the image. The sharpness of the image is related to the content of high-frequency components. The more high-frequency components, the sharper the image, and the lower the degree of image defocus. Commonly used frequency domain analysis methods include Fourier transform and wavelet transform. The above traditional defocus evaluation methods based on image sharpness usually rely on manually designed image features, which have high efficiency but poor accuracy of defocus evaluation. With the development of deep learning, defocus evaluation methods based on deep networks have attracted attention. Defocus evaluation methods based on lightweight networks improve the convenience of deployment and application by virtue of the lightweight structure of the network, but their accuracy is low. Therefore, how to design an accurate defocus evaluation method based on lightweight networks still needs further research. SUMMARY

[0003] In view of the above problems, the present application provides an industrial image defocus evaluation method and device, an electronic device, and a storage medium.

[0004] According to a first aspect of the present application, an industrial image defocus evaluation method is provided, which comprises:

[0005] An industrial image is obtained by a visual sensor, and the industrial image is divided into blocks to obtain a set of image blocks.

[0006] inputting each image block in the image block set into the trained image block defocus evaluation network respectively, obtaining the defocus evaluation result of each image block, and taking the mean value of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image;

[0007] The image block defocus evaluation network comprises a multi-path feature extractor, a feature refining unit and a quantization unit.

[0008] The multi-path feature extractor comprises a plurality of hollow convolution, a feature splicing layer and a maximum pooling layer.

[0009] The feature refining unit comprises a double-path channel attention and a convolution layer connected in sequence, and the double-path channel attention comprises a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature splicing and flattening layer and a first full connection layer.

[0010] The quantization unit comprises a second adaptive maximum pooling layer, a feature flattening layer and a second full connection layer connected in sequence.

[0011] According to the embodiment of the present application, the above-mentioned inputting each image block in the image block set into the trained image block defocus evaluation network respectively, obtaining the defocus evaluation result of each image block comprises:

[0012] The image block defocus evaluation network is trained by using the pre-constructed training sample and combining the pre-set loss function, and the trained image block defocus evaluation network is obtained.

[0013] The image block in the image block set is subjected to feature extraction by using the trained multi-path feature extractor, the plurality of features extracted are spliced, a maximum pooling operation is performed on the spliced result, and an initial feature map is obtained.

[0014] The maximum pooling operation and the average pooling operation are performed on the initial feature map by using the trained feature refining unit, the results of the pooling operations are subjected to splicing and flattening operations, and the results of the splicing and flattening operations are subjected to full connection operation and convolution operation, and an intermediate feature map is obtained.

[0015] The maximum pooling operation is performed on the intermediate feature map by using the trained quantization unit, and the full connection operation is performed on the result of the maximum pooling operation after the flattening operation, and the defocus evaluation result of the image block is obtained.

[0016] According to the embodiment of the present application, the above-mentioned using the trained multi-path feature extractor to extract features of the image block in the image block set, splicing the plurality of features extracted, and performing a maximum pooling operation on the spliced result to obtain an initial feature map comprises:

[0017] The image block is feature-extracted by using multiple hollow convolution, multiple feature maps are obtained, and the multiple feature maps are spliced along the spatial dimension by using a feature splicing layer to obtain a first feature map;

[0018] The first feature map is subjected to a maximum pooling operation along the height dimension and the width dimension by using a maximum pooling layer to obtain a second feature map, and the second feature map is input into the trained feature refining unit as an initial feature map.

[0019] According to the embodiment of the present application, the above-mentioned maximum pooling operation and average pooling operation are performed on the initial feature map by using the trained feature refining unit, the results of the pooling operations are spliced and flattened, and the results of the splicing and flattening operations are subjected to full connection operation and convolution operation to obtain an intermediate feature map, including:

[0020] The initial feature map is subjected to adaptive maximum pooling operation and adaptive average pooling operation by using the first adaptive maximum pooling layer and the first adaptive average pooling layer of the trained double-path channel attention, to obtain a third feature map and a fourth feature map;

[0021] The third feature map and the fourth feature map are spliced along the channel dimension by using the feature splicing and flattening layer of the trained double-path channel attention to obtain a fifth feature map, and the fifth feature map is flattened to obtain a sixth feature map;

[0022] The sixth feature map is subjected to multi-layer linear processing by using the first full connection layer of the trained double-path channel attention to obtain a seventh feature map, and the seventh feature map is multiplied with the initial feature map to obtain an eighth feature map;

[0023] The eighth feature map is subjected to convolution processing by using the convolution layer of the trained feature refining unit to obtain a ninth feature map, and the ninth feature map is input into the trained quantization unit as an intermediate feature map.

[0024] According to the embodiment of the present application, the above-mentioned maximum pooling operation is performed on the intermediate feature map by using the trained quantization unit, and the result of the maximum pooling operation is subjected to full connection operation after flattening operation to obtain the out-of-focus evaluation result of the image block, including:

[0025] The intermediate feature map is subjected to adaptive maximum pooling operation by using the second adaptive maximum pooling layer of the trained quantization unit to obtain a tenth feature map;

[0026] The tenth feature map is subjected to flattening operation by using the feature flattening layer of the trained quantization unit to obtain an eleventh feature map;

[0027] The eleventh feature map is subjected to multi-layer linear processing by using the second full connection layer of the trained quantization unit to obtain the out-of-focus evaluation result of the image block.

[0028] According to a second aspect of the present application, there is provided an industrial image defocus evaluation device, comprising:

[0029] an image block acquisition module, configured to acquire an industrial image through a visual sensor, and divide the industrial image into blocks to obtain a set of image blocks;

[0030] a defocus evaluation module, configured to input each image block in the set of image blocks into a trained image block defocus evaluation network respectively, obtain a defocus evaluation result of each image block, and take the mean of the defocus evaluation results of all the image blocks as a defocus evaluation result of the industrial image;

[0031] wherein the image block defocus evaluation network comprises a multi-path feature extractor, a feature refining unit and a quantization unit;

[0032] wherein the multi-path feature extractor comprises a plurality of hollow convolution, a feature concatenation layer and a max-pooling layer;

[0033] wherein the feature refining unit comprises a double-path channel attention and a convolution layer connected in sequence, and the double-path channel attention comprises a first adaptive max-pooling layer, a first adaptive average-pooling layer, a feature concatenation and flattening layer and a first full connection layer;

[0034] wherein the quantization unit comprises a second adaptive max-pooling layer, a feature flattening layer and a second full connection layer connected in sequence.

[0035] A third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0036] A fourth aspect of the present application also provides a computer-readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0037] The present application can improve the precision of defocus evaluation while facilitating deployment and application, and provides technical support for industrial precision detection. BRIEF DESCRIPTION OF DRAWINGS

[0038] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0039] Figure 1 is an application scenario of an industrial image defocus evaluation method according to an embodiment of the present application;

[0040] Figure 2 is a flowchart of an industrial image defocus evaluation method according to an embodiment of the present application;

[0041] Figure 3 is a structural schematic diagram of an industrial image defocus evaluation device according to an embodiment of the present application;

[0042] Figure 4 is a block diagram of an electronic device suitable for implementing an industrial image defocus evaluation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely illustrative and is not intended to limit the scope of the present application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0044] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present application. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted in the context of the present specification and should not be interpreted in an idealized or overly formal manner.

[0046] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should be generally interpreted as including at least one of the items, unless otherwise defined (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C together, etc.).

[0047] To solve the above problems in the prior art, i.e., to solve the problem of low accuracy of the existing light network-based defocus evaluation method, the present application proposes an industrial image defocus evaluation method.

[0048] Figure 1 is an application scenario of the industrial image defocus evaluation method according to an embodiment of the present application.

[0049] As shown in Figure 1 , the application scenario 100 according to this embodiment can include the field of industrial precision detection technology. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0050] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0051] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers, etc.

[0052] The server 105 can be a server that provides various services, such as a background management server that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0053] It should be noted that the industrial image defocus evaluation method provided by the embodiments of the present application can generally be executed by the server 105. Correspondingly, the industrial image defocus evaluation device provided by the embodiments of the present application can generally be arranged in the server 105. The industrial image defocus evaluation method provided by the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the industrial image defocus evaluation device provided by the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0054] It should be understood that,Figure 1 The number of terminal devices, networks and servers in the system is only illustrative. Any number of terminal devices, networks and servers can be provided according to implementation needs.

[0055] Figure 2 is a flowchart of an industrial image defocus evaluation method according to an embodiment of the present application.

[0056] As shown in Figure 2 The industrial image defocus evaluation method comprises the following steps:

[0057] In step S10, an industrial image is acquired by a vision sensor, and the industrial image is blocked to obtain a set of image blocks.

[0058] In step S10, the industrial image acquired by the vision sensor is blocked at equal intervals to obtain the set of image blocks of the industrial image.

[0059] In step S20, each image block in the set of image blocks is input into the trained image block defocus evaluation network to obtain the defocus evaluation result of each image block, and the average of the defocus evaluation results of all image blocks is taken as the defocus evaluation result of the industrial image.

[0060] In step S20, the image blocks in the set of image blocks are sequentially processed by the trained image block defocus evaluation network to obtain the defocus evaluation results of all image blocks in the set of image blocks, and the average of the defocus evaluation results of all image blocks is taken as the defocus evaluation result of the industrial image.

[0061] The image block defocus evaluation network comprises a multi-path feature extractor, a feature refining unit and a quantization unit.

[0062] The multi-path feature extractor comprises a plurality of hollow convolution, a feature splicing layer and a maximum pooling layer.

[0063] The feature refining unit comprises a double-path channel attention and a convolution layer connected in sequence, and the double-path channel attention comprises a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature splicing and flattening layer and a first full connection layer.

[0064] The quantization unit comprises a second adaptive maximum pooling layer, a feature flattening layer and a second full connection layer connected in sequence.

[0065] The application can improve the precision of defocus evaluation, and provide technical support for industrial precision detection while being convenient to deploy and apply.

[0066] The embodiment is a relatively optimal implementation manner, and an image block defocus evaluation network including a multi-path feature extractor, a feature refining unit and a quantization unit is constructed in advance. The image block defocus evaluation network is trained based on pre-constructed training samples and in combination with a pre-set loss function, parameters of the multi-path feature extractor, the feature refining unit and the quantization unit are obtained, and the parameters are applied to the industrial image defocus evaluation method of the application.

[0067] According to the embodiment of the application, the image block defocus evaluation network includes a multi-path feature extractor, a feature refining unit and a quantization unit. The multi-path feature extractor includes multiple hole convolutions, a feature splicing layer and a maximum pooling layer. The feature refining unit includes a double-path channel attention and a convolution layer connected in sequence. The double-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature splicing and flattening layer and a first full connection layer. The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second full connection layer connected in sequence.

[0068] The multi-path feature extractor can alternatively include six hole convolutions Conv1, Conv2, Conv3, Conv4, Conv5 and Conv6, a feature splicing layer FCon and a maximum pooling layer Max_Pool_1.

[0069] The feature refining unit can alternatively include a double-path channel attention DC_Attn and a convolution layer Conv7, and the two parts are connected in sequence. The double-path channel attention DC_Attn can alternatively include a first adaptive maximum pooling layer Adp_Max_Pool_1, a first adaptive average pooling layer Adp_Avg_Pool_1, a feature splicing and flattening layer and a first full connection layer FC1.

[0070] The quantization unit can alternatively include a second adaptive maximum pooling layer Adp_Max_Pool_2, a feature flattening layer and a second full connection layer FC2, and the three parts are connected in sequence.

[0071] The multi-channel feature extractor includes six hole convolutions Conv1, Conv2, Conv3, Conv4, Conv5, and Conv6, a feature splicing layer FCon, and a maximum pooling layer Max_Pool_1. The six hole convolutions respectively extract features from the input image block and send the respective output features to the feature splicing layer for splicing. The spliced features are sent to the maximum pooling layer to obtain multi-channel features. The six hole convolutions all use a convolution kernel with a size of 3x3x3x8. The parameter format of the convolution kernel is heightxwidthxinput channelxoutput channel. The hole intervals are (1, 1), (2, 2), (4, 4), (6, 6), (8, 8), and (10, 10), respectively. The convolution step sizes are all (2, 2). The padding sizes are (1, 1), (2, 2), (4, 4), (6, 6), (8, 8), and (10, 10), respectively. The padding mode is zero padding. The six hole convolutions respectively extract features from the input image block to obtain six feature maps with the same size, which are sent to the feature splicing layer FCon. FCon uses the torch.cat function in Pytorch to splice along the spatial dimension to obtain a first feature map F1. The first feature map F1 is sent to the maximum pooling layer Max_Pool_1. Max_Pool_1 performs a maximum pooling operation along the dimensions of the height and width of F1. The pooling size is (2, 2), so that the width and height of F1 are reduced by half to obtain a second feature map F2. For an input image block with a size of (highxwidthxchannel), after processing by Conv1, Conv2, Conv3, Conv4, Conv5, Conv6, and FCon, a first feature map F1 with a size of is obtained. After processing by Max_Pool_1, a second feature map F2 with a size of is obtained.

[0072] The second feature map F2 is sent to the feature refining unit. The feature refining unit includes a double-channel channel attention DC_Attn and a convolution layer Conv7. The two parts are connected in sequence.

[0073] The double-channel channel attention DC_Attn includes a first adaptive max-pooling layer Adp_Max_Pool_1, a first adaptive average-pooling layer Adp_Avg_Pool_1, a feature splicing and flattening layer, and a first fully connected layer FC1. The Adp_Max_Pool_1 performs an adaptive max-pooling operation along the dimensions of the height and width of F2; the Adp_Avg_Pool_1 performs an adaptive average-pooling operation along the dimensions of the height and width of F2. The second feature map F2 is input into the double-channel channel attention DC_Attn, and after being processed by the Adp_Max_Pool_1 and the Adp_Avg_Pool_1, the third feature map F3 and the fourth feature map F4 are obtained, both of which have a size of 48x1. Subsequently, the F3 and F4 features are input into the feature splicing and flattening layer for splicing and flattening operations: first, the torch.cat function in Pytorch is used to splice along the channel dimensions of F3 and F4 to obtain a fifth feature map F5, which has a size of 96x1; then the fifth feature map F5 is flattened by the torch.view function in Pytorch to obtain a sixth feature map F6, which has a dimension of 1 and a total of 96 features. The sixth feature map F6 is input into the first fully connected layer FC1. The first fully connected layer FC1 includes a first linear layer Linear1 and a second linear layer Linear2 connected in sequence. The first linear layer Linear1 has a size of 96x24 (input feature numberxoutput feature number), has a learnable bias parameter, and uses a ReLu activation function; the second linear layer Linear2 has a size of 24x48 (input feature numberxoutput feature number), has a learnable bias parameter, and uses a Sigmoid activation function. The sixth feature map F6 is processed by the FC1 to obtain a seventh feature map F7, which has a dimension of 1 and a total of 48 features. The seventh feature map F7 and the second feature map F2 are multiplied to obtain an eighth feature map F8, which has a size of . The F8 is the output of the double-channel channel attention DC_Attn, and the output is input into a convolutional layer Conv7. The convolutional layer Conv7 uses a 1x1x48x48 convolutional kernel with a step size of (1, 1). The eighth feature map F8 is processed by the Conv7 to obtain a ninth feature map F9, which has a size of .

[0074] The ninth feature map F9 is input into a quantization unit, which includes a second adaptive max-pooling layer Adp_Max_Pool_2, a feature flattening layer, and a second fully connected layer FC2, which are connected in sequence.

[0075] The second adaptive max-pooling layer Adp_Max_Pool_2 performs an adaptive max-pooling operation along the dimensions of the height and width of F9 to obtain a tenth feature map F 10 , which has a size of 48x1. The tenth feature map F 10The data is fed into the feature flattening layer for flattening, specifically using the `torch.view` function in PyTorch to flatten the F-shape. 10 Flattening the image yields the eleventh feature map F. 11 The eleventh feature map F has a dimension of 1 and a total of 48 features. 11 The data is fed into the second fully connected layer FC2, which contains a third linear layer (Linear3) and a fourth linear layer (Linear4) connected sequentially. The third linear layer (Linear3) has a size of 48×8 (input features × output features), learnable bias parameters, and uses the ReLU activation function; the fourth linear layer (Linear4) has a size of 8×1 (input features × output features), learnable bias parameters, and no activation function. The eleventh feature map F... 11 The output of the quantization unit is obtained after processing by the second fully connected layer FC2, which is the output of the image patch defocus evaluation network.

[0076] The training samples and loss function of the image patch defocus evaluation network, which includes a multi-path feature extractor, a feature refinement unit, and a quantization unit, according to an embodiment of the present invention, are described below.

[0077] The image patch defocus evaluation network is trained on the WPSurface dataset (https: / / pan.baidu.com / share / init?surl=AJORPtKkikQcXhI9gFbvsg&pwd=0rvz). For each training image in the WPSurface dataset, a patch defocus evaluation network is obtained by random cropping. The image patch is used as training sample data. For specific cropping methods, please refer to https: / / blog.csdn.net / yanzhiwen2 / article / details / 123870111. and In this invention, the weights are preferably set to 50 and 50 respectively. The training process uses the Adam optimizer, with an initial learning weight of 0.01. Every 60 generations, the weights decay to 0.1 times their current value. Batch training is used, with a batch size of 50, for 120 generations. The loss function is the mean squared error loss. The calculation is shown in formula (1):

[0078] (1),

[0079] in, The number of training samples, For the first The output of the image patch defocusing evaluation network is evaluated based on a training sample. For the first A true value corresponding to each training sample.

[0080] The trained lightweight defocus evaluation network is applied to the industrial image defocus evaluation method of the present application, and the specific steps are as follows:

[0081] In step S10, an industrial image is acquired by a visual sensor, and the industrial image is blocked to obtain a set of image blocks.

[0082] In this embodiment, the industrial image is acquired by the visual sensor In the present application, the camera and lens of the visual sensor are preferably Hikvision MV-CH250-90UC and XF-PTL03719-C-VI, respectively; the industrial image is divided into image blocks at equal intervals to form a set of image blocks , wherein the size of each image block is , and In the present application, they are preferably set to 7 and 10, respectively.

[0083] In step S20, each image block in the set of image blocks is input into the trained image block defocus evaluation network to obtain the defocus evaluation result of each image block, and the average of the defocus evaluation results of all image blocks is taken as the defocus evaluation result of the industrial image.

[0084] In this embodiment, each image block in the set of image blocks is input into the trained image block defocus evaluation network to obtain the defocus evaluation result of each image block, denoted as , and the average of all image block defocus evaluation results, i.e. , is taken as the defocus evaluation result of the industrial image .

[0085] The present application can provide more accurate defocus evaluation results while being convenient to deploy and apply, achieve good technical effects, and provide technical support for industrial precision detection.

[0086] Figure 3 is a structural schematic diagram of an industrial image defocus evaluation device according to an embodiment of the present application.

[0087] As shown in Figure 3 , an industrial image defocus evaluation device 300 is provided, which comprises an image block acquisition module 310 and a defocus evaluation module 320.

[0088] The image block obtaining module 310 is configured to obtain an industrial image by using a visual sensor, and divide the industrial image into a set of image blocks. In an embodiment, the image block obtaining module 310 can be configured to perform the step S10 described above, and details are not repeated here.

[0089] The defocus evaluation module 320 is configured to input each image block in the set of image blocks into the trained image block defocus evaluation network respectively, obtain a defocus evaluation result of each image block, and take the average of the defocus evaluation results of all the image blocks as the defocus evaluation result of the industrial image. In an embodiment, the defocus evaluation module 320 can be configured to perform the step S20 described above, and details are not repeated here.

[0090] According to an embodiment of the present application, the image block obtaining module 310 and the defocus evaluation module 320 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the image block obtaining module 310 and the defocus evaluation module 320 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. hardware or firmware, or any one of software, hardware and firmware or any appropriate combination of several of them. Alternatively, at least one of the image block obtaining module 310 and the defocus evaluation module 320 can be at least partially implemented as a computer program module which can perform corresponding functions when running.

[0091] Figure 4 is a block diagram of an electronic device suitable for implementing the industrial image defocus evaluation method according to an embodiment of the present application.

[0092] As shown in Figure 4 The electronic device 400 according to an embodiment of the present application includes a processor 401 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 402 or loaded from a storage portion 408 into a random access memory (RAM) 403. The processor 401 may, for example, include a general-purpose microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 401 can also include an on-board memory for cache use. The processor 401 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present application.

[0093] In the RAM 403, various programs and data required for the operation of the electronic device 400 are stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other via the bus 404. The processor 401 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 402 and / or the RAM 403. It should be noted that the programs can also be stored in one or more memories other than the ROM 402 and the RAM 403. The processor 401 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0094] According to the embodiments of the present application, the electronic device 400 can further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 can further include one or more of the following components connected to the input / output (I / O) interface 405: an input part 406 including a keyboard, a mouse, etc.; an output part 407 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 408 including a hard disk, etc.; and a communication part 409 including a network interface card such as a LAN card, a modem, etc. The communication part 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as necessary. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 410 as necessary, so that a computer program read therefrom is installed in the storage part 408 as necessary.

[0095] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0096] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories of the ROM 402 and / or the RAM 403 described above and / or one or more memories other than the ROM 402 and the RAM 403.

[0097] Embodiments of the present application also include a computer program product comprising a computer program containing program code for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program code is used to make the computer system implement the industrial image defocus evaluation method provided by the embodiments of the present application.

[0098] The above-described functions defined in the system / device / apparatus of the embodiments of the present application are performed when the computer program is executed by the processor 401. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0099] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium and installed and downloaded through the communication part 409 and / or installed from the detachable medium 411. The program code contained in the computer program can be transmitted by any suitable network medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0100] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409 and / or installed from the detachable medium 411. When the computer program is executed by the processor 401, the above-described functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0101] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language can include, but is not limited to, Java, C++, python, "C" language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0102] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0103] Those skilled in the art will appreciate that the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly noted in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways without departing from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0104] The embodiments of the present application have been described above. However, these embodiments are merely for the purpose of illustration and are not intended to limit the scope of the present application. Although the embodiments are described separately above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present application, and these substitutions and modifications should fall within the scope of the present application.

Claims

1. A method of industrial image defocus assessment, characterized in that, The method comprises: obtaining an industrial image through a visual sensor, and dividing the industrial image into blocks to obtain a set of image blocks; inputting each image block in the set of image blocks into a trained image block defocus evaluation network respectively to obtain a defocus evaluation result of each image block, and taking the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image; wherein the image block defocus evaluation network comprises a multi-path feature extractor, a feature refining unit and a quantization unit; wherein the multi-path feature extractor comprises a plurality of hollow convolutional layers, a feature concatenation layer and a max-pooling layer; wherein the feature refining unit comprises a double-path channel attention and a convolutional layer connected in sequence, and the double-path channel attention comprises a first adaptive max-pooling layer, a first adaptive average-pooling layer, a feature concatenation and flattening layer and a first fully connected layer; wherein the quantization unit comprises a second adaptive max-pooling layer, a feature flattening layer and a second fully connected layer connected in sequence; wherein inputting each image block in the set of image blocks into the trained image block defocus evaluation network respectively to obtain a defocus evaluation result of each image block comprises: training the image block defocus evaluation network using pre-constructed training samples and combining a pre-set loss function to obtain the trained image block defocus evaluation network; extracting features of the image blocks in the set of image blocks using the trained multi-path feature extractor, concatenating the extracted multiple features, and performing a max-pooling operation on the concatenation result to obtain an initial feature map; performing a max-pooling operation and an average-pooling operation on the initial feature map using the trained feature refining unit, concatenating and flattening the results of the pooling operations, and performing a fully connected operation and a convolutional operation on the results of the concatenation and flattening operations to obtain an intermediate feature map; performing a max-pooling operation on the intermediate feature map using the trained quantization unit, and performing a fully connected operation on the result of the max-pooling operation after flattening to obtain the defocus evaluation result of the image block.

2. The method of claim 1, wherein, extracting features of the image blocks in the set of image blocks using the trained multi-path feature extractor, concatenating the extracted multiple features, and performing a max-pooling operation on the concatenation result to obtain an initial feature map comprises: extracting features of the image blocks using the plurality of hollow convolutional layers to obtain a plurality of feature maps, and concatenating the plurality of feature maps along the spatial dimension using the feature concatenation layer to obtain a first feature map; performing a max-pooling operation on the first feature map along the height dimension and the width dimension using the max-pooling layer to obtain a second feature map, and inputting the second feature map into the trained feature refining unit as the initial feature map.

3. The method of claim 2, wherein, performing a max-pooling operation and an average-pooling operation on the initial feature map using the trained feature refining unit, concatenating and flattening the results of the pooling operations, and performing a fully connected operation and a convolutional operation on the results of the concatenation and flattening operations to obtain an intermediate feature map comprises: The first adaptive max-pooling layer and the first adaptive average-pooling layer of the trained dual-path channel attention are used to perform adaptive max-pooling operation and adaptive average-pooling operation on the initial feature map, respectively, to obtain a third feature map and a fourth feature map; The feature splicing and flattening layer of the trained dual-path channel attention is used to splice the third feature map and the fourth feature map along the channel dimension to obtain a fifth feature map, and the fifth feature map is flattened to obtain a sixth feature map; The first full connection layer of the trained dual-path channel attention is used to perform multi-layer linear processing on the sixth feature map to obtain a seventh feature map, and the seventh feature map is multiplied by the initial feature map to obtain an eighth feature map; The convolution layer of the trained feature refining unit is used to perform convolution processing on the eighth feature map to obtain a ninth feature map, and the ninth feature map is input into the trained quantization unit as the intermediate feature map.

4. The method of claim 3, wherein, The trained quantization unit is used to perform max-pooling operation on the intermediate feature map, and the result of the max-pooling operation is flattened and then subjected to full connection operation to obtain the out-of-focus evaluation result of the image block, including: The second adaptive max-pooling layer of the trained quantization unit is used to perform adaptive max-pooling operation on the intermediate feature map to obtain a tenth feature map; The feature flattening layer of the trained quantization unit is used to perform flattening operation on the tenth feature map to obtain an eleventh feature map; The second full connection layer of the trained quantization unit is used to perform multi-layer linear processing on the eleventh feature map to obtain the out-of-focus evaluation result of the image block.

5. An industrial image defocus evaluation apparatus, characterized by, The device comprises: An image block acquisition module is configured to acquire an industrial image through a visual sensor and divide the industrial image into blocks to obtain a set of image blocks; An out-of-focus evaluation module is configured to input each image block in the set of image blocks into a trained image block out-of-focus evaluation network, obtain an out-of-focus evaluation result of each image block, and take the average of the out-of-focus evaluation results of all image blocks as an out-of-focus evaluation result of the industrial image; The image block out-of-focus evaluation network comprises a multi-path feature extractor, a feature refining unit, and a quantization unit; The multi-path feature extractor comprises a plurality of hollow convolution, a feature splicing layer, and a max-pooling layer; The feature refining unit comprises a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention comprises a first adaptive max-pooling layer, a first adaptive average-pooling layer, a feature splicing and flattening layer, and a first full connection layer; The quantization unit comprises a second adaptive max-pooling layer, a feature flattening layer, and a second full connection layer connected in sequence; The image block out-of-focus evaluation network is trained by using pre-constructed training samples and combining a pre-set loss function to obtain the trained image block out-of-focus evaluation network. ​ The multi-path feature extractor trained is used to extract features of the image blocks in the image block set, the extracted features are spliced, a maximum pooling operation is performed on the spliced result, and an initial feature map is obtained; The feature refining unit trained is used to perform a maximum pooling operation and an average pooling operation on the initial feature map, the results of the pooling operations are spliced and flattened, the spliced and flattened results are subjected to a full connection operation and a convolution operation, and an intermediate feature map is obtained; The quantization unit trained is used to perform a maximum pooling operation on the intermediate feature map, and a full connection operation is performed on the result of the maximum pooling operation after flattening, so as to obtain the out-of-focus evaluation result of the image block. 6.An electronic device comprising: one or more processors; memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-4.

7. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instruction is executed by the processor to implement the steps of the method according to any one of claims 1-4.

Images