Industrial image defocus evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides an industrial image defocus evaluation method which can be applied to the technical field of computer vision and the technical field of industrial precision detection. The method comprises the following steps: acquiring an industrial image through a visual sensor, and partitioning the industrial image to obtain an image block set; and inputting each image block in the image block set into the trained image block out-of-focus evaluation network to obtain an out-of-focus evaluation result of each image block, and taking a mean value of the out-of-focus evaluation results of all the image blocks as an out-of-focus evaluation result of the industrial image. The invention further provides an industrial image defocus evaluation device and equipment and a storage medium. According to the method, the defocus evaluation precision can be improved while deployment and application are convenient.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision technology and industrial precision detection technology, and more specifically to an industrial image defocus evaluation method and device, electronic equipment and storage medium. Background Art

[0002] In recent years, computer vision technology has entered people's daily work and life. In the field of industrial precision detection, the high-precision cameras used often have a fixed focal length and a shallow depth of field, which makes the range of clear imaging small, and it is easy to have defocus blur. Therefore, it is of great significance to evaluate the defocus of industrial images. Since the image defocus degree is positively correlated with the image clarity and is robust to exposure conditions, the defocus evaluation method based on image clarity has attracted much attention. Traditional defocus evaluation methods based on image clarity usually rely on image feature extraction and quantitative analysis, mainly including two categories: defocus evaluation methods based on image gradient information and defocus evaluation methods based on image frequency domain analysis. The defocus evaluation method based on image gradient information evaluates the defocus degree of the image by calculating the gradient information of the pixel points in the image. Commonly used gradient information extraction operators include Sobel operator, Scharr operator, Laplacian operator, etc. These operators perform convolution operations on the image to obtain the gradient information of the pixel points in the image, complete the characterization of the image clarity, and then obtain the defocus evaluation result. In addition, other features, such as entropy and contrast, can be combined to further improve the accuracy of image defocus assessment. The defocus assessment method based on image frequency domain analysis converts the image from the spatial domain to the frequency domain and analyzes the frequency components of the image. The clarity of an image is related to the content of high-frequency components. The more high-frequency components there are, the clearer the image is and the lower the degree of defocus of the image is. Commonly used frequency domain analysis methods include Fourier transform and wavelet transform. The above traditional defocus assessment methods based on image clarity usually rely on manually designed image features. Although they are more efficient, the accuracy of defocus assessment is poor. With the development of deep learning, defocus assessment methods based on deep networks have attracted attention. Among them, the defocus assessment method based on lightweight networks improves the convenience of deployment and application by virtue of the lightweight network structure, but its accuracy is low. To this end, how to design an accurate defocus assessment method based on lightweight networks remains to be further studied. Summary of the invention

[0003] In view of the above problems, the present invention provides an industrial image defocus assessment method and device, an electronic device and a storage medium.

[0004] According to a first aspect of the present invention, there is provided an industrial image defocus assessment method, the method comprising:

[0005] Acquire industrial images through visual sensors, and divide the industrial images into blocks to obtain a set of image blocks;

[0006] Input each image block in the image block set into the trained image block defocus evaluation network to obtain a defocus evaluation result of each image block, and take the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image;

[0007] The image patch defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit;

[0008] Among them, the multi-channel feature extractor includes multiple dilated convolutions, feature concatenation layers, and maximum pooling layers;

[0009] The feature refinement unit includes a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature concatenation and flattening layer, and a first fully connected layer;

[0010] The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer which are connected in sequence.

[0011] According to an embodiment of the present invention, the step of inputting each image block in the image block set into a trained image block defocus evaluation network to obtain a defocus evaluation result of each image block includes:

[0012] Using pre-constructed training samples and a pre-set loss function to train the image block defocus assessment network, to obtain a trained image block defocus assessment network;

[0013] The trained multi-channel feature extractor is used to extract features from the image blocks in the image block set, multiple features extracted are spliced, and a maximum pooling operation is performed on the spliced ​​results to obtain an initial feature map;

[0014] The trained feature refinement unit is used to perform maximum pooling and average pooling operations on the initial feature map, the results of the pooling operations are concatenated and flattened, and the results of the concatenation and flattening operations are fully connected and convolved to obtain an intermediate feature map;

[0015] The trained quantization unit is used to perform a maximum pooling operation on the intermediate feature map. The result of the maximum pooling operation is flattened and then fully connected to obtain the defocus evaluation result of the image block.

[0016] According to an embodiment of the present invention, the above-mentioned multi-channel feature extractor that has been trained is used to extract features from image blocks in the image block set, multiple features extracted are spliced, and a maximum pooling operation is performed on the splicing result to obtain an initial feature map including:

[0017] Using multiple dilated convolutions to extract features from the image block to obtain multiple feature maps, and using a feature concatenation layer to concatenate the multiple feature maps along the spatial dimension to obtain a first feature map;

[0018] The maximum pooling layer is used to perform a maximum pooling operation on the first feature map along the height dimension and the width dimension to obtain a second feature map, and the second feature map is input into the trained feature refinement unit as the initial feature map.

[0019] According to an embodiment of the present invention, the feature refinement unit using the training is used to perform a maximum pooling operation and an average pooling operation on the initial feature map, the results of the pooling operation are spliced ​​and flattened, and the results of the splicing and flattening operation are fully connected and convolved to obtain an intermediate feature map including:

[0020] Using the first adaptive maximum pooling layer and the first adaptive average pooling layer of the trained dual-pathway channel attention, respectively, an adaptive maximum pooling operation and an adaptive average pooling operation are performed on the initial feature map to obtain a third feature map and a fourth feature map;

[0021] The third feature map and the fourth feature map are concatenated along the channel dimension using the feature concatenation and flattening layer of the trained dual-path channel attention to obtain a fifth feature map, and the fifth feature map is flattened to obtain a sixth feature map;

[0022] Perform multi-layer linear processing on the sixth feature map using the first fully connected layer of the trained dual-path channel attention to obtain a seventh feature map, and multiply the seventh feature map by the initial feature map to obtain an eighth feature map;

[0023] The eighth feature map is convolved using the convolution layer of the trained feature refinement unit to obtain a ninth feature map, and the ninth feature map is input as an intermediate feature map into the trained quantization unit.

[0024] According to an embodiment of the present invention, the above-mentioned maximum pooling operation is performed on the intermediate feature map using the trained quantization unit, and the result of the maximum pooling operation is flattened and then fully connected to obtain the defocus evaluation result of the image block, which includes:

[0025] Perform an adaptive maximum pooling operation on the intermediate feature map using the second adaptive maximum pooling layer of the trained quantization unit to obtain a tenth feature map;

[0026] Using the feature flattening layer of the trained quantization unit to flatten the tenth feature map, to obtain an eleventh feature map;

[0027] The eleventh feature map is subjected to multi-layer linear processing using the second fully connected layer of the trained quantization unit to obtain a defocus evaluation result of the image block.

[0028] According to a second aspect of the present invention, there is provided an industrial image defocus assessment device, the device comprising:

[0029] An image block acquisition module is used to acquire industrial images through a visual sensor and divide the industrial images into blocks to obtain an image block set;

[0030] A defocus evaluation module is used to input each image block in the image block set into the trained image block defocus evaluation network, obtain the defocus evaluation result of each image block, and take the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image;

[0031] The image patch defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit;

[0032] Among them, the multi-channel feature extractor includes multiple dilated convolutions, feature concatenation layers, and maximum pooling layers;

[0033] The feature refinement unit includes a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature concatenation and flattening layer, and a first fully connected layer;

[0034] The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer which are connected in sequence.

[0035] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0036] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instructions are stored, and the computer program or instructions are executed by a processor to implement the steps of the above method.

[0037] The present invention divides industrial images into equally spaced blocks, and uses an image block defocus assessment network that integrates a multi-channel feature extractor, a feature refinement unit, and a quantization unit to perform defocus assessment on each image block, and combines the defocus assessment results of each image block to form a defocus assessment result for the entire image. The present invention can improve the accuracy of defocus assessment while being convenient to deploy and apply, and provide technical support for industrial precision detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0039] Figure 1 is an application scenario diagram of the industrial image defocus assessment method according to an embodiment of the present invention;

[0040] Figure 2 is a flow chart of an industrial image defocus assessment method according to an embodiment of the present invention;

[0041] Figure 3 is a schematic structural diagram of an industrial image defocus assessment device according to an embodiment of the present invention;

[0042] Figure 4 4 is a block diagram of an electronic device suitable for implementing an industrial image defocus evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0044] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0045] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0046] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0047] In order to solve the above-mentioned problems in the prior art, that is, to solve the problem of low accuracy of the existing defocus assessment method based on lightweight network, the present invention proposes an industrial image defocus assessment method.

[0048] Figure 1 2 is an application scenario diagram of the industrial image defocus assessment method according to an embodiment of the present invention.

[0049] like Figure 1 As shown, the application scenario 100 according to this embodiment may include the field of industrial precision detection technology. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0050] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0051] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0052] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0053] It should be noted that the industrial image defocus assessment method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the industrial image defocus assessment device provided in the embodiment of the present invention can generally be arranged in the server 105. The industrial image defocus assessment method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the industrial image defocus assessment device provided in the embodiment of the present invention can also be arranged in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0054] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0055] Figure 2 4 is a flow chart of an industrial image defocus assessment method according to an embodiment of the present invention.

[0056] like Figure 2 As shown, the above industrial image defocus assessment method includes the following steps:

[0057] In step S10, an industrial image is acquired by a visual sensor, and the industrial image is divided into blocks to obtain an image block set.

[0058] In the above step S10, the industrial image acquired by the visual sensor is divided into blocks at equal intervals to obtain an image block set of the industrial image.

[0059] In step S20, each image block in the image block set is input into the trained image block defocus evaluation network to obtain a defocus evaluation result of each image block, and the average of the defocus evaluation results of all image blocks is used as the defocus evaluation result of the industrial image.

[0060] In the above step S20, the image blocks in the image block set are processed in sequence using the trained image block defocus assessment network to obtain defocus assessment results of all image blocks in the image block set, and the average of the defocus assessment results of all image blocks is used as the defocus assessment result of the industrial image.

[0061] Among them, the image block defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit.

[0062] Among them, the multi-channel feature extractor includes multiple hole convolutions, feature splicing layers and maximum pooling layers.

[0063] Among them, the feature refinement unit includes a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature splicing and flattening layer and a first fully connected layer.

[0064] The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer which are connected in sequence.

[0065] The present invention divides industrial images into equally spaced blocks, and uses an image block defocus assessment network that integrates a multi-channel feature extractor, a feature refinement unit, and a quantization unit to perform defocus assessment on the image blocks, and combines the defocus assessment results of each image block to form a defocus assessment result of the entire industrial image. The present invention can improve the accuracy of defocus assessment while being convenient to deploy and apply, and provide technical support for industrial precision detection.

[0066] This embodiment is a preferred implementation method, which pre-constructs an image block defocus assessment network including a multi-channel feature extractor, a feature refinement unit and a quantization unit. The image block defocus assessment network is trained based on pre-constructed training samples and in combination with a pre-set loss function, and the parameters of the multi-channel feature extractor, the feature refinement unit and the quantization unit are obtained and applied to the industrial image defocus assessment method of the present invention.

[0067] According to an embodiment of the present invention, the above-mentioned image block defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit; wherein the multi-channel feature extractor includes multiple hole convolutions, feature splicing layers and maximum pooling layers; wherein the feature refinement unit includes dual-channel channel attention and convolution layers connected in sequence, and the dual-channel channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature splicing and flattening layer and a first fully connected layer; wherein the quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer connected in sequence.

[0068] The above-mentioned multi-channel feature extractor may optionally include six dilated convolutions Conv1, Conv2, Conv3, Conv4, Conv5, Conv6, a feature concatenation layer FCon and a maximum pooling layer Max_Pool_1.

[0069] The feature refinement unit may optionally include a dual-path channel attention DC_Attn and a convolution layer Conv7, which are connected in sequence. The dual-path channel attention DC_Attn may optionally include a first adaptive maximum pooling layer Adp_Max_Pool_1, a first adaptive average pooling layer Adp_Avg_Pool_1, a feature concatenation and flattening layer and a first fully connected layer FC1.

[0070] The quantization unit may optionally include a second adaptive maximum pooling layer Adp_Max_Pool_2, a feature flattening layer and a second fully connected layer FC2, and these three parts are connected in sequence.

[0071] The multi-channel feature extractor includes six dilated convolutions Conv1, Conv2, Conv3, Conv4, Conv5, and Conv6, a feature concatenation layer FCon, and a maximum pooling layer Max_Pool_1. The six dilated convolutions extract features from the input image blocks respectively and send their output features to the feature concatenation layer for concatenation. The concatenated features are sent to the maximum pooling layer to obtain multi-channel features. The six dilated convolutions all use a convolution kernel of size 3×3×3×8, and the convolution kernel parameter format is height×width×input channel×output channel; the dilated intervals are (1, 1), (2, 2), (4, 4), (6, 6), (8, 8), and (10, 10), and the convolution step size is (2, 2); the padding sizes are (1, 1), (2, 2), (4, 4), (6, 6), (8, 8), and (10, 10), and the padding method is zero padding. Six dilated convolutions extract features from the input image blocks respectively to obtain six feature maps of the same size, which are sent to the feature concatenation layer FCon. FCon uses the torch.cat function in Pytorch to concatenate along the spatial dimension to obtain the first feature map F1. The first feature map F1 is sent to the maximum pooling layer Max_Pool_1, which performs the maximum pooling operation along the dimensions of the height and width of F1. The pooling size is (2, 2), which doubles the width and height of F1 to obtain the second feature map F2. For a size of The input image block of size (height × width × channels) is processed by Conv1, Conv2, Conv3, Conv4, Conv5, Conv6 and FCon to obtain a block of size The first feature map F1 of F1 is processed by Max_Pool_1 to obtain a size of The second feature map F2.

[0072] The second feature map F2 is sent to the feature refinement unit, which includes a dual-path channel attention DC_Attn and a convolutional layer Conv7, which are connected in sequence.

[0073] The dual-path channel attention DC_Attn includes the first adaptive maximum pooling layer Adp_Max_Pool_1, the first adaptive average pooling layer Adp_Avg_Pool_1, the feature concatenation and flattening layer, and the first fully connected layer FC1. Adp_Max_Pool_1 performs an adaptive maximum pooling operation along the height and width dimensions of F2; Adp_Avg_Pool_1 performs an adaptive average pooling operation along the height and width dimensions of F2. The second feature map F2 is sent to the dual-path channel attention DC_Attn, and the third feature map F3 and the fourth feature map F4 are obtained after being processed by Adp_Max_Pool_1 and Adp_Avg_Pool_1, respectively, with a size of 48×1. Subsequently, the features F3 and F4 are sent to the feature concatenation and flattening layer for concatenation and flattening operations: first, the torch.cat function in Pytorch is used to concatenate along the channel dimension of F3 and F4 to obtain the fifth feature map F5, with a size of 96×1, and then the fifth feature map F5 is flattened by the torch.view function in Pytorch to obtain the sixth feature map F6, with a dimension of 1, with a total of 96 features. The sixth feature map F6 is sent to the first fully connected layer FC1. The first fully connected layer FC1 contains the first linear layer Linear1 and the second linear layer Linear2 connected sequentially. The first linear layer Linear1 has a size of 96×24 (input feature number × output feature number), with a learnable bias parameter, and uses the ReLu activation function; the second linear layer Linear2 has a size of 24×48 (input feature number × output feature number), with a learnable bias parameter, and uses the Sigmoid activation function. After the sixth feature map F6 is processed by FC1, the seventh feature map F7 is obtained, with a dimension of 1, and a total of 48 features. The seventh feature map F7 and the second feature map F2 are multiplied to obtain the eighth feature map F8, the size of which is . F8 is the output of the dual-channel channel attention DC_Attn, which is fed into the convolution layer Conv7. The convolution layer Conv7 uses a 1×1×48×48 convolution kernel with a step size of (1, 1). The eighth feature map F8 is processed by Conv7 to obtain the ninth feature map F9, with a size of .

[0074] The ninth feature map F9 is sent to the quantization unit, where the quantization unit includes the second adaptive maximum pooling layer Adp_Max_Pool_2, a feature flattening layer and the second fully connected layer FC2, and these three parts are connected in sequence.

[0075] The second adaptive maximum pooling layer Adp_Max_Pool_2 performs an adaptive maximum pooling operation along the height and width dimensions of F9 to obtain the tenth feature map F 10 , with a size of 48×1. 10Send it to the feature flattening layer for flattening operation, that is, use the torch.view function in Pytorch to flatten F 10 Flatten it to get the eleventh feature map F 11 , with a dimension of 1 and a total of 48 features. 11 The second fully connected layer FC2 contains the third linear layer Linear3 and the fourth linear layer Linear4 connected in sequence. The third linear layer Linear3 has a size of 48×8 (input feature number × output feature number), with a learnable bias parameter, and uses a ReLu activation function; the fourth linear layer Linear4 has a size of 8×1 (input feature number × output feature number), with a learnable bias parameter, and no activation function. The eleventh feature map F 11 After being processed by the second fully connected layer FC2, the output of the quantization unit is obtained, which is also the output of the image block defocus evaluation network.

[0076] The following describes the training samples and loss function of the image block defocus assessment network including a multi-channel feature extractor, a feature refinement unit, and a quantization unit in an embodiment of the present invention.

[0077] The image patch defocus evaluation network is trained based on the WPSurface dataset (https: / / pan.baidu.com / share / init?surl=AJORPtKkikQcXhI9gFbvsg&pwd=0rvz). For each training image in the WPSurface dataset, a random cropping method is used to obtain a The image block is used as a training sample data. For the specific cropping method, please refer to https: / / blog.csdn.net / yanzhiwen2 / article / details / 123870111. and In the present invention, they are preferably set to 50 and 50 respectively. The training process uses the Adam optimizer, the initial learning weight is 0.01, and the weight decays to 0.1 times the current weight after every 60 generations of training. The batch training method is adopted, the batch size is set to 50, and the training is 120 generations. The loss function uses the mean square error loss , calculated as shown in formula (1):

[0078] (1),

[0079] in, is the number of training samples, Based on training samples, the output of the image patch defocus estimation network, For the The true value corresponding to the training samples.

[0080] The trained lightweight defocus evaluation network is applied to the industrial image defocus evaluation method of the present invention, and the specific steps are as follows:

[0081] Step S10: acquiring an industrial image through a visual sensor, and dividing the industrial image into blocks to obtain an image block set.

[0082] In this embodiment, industrial images are acquired through visual sensors. In the present invention, the camera and lens of the visual sensor are preferably Hikvision MV-CH250-90UC and XF-PTL03719-C-VI respectively; for industrial image Equally spaced image blocks, forming an image block set , where the size of each image block is , and In the present invention, they are preferably set to 7 and 10 respectively.

[0083] Step S20, input each image block in the image block set into the trained image block defocus evaluation network, obtain the defocus evaluation result of each image block, and take the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image.

[0084] In this embodiment, the image block set Each image block in is input into the trained image block defocus evaluation network to obtain the defocus evaluation result of each image block, which is recorded as , the mean of the defocus evaluation results of all image blocks is As industrial image Defocus evaluation results.

[0085] The present invention can provide more accurate defocus evaluation results while being convenient to deploy and apply, achieve better technical effects, and provide technical support for industrial precision detection.

[0086] Figure 3 Schematic diagram of the structure of an industrial image defocus evaluation device according to an embodiment of the present invention.

[0087] like Figure 3 As shown, an industrial image defocus assessment device 300 is provided, which includes an image block acquisition module 310 and a defocus assessment module 320.

[0088] The image block acquisition module 310 is used to acquire an industrial image through a visual sensor and divide the industrial image into blocks to obtain an image block set. In one embodiment, the image block acquisition module 310 can be used to execute the step S10 described above, which will not be described in detail here.

[0089] The defocus evaluation module 320 is used to input each image block in the image block set into the trained image block defocus evaluation network, obtain the defocus evaluation result of each image block, and use the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image. In one embodiment, the defocus evaluation module 320 can be used to execute step S20 described above, which will not be repeated here.

[0090] According to an embodiment of the present invention, the image block acquisition module 310 and the defocus assessment module 320 may be combined in one module, or any one of the modules may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the image block acquisition module 310 and the defocus assessment module 320 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or encapsulating the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the image block acquisition module 310 and the defocus assessment module 320 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function may be executed.

[0091] Figure 4 4 is a block diagram of an electronic device suitable for implementing an industrial image defocus evaluation method according to an embodiment of the present invention.

[0092] like Figure 4 As shown, the electronic device 400 according to an embodiment of the present invention includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 to a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0093] In RAM 403, various programs and data required for the operation of electronic device 400 are stored. Processor 401, ROM 402 and RAM 403 are connected to each other via bus 404. Processor 401 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 402 and / or RAM 403. It should be noted that the program can also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0094] According to an embodiment of the present invention, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may further include one or more of the following components connected to the input / output (I / O) interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.

[0095] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0096] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 402 and / or RAM 403 described above and / or one or more memories other than ROM 402 and RAM 403.

[0097] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the industrial image defocus assessment method provided by the embodiment of the present invention.

[0098] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the processor 401 executes the computer program. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0099] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 409, and / or installed from the removable medium 411. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0100] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the processor 401, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0101] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0104] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for evaluating defocus of an industrial image, characterized in that: The method comprises: Acquire an industrial image through a visual sensor, and divide the industrial image into blocks to obtain an image block set; Inputting each image block in the image block set into a trained image block defocus evaluation network, respectively, to obtain a defocus evaluation result of each image block, and taking the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image; Wherein, the image block defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit; Wherein, the multi-channel feature extractor includes multiple dilated convolutions, feature concatenation layers and maximum pooling layers; The feature refinement unit includes a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature concatenation and flattening layer, and a first fully connected layer; The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer which are connected in sequence.

2. The method according to claim 1, characterized in that Inputting each image block in the image block set into a trained image block defocus evaluation network respectively, and obtaining a defocus evaluation result of each image block comprises: Using pre-constructed training samples and combining with a pre-set loss function to train the image block defocus assessment network, to obtain the trained image block defocus assessment network; Using the trained multi-channel feature extractor to extract features from the image blocks in the image block set, splicing the extracted multiple features, and performing a maximum pooling operation on the splicing result to obtain an initial feature map; Using the trained feature refinement unit to perform a maximum pooling operation and an average pooling operation on the initial feature map, concatenating and flattening the results of the pooling operation, and performing a full connection operation and a convolution operation on the results of the concatenation and flattening operation to obtain an intermediate feature map; The trained quantization unit is used to perform a maximum pooling operation on the intermediate feature map, and the result of the maximum pooling operation is flattened and then fully connected to obtain a defocus evaluation result of the image block.

3. The method according to claim 2, characterized in that The trained multi-channel feature extractor is used to extract features from the image blocks in the image block set, the extracted multiple features are spliced, and a maximum pooling operation is performed on the splicing result to obtain an initial feature map including: Using the multiple dilated convolutions to extract features from the image block to obtain multiple feature maps, and using the feature splicing layer to splice the multiple feature maps along the spatial dimension to obtain a first feature map; The maximum pooling layer is used to perform a maximum pooling operation on the first feature map along the height dimension and the width dimension to obtain a second feature map, and the second feature map is input into the trained feature refinement unit as the initial feature map.

4. The method according to claim 3, characterized in that The trained feature refinement unit is used to perform a maximum pooling operation and an average pooling operation on the initial feature map, the results of the pooling operation are concatenated and flattened, and the results of the concatenation and flattening operation are fully connected and convolved to obtain an intermediate feature map including: Using the trained first adaptive maximum pooling layer and the first adaptive average pooling layer of the dual-pathway channel attention, respectively, an adaptive maximum pooling operation and an adaptive average pooling operation are performed on the initial feature map to obtain a third feature map and a fourth feature map; Using the trained dual-path channel attention feature concatenation and flattening layer, the third feature map and the fourth feature map are concatenated along the channel dimension to obtain a fifth feature map, and the fifth feature map is flattened to obtain a sixth feature map; Using the first fully connected layer of the trained dual-pathway channel attention to perform multi-layer linear processing on the sixth feature map to obtain a seventh feature map, and multiplying the seventh feature map with the initial feature map to obtain an eighth feature map; The eighth feature map is convolved using the convolution layer of the trained feature refinement unit to obtain a ninth feature map, and the ninth feature map is input into the trained quantization unit as the intermediate feature map.

5. The method according to claim 4, characterized in that The trained quantization unit is used to perform a maximum pooling operation on the intermediate feature map, and the result of the maximum pooling operation is flattened and then fully connected to obtain a defocus evaluation result of the image block, including: Using the second adaptive maximum pooling layer of the trained quantization unit to perform an adaptive maximum pooling operation on the intermediate feature map to obtain a tenth feature map; Using the feature flattening layer of the trained quantization unit to flatten the tenth feature map, to obtain an eleventh feature map; The eleventh feature map is subjected to multi-layer linear processing using the second fully connected layer of the trained quantization unit to obtain a defocus evaluation result of the image block.

6. An industrial image defocus assessment device, characterized in that: The device comprises: An image block acquisition module is used to acquire an industrial image through a visual sensor and divide the industrial image into blocks to obtain an image block set; A defocus evaluation module, used for inputting each image block in the image block set into a trained image block defocus evaluation network, obtaining a defocus evaluation result of each image block, and taking the average of the defocus evaluation results of all image blocks as the defocus evaluation result of the industrial image; Wherein, the image block defocus assessment network includes a multi-channel feature extractor, a feature refinement unit and a quantization unit; Wherein, the multi-channel feature extractor includes multiple dilated convolutions, feature concatenation layers and maximum pooling layers; The feature refinement unit includes a dual-path channel attention and a convolution layer connected in sequence, and the dual-path channel attention includes a first adaptive maximum pooling layer, a first adaptive average pooling layer, a feature concatenation and flattening layer, and a first fully connected layer; The quantization unit includes a second adaptive maximum pooling layer, a feature flattening layer and a second fully connected layer which are connected in sequence.

7. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instructions are executed by a processor to implement the steps of the method according to any one of claims 1 to 5.

Images