Image quality control method and device based on lightweight convolutional neural network and medium

Abstract

The application discloses a kind of image quality control methods, device and medium based on light weight convolutional neural network, applied to image analysis technical field.The method first obtains the drug sensitive holographic image to be detected;Then drug sensitive holographic image is input drug sensitive holographic image quality determination model.Finally, according to the output result of drug sensitive holographic image quality determination model, the quality of drug sensitive holographic image is determined.Because the drug sensitive holographic image quality determination model proposed in the present application will fuse each feature map of different scales to obtain the final target feature map, higher recognition accuracy can be obtained under a shallower number of neural network layers, and since the number of neural network layers is small, the calculation time is reduced, so the image quality recognition time is also faster.Compared with the traditional scheme, the technical scheme simultaneously ensures the accuracy and speed of image quality recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, in particular to an image quality control method and device based on a light-weight convolutional neural network and a medium. BACKGROUND

[0002] With the development of image deep learning technology, image-based analysis, detection, interpretation and screening are increasingly significant in various fields, such as disease screening based on images in the medical field. The automatic pipeline drug sensitivity analysis platform served by the present technology needs to continuously capture images of multiple antibiotics at different concentration gradients with bacteria. In order to ensure accuracy and efficiency, quality judgment needs to be made on each image, and image capture at the next concentration gradient is only performed after passing the quality detection. Image quality control analysis plays a key role in this process, and needs to be able to quickly and accurately evaluate the image to determine whether its quality is qualified, so as to avoid capturing low-quality images and improve the efficiency and accuracy of the entire pipeline. At the same time, the requirements for image quality control analysis in applications are high, mainly in two aspects, one is the need for high accuracy, and the other is the need for high recognition speed.

[0003] Based on traditional machine vision detection technology, only single type can be applied, and it is affected by image perspective change, noise, light change and other interference, resulting in low recognition accuracy. The recognition accuracy of classic convolutional neural networks such as resnet18, vgg is high but the recognition time is long. Although the classic light-weight networks mobilenet_v2, shufflenet meet the requirement of recognition time, the recognition accuracy is low.

[0004] Therefore, how to ensure the accuracy and speed of image quality recognition at the same time is a problem to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide an image quality control method and device based on a light-weight convolutional neural network and a medium to solve the problem that current image quality recognition technology cannot meet the requirements of speed and accuracy at the same time.

[0006] To solve the above technical problems, the present application provides an image quality control method based on a light-weight convolutional neural network, comprising:

[0007] Obtaining a drug sensitivity holographic image to be detected;

[0008] input the drug sensitivity holographic image into a drug sensitivity holographic image quality judgment model; wherein the drug sensitivity holographic image quality judgment model inputs a feature map of the drug sensitivity holographic image into a plurality of backbone feature extraction modules connected in turn; each layer of the backbone feature extraction module extracts the features of the received feature map to form a new feature map and output to the next layer; and the feature map size output by each backbone feature extraction module decreases layer by layer; the drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, and determines the output result according to the target feature map;

[0009] determine the quality of the drug sensitivity holographic image according to the output result of the drug sensitivity holographic image quality judgment model.

[0010] Preferably, the drug sensitivity holographic image quality judgment model comprises a plurality of backbone feature extraction modules and a plurality of branch feature extraction modules connected with each backbone feature extraction module;

[0011] Each backbone feature extraction module is used to extract a deep semantic feature map layer by layer, and each branch feature extraction module is used to obtain a feature map of different scales output by each backbone feature extraction module;

[0012] The drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, comprising:

[0013] Fusing the semantic feature map finally extracted by each backbone feature extraction module and the feature map of different scales obtained by each branch feature extraction module to obtain a final target feature map.

[0014] Preferably, the drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, comprising:

[0015] Adding the feature values of the corresponding positions in each feature map to obtain a final target feature map.

[0016] Preferably, the drug sensitivity holographic image quality judgment model performs a pretreatment operation on the drug sensitivity holographic image through a convolution layer and a max-pooling layer to obtain an initial feature map.

[0017] Preferably, the backbone feature extraction module is a ResCSPBlock feature extraction module.

[0018] Each backbone feature extraction module outputs a feature map with a reduced size through a max-pooling operation.

[0019] Preferably, the drug sensitivity holographic image quality judgment model adopts a SiLu activation function to process the output feature map after each convolution operation.

[0020] Preferably, training the drug sensitivity holographic image quality judgment model comprises:

[0021] The drug sensitivity holographic image quality judgment model is updated step by step by using a gradient descent algorithm to minimize the cross-entropy loss function of the model.

[0022] To solve the above technical problems, the present application also provides an image quality control device based on a lightweight convolutional neural network, comprising:

[0023] An acquisition module is configured to acquire a drug sensitivity holographic image to be detected.

[0024] An input module is configured to input the drug sensitivity holographic image into a drug sensitivity holographic image quality judgment model; wherein the drug sensitivity holographic image quality judgment model inputs a feature map of the drug sensitivity holographic image into a plurality of layers of sequentially connected backbone feature extraction modules; each layer of the backbone feature extraction modules extracts features of the received feature map to form a new feature map and output to the next layer; and the feature map size output by each of the backbone feature extraction modules decreases layer by layer; the drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, and determines an output result according to the target feature map;

[0025] An interpretation module is configured to determine the quality of the drug sensitivity holographic image according to the output result of the drug sensitivity holographic image quality judgment model.

[0026] To solve the above technical problems, the present application also provides an image quality control device based on a lightweight convolutional neural network, comprising: a memory for storing a computer program;

[0027] A processor is configured to execute the computer program to implement the steps of the above-mentioned image quality control method based on a lightweight convolutional neural network.

[0028] To solve the above technical problems, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned image quality control method based on a lightweight convolutional neural network.

[0029] The image quality control method based on the lightweight convolutional neural network provided in the application first acquires a drug sensitivity holographic image to be detected; and then inputs the drug sensitivity holographic image into a drug sensitivity holographic image quality judgment model. The drug sensitivity holographic image quality judgment model inputs a feature map of the drug sensitivity holographic image into a plurality of sequentially connected backbone feature extraction modules; each layer of the backbone feature extraction module extracts features of the received feature map to form a new feature map and output to the next layer; and the feature map size output by each backbone feature extraction module decreases layer by layer; the drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, and determines an output result according to the target feature map. Finally, the quality of the drug sensitivity holographic image is determined according to the output result of the drug sensitivity holographic image quality judgment model. Because the drug sensitivity holographic image quality judgment model proposed in the application fuses each feature map of different scales to obtain a final target feature map, a higher recognition accuracy can be obtained under a shallower number of neural network layers, and because the number of neural network layers is small, the calculation time is reduced, so the image quality recognition time is also faster. Compared with the traditional scheme, the technical scheme simultaneously ensures the accuracy and speed of image quality recognition.

[0030] The application also provides an image quality control device based on a lightweight convolutional neural network and a computer readable storage medium, which correspond to the above method and have the same beneficial effects as the above method. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flowchart of the image quality control method based on the lightweight convolutional neural network provided in the embodiments of the application;

[0033] Figure 2 A structure diagram of a lightweight model provided in the embodiments of the application;

[0034] Figure 3 A structure diagram of a ResCSPBlock feature extraction module provided in the embodiments of the application;

[0035] Figure 4 A structure diagram of a ResBlock residual block provided in the embodiments of the application;

[0036] Figure 5 A structure diagram of a convolution module provided in the embodiments of the application;

[0037] Figure 6 A general flowchart for obtaining a drug sensitivity holographic image quality judgment model is provided for an embodiment of the present application.

[0038] Figure 7 A structural diagram of an image quality control device based on a lightweight convolutional neural network is provided for an embodiment of the present application.

[0039] Figure 8 A structural diagram of an image quality control device based on a lightweight convolutional neural network is provided for another embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0041] The core of the present application is to provide an image quality control method, device and medium based on a lightweight convolutional neural network, to solve the problem that current image quality recognition technology cannot simultaneously meet speed and accuracy requirements.

[0042] In order to enable those skilled in the art to better understand the present application scheme, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0043] With the development of image deep learning technology, image-based analysis, detection, interpretation and screening are increasingly significant in various fields, such as disease screening based on images in the medical field, product quality control based on images in the product production field, face recognition and behavior monitoring based on images in the security field, traffic monitoring and intelligent traffic management based on images in the transportation field, etc. Before image application, image quality control analysis is usually required. Image quality control analysis can exclude low-quality image data, such as blur, noise, underexposure or overexposure, etc., and can also detect and exclude missing or damaged image data. Thus, the quality of the image is ensured to be good when subsequent application is performed, and the reliability and accuracy of the performance of the subsequent image application system are ensured.

[0044] The automatic pipeline drug sensitivity analysis platform of the present technology needs to continuously take images of multiple antibiotics at different concentration gradients with bacteria. In order to ensure accuracy and efficiency, quality judgment needs to be performed on each image, and image taking at the next concentration gradient is only performed after passing the quality detection. The image quality is judged by image quality control analysis. The requirements for image quality control analysis are high, mainly in two aspects. First, high accuracy is required to ensure accurate judgment of image quality to avoid errors caused by low-quality images. Second, high recognition speed is required to analyze and judge the image within a short time to ensure efficient operation of the pipeline. Therefore, a highly accurate and high-speed image quality control analysis system is needed to ensure that the automatic pipeline drug sensitivity analysis platform can operate at high accuracy and recognition speed.

[0045] The embodiment of the present application provides an image quality control method based on a lightweight convolutional neural network, which can accurately realize quality prediction of drug sensitivity holographic images and determine whether the drug sensitivity images are qualified according to the quality prediction result. The embodiment of the present application also designs a lightweight model framework. Lightweight means reducing model parameters and computational complexity while maintaining high performance, thereby improving the inference speed of the model. The present scheme reduces the model parameter amount, reduces the memory occupation amount, and improves the recognition speed of the model by introducing a multi-stage feature map fusion strategy, a convolution input / output channel number same strategy, and the like, while ensuring the performance of the model. Figure 1 A flowchart of an image quality control method based on a lightweight convolutional neural network provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in the figure, the method comprises the following steps:

[0046] S10: Obtain a drug sensitivity holographic image to be detected.

[0047] S11: Input the drug sensitivity holographic image into a drug sensitivity holographic image quality determination model.

[0048] S12: Determine the quality of the drug sensitivity holographic image according to the output result of the drug sensitivity holographic image quality determination model.

[0049] This technology proposes an AI-based image quality control method capable of quality control of drug sensitivity holographic images. By designing a lightweight convolutional neural network framework, it enables rapid and accurate quality assessment of drug sensitivity holographic images, thereby providing qualified drug sensitivity holographic images for subsequent drug sensitivity interpretation systems and improving the accuracy of the interpretation system. The core of this application lies in establishing and training a drug sensitivity holographic image quality judgment model. By inputting the drug sensitivity holographic image to be detected into the model and obtaining the output result, the quality of the drug sensitivity holographic image is determined. Specifically, the drug sensitivity holographic image quality judgment model typically first uses convolutional layers and max-pooling layers to preprocess the drug sensitivity holographic image to obtain an initial feature map. Then, the obtained feature map is input into a multi-layered, sequentially connected backbone feature extraction module. Each backbone feature extraction module extracts features from the received feature maps to form a new feature map and outputs it to the next layer. The size of the feature maps output by each backbone feature extraction module decreases layer by layer. The drug sensitivity holographic image quality judgment model fuses the feature maps of different scales to obtain the final target feature map and determines the output result based on the target feature map.

[0050] The following example illustrates one lightweight model framework of this application. Figure 2 This is a schematic diagram of the structure of a lightweight model provided in an embodiment of this application; as shown below. Figure 2 As shown, the model mainly consists of three parts: a backbone feature extraction module, a branch feature extraction module, and a classifier. Here, H is the image height, W is the width, k is the convolution kernel, s is the stride, and Cout is the dimension of the output feature map.

[0051] The backbone feature extraction module consists of convolutional layers and cross-stage local connections. First, a standard 3x3 convolution (Conv) with a stride of 2 and a max-pooling operation preprocess the image, learning shallow local features such as texture and edges. Then, through the stacking of four ResCSPBlock feature extraction modules (i.e., the backbone feature extraction module), higher-level semantic features of the image, such as structure and category, are further learned. Starting from the second ResCSPBlock feature extraction module, each subsequent module is followed by a max-pooling operation. Pooling layers can expand the receptive field of the network model, reduce the size of the feature maps, reduce computational cost, and reduce memory consumption, among other benefits.

[0052] Figure 2 The three branch feature extraction modules on the right side of the middle section obtain data feature information from different network depths and different scales. They are composed of three 1*1 convolutional modules with strides of 2, 4 and 8 respectively, and the number of output channels is 1024.

[0053] Multi-stage feature fusion: the feature maps extracted by the backbone feature extraction module and the feature maps obtained by the branch feature extraction module are directly added element by element (Add) to obtain a new feature map (i.e., a target feature map), thereby realizing the fusion and mutual complementation of features, and at the same time, different levels of feature information can be utilized to enhance the diversity of features and better capture different features in the image, thereby improving the expression ability and robustness of the model. And the parameter amount does not increase, and the calculation amount is small.

[0054] The classifier is composed of a global pooling layer (GlobalAvgPool) and a fully connected layer (Linear). The global pooling layer replaces the fully connected layer to reduce the parameter amount and mainly solves the problem of full connection to generate a feature vector with a length of 1024. Then, a fully connected layer is used to map the features to a class space for image classification, which is 2 classes in this technology. Finally, the probability value of each class is calculated in the softmax, and the sum of the probabilities is 1.

[0055] Figure 3 A structural diagram of a ResCSPBlock feature extraction module provided by an embodiment of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, the ResCSPBlock feature extraction module is a basic module of the lightweight convolutional neural network, which adopts the idea of the CSP network (i.e., the cross-stage local network). First, the basic feature map BaseLayer is divided into two branches. One branch first adopts a 1*1 convolution, and then a N ResBlock residual block or other feature extraction module (such as a Bottleneck), where N can be one or more. In this technology, a standard residual block is adopted (such as Figure 4 ). The other branch directly adopts a 1*1 convolution to extract low-level detailed features. A cross-stage connection is introduced between them, and the feature maps of the two branches are fused by the Concat feature splicing method, which can retain more feature information, provide richer feature representation ability and anti-interference ability, and optimize the accuracy and robustness of the algorithm. At the same time, this structure also realizes efficient reuse of features, reduces repeated calculation, and reduces the parameter amount and FLOPS value of the model, improving the calculation efficiency of the network. Wherein, Cin is the dimension of the input feature map.

[0056] Figure 4 A structural diagram of a ResBlock residual block provided by an embodiment of the present application is shown in FIG. 2. Figure 4As shown, the ResBlock residual block is composed of three convolutional layers, i.e. two pointwise convolutional intermediate plus a 3*3 convolutional layer. The first layer of pointwise convolution is to reduce the channel number of the input, so the number of convolutional kernels of this layer is less. The middle 3*3 convolution is a feature extraction layer, and the number of convolutional kernels thereof is consistent with the number of the previous layer. Finally, the output channel number is increased by increasing the number of pointwise convolutional kernels. Therefore, the process of the standard residual block can be summarized as "dimension reduction"-"convolution"-"dimension increase". This process reduces the dimension of the input, reduces some input features when extracting features by convolution, and thus improves the operation efficiency.

[0057] The modules requiring feature extraction in the present application are implemented using convolution modules. Figure 5 A structural schematic diagram of a convolution module provided for an embodiment of the present application is shown in Figure 2. Figure 5 As shown, the convolution module (such as 1*1 or 3*3 convolution) used in the present technology is composed of convolution, batch processing (BN) and SiLu activation. That is, the SiLu activation function is used to process the output feature map after each convolution operation of the drug sensitivity holographic image quality judgment model. Of course, different activation functions (such as Relu, etc.) can also be used according to the actual training effect in actual application.

[0058] The SiLu (Sigmoid Linear Unit) activation function is also called Swish activation function, which is a smooth nonlinear function, and the formula is as follows:

[0059]

[0060] Where x is the input and f(x) is the output. SiLu has the characteristics of smoothness and nonlinearity, and the derivative exists in the entire real number domain. Due to its smoothness, it is more stable during gradient descent, which helps to avoid the problems of gradient disappearance and gradient explosion. It can limit the activation value to (0, 1) when the input is large or small, and is suitable for scenarios that require smoothness and output range in (0, 1), such as image classification, target detection, etc.

[0061] Figure 6 A general flowchart for obtaining a drug sensitivity holographic image quality judgment model is provided for an embodiment of the present application, as shown in Figure 3. Figure 6As shown, including steps S20: data collection. S21: model training. S22: model testing. The model training part includes constructing the model dataset, including the training set and the test set; loading the lightweight model and setting the initial training parameters, then inputting the training set data after data preprocessing into the lightweight neural network model for forward propagation to obtain the recognition result of the model; comparing the recognition result with the true label of the data to calculate the error, and updating the lightweight model parameters using the back propagation algorithm; repeat the above steps until the prediction error of the model meets the requirements or the iteration is completed. The model testing part mainly loads the lightweight model and the optimized model parameters; obtains the drug sensitivity holographic image to be quality controlled, which is subjected to the same data preprocessing as the training model; then input into the optimized model for quality identification, and the probability belonging to different qualities is obtained through the softmax activation function, and the sum of the probabilities of each type is 1. In this technology, there are only two quality types (qualified and unqualified).

[0062] The data collection process mainly collects the drug sensitivity holographic images required by the present technology, including clear, blurred, and incomplete field of view drug sensitivity holographic images under different concentration gradients. Then the images are manually labeled, wherein the clear and non-incomplete field of view drug sensitivity holographic images belong to qualified, and the blurred and incomplete field of view images belong to unqualified. The data preprocessing process first normalizes the data, and then uses data enhancement methods to expand the diversity of the data during the training process. The data enhancement method specifically includes image mirroring, slight image blurring, image contrast, and saturation processing.

[0063] The model testing process is model measurement. In actual application, the gradient descent algorithm can be used to gradually update the drug sensitivity holographic image quality judgment model to minimize the cross-entropy loss function (Cross-Entropy Loss) of the model. The cross-entropy loss function is suitable for classification problems, and the loss L measures the difference between the predicted result and the true label, as follows:

[0064] L = -(y * log p + (1 - y) * log (1 - p)) (2)

[0065] Wherein, y is the true label (0 or 1), and p is the predicted probability of the model.

[0066] Label smoothing is a commonly used technique. Label smoothing can reduce the influence of noise or abnormal samples in the training data on the model, thereby improving the stability of the model. Label smoothing can be regarded as a regularization method, which limits the complexity of the model by introducing smooth prior knowledge. This regularization effect can reduce the variance of the model, thereby reducing the risk of overfitting. The formula of label smoothing is (3).

[0067] y = (1 - a) y onehot + a / k (3)

[0068] where y is the smoothed label vector, y onehot is the traditional one-hot label vector, a is the weight of the smoothing term, which is a small hyperparameter (0.1 can be taken, and it can also be adjusted according to the experimental performance), and k refers to the number of categories (2 can be taken).

[0069] In the model training process, the loss function is gradually minimized using optimization algorithms such as gradient descent to improve the accuracy of the model. In this technology, the Adam optimizer can be used, which can adaptively adjust the learning rate and has good convergence performance.

[0070] The present technology realizes the quality control of drug sensitivity holographic images by designing a lightweight convolutional neural network, which improves the recognition accuracy compared with traditional machine vision detection technology, and improves the image recognition speed on the basis of ensuring accuracy compared with the classic convolutional neural network. The specific advantages are as follows: 1. Fast speed: under the same hardware conditions, the speed of this method for reasoning an image is the fastest, which is conducive to engineering deployment. 2. High accuracy: the accuracy of this method on the validation set is higher than that of resnet18 and mobilenet_v2, and slightly lower than that of resnet50.

[0071] It can be seen that the present embodiment proposes a lightweight convolutional neural network to realize the quality control of drug sensitivity holographic images, but it should be noted that the backbone feature extraction module of the lightweight convolutional neural network mentioned in the foregoing is not limited to ResCSPBlock, and other forms of feature extraction modules, such as ResXCSPBlock, DenseBlock, Darknetbottleneck, ShuffleBlock, etc. The present technology uses a label smoothing optimization strategy to improve the performance of the model, and other strategies that can improve the performance of the model, such as knowledge distillation, can also be used. The image to be quality controlled can be a drug sensitivity holographic image, other forms of natural images, and microscope imaging, etc.

[0072] The image quality control method based on the lightweight convolutional neural network provided by the embodiment of the application comprises the following steps: first, a drug sensitivity holographic image to be detected is acquired; and then the drug sensitivity holographic image is input into a drug sensitivity holographic image quality judgment model. In the drug sensitivity holographic image quality judgment model, the feature map of the drug sensitivity holographic image is input into a plurality of sequentially connected backbone feature extraction modules; each backbone feature extraction module extracts the features of the received feature map to form a new feature map and output the new feature map to the next layer; the size of the feature map output by each backbone feature extraction module is reduced layer by layer; the drug sensitivity holographic image quality judgment model fuses the feature maps of different scales to obtain a final target feature map, and determines an output result according to the target feature map. Finally, the quality of the drug sensitivity holographic image is determined according to the output result of the drug sensitivity holographic image quality judgment model. Because the drug sensitivity holographic image quality judgment model proposed in the application fuses the feature maps of different scales to obtain the final target feature map, a higher recognition accuracy can be obtained under a shallower number of neural network layers, and because the number of neural network layers is small, the calculation time is reduced, so the image quality recognition time is also faster. Compared with the traditional scheme, the technical scheme simultaneously guarantees the accuracy and speed of image quality recognition.

[0073] The drug sensitivity holographic image quality judgment model in the embodiment of the application comprises a plurality of backbone feature extraction modules and a plurality of branch feature extraction modules connected with the backbone feature extraction modules, and specifically as shown in Figure 2 Each backbone feature extraction module is used to extract a deep semantic feature map layer by layer, and each branch feature extraction module is used to acquire a feature map of different scales output by each backbone feature extraction module; the drug sensitivity holographic image quality judgment model specifically fuses the semantic feature map finally extracted by each backbone feature extraction module and the feature maps of different scales acquired by each branch feature extraction module to obtain a final target feature map, and the specific fusion manner can be to add the feature values at the corresponding positions in each feature map.

[0074] In the above embodiment, it is mentioned that the drug sensitivity holographic image quality judgment model performs a preprocessing operation on the drug sensitivity holographic image through a convolution layer and a max-pooling layer to acquire an initial feature map, then inputs the acquired feature map into a first layer backbone feature extraction module, each sequentially connected backbone feature extraction module extracts a deeper semantic feature layer by layer, and the backbone feature extraction module can specifically adopt a ResCSPBlock feature extraction module and can output a feature map with a reduced size through a max-pooling operation.

[0075] The scheme has the following advantages: 1: fast detection speed, on an Intel(R) Core(TM) i5-7400 CPU@3.00GHz processor, the method can infer an image at a speed of 7.41ms. While resnet18 needs 19.78ms to infer the same image, resnet50 needs 44.89ms, and mobilenet_v2 needs 7.93ms. 2: high accuracy, the accuracy of the method on the validation set during training reaches 99.56%. While the accuracy of resnet18 on the validation set reaches 99.55%, the accuracy of resnet50 on the validation set reaches 99.67%, and the accuracy of mobilenet_v2 on the validation set reaches 95.71%. 3: The scheme is based on deep learning to realize the quality control of drug sensitivity holographic images, and there is an automatic photographing platform to photograph each microwell. The whole process does not require human intervention, which is convenient and time-saving. 4: The lightweight model structure proposed in the present technology can extract sufficient features, has small parameter quantity, and can ensure the inference speed while ensuring the accuracy.

[0076] In the above embodiment, the image quality control method based on the lightweight convolutional neural network is described in detail, and the present application also provides corresponding embodiments of the image quality control device based on the lightweight convolutional neural network. It should be noted that the embodiments of the device part are described from two angles, one is based on the functional module angle, and the other is based on the hardware angle.

[0077] Based on the functional module angle, the present embodiment provides an image quality control device based on a lightweight convolutional neural network, Figure 7 The structure diagram of the image quality control device based on the lightweight convolutional neural network provided by the present embodiment is shown in Figure 7 As shown in the figure, the device comprises:

[0078] The acquisition module 10 is used to acquire the drug sensitivity holographic image to be detected;

[0079] The input module 11 is used to input the drug sensitivity holographic image into the drug sensitivity holographic image quality judgment model; wherein the drug sensitivity holographic image quality judgment model inputs the feature map of the drug sensitivity holographic image into the backbone feature extraction module connected in turn in multiple layers; each layer of the backbone feature extraction module extracts the features of the received feature map to form a new feature map and outputs to the next layer; and the feature map size output by each backbone feature extraction module decreases layer by layer; the drug sensitivity holographic image quality judgment model fuses each feature map of different scales to obtain a final target feature map, and determines an output result according to the target feature map;

[0080] The interpretation module 12 is used to determine the quality of the drug sensitivity holographic image according to the output result of the drug sensitivity holographic image quality judgment model.

[0081] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and will not be described here.

[0082] The image quality control device based on the lightweight convolutional neural network provided in the embodiment corresponds to the method described above, and therefore has the same beneficial effects as the method described above.

[0083] From the perspective of hardware, the embodiment provides another image quality control device based on a lightweight convolutional neural network, Figure 8 The structure diagram of the image quality control device based on the lightweight convolutional neural network provided in another embodiment of the present application is shown in Figure 8 The image quality control device based on the lightweight convolutional neural network includes a memory 20 for storing a computer program.

[0084] The processor 21 is used to execute the computer program to realize the steps of the image quality control method based on the lightweight convolutional neural network mentioned in the above embodiments.

[0085] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a graphics processor (GPU) for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 can also include an artificial intelligence (AI) processor for processing machine learning-related computing operations.

[0086] The memory 20 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 20 can also include high-speed random access memory and nonvolatile, computer-readable storage media such as one or more magnetic disk storage devices, flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, wherein the computer program is loaded and executed by the processor 21 and can implement the steps of the image quality control method based on the lightweight convolutional neural network disclosed in any of the preceding embodiments. In addition, the resources stored in the memory 20 can also include an operating system 202 and data 203, and the storage mode can be temporary storage or permanent storage. The operating system 202 can include Windows, Unix, Linux, etc. The data 203 can include but is not limited to data related to the image quality control method based on the lightweight convolutional neural network.

[0087] In some embodiments, the image quality control device based on the lightweight convolutional neural network can further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0088] Those skilled in the art can understand that the structure shown in the figure does not constitute a limitation on the image quality control device based on the lightweight convolutional neural network, and can include more or fewer components than those shown in the figure.

[0089] The image quality control device based on the lightweight convolutional neural network provided by the embodiments of the present application includes a memory and a processor. When the processor executes the program stored in the memory, the following method can be implemented: the image quality control method based on the lightweight convolutional neural network.

[0090] The image quality control device based on the lightweight convolutional neural network provided by the embodiments of the present application corresponds to the above method, and therefore has the same beneficial effects as the above method.

[0091] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps as described in the above method embodiment.

[0092] It can be understood that if the method in the above embodiment is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0093] The computer readable storage medium provided by the embodiment corresponds to the above method, and therefore has the same beneficial effects as the above method.

[0094] The above describes in detail the method, device and medium for image quality control based on a lightweight convolutional neural network. The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0095] It should be further noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or equipment including the above element.

Claims

1. An image quality control method based on a lightweight convolutional neural network, characterized in that, include: Acquire the holographic image of the drug susceptibility to be detected; The drug sensitivity holographic image is input into a drug sensitivity holographic image quality assessment model. The model inputs the feature map of the drug sensitivity holographic image into a first-layer backbone feature extraction module in a multi-layered, sequentially connected backbone feature extraction module. Each layer's backbone feature extraction module extracts features from the received feature map to form a new feature map and outputs it to the next layer. The size of the feature map output by each backbone feature extraction module decreases progressively with each layer. The drug sensitivity holographic image quality assessment model fuses feature maps of different scales to obtain the final target feature map and determines the output result based on the target feature map. The quality of the drug sensitivity holographic image is determined based on the output of the drug sensitivity holographic image quality assessment model. The drug sensitivity holographic image quality assessment model includes multiple main feature extraction modules and multiple branch feature extraction modules connected to each of the main feature extraction modules; Each of the main feature extraction modules is used to extract semantic feature maps to a deeper level layer by layer, and each of the branch feature extraction modules is used to obtain feature maps of different scales output by each of the main feature extraction modules. The drug sensitivity holographic image quality assessment model fuses feature maps of different scales to obtain the final target feature map, which includes: The semantic feature maps finally extracted by each of the main feature extraction modules and the feature maps of different scales obtained by each of the branch feature extraction modules are fused to obtain the final target feature map; The multiple branch feature extraction modules are 1*1 convolutional modules with different strides.

2. The image quality control method based on a lightweight convolutional neural network according to claim 1, characterized in that, The drug sensitivity holographic image quality assessment model fuses feature maps of different scales to obtain the final target feature map, which includes: The final target feature map is obtained by adding the feature values ​​at corresponding positions in each feature map.

3. The image quality control method based on a lightweight convolutional neural network according to claim 2, characterized in that, The drug sensitivity holographic image quality assessment model preprocesses the drug sensitivity holographic image through convolutional layers and max pooling layers to obtain an initial feature map.

4. The image quality control method based on a lightweight convolutional neural network according to claim 3, characterized in that, The backbone feature extraction module is the ResCSPBlock feature extraction module; Each of the main feature extraction modules outputs a feature map with a reduced size through max pooling.

5. The image quality control method based on a lightweight convolutional neural network according to claim 4, characterized in that, The feature map output by the drug sensitivity holographic image quality assessment model is processed by the SiLu activation function after each convolution operation.

6. The image quality control method based on a lightweight convolutional neural network according to claim 5, characterized in that, Training the drug sensitivity holographic image quality assessment model includes: The gradient descent algorithm is used to progressively update the drug sensitivity holographic image quality assessment model in order to minimize the model's cross-entropy loss function.

7. An image quality control device based on a lightweight convolutional neural network, characterized in that, include: The acquisition module is used to acquire the drug sensitivity holographic image to be detected; An input module is used to input the drug sensitivity holographic image into a drug sensitivity holographic image quality assessment model. The drug sensitivity holographic image quality assessment model inputs the feature map of the drug sensitivity holographic image into a first-layer backbone feature extraction module in a multi-layered, sequentially connected backbone feature extraction module. Each layer of the backbone feature extraction module extracts features from the received feature map to form a new feature map and outputs it to the next layer. Furthermore, the size of the feature map output by each backbone feature extraction module decreases progressively with each layer. The drug sensitivity holographic image quality assessment model fuses the feature maps of different scales to obtain the final target feature map and determines the output result based on the target feature map. The interpretation module is used to determine the quality of the drug sensitivity holographic image based on the output of the drug sensitivity holographic image quality judgment model; The drug sensitivity holographic image quality assessment model includes multiple main feature extraction modules and multiple branch feature extraction modules connected to each of the main feature extraction modules; Each of the main feature extraction modules is used to extract semantic feature maps to a deeper level layer by layer, and each of the branch feature extraction modules is used to obtain feature maps of different scales output by each of the main feature extraction modules. The drug sensitivity holographic image quality assessment model fuses feature maps of different scales to obtain the final target feature map, which includes: The semantic feature maps finally extracted by each of the main feature extraction modules and the feature maps of different scales obtained by each of the branch feature extraction modules are fused to obtain the final target feature map; The multiple branch feature extraction modules are 1*1 convolutional modules with different strides.

8. An image quality control device based on a lightweight convolutional neural network, characterized in that, Includes memory used to store computer programs; A processor, configured to implement the steps of the image quality control method based on a lightweight convolutional neural network as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image quality control method based on a lightweight convolutional neural network as described in any one of claims 1 to 6.

Images