Method, apparatus, and system for detecting mist deposition

Abstract

The disclosure provides a kind of fog drop deposition detection method, device, system, comprising: obtaining the target fog drop deposition image to be detected, based on the fog drop classification model of pre-training, the adhesion fog drop and non-adhesion fog drop in target fog drop deposition image are classified and handled, obtain target adhesion fog drop image and target non-adhesion fog drop image, fog drop classification model is based on sample fog drop deposition image to first base network model training obtains, first base network model includes first main network, training obtains fog drop classification model includes first freeze stage and first unfreezing stage, in first freeze stage, the network parameter of first main network does not change, in first unfreezing stage, the network parameter of first main network changes, target adhesion fog drop image and target non-adhesion fog drop image are analyzed, obtain analysis result, analysis result includes the fog drop coverage of target fog drop deposition image and / or fog drop deposition parameter, to improve the accuracy of detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of deep learning and image processing, and particularly relates to a fog droplet deposition detection method, device and system. BACKGROUND

[0002] In modern agriculture, spraying operation of target objects (such as pesticides) is an important link of agricultural production, and the distribution of fog droplet deposition generated by spraying operation is a key to evaluate the effect of spraying operation.

[0003] In the related art, fog droplet deposition detection can be realized by manual method or by computer image processing algorithm.

[0004] However, the reliability of fog droplet deposition detection realized by manual method is relatively low, and it is urgent to improve the reliability of fog droplet deposition detection.

[0005] The content in the background section is only the information known by the inventor, and does not mean that the above information has entered the public domain before the filing date of the present disclosure, nor does it mean that it can be prior art of the present disclosure. SUMMARY

[0006] The present disclosure provides a fog droplet deposition detection method, device and system to improve the reliability of fog droplet deposition detection.

[0007] In a first aspect, the present disclosure provides a fog droplet deposition detection method, comprising:

[0008] obtaining a target fog droplet deposition image to be detected;

[0009] performing classification processing on the adherent fog droplets and the non-adherent fog droplets in the target fog droplet deposition image based on a pre-trained fog droplet classification model, to obtain a target adherent fog droplet image and a target non-adherent fog droplet image, wherein the fog droplet classification model is obtained by training a first base network model based on a sample fog droplet deposition image, the first base network model comprises a first backbone network, and the training of the fog droplet classification model comprises a first freezing stage and a first unfreezing stage, in the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change;

[0010] performing analysis on the target adherent fog droplet image and the target non-adherent fog droplet image to obtain an analysis result, wherein the analysis result comprises a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image.

[0011] In some embodiments, the classifying process of the adhered droplets and the non-adhesive droplets in the target droplet deposition image based on the pre-trained droplet classification model to obtain the target adhered droplet image and the target non-adhesive droplet image includes:

[0012] performing feature extraction processing on the target droplet deposition image based on the first backbone network in the droplet classification model to obtain target image features of the target droplet deposition image;

[0013] Based on the droplet classification model and the target image features, the target droplet deposition image is classified to obtain the target adhesion droplet image and the target non-adhesion droplet image.

[0014] In some embodiments, the analyzing the target adhering droplet image and the target non-adhering droplet image to obtain an analysis result includes:

[0015] Performing non-adhesion segmentation processing on the target adhesion droplet image to obtain a target segmented image;

[0016] Merging the target segmented image and the target non-adhesive droplet image to obtain a merged image;

[0017] Parameter calculation processing is performed on the merged image to obtain the analysis result.

[0018] In some embodiments, performing non-adhesion segmentation processing on the target adhesion droplet image to obtain a target segmented image includes:

[0019] Identifying a target concave point in the target adhesion droplet image based on a pre-trained concave point detection model, wherein the concave point detection model is obtained by training a second basic network model based on the sample adhesion droplet image, the second basic network model includes a second backbone network, and the training of the concave point detection model includes a second freezing stage and a second thawing stage, in which the network parameters of the second backbone network do not change, and in the second thawing stage, the network parameters of the second backbone network change;

[0020] The target adhesion droplet image is segmented based on a connection operation on the target concave points to obtain the target segmented image.

[0021] In some embodiments, performing parameter calculation processing on the merged image to obtain the analysis result includes:

[0022] Performing edge detection on the merged image to obtain a set of fog droplet outline pixel points of the merged image;

[0023] Based on the set of droplet outline pixels, the total number of droplets in the merged image and the pixel area corresponding to each droplet are calculated;

[0024] The analysis result is calculated based on the total number of droplets and the area of ​​each pixel, wherein the analysis result includes the droplet coverage and the droplet deposition parameters, and the droplet deposition parameters include droplet deposition density.

[0025] In some embodiments, obtaining a target droplet deposition image to be detected includes:

[0026] Acquire initial droplet deposition images;

[0027] The initial droplet deposition image is corrected based on a local histogram equalization algorithm to obtain the target droplet deposition image.

[0028] In some embodiments, the correcting the initial droplet deposition image based on a local histogram equalization algorithm to obtain the target droplet deposition image includes:

[0029] Converting the initial droplet deposition image from RGB color space to HSV color space;

[0030] Segmenting the initial brightness channel of the HSV color space into a plurality of uniform regions;

[0031] For each of the multiple regions, calculate and obtain a cumulative histogram and a total number of pixels corresponding to the region;

[0032] Performing brightness equalization processing on the initial brightness channel based on the cumulative histogram and the total number of pixels corresponding to each region to obtain a target brightness channel;

[0033] The target droplet deposition image is generated based on an initial hue channel, an initial saturation channel, and the target brightness channel, and the target droplet deposition image is converted into an RGB color space, wherein the HSV color space includes the initial hue channel and the initial saturation channel.

[0034] In a second aspect, the present disclosure provides a droplet deposition detection device, comprising:

[0035] An acquisition unit, configured to acquire a target droplet deposition image to be detected;

[0036] The classification unit is configured to perform classification processing on the target fog droplet deposition image based on a pre-trained fog droplet classification model to obtain a target agglomerated fog droplet image and a target non-agglomerated fog droplet image, wherein the fog droplet classification model is obtained by training a first base network model based on sample fog droplet deposition images, the first base network model comprises a first backbone network, and the training of the fog droplet classification model comprises a first freezing stage and a first unfreezing stage, in the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change.

[0037] The analysis unit is configured to analyze the target agglomerated fog droplet image and the target non-agglomerated fog droplet image to obtain an analysis result, wherein the analysis result comprises a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image.

[0038] In some embodiments, the classification unit comprises:

[0039] The extraction subunit is configured to perform feature extraction processing on the target fog droplet deposition image based on the first backbone network in the fog droplet classification model to obtain target image features of the target fog droplet deposition image.

[0040] The classification subunit is configured to perform classification processing on the target fog droplet deposition image based on the fog droplet classification model and the target image features to obtain the target agglomerated fog droplet image and the target non-agglomerated fog droplet image.

[0041] In some embodiments, the analysis unit comprises:

[0042] The segmentation subunit is configured to perform non-agglomeration segmentation processing on the target agglomerated fog droplet image to obtain a target segmented image.

[0043] The merging subunit is configured to perform merging processing on the target segmented image and the target non-agglomerated fog droplet image to obtain a merged image.

[0044] The calculation subunit is configured to perform parameter calculation processing on the merged image to obtain the analysis result.

[0045] In some embodiments, the segmentation subunit is configured to identify target pits in the target adhesion fog droplet image based on a pre-trained pit detection model, wherein the pit detection model is obtained by training a second basic network model based on sample adhesion fog droplet images, the second basic network model comprises a second backbone network, and the training of the pit detection model comprises a second freezing stage and a second unfreezing stage, in the second freezing stage, the network parameters of the second backbone network do not change, and in the second unfreezing stage, the network parameters of the second backbone network change; and segment the target adhesion fog droplet image based on a connection operation on the target pits to obtain the target segmented image.

[0046] In some embodiments, the calculation subunit is configured to perform edge detection on the merged image to obtain a set of fog droplet contour pixel points of the merged image; based on the set of fog droplet contour pixel points, calculate the total number of fog droplets in the merged image and the respective pixel areas corresponding to each fog droplet; and based on the total number of fog droplets and the respective pixel areas, calculate the analysis result, wherein the analysis result comprises the fog droplet coverage rate and the fog droplet deposition parameter, and the fog droplet deposition parameter comprises the fog droplet deposition density.

[0047] In some embodiments, the obtaining unit comprises:

[0048] The acquisition subunit is configured to acquire an initial fog droplet deposition image.

[0049] The correction subunit is configured to perform correction processing on the initial fog droplet deposition image based on a local histogram equalization algorithm to obtain the target fog droplet deposition image.

[0050] In some embodiments, the correction subunit is configured to convert the initial fog droplet deposition image from an RGB color space to an HSV color space; divide the initial luminance channel of the HSV color space into a plurality of uniform regions; for each region in the plurality of regions, calculate a cumulative histogram and a total number of pixels corresponding to the region; perform luminance equalization processing on the initial luminance channel based on the cumulative histogram and the total number of pixels corresponding to each region to obtain a target luminance channel; generate the target fog droplet deposition image based on an initial hue channel, an initial saturation channel, and the target luminance channel, and convert the target fog droplet deposition image to the RGB color space, wherein the HSV color space comprises the initial hue channel and the initial saturation channel.

[0051] In a third aspect, the present disclosure provides a fog droplet deposition detection system, the system comprising: a fog droplet collection device, an image acquisition device, a lower computer, a cloud server, and a user equipment, wherein,

[0052] The droplet collecting device is used to form a target droplet deposition image based on the droplets;

[0053] The user device is configured to obtain and send an image acquisition request to the cloud server;

[0054] The cloud server is used to forward the image acquisition request to the slave computer;

[0055] The lower computer is configured to control the image acquisition device to acquire the target droplet deposition image based on the image acquisition request;

[0056] The image acquisition device is used to acquire the target droplet deposition image and send the target droplet deposition image to the slave computer;

[0057] The lower computer is further configured to send the target droplet deposition image, acquisition time, and acquisition location to the cloud server, wherein the acquisition time is the time when the lower computer receives the image acquisition request, and the acquisition location is the location of the lower computer;

[0058] The cloud server is further configured to determine an analysis result of the target droplet deposition image based on the method, and send the analysis result, the acquisition time, and the acquisition location to the user device;

[0059] The user equipment is further configured to display the analysis result, the collection time, and the collection location.

[0060] In a fourth aspect, the present disclosure provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0061] The memory stores computer-executable instructions;

[0062] The processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of the first aspects above.

[0063] In a fifth aspect, the present disclosure provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method described in the first aspect above.

[0064] In a sixth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0065] The disclosure provides a fog droplet deposition detection method, device and system, including: obtaining a target fog droplet deposition image to be detected, classifying adhesion fog droplets and non-adhesion fog droplets in the target fog droplet deposition image based on a pre-trained fog droplet classification model to obtain a target adhesion fog droplet image and a target non-adhesion fog droplet image, wherein the fog droplet classification model is obtained by training a first basic network model based on a sample fog droplet deposition image, the first basic network model includes a first backbone network, and the training of the fog droplet classification model includes a first freezing stage and a first unfreezing stage, in the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change, analyzing the target adhesion fog droplet image and the target non-adhesion fog droplet image to obtain an analysis result, wherein the analysis result includes a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image, in the embodiment, the detection device uses the fog droplet classification model to determine the target adhesion fog droplet image and the target non-adhesion fog droplet image, compared with the way of classifying and processing by using artificial or computer image processing algorithm in the related art, the image containing only non-adhesion fog droplets (i.e. the target non-adhesion fog droplet image) and the image containing only adhesion fog droplets (i.e. the target adhesion fog droplet image) can be separated more accurately and efficiently, that is, the efficiency and reliability of classifying the target fog droplet deposition image are improved, so that when analyzing the target non-adhesion fog droplet image and the target adhesion fog droplet image with high reliability, the accuracy of the analysis can be improved, and the analysis result obtained by the analysis has high reliability. BRIEF DESCRIPTION OF DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure, together with the description.

[0067] Figure 1 A schematic diagram of a fog droplet deposition detection method according to an embodiment of the present disclosure;

[0068] Figure 2 A schematic diagram of the principle of training a fog droplet classification model according to the present disclosure;

[0069] Figure 3 A schematic diagram of a fog droplet deposition detection method according to another embodiment of the present disclosure;

[0070] Figure 4 A schematic diagram of the principle of training a concave point detection model according to the present disclosure;

[0071] Figure 5 A schematic diagram of a fog droplet deposition detection system according to an embodiment of the present disclosure;

[0072] Figure 6 A schematic diagram of the structure of a lower computer according to an embodiment of the present disclosure;

[0073] Figure 7 A schematic diagram of a droplet deposition detection device according to an embodiment of the present disclosure;

[0074] Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present disclosure.

[0075] The specific embodiments of the present disclosure have been shown by way of example in the drawings and more details will be described in the following. These drawings and the written description are not intended to restrict the scope of the present disclosure in any way and are merely meant to illustrate the concepts of the present disclosure to a person skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0076] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following description is not meant to limit the present disclosure in any way, but merely to illustrate the concepts of the present disclosure to a person skilled in the art.

[0077] It should be understood that the terms "comprises" and "comprising", when used in the present disclosure, are intended to cover both the case where one or more components are included in the product or device, and the case where one or more components are not included in the product or device. For example, a product or device that "comprises" one or more components would also typically be viewed as "consisting essentially of" and "consisting of" the one or more components.

[0078] The term "and / or", when used in the present disclosure, means that the associated objects can exist in three cases, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally means that the associated objects are in an "or" relationship.

[0079] The term "a plurality of" in the present disclosure means two or more, and other quantifiers are similar to it.

[0080] The terms "first", "second", "third", and the like in the present disclosure are used to distinguish similar or like objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, for example, those other than the order given in the illustration or description of the present disclosure can be implemented.

[0081] The term "unit / module" used in the present disclosure refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code capable of performing a function associated with the element.

[0082] For the reader's understanding of the present disclosure, at least part of the terms involved in the present disclosure are explained as follows:

[0083] Artificial Intelligence (AI) technology refers to the technology of researching and developing theories, methods, techniques, and application systems for simulating, extending, and expanding human intelligence.

[0084] Deep Learning (DL) is a subfield of Machine Learning (ML) that learns the internal rules and representation levels of sample data, and the information obtained in the learning process is very helpful for the interpretation of data such as text, images, and sound.

[0085] Artificial Neural Network (ANN), also known as Neural Network (NN), is a mathematical model or computational model that simulates the structure and function of a biological neural network.

[0086] Transfer learning is a machine learning method that uses a model developed for task A as an initial point to reuse in the process of developing a model for task B.

[0087] Image processing, also known as image processing, is a technique of analyzing images with a computer to achieve the desired results.

[0088] Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise and widely used in the noise reduction process of image processing. In simple terms, Gaussian filtering is a process of weighted average of the entire image, and the value of each pixel point is obtained by weighted average of itself and other pixel values in the neighborhood.

[0089] Specifically, the operation of Gaussian filtering is to scan each pixel in the image with a template (or convolution, mask), and replace the value of the center pixel point with the weighted average gray value of the pixels in the neighborhood determined by the template.

[0090] Binaryzation is an image processing technique, also known as black and white, which converts grayscale images into binary images using only two colors such as white or black. That is, binaryzation can convert grayscale images into binary images.

[0091] Charge Coupled Device (CCD), which is a kind of semiconductor imaging device, has the advantages of high sensitivity, strong light resistance, small distortion, small size, long service life, and anti-vibration.

[0092] 5th Generation Mobile Communication Technology (5G) is a new generation of broadband mobile communication technology with high speed, low latency and large connection characteristics. 5G communication facilities are network infrastructure for realizing man-machine and Internet of Things.

[0093] 6th Generation Mobile Communication Technology (6G) is also known as the sixth generation of mobile communication technology, which can promote the development of industrial Internet and Internet of Things.

[0094] Universal Serial Bus (USB) is a serial bus standard and a technical specification for input and output interfaces. It is widely used in personal computers and mobile devices, and is expanding to photography equipment, digital television (set-top box), game consoles and other related fields.

[0095] Water-sensitive paper, also known as moisture-sensitive paper and water-sensing paper, is a special paper with special impedance characteristics. It contains a large number of nanometer structure fibers and metal powder that have been oriented and processed, and has high moisture sensitivity. It can be used for detection and protection of water systems and automatic control of other electronic devices.

[0096] Based on the above characteristics of water-sensitive paper, water-sensitive paper can be used for detection of fog droplet deposition in agricultural production scenarios.

[0097] It can be understood that the spraying operation of the target object (such as pesticide) is an important part of agricultural production, and the distribution of fog droplet deposition generated by the spraying operation is the key to evaluating the effect of the spraying operation. Efficient fog droplet deposition detection is of great significance to precision spraying, plant protection equipment and other research. Promoting efficient and precise spraying, promoting pesticide reduction and efficiency, and relying on the improvement of precision spraying technology, higher requirements are put forward for fog droplet deposition detection technology.

[0098] In related technologies, fog droplet deposition detection is mainly achieved by using water-sensitive paper to collect fog droplets, and then manually measuring or using computer image processing algorithms to calculate fog droplet deposition parameters. In the agricultural production scenario, water-sensitive paper is a yellow test paper that changes color when it comes into contact with water, and has the characteristics of high sensitivity and simple operation.

[0099] In comparison, the manual measurement has a complicated operation process and low measurement accuracy. Although the computer image processing algorithm can effectively improve the detection speed and accuracy compared with the manual measurement, the processing capability for the adhered fog droplets is limited, thereby resulting in a low accuracy of the calculation of the fog droplet deposition parameters.

[0100] Therefore, in order to avoid at least one of the above technical problems, the present disclosure provides a technical concept with creative labor: the detection system for realizing the fog droplet deposition detection can obtain the fog droplet classification model based on the deep learning in advance, or can obtain or call the fog droplet classification model from other systems, so as to perform the classification processing on the fog droplet deposition image to be detected based on the fog droplet classification model, to obtain the adhered fog droplet image and the non-adhered fog droplet image, and to obtain the analysis result including the fog droplet coverage and / or the fog droplet deposition parameter through the analysis of the adhered fog droplet image and the non-adhered fog droplet image.

[0101] It should be understood that the content of the above related technical part is only the information known by the inventor personally, and does not mean that the above information has entered the public domain before the filing date of the present disclosure, nor does it mean that it can be prior art of the present disclosure.

[0102] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in combination with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, and not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative labor are within the scope of protection of the present disclosure.

[0103] Based on the above technical concept, the present disclosure provides a fog droplet deposition detection method.

[0104] Please refer to Figure 1 , Figure 1 The figure is a schematic diagram of the fog droplet deposition detection method according to an embodiment of the present disclosure. As shown in the figure, Figure 1 The method comprises the following steps:

[0105] S101: obtaining a target fog droplet deposition image to be detected.

[0106] Exemplarily, the execution subject of the present embodiment can be a fog droplet deposition detection device (hereinafter referred to as a detection device), the detection device can be a server (such as a local server or a cloud server, such as an independent server or a server cluster), can also be a terminal device, can also be a processor, can also be a chip, etc., and the present embodiment is not limited.

[0107] The present embodiment does not limit the way in which the detection device obtains the target fog droplet deposition image, which can be implemented by the following examples:

[0108] In one example, the detection device can be connected with the image acquisition device and receive the target fog droplet deposition image sent by the image acquisition device.

[0109] In another example, the detection device can provide a tool for loading the target fog droplet deposition image, and a user can transmit the target fog droplet deposition image to the detection device through the tool for loading the target fog droplet deposition image.

[0110] The tool for loading the target fog droplet deposition image can be an interface for connecting with an external device, such as an interface for connecting with another storage device, through which the target fog droplet deposition image transmitted by the external device is obtained. The tool for loading the target fog droplet deposition image can also be a display device, such as an interface for loading the target fog droplet deposition image function input by the detection device on the display device, through which the user can import the target fog droplet deposition image into the detection device, and the detection device obtains the imported target fog droplet deposition image.

[0111] S102: Classify the adherent fog droplets and the non-adherent fog droplets in the target fog droplet deposition image based on the pre-trained fog droplet classification model to obtain a target adherent fog droplet image and a target non-adherent fog droplet image, wherein the fog droplet classification model is obtained by training a first base network model based on a sample fog droplet deposition image, the first base network model includes a first backbone network, and the training of the fog droplet classification model includes a first freezing stage and a first unfreezing stage. In the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change.

[0112] The execution subject of the training of the fog droplet classification model is not limited in the embodiment, which can be the detection device or other devices.

[0113] For example, the execution subject of the training of the fog droplet classification model is the detection device, the detection device can pre-train and store the fog droplet classification model, so as to call the stored fog droplet classification model to detect the target fog droplet deposition image when the target fog droplet deposition image needs to be detected.

[0114] For another example, if the training entity for the droplet classification model is another device (such as a training device), a communication link is established between the training device and the detection device. After training the droplet classification model, the training device can transmit the droplet classification model to the detection device based on the communication link, so that the detection device can perform droplet deposition detection on the target droplet deposition image based on the droplet classification model. Alternatively, after training the droplet classification model, the training device stores the droplet classification model in local memory, and a calling interface is established between the training device and the detection device. When the detection device needs to perform a droplet deposition detection operation, the droplet classification model stored in the training device is called through the calling interface to perform droplet deposition detection on the target droplet deposition image based on the called droplet classification model.

[0115] This embodiment uses the example of a detection device as the execution entity of the trained droplet classification model. This embodiment does not limit the type of the first basic network model or the number of sample droplet deposition images, which can be determined by the detection device based on requirements, historical records, and experiments.

[0116] For example, the droplet classification model is obtained by training a first basic network through transfer learning. The first basic network can be a U-Net network, a deep learning network for semantic segmentation under a fully convolutional network. The first backbone network can be a deep convolutional neural network architecture (Visual Geometry Group, VGG), such as the VGG16 network.

[0117] To facilitate readers' understanding of this disclosure, Figure 2 The principle of training the U-Net network to obtain the droplet classification model of the detection device is explained in detail.

[0118] The U-Net network consists of two parts: Encoder-Decoder. Encoder is used for downsampling and Decoder is used for upsampling. Figure 2 Each arrow in corresponds to an operation, such as Figure 2 The convolution operation, maximum pooling operation, upsampling operation, and skip connection operation shown in the figure, and the long bars and rectangular boxes at both ends of the arrows represent a feature layer.

[0119] like Figure 2As shown, the input layer inputs a sample droplet deposition image, and the sample droplet deposition image can be a 256*256 (pixel) image. The U-Net network performs twice convolution operation on the sample droplet deposition image, and then performs a skip connection operation (for the sake of distinction, we call this skip connection operation a first skip connection operation) on one branch and a max pooling operation (for the sake of distinction, we call this max pooling operation a first max pooling operation) on another branch. Then, on the branch of the first max pooling operation, twice convolution operation is performed, and then a skip connection operation (for the sake of distinction, we call this skip connection operation a second skip connection operation) is performed on one branch and a max pooling operation (for the sake of distinction, we call this max pooling operation a second max pooling operation) is performed on another branch. Then, on the branch of the second max pooling operation, thrice convolution operation is performed, and then a skip connection operation (for the sake of distinction, we call this skip connection operation a third skip connection operation) is performed on one branch and a max pooling operation (for the sake of distinction, we call this max pooling operation a third max pooling operation) is performed on another branch. Then, on the branch of the third max pooling operation, thrice convolution operation is performed, and then a skip connection operation (for the sake of distinction, we call this skip connection operation a fourth skip connection operation) is performed on one branch and a max pooling operation (for the sake of distinction, we call this max pooling operation a fourth max pooling operation) is performed on another branch. Then, on the branch of the fourth max pooling operation, thrice convolution operation is performed, and then up-sampling operation is performed. The result of the fourth skip connection operation is fused, and after twice convolution operation, up-sampling operation is performed to fuse the result of the third skip connection operation, and after twice convolution operation, up-sampling operation is performed to fuse the result of the second skip connection operation, and after twice convolution operation, up-sampling operation is performed to fuse the result of the first skip connection operation. After thrice convolution operation, the output layer outputs a prediction result, which is a classification result of the sample droplet deposition image, such as a sample coalesced droplet image and a sample non-coalesced droplet image in the sample droplet deposition image.

[0120] After the detection device predicts the prediction result, a loss function between the prediction result and a pre-labeled classification true value can be constructed, and the U-Net network can be iteratively trained based on the loss function to obtain a droplet classification model.

[0121] That is to say, the input of the first basic network model is the sample droplet deposition image, and the output is the classification results of the sample adhesion droplet image and the sample non-adhesion droplet image of the sample droplet deposition image, so as to train the first basic network model to distinguish between adhesion droplets and non-adhesion droplets, and iteratively train the first basic network model based on the classification results, so that the first basic network model can classify adhesion droplets and non-adhesion droplets in the target droplet deposition image, thereby obtaining the target adhesion droplet image and the target non-adhesion droplet image.

[0122] It should be understood that Figure 2 The structure shown and the Figure 2 The above description is only used to exemplify possible structures and parameters of the droplet classification model and should not be understood as a limitation on the droplet classification model.

[0123] In some embodiments, the training of the droplet classification model can be implemented by the detection device based on the weights obtained by pre-training, and the data set used for pre-training can be an enhanced data set, such as the Pattern Analysis Statistical Modeling and Computational Learning Visual Obje Classes (PASCAL VOC) data set, specifically PASCAL VOC 2012.

[0124] The detection device trains the droplet classification model in two stages, namely the first freezing stage and the first thawing stage. In the first freezing stage, the VGG16 network (i.e., the first backbone network) is frozen, and the parameters of the VGG16 network do not change. In the first thawing stage, the VGG16 network is thawed, and the parameters of the VGG16 network change.

[0125] The training cycles may be different in different stages of training. For example, in the first freezing stage, the training cycles may be 50 times, and in the first thawing stage, the training cycles may be 100 times.

[0126] In this embodiment, the detection device combines the droplet classification model to identify the adhered droplets and non-adhesive droplets in the target droplet deposition image to separate the images containing non-adhesive droplets (i.e., the target non-adhesive droplet image) and the images containing adhered droplets (i.e., the target adherent droplet image) to improve the efficiency and reliability of the classification of the target droplet deposition image. The detection device can also obtain the droplet classification model based on transfer learning training to improve the generalization ability of the droplet classification model when the data set (i.e., the sample droplet deposition image) is limited, thereby improving the accuracy of subsequent analysis results.

[0127] S103: analyzing the target adherent fog droplet image and the target non-adherent fog droplet image to obtain an analysis result, wherein the analysis result includes a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image.

[0128] The embodiment does not limit the method used by the detection device to analyze the target adherent fog droplet image and the target non-adherent fog droplet image. For example, the analysis result can be obtained by calculating the fog droplet coverage rate and / or the fog droplet deposition parameter. Alternatively, the target adherent fog droplet image and the target non-adherent fog droplet image can be further processed by image processing to determine the analysis result based on the result of the image processing.

[0129] Based on the above analysis, the present disclosure provides a fog droplet deposition detection method, which includes: obtaining a target fog droplet deposition image to be detected, performing classification processing on adherent fog droplets and non-adherent fog droplets in the target fog droplet deposition image based on a pre-trained fog droplet classification model to obtain a target adherent fog droplet image and a target non-adherent fog droplet image, wherein the fog droplet classification model is obtained by training a first base network model based on a sample fog droplet deposition image, the first base network model includes a first backbone network, and the training of the fog droplet classification model includes a first freezing stage and a first unfreezing stage. In the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change. The target adherent fog droplet image and the target non-adherent fog droplet image are analyzed to obtain an analysis result, wherein the analysis result includes a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image. In the embodiment, the detection device uses the fog droplet classification model to determine the target adherent fog droplet image and the target non-adherent fog droplet image. Compared with the related art, which uses manual or computer image processing algorithm for classification processing, the target non-adherent fog droplet image containing non-adherent fog droplets and the target adherent fog droplet image containing adherent fog droplets can be more accurately and efficiently separated, that is, the efficiency and reliability of the classification of the target fog droplet deposition image are improved, so that the accuracy of the analysis is improved when the target non-adherent fog droplet image and the target adherent fog droplet image with high reliability are analyzed, and the analysis result obtained by the analysis has high reliability.

[0130] To help the reader have a deeper understanding of the fog droplet deposition detection method of the present disclosure, the following will describe the fog droplet deposition detection method of the present disclosure in conjunction with the accompanying drawings. Figure 3 The fog droplet deposition detection method of the present disclosure will be described in more detail. In the following description, Figure 3 The fog droplet deposition detection method of another embodiment of the present disclosure is shown in FIG. 4, which includes the following steps. Figure 3 The fog droplet deposition detection method of another embodiment of the present disclosure is shown in FIG. 4, which includes the following steps.

[0131] S301: obtaining an initial fog droplet deposition image.

[0132] It should be understood that, in order to avoid tedious description, the same technical features as the above embodiments are not limited in this embodiment. For example, the execution subject of this embodiment and the method for the detection device to obtain the initial fog droplet deposition image can be referred to the above examples, which will not be described here.

[0133] S302: correcting the initial fog droplet deposition image based on a local histogram equalization algorithm to obtain a target fog droplet deposition image.

[0134] In combination with the above analysis, this embodiment can be understood as that the target fog droplet deposition image is an image obtained by the detection device pre-processing the initial fog droplet deposition image, and the pre-processing is specifically implemented based on the local histogram equalization algorithm. In this embodiment, by using the local histogram equalization algorithm to pre-process the initial fog droplet deposition image by the detection device, the problem of uneven brightness of the collected image caused by environmental light and shielding in actual operation can be solved, that is, the image brightness of the target fog droplet deposition image is more balanced relative to the initial fog droplet deposition image, so that when the detection device performs subsequent operations such as classification processing and analysis on the target fog droplet deposition image, the accuracy of the subsequent operations of the detection device can be improved.

[0135] In some embodiments, S302 can include the following steps:

[0136] First step: converting the initial fog droplet deposition image from RGB color space to HSV color space.

[0137] Among them, the RGB color space is based on three basic colors of red (Red, R), green (Green, G), and blue (Blue, B), and different degrees of superposition to produce rich and extensive colors, so it is commonly known as three primary color mode. HSV (Hue, Saturation, Value) color space is a color space created from the intuitive properties of color, also known as hexagonal cone model (Hexcone Model). HSV color space refers to a visible light subset in a three-dimensional color space of hue H, saturation S, and brightness V, which contains all colors in a certain color domain.

[0138] Second step: dividing the initial brightness channel based on the HSV color space into a plurality of uniform regions.

[0139] For example, the detection device can extract the initial brightness channel from the HSV color space and divide the initial brightness channel to obtain a plurality of regions. In this embodiment, the granularity of the division is not limited, for example, the detection device can divide the initial brightness channel into a plurality of regions of 5x5 (pixels).

[0140] Third step: for each region in the plurality of regions, calculate the cumulative histogram and the total number of pixels corresponding to the region.

[0141] Exemplarily, for each of the multiple regions, the detection device calculates the histogram within the region, obtains all histograms within the region, and determines a cumulative histogram within the region based on all histograms within the region. The cumulative histogram represents the cumulative probability distribution of the image components at grayscale levels, with each probability value representing a probability less than or equal to that grayscale value. Accordingly, the cumulative histogram within each region represents the cumulative probability distribution of the image components at grayscale levels within the region. Similarly, for each of the multiple regions, the detection device can determine the total number of pixels within the region.

[0142] Step 4: Perform brightness equalization on the initial brightness channel based on the cumulative histogram and total number of pixels corresponding to each region to obtain the target brightness channel.

[0143] Exemplarily, for each of the multiple areas, the detection device calculates a grayscale mapping function based on the cumulative histogram and the total number of pixels in the area, and applies the mapping function to each pixel to obtain a balanced brightness channel (i.e., a target brightness channel).

[0144] Step 5: Generate a target droplet deposition image based on the initial hue channel, the initial saturation channel, and the target brightness channel, and convert the target droplet deposition image into an RGB color space, wherein the HSV color space includes an initial hue channel and an initial saturation channel.

[0145] Exemplarily, the detection device merges the initial hue channel, the initial saturation channel, and the target brightness channel to obtain a target droplet deposition image, and converts the target droplet deposition image in the HSV color space back to the target droplet deposition image in the RGB color space.

[0146] In this embodiment, the detection device determines the target droplet deposition image through operations such as color space conversion, region segmentation, cumulative histogram determination, brightness equalization processing, and merging processing, so that the brightness reliability of the target droplet deposition image is higher than that of the initial target droplet deposition image, thereby avoiding the problem of uneven brightness of the initial target droplet deposition image.

[0147] In some embodiments, after obtaining the target droplet deposition image, the detection device may further process the target droplet deposition image, such as performing Gaussian filtering and binarization on the target droplet deposition image in sequence, thereby obtaining the target droplet deposition image for performing subsequent operations.

[0148] The Gaussian filtering method can include that the detection device can call an image convolution function (GaussianBlur function) of an Open Computer Vision Library (OpenCV) to perform Gaussian filtering on the target fog droplet deposition image with a Gaussian kernel of 9x9.

[0149] The binarization method can include that the detection device can complete Otsu binarization of the target fog droplet deposition image by using a binarization processing function (threshold function).

[0150] S303: performing feature extraction processing on the target fog droplet deposition image based on a first backbone network in the pre-trained fog droplet classification model to obtain target image features of the target fog droplet deposition image.

[0151] For example, according to the above analysis, the fog droplet classification model includes the first backbone network, the detection device can input the target fog droplet deposition image into the fog droplet classification model and run the fog droplet classification model, the first backbone network in the fog droplet classification model performs feature extraction processing on the target fog droplet deposition image to obtain target image features. The target image features are used to represent the features of the foreground of the target fog droplet deposition image. The features of the foreground can be understood as the respective features of each fog droplet, such as the respective size and dimension features of each fog droplet, which can be the perimeter and circularity and the like.

[0152] S304: performing classification processing on the target fog droplet deposition image based on the fog droplet classification model and the target image features to obtain a target coalesced fog droplet image and a target non-coalesced fog droplet image.

[0153] For example, the fog droplet classification model can further perform classification processing on the target fog droplet deposition image based on the target image features to obtain the target coalesced fog droplet image and the target non-coalesced fog droplet image.

[0154] In this embodiment, the detection device runs the first backbone network to extract the target image features, so as to further perform classification processing based on the target image features, thereby improving the effectiveness and reliability of the classification processing.

[0155] S305: performing non-coalesced segmentation processing on the target coalesced fog droplet image to obtain a target segmented image.

[0156] The detection device can perform non-coalesced segmentation processing in various ways, such as by using image analysis or by using a network model.

[0157] For example, when the detection device performs non-coalesced segmentation processing by using a network model, S306 can include the following steps:

[0158] The first step: identifying target concave points in the target adhesion droplet image based on a pre-trained concave point detection model, wherein the concave point detection model is obtained by training a second basic network model based on the sample adhesion droplet image, the second basic network model includes a second backbone network, and the trained concave point detection model includes a second freezing stage and a second thawing stage. In the second freezing stage, the network parameters of the second backbone network do not change, and in the second thawing stage, the network parameters of the second backbone network change.

[0159] Similarly, in this embodiment, the concave point detection model can be pre-trained by the detection device, or trained by other devices, which is not limited in this embodiment. Figure 4 The method for training the detection device to obtain the concave point detection model is described in detail. In this embodiment, the type of the second basic network model and the number of sample adhesion droplet images are not limited, and can be determined by the detection device based on needs, historical records, and experiments.

[0160] For example, the concave point detection model is obtained by training the second basic network model by the detection device through transfer learning. The second basic network can be a target detection network (CenterNet), and the second backbone network can be a residual network (ResNet), such as ResNet50. The concave point detection model can be pre-trained by the detection device using the VOC07+12 training set.

[0161] Among them, the detection device trains the concave point detection model through transfer learning, which can improve the generalization ability of the concave point detection model when the data set (such as sample adhesion droplet image) is limited, thereby improving the accuracy of the analysis results determined on this basis.

[0162] like Figure 4 As shown in the figure, the second basic network model includes: input layer, residual network model, transposed convolution layer, regularization layer, convolution layer, linear layer, activation function layer (Sigmoid), bias output, and heat map output.

[0163] Among them, the input of the input layer is the sample adhesion droplet image; the transposed convolution layer can be a 4*4 convolution; the regularization layer can include a linear rectification function (ReLU); the convolution layer between the two regularization layers can include a 3*3*64 convolution, and the other two convolution layers can include a 1*1*2 convolution.

[0164] It is worth noting that in the related art, the CenterNet network includes three branches of output, and the outputs of the three branches are: detection box width and height output branch, heat map output branch, bias (or offset) output branch, while in this embodiment, when the second basic network model is the CenterNet network, the second basic network model includes two branches of output, and the outputs of the two branches are: heat map output branch and offset output branch, that is, the detection box width and height output branch is removed.

[0165] That is to say, in this embodiment, the detection device obtains a concave point detection model by combining the heat map output branch and the offset output branch for training. While avoiding the complex processing caused by the detection frame width and height output branches, it can achieve the effectiveness and reliability of combining the heat map output branch and the offset output branch for training, thereby making the concave point detection model relatively simple while meeting the prediction requirements.

[0166] After the detection device predicts the heat map output branch and the offset output branch, a loss function can be constructed between the heat map output branch and the offset output branch and the pre-labeled true value, and the CenterNet network can be iteratively trained based on the loss function to obtain a concave point detection model.

[0167] That is to say, the input of the second basic network model is the sample adhesion droplet image, and the output is the sample concave points, so as to train the second basic network model's ability to distinguish concave points, so that the second basic network model can identify the concave points in the target droplet deposition image (i.e., the target concave points).

[0168] Similarly, Figure 4 The structure shown and the Figure 4 The above description is only used to exemplify the possible structure and parameters of the concave point detection model and should not be understood as limiting the concave point detection model. The detection device trains the concave point detection model in two stages: a second freezing stage and a second thawing stage. In the second freezing stage, ResNet50 (i.e., the second backbone network) is frozen and the parameters of ResNet50 do not change. In the second thawing stage, ResNet50 is thawed and the parameters of ResNet50 are changed.

[0169] The training cycles may be different in different stages of training. For example, in the first freezing stage, the training cycles may be 50 times, and in the first thawing stage, the training cycles may be 200 times.

[0170] The second step: segment the target adhesion droplet image based on the connection operation of the target concave points to obtain the target segmented image.

[0171] Exemplarily, the detection apparatus connects the target concave points to complete image segmentation on the target image including the adhered fog droplets, to obtain a segmented image without the adhered fog droplets (i.e., a target segmented image).

[0172] In this embodiment, the detection apparatus performs concave point detection through the concave point detection model, which can improve the accuracy and speed of the concave point detection. Since the concave point detection has high accuracy, the target segmented image obtained based on the target concave points also has relatively high accuracy, so that the analysis result obtained on this basis has high reliability. Moreover, since the concave point detection is fast, the efficiency of the fog droplet deposition detection based on this can be improved.

[0173] S306: performing merging processing on the target segmented image and the target non-adhered fog droplet image to obtain a merged image.

[0174] In this embodiment, the merging processing of the detection apparatus can be understood as an "or operation". For example, the detection apparatus superimposes the content of the target segmented image and the content of the target non-adhered fog droplet image in one image, and the obtained image is the merged image. That is, the merged image includes the content of the target segmented image and the content of the target non-adhered fog droplet image, so that the merged image is highly consistent with the content in the real scene, thereby improving the authenticity and reliability of the subsequent determination of the analysis result.

[0175] S307: performing parameter calculation processing on the merged image to obtain an analysis result, wherein the analysis result includes a fog droplet coverage rate and / or a fog droplet deposition parameter of a target fog droplet deposition image.

[0176] In this embodiment, the detection apparatus performs segmentation processing on the image including the adhered fog droplets (i.e., a target adhered fog droplet image) to obtain a non-adhered image (a target segmented image), and performs merging on the non-adhered image (including the target segmented image and a target non-adhered fog droplet image) to analyze the merged image to obtain an analysis result. This can avoid the low accuracy caused by directly analyzing the image including the adhered fog droplets, thereby improving the accuracy and reliability of the analysis result.

[0177] In some embodiments, S307 can include the following steps:

[0178] First step: performing edge detection on the merged image to obtain a set of fog droplet contour pixel points of the merged image.

[0179] The set of fog droplet contour pixel points includes contour pixel points corresponding to each fog droplet in the merged image.

[0180] The detection device is not limited in terms of edge detection, such as edge detection based on a gray histogram, edge detection based on a gradient, edge detection based on a second derivative, edge detection based on fuzzy reasoning, edge detection based on a neural network, and edge detection based on a genetic algorithm.

[0181] In a second step, the total number of fog droplets in the merged image and the pixel area corresponding to each fog droplet are calculated based on the set of fog droplet contour pixel points.

[0182] For example, when the detection device determines the set of fog droplet contour pixel points, which includes the contour pixel points corresponding to each fog droplet in the merged image, the detection device can further calculate the total number of fog droplets based on each fog droplet in the merged image, and calculate the pixel area corresponding to each fog droplet based on the contour pixel points corresponding to each fog droplet.

[0183] In a third step, the analysis result is calculated based on the total number of fog droplets and the pixel area, and the analysis result includes the fog droplet coverage rate of the target fog droplet deposition image and the fog droplet deposition parameter, and the fog droplet deposition parameter includes the fog droplet deposition density.

[0184] For example, after obtaining the pixel area, the detection device can calculate the sum of the pixel area, i.e., the total pixel area of all fog droplets, and calculate the fog droplet coverage rate based on the total pixel area of all fog droplets and the total pixel area of the merged image. For example, the detection device can calculate the fog droplet coverage rate C based on formula 1:

[0185]

[0186] where As is the total pixel area of all fog droplets, and Ap is the total pixel area of the merged image.

[0187] In some embodiments, the detection device can calculate the fog droplet deposition density D based on formula 2:

[0188]

[0189] where N is the total number of fog droplets, and S is the real area of the merged image in the real scene.

[0190] In this embodiment, the detection device obtains a droplet contour pixel point set including pixel points of the contour of each droplet through edge detection, calculates the total number of droplets and two-dimensional parameters of the pixel area corresponding to each droplet based on the droplet contour pixel point set, and obtains the analysis result by combining the parameters of the two dimensions. Since the merged image has high accuracy, the droplet contour pixel point set determined by the detection device has high reliability, so that the two-dimensional parameters determined based on the droplet contour pixel point set can be more accurate, thereby improving the reliability and effectiveness of the analysis results.

[0191] In some embodiments, the detection device may also output analysis results so that the user can view the analysis results.

[0192] Exemplarily, the detection device includes a display component that displays the analysis results. Accordingly, a user can observe the analysis results through the display component and take adaptive measures based on the analysis results. Alternatively, the detection device can be connected to other devices, such as a user device, to transmit the analysis results to the user device, so that the analysis results can be displayed on the user device.

[0193] In some embodiments, the detection device can also obtain the acquisition time and acquisition position corresponding to the target droplet deposition image, and output the acquisition time and acquisition position when outputting the analysis results, so that the user can obtain relatively more complete information.

[0194] Based on the above technical concept, the present disclosure also provides a droplet deposition detection system.

[0195] See also Figure 5 , Figure 5 is a schematic diagram of a droplet deposition detection system 500 according to an embodiment of the present disclosure, as shown in FIG. Figure 5 As shown, the system 500 includes: a droplet collection device 501, an image acquisition device 502, a lower computer 503, a cloud server 504, and a user device 505, wherein:

[0196] The droplet collecting device 501 is used to form a target droplet deposition image based on the droplets.

[0197] The droplet collection device 501 can be understood as a device for collecting droplets in a scene where a target object is sprayed, and forming a target droplet deposition image based on the droplets. For example, the droplet collection device 501 can be water-sensitive paper.

[0198] The user device 505 is used to obtain and send an image acquisition request to the cloud server 504 .

[0199] The user equipment 505 can be a device providing voice and / or data connectivity to a user, a handheld device having wireless connection capability, or other processing devices connected to a wireless modem, etc. For example, the user equipment can be a mobile phone and a palmtop computer.

[0200] Taking the mobile phone as an example, the mobile phone can be provided with an applet, and the user can send an image collection request to the cloud server 504 through the applet. The image collection request can be understood as a request for collecting a target fog droplet deposition image, or can also be understood as a request for triggering fog droplet deposition detection.

[0201] The cloud server 504 is configured to forward the image collection request to the lower machine 503.

[0202] The lower machine 503 is configured to control the image collection device 502 to collect a target fog droplet deposition image based on the image collection request.

[0203] The image collection device 502 is configured to collect a target fog droplet deposition image and send the target fog droplet deposition image to the lower machine 503.

[0204] The image collection device 502 can be a device having an image collection function, that is, the image collection device 502 can collect a target fog droplet deposition image formed by the fog droplet collection device 501. For example, the image collection device 502 can be a CCD camera.

[0205] The lower machine 503 is further configured to send the target fog droplet deposition image, the collection time, and the collection position to the cloud server 504.

[0206] The collection time can be understood as the time when the lower machine 503 receives the image collection request, and the collection position can be understood as the position of the lower machine 503.

[0207] For example, the lower machine 503 has a function of temporarily storing data, so as to store the target fog droplet deposition image, the collection time, and the collection position, and forward the target fog droplet deposition image, the collection time, and the collection position temporarily stored by the lower machine 503 to the cloud server 504.

[0208] The cloud server 504 is further configured to determine an analysis result of the target fog droplet deposition image based on the fog droplet deposition detection method according to any one of the above embodiments, and send the analysis result, the collection time, and the collection position to the user equipment 505.

[0209] For example, the detection device can be the cloud server 504, and the cloud server 504 obtains the analysis result by executing the fog droplet deposition detection method according to the above embodiments, so as to transmit the analysis result, the collection time, and the collection position to the user equipment 505.

[0210] For example, the cloud server 504 stores a computer program for performing the fog droplet deposition detection method as described in any of the above embodiments. When the cloud server 504 receives a target fog droplet deposition image, the cloud server 504 can call and run the computer program to obtain an analysis result.

[0211] The user device 505 is further configured to display the analysis result, the collection time, and the collection location.

[0212] It is worth noting that, Figure 5 The system shown and the above description of the system 500 are only used to exemplarily illustrate the components and operation modes that the system 500 can include, and cannot be understood as a limitation of the system 500.

[0213] For example, the system 500 can only include the fog droplet collection device 501, the image acquisition device 502, and the cloud server 504. The fog droplet collection device 501 is configured to form a target fog droplet deposition image, the image acquisition device 502 is configured to acquire the target fog droplet deposition image, and the cloud server 504 is configured to determine an analysis result of the target fog droplet deposition image based on the fog droplet deposition detection method as described in the above embodiments, and transmit the analysis result to the user device 505 outside the system 500.

[0214] For another example, the system 500 can only include the fog droplet collection device 501, the image acquisition device 502, and the lower machine 503. The lower machine 503 is configured to perform the fog droplet deposition detection method as described in the above embodiments.

[0215] For another example, the system 500 can only include the fog droplet collection device 501 and the image acquisition device 502. The image acquisition device 502 has both an image acquisition function and an analysis function. The analysis function is embodied in that the image acquisition device 502 can perform the fog droplet deposition detection method as described in the above embodiments.

[0216] For example, the user device 505 can directly send an image acquisition request to the lower machine 503 without forwarding by the cloud server 504. For another example, the user device 505 can access the cloud server 504 to obtain the analysis result, the collection time, and the collection location, and display the analysis result, the collection time, and the collection location on the front end of the applet, instead of the cloud server 504 actively sending the analysis result, the collection time, and the collection location to the user device 505.

[0217] In some embodiments, the lower computer 503 includes a Raspberry Pi and a communication module, wherein the communication module can be a 5G communication module used to connect the Raspberry Pi to a 5G network; the Raspberry Pi connects the CCD camera through a USB interface, controls the CCD camera to collect target fog droplet deposition images, temporarily stores the collected target fog droplet deposition images, collection time and collection position, and uploads the target fog droplet deposition images, collection time and collection position to the cloud server 504 through the 5G communication module.

[0218] Please refer to Figure 6 , Figure 6 The structure diagram of the lower computer 503 of the embodiment of the present disclosure is as shown in Figure 6 The 5G communication module 5031 is connected to the 5G communication expansion board 5032, the 5G communication expansion board 5032 communicates with the Raspberry Pi 5034 through a serial port connection, and the copper column 5033 is used to fixedly connect the 5G communication expansion board 5031 and the Raspberry Pi 5034.

[0219] In the present embodiment, the lower computer 503 connects to the 5G network through the 5G communication module 5031, establishes communication with the cloud server 504, uses the low delay advantage of the 5G communication technology to improve the communication speed between the lower computer 503 and the user equipment 505 (such as an applet), reduces the image collection delay, uses the high speed feature of the 5G communication technology to improve the upload speed of the collected images, and improves the collection speed.

[0220] The cloud server 504 can be a server built with a file transfer protocol (FTP), and the lower computer 503 uses the FTP protocol to upload the collected data to the cloud server 504.

[0221] Similarly, Figure 6 The lower computer 503 and the above description of the lower computer 503 are only used to exemplarily illustrate the components and operating modes that the lower computer 503 can include, and cannot be understood as a limitation of the lower computer 503. For example, the communication module in the lower computer 503 can be a 6G communication module, and the like, which will not be listed one by one here.

[0222] It is worth mentioning that in the system 500 provided by the embodiment, the target fog droplet deposition image (or other information such as collection time and collection position can also be included) is collected, transmitted and stored by the user equipment 505 (such as an applet in the user equipment) cooperating with the cloud server 504, the lower machine 503 and the image acquisition device 502 (such as a CCD camera), so as to improve the data collection speed, simplify the collection operation process, improve the automation degree of the system, avoid damage to the collection result in the manual recovery process, reduce the damage to the fog droplet collection device 501 (such as water-sensitive paper) in the data collection process, save the labor and time cost of the traditional collection scheme; the image collection quality is improved by using the fog droplet deposition detection method described in the above embodiment, and the fog droplets are classified and segmented based on the deep learning algorithm, so as to effectively improve the processing precision and speed of the fog droplet deposition image, provide reliable plant protection spraying operation evaluation data to guide the actual plant protection spraying operation, and promote the reduction of the target object (such as pesticide) and the increase of efficiency.

[0223] Based on the above technical concept, the disclosure also provides a fog droplet deposition detection device.

[0224] Please refer to Figure 7 , Figure 7 The fog droplet deposition detection device 700 of the embodiment of the disclosure is shown in Figure 7 The fog droplet deposition detection device 700 comprises:

[0225] The obtaining unit 701 is configured to obtain a target fog droplet deposition image to be detected.

[0226] In some embodiments, the obtaining unit 701 comprises:

[0227] The acquisition sub-unit 7011 is configured to acquire an initial fog droplet deposition image.

[0228] The correction sub-unit 7012 is configured to perform correction processing on the initial fog droplet deposition image based on a local histogram equalization algorithm to obtain the target fog droplet deposition image.

[0229] In some embodiments, the correction sub-unit is configured to convert the initial fog droplet deposition image from an RGB color space to an HSV color space; divide an initial luminance channel of the HSV color space into a plurality of uniform regions; for each region of the plurality of regions, calculate a corresponding cumulative histogram and a total number of pixels of the region; perform luminance equalization processing on the initial luminance channel based on the cumulative histogram and the total number of pixels corresponding to each region to obtain a target luminance channel; generate the target fog droplet deposition image based on an initial hue channel, an initial saturation channel and the target luminance channel, and convert the target fog droplet deposition image to the RGB color space, wherein the HSV color space comprises the initial hue channel and the initial saturation channel.

[0230] The classification unit 702 is configured to perform classification processing on the target fog droplet deposition image based on a pre-trained fog droplet classification model to obtain a target agglomerated fog droplet image and a target non-agglomerated fog droplet image, wherein the fog droplet classification model is obtained by training a first base network model based on sample fog droplet deposition images, and the first base network model comprises a first backbone network, and the training of the fog droplet classification model comprises a first freezing stage and a first unfreezing stage, in the first freezing stage, the network parameters of the first backbone network do not change, and in the first unfreezing stage, the network parameters of the first backbone network change.

[0231] In some embodiments, the classification unit 702 comprises:

[0232] The extraction subunit 7021 is configured to perform feature extraction processing on the target fog droplet deposition image based on the first backbone network in the fog droplet classification model to obtain target image features of the target fog droplet deposition image.

[0233] The classification subunit 7022 is configured to perform classification processing on the target fog droplet deposition image based on the fog droplet classification model and the target image features to obtain the target agglomerated fog droplet image and the target non-agglomerated fog droplet image.

[0234] The analysis unit 703 is configured to analyze the target agglomerated fog droplet image and the target non-agglomerated fog droplet image to obtain an analysis result, wherein the analysis result comprises a fog droplet coverage rate and / or a fog droplet deposition parameter of the target fog droplet deposition image.

[0235] In some embodiments, the analysis unit 703 comprises:

[0236] The segmentation subunit 7031 is configured to perform non-agglomeration segmentation processing on the target agglomerated fog droplet image to obtain a target segmented image.

[0237] In some embodiments, the segmentation subunit 7031 is configured to identify a target concave point in the target agglomerated fog droplet image based on a pre-trained concave point detection model, wherein the concave point detection model is obtained by training a second base network model based on sample agglomerated fog droplet images, the second base network model comprises a second backbone network, and the training of the concave point detection model comprises a second freezing stage and a second unfreezing stage, in the second freezing stage, the network parameters of the second backbone network do not change, and in the second unfreezing stage, the network parameters of the second backbone network change; and segmenting the target agglomerated fog droplet image based on a connection operation on the target concave point to obtain the target segmented image.

[0238] The merging subunit 7032 is configured to merge the target segmented image and the target non-adhesion fog droplet image to obtain a merged image.

[0239] The calculating subunit 7033 is configured to perform parameter calculation processing on the merged image to obtain the analysis result.

[0240] In some embodiments, the calculating subunit 7033 is configured to perform edge detection on the merged image to obtain a set of fog droplet contour pixel points of the merged image; calculate a total number of fog droplets in the merged image and a pixel area corresponding to each fog droplet based on the set of fog droplet contour pixel points; and calculate the analysis result based on the total number of fog droplets and the pixel area corresponding to each fog droplet, wherein the analysis result includes the fog droplet coverage rate and the fog droplet deposition parameter, and the fog droplet deposition parameter includes a fog droplet deposition density.

[0241] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0242] According to embodiments of the present disclosure, the present disclosure further provides a computer program product, which includes a computer program stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to enable the electronic device to perform the scheme provided in any of the above embodiments.

[0243] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0244] As Figure 8As shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0245] A plurality of components in the device 800 are connected to the I / O interface 805, including an input unit 806 such as a keyboard, a mouse, etc., an output unit 807 such as various types of displays, speakers, etc., a storage unit 808 such as a magnetic disk, an optical disk, etc., and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0246] The computing unit 801 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as the mist deposition detection method. For example, in some embodiments, the mist deposition detection method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the mist deposition detection method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the mist deposition detection method by any other appropriate means, such as by means of firmware.

[0247] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0248] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0249] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0250] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0251] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0252] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server, or VPS for short). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0253] Those skilled in the art will appreciate that embodiments of the disclosure can be supplied as a method, a system, or a computer program product. Thus, the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, magnetic disks and optical storage media) embodying computer programs, code, or instructions.

[0254] The computer executable instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operations steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0255] These processor-executable instructions can also be stored in a processor-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0256] These processor-executable instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operations steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 Figure 1 one or more flow or blocks.

[0257] Obviously, numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the present disclosure can be practiced otherwise than as specifically described.

Claims

1. A method for detecting droplet deposition, characterized in that: The method comprises: Obtaining a target droplet deposition image to be detected; Based on a pre-trained droplet classification model, the adhering droplets and non-adhering droplets in the target droplet deposition image are classified to obtain a target adhering droplet image and a target non-adhering droplet image, wherein the droplet classification model is obtained by training a first basic network model based on the sample droplet deposition image, the first basic network model includes a first backbone network, and the droplet classification model obtained by training includes a first freezing stage and a first thawing stage, in the first freezing stage, the network parameters of the first backbone network do not change, and in the first thawing stage, the network parameters of the first backbone network change; Analyzing the target adhered droplet image and the target non-adhesive droplet image to obtain an analysis result, wherein the analysis result includes a droplet coverage rate and / or a droplet deposition parameter of the target droplet deposition image; The analyzing the target adhering mist droplet image and the target non-adhering mist droplet image to obtain the analysis result includes: Performing non-adhesion segmentation processing on the target adhesion droplet image to obtain a target segmented image; Merging the target segmented image and the target non-adhesive droplet image to obtain a merged image; performing parameter calculation processing on the merged image to obtain the analysis result; The step of performing non-adhesion segmentation processing on the target adhesion droplet image to obtain a target segmented image includes: Identifying a target concave point in the target adhesion droplet image based on a pre-trained concave point detection model, wherein the concave point detection model is obtained by training a second basic network model based on the sample adhesion droplet image, the second basic network model includes a second backbone network, and the training of the concave point detection model includes a second freezing stage and a second thawing stage, in which the network parameters of the second backbone network do not change, and in the second thawing stage, the network parameters of the second backbone network change; The target adhesion droplet image is segmented based on a connection operation on the target concave points to obtain the target segmented image.

2. The method according to claim 1, characterized in that The method of classifying the adhered droplets and the non-adhesive droplets in the target droplet deposition image based on the pre-trained droplet classification model to obtain the target adhered droplet image and the target non-adhesive droplet image includes: performing feature extraction processing on the target droplet deposition image based on the first backbone network in the droplet classification model to obtain target image features of the target droplet deposition image; Based on the droplet classification model and the target image features, the target droplet deposition image is classified to obtain the target adhesion droplet image and the target non-adhesion droplet image.

3. The method according to claim 1, characterized in that The performing parameter calculation processing on the merged image to obtain the analysis result includes: Performing edge detection on the merged image to obtain a set of fog droplet outline pixel points of the merged image; Based on the set of droplet outline pixels, the total number of droplets in the merged image and the pixel area corresponding to each droplet are calculated; The analysis result is calculated based on the total number of droplets and the area of ​​each pixel, wherein the analysis result includes the droplet coverage and the droplet deposition parameters, and the droplet deposition parameters include droplet deposition density.

4. The method according to any one of claims 1 to 3, characterized in that The step of obtaining a target droplet deposition image to be detected includes: Acquire initial droplet deposition images; The initial droplet deposition image is corrected based on a local histogram equalization algorithm to obtain the target droplet deposition image.

5. The method according to claim 4, characterized in that The correcting process of the initial droplet deposition image based on the local histogram equalization algorithm to obtain the target droplet deposition image includes: Converting the initial droplet deposition image from RGB color space to HSV color space; Segmenting the initial brightness channel of the HSV color space into a plurality of uniform regions; For each of the multiple regions, calculate and obtain a cumulative histogram and a total number of pixels corresponding to the region; Performing brightness equalization processing on the initial brightness channel based on the cumulative histogram and the total number of pixels corresponding to each region to obtain a target brightness channel; The target droplet deposition image is generated based on an initial hue channel, an initial saturation channel, and the target brightness channel, and the target droplet deposition image is converted into an RGB color space, wherein the HSV color space includes the initial hue channel and the initial saturation channel.

6. A droplet deposition detection device, characterized in that: The device comprises: An acquisition unit, configured to acquire a target droplet deposition image to be detected; a classification unit, configured to classify adhered droplets and non-adhesive droplets in the target droplet deposition image based on a pre-trained droplet classification model to obtain a target adhered droplet image and a target non-adhesive droplet image, wherein the droplet classification model is obtained by training a first basic network model based on a sample droplet deposition image, the first basic network model including a first backbone network, and the droplet classification model obtained by training includes a first freezing stage and a first thawing stage, wherein in the first freezing stage, the network parameters of the first backbone network do not change, and in the first thawing stage, the network parameters of the first backbone network change; an analyzing unit, configured to analyze the target adhering droplet image and the target non-adhering droplet image to obtain an analysis result, wherein the analysis result includes a droplet coverage rate and / or droplet deposition parameters of the target droplet deposition image; Wherein, the analysis unit includes: a segmentation subunit, configured to perform non-adhesion segmentation processing on the target adhesion droplet image to obtain a target segmented image; a merging subunit, configured to merge the target segmented image and the target non-adhesive droplet image to obtain a merged image; a calculation subunit, configured to perform parameter calculation processing on the merged image to obtain the analysis result; Wherein, the segmentation subunit is used to identify the target concave points in the target adhesion droplet image based on a pre-trained concave point detection model, wherein the concave point detection model is obtained by training a second basic network model based on the sample adhesion droplet image, and the second basic network model includes a second backbone network. The trained concave point detection model includes a second freezing stage and a second thawing stage. In the second freezing stage, the network parameters of the second backbone network do not change, and in the second thawing stage, the network parameters of the second backbone network change; and based on the connection operation of the target concave points, the target adhesion droplet image is segmented to obtain the target segmented image.

7. A droplet deposition detection system, characterized in that: The system includes: a droplet collection device, an image acquisition device, a lower computer, a cloud server, and a user device, wherein: The droplet collecting device is used to form a target droplet deposition image based on the droplets; The user device is used to obtain and send an image acquisition request to the cloud server; The cloud server is used to forward the image acquisition request to the slave computer; The lower computer is configured to control the image acquisition device to acquire the target droplet deposition image based on the image acquisition request; The image acquisition device is used to acquire the target droplet deposition image and send the target droplet deposition image to the slave computer; The lower computer is further configured to send the target droplet deposition image, acquisition time, and acquisition location to the cloud server, wherein the acquisition time is the time when the lower computer receives the image acquisition request, and the acquisition location is the location of the lower computer; The cloud server is further configured to determine an analysis result of the target droplet deposition image based on the method according to any one of claims 1 to 5, and send the analysis result, the acquisition time, and the acquisition location to the user device; The user equipment is further configured to display the analysis result, the collection time, and the collection location.

8. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 5.

Images