Spray detection method, device, computer equipment, storage medium and program product

Abstract

The present application discloses a spray detection method, device, computer equipment, storage medium and program product. The method comprises: collecting an initial spray image; the initial spray image comprises the spray condition of at least one nozzle; segmenting the initial spray image to obtain the spray image to be detected corresponding to each nozzle; using a detection model to detect the spray condition of the spray image to be detected to obtain the spray detection result of the nozzle corresponding to the spray image to be detected; the detection model is trained based on the labeled sample spray image; the spray detection result comprises the spray uniformity, the fan angle and the spray condition, and the spray condition is used to characterize the spray effect level. The use of this method can effectively improve the detection efficiency of the spray condition of the nozzle.

Description

Technical Field

[0001] The present application relates to the field of detection technology, and in particular to a spray detection method, device, computer equipment, storage medium and program product. Background Art

[0002] Coal is mostly transported in uncovered open trucks. During the transportation process, the relative movement of airflow and wind cause a large amount of coal powder to fly, which not only causes coal loss but also causes coal dust pollution along the line.

[0003] Spraying dust suppressants is an effective means of controlling coal dust. That is, the dust suppressants are sprayed on coal transport vehicles through spraying equipment to form a crust on the surface of the coal, thereby suppressing the flying of dust. The spraying effect of the spraying equipment is a key factor affecting the dust suppression effect. Therefore, monitoring the spraying effect of the dust suppressant spraying equipment is a necessary means.

[0004] Currently, the spraying effect of spraying equipment is often monitored through manual inspection and human observation, but manual monitoring has the problem of low efficiency. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a spray detection method, apparatus, computer equipment, storage medium and program product that can effectively improve the detection efficiency of the spraying condition of the sprinkler.

[0006] In a first aspect, the present application provides a spray detection method. The method comprises:

[0007] Acquiring an initial spraying image; the initial spraying image includes the spraying condition of at least one nozzle;

[0008] The initial spraying image is segmented to obtain the spraying image to be detected corresponding to each nozzle;

[0009] The detection model is used to detect the spraying condition of the spray image to be detected, and the spraying detection result of the nozzle corresponding to the spray image to be detected is obtained; the detection model is trained based on the labeled sample spray image; the spraying detection result includes spraying uniformity, fan angle and spraying condition, and the spraying condition is used to characterize the spraying effect level.

[0010] In combination with the first aspect, in a possible implementation, the spraying detection method further includes: performing edge detection processing on the thermal imaging image and the visible light image in the initial spraying image to obtain position information of each nozzle in the thermal imaging image and the visible light image; the position information includes the coordinates of each nozzle in the thermal imaging image and the visible light image, and the coordinates of the intersection of the spraying fans of adjacent nozzles; according to the position information of the thermal imaging image and the visible light image, performing geometric transformation processing on the thermal imaging image and / or the visible light image to obtain two candidate images, the spraying fans of the nozzles in the two candidate images are the same, the two candidate images include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image after the geometric transformation processing and the visible light image after the geometric transformation processing; fitting processing is performed on the two candidate images, and the fitted image is updated to the initial spraying image.

[0011] In combination with the first aspect, in a possible implementation, the initial spray image is segmented, including: determining the midpoint position between adjacent nozzles and the intersection position of the spray fan between adjacent nozzles; and segmenting the initial spray image based on the straight line where the midpoint position and the intersection position are located.

[0012] In combination with the first aspect, in a possible implementation method, the training process of the detection model includes: obtaining multiple initial sample spray images; the initial sample spray images include the spraying conditions of at least one nozzle; segmenting the initial sample spray images to obtain sample spray images corresponding to each nozzle; determining the label of each sample spray image; the label of the sample spray image includes the spray uniformity label, fan angle label and spray condition label of the nozzle; inputting multiple sample spray images and the label of each sample spray image into the initial model for feature extraction and feature mapping to obtain the predicted spray uniformity, predicted fan angle and predicted spray condition output by the initial model; based on the differences between the predicted spray uniformity, predicted fan angle and predicted spray condition and the spray uniformity label, fan angle label and spray condition label of the nozzle, iteratively training the initial model until the difference meets the preset difference condition.

[0013] In combination with the first aspect, in a possible implementation, determining the label of each sample spray image includes: determining the spray uniformity label of the nozzle based on the temperature data of the thermal imaging image of the sample spray image, the temperature data is the temperature data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle; or, determining the spray uniformity label of the nozzle based on the brightness data of the visible light image of the sample spray image, the brightness data is the brightness data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle.

[0014] In combination with the first aspect, in a possible implementation, the spraying detection method also includes: obtaining a target image in which the spraying condition does not meet a preset condition; and iteratively training the detection model using the target image as a model sample until a preset number of iterations is reached or the model accuracy reaches a preset accuracy.

[0015] In a second aspect, the present application also provides a spray detection device. The device comprises:

[0016] A collection module, used for collecting an initial spraying image; the initial spraying image includes the spraying condition of at least one nozzle;

[0017] A segmentation module is used to segment the initial spraying image to obtain a spraying image to be detected corresponding to each nozzle;

[0018] The detection module is used to use the detection model to perform spraying condition detection on the spray image to be detected, and obtain the spraying detection result of the nozzle corresponding to the spray image to be detected; the detection model is trained based on the labeled sample spray image; the spraying detection result includes spraying uniformity, fan angle and spraying condition, and the spraying condition is used to characterize the spraying effect level.

[0019] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0021] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0022] The embodiments of the present application provide a spray detection method, device, computer equipment, storage medium and program product, which can capture the spraying conditions of the nozzles to obtain an initial spraying image, and then segment the initial spraying image to obtain the spraying images to be detected corresponding to each nozzle, and then input the spraying images to be detected into the detection model for spraying condition detection to obtain information such as the spraying uniformity, fan angle and spraying condition of the nozzles. The embodiments of the present application can detect images containing the spraying conditions of the nozzles through a pre-trained detection model to obtain the spraying conditions of each nozzle, which effectively improves the detection efficiency of the nozzle spraying conditions. Moreover, the present application represents the spraying conditions of the nozzles based on multiple information such as spraying uniformity, fan angle, spraying condition, etc., which improves the accuracy and diversity of the nozzle spraying detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0024] Figure 1 A schematic diagram of a spray detection method according to an embodiment;

[0025] Figure 2 Another schematic diagram of a spray detection method in one embodiment;

[0026] Figure 3 Another schematic diagram of a spray detection method in one embodiment;

[0027] Figure 4 Another schematic diagram of a spray detection method in one embodiment;

[0028] Figure 5 Another schematic diagram of a spray detection method in one embodiment;

[0029] Figure 6 is a structural block diagram of a spray detection device in one embodiment;

[0030] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0032] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. In addition, the term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects.

[0033] In one embodiment, Figure 1As shown, a spray detection method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0034] Step 101: Collect an initial spraying image.

[0035] In the embodiment of the present application, a camera can be arranged above the nozzle of the spraying device, and when the spraying device is working, that is, when the nozzle sprays the inhibitor (for example, it can be detected that the spraying flow rate collected by the flow meter of the spraying device is within the preset flow rate range), the initial spraying image including the spraying situation of the nozzle is collected. Among them, the image collection frequency can be specifically set according to the monitoring requirements, for example, it can be specifically set according to the regular spraying time of the nozzle.

[0036] Based on the different distances between the camera and the spraying equipment, the initial spraying image may include a spraying image of one nozzle, and in order to reduce the monitoring cost, the initial spraying image may also include spraying images of multiple nozzles.

[0037] Among them, the camera may include at least one of a thermal imaging camera and a visible light camera, and correspondingly, the initial spraying image includes at least one of a thermal imaging image and a visible light image. That is, the thermal imaging image of the nozzle during spraying can be collected by a thermal imaging camera, the visible light image of the nozzle during spraying can be collected by a visible light camera, or the thermal imaging image and the visible light image of the nozzle during spraying can be collected simultaneously by a thermal imaging camera and a visible light camera. Alternatively, the camera can also be a multi-spectral camera that has both visible light and thermal imaging shooting capabilities. It can be understood that the image data of the thermal imaging image includes temperature data, and the image data of the visible light image includes three primary color data (i.e., RGB), and the brightness data of each pixel of the visible light image can be obtained based on the three primary color data.

[0038] Step 102: segment the initial spraying image to obtain the spraying image to be detected corresponding to each nozzle.

[0039] In the embodiment of the present application, after the initial spraying image is collected, the initial spraying image can be segmented according to the boundaries between adjacent nozzles, thereby obtaining a plurality of spraying images to be detected.

[0040] The spraying image to be detected includes the spraying situation corresponding to a single nozzle.

[0041] In a possible implementation, the resolution and data type of the spray images to be detected corresponding to each nozzle obtained by segmentation may be corrected and normalized so that the processed spray images to be detected meet the data processing standard of the detection model.

[0042] Step 103: Use the detection model to perform spraying condition detection on the spraying image to be detected, and obtain the spraying detection result of the nozzle corresponding to the spraying image to be detected.

[0043] In an embodiment of the present application, a detection model can be trained based on sample spray images marked with spray uniformity, fan angle and spray conditions, and then the detection model can be used to perform spray condition detection on the above-mentioned spray images to be detected to obtain the spray detection results of the nozzle corresponding to each spray image to be detected, that is, the spray uniformity, fan angle and spray condition of the nozzle.

[0044] The spraying condition is used to characterize the spraying effect level. For example, the spraying condition may include qualified spraying effect, unqualified spraying effect, and average spraying effect; the spraying condition may also be any number between 0 and 1. The closer the spraying condition is to 1, the better the spraying effect is, and the closer it is to 0, the worse the spraying effect is.

[0045] In a possible implementation, the sprinkler heads whose spraying detection results do not meet the expected results may be recorded and maintenance prompt information may be sent to the user terminal.

[0046] The embodiment of the present application provides a spray detection method, which can capture the spraying situation of the nozzle to obtain an initial spraying image, and then segment the initial spraying image to obtain the spraying image to be detected corresponding to each nozzle, and then input the spraying image to be detected into the detection model for spraying condition detection to obtain information such as the spraying uniformity, fan angle, and spraying condition of the nozzle. The embodiment of the present application can detect images containing the spraying situation of the nozzle through a pre-trained detection model to obtain the spraying condition of each nozzle, which effectively improves the detection efficiency of the nozzle spraying condition. Moreover, the present application represents the spraying situation of the nozzle based on multiple information such as spraying uniformity, fan angle, and spraying condition, thereby improving the accuracy and diversity of the nozzle spraying detection results.

[0047] In the above-mentioned embodiment, the initial spray image includes at least one of a thermal imaging image and a visible light image. In another embodiment of the present application, when the initial spray image includes a thermal imaging image and a visible light image, the two need to be fitted and then segmented. This embodiment includes the following: Figure 2 Steps shown:

[0048] Step 201 : performing edge detection processing on the thermal imaging image and the visible light image in the initial spraying image to obtain position information of each nozzle in the thermal imaging image and the visible light image.

[0049] The position information includes the coordinates of each nozzle in the thermal imaging image and the visible light image, and the coordinates of the intersection points of the spraying fans of adjacent nozzles.

[0050] It should be noted that when the camera includes a thermal imaging camera and a visible light camera, the optical axes of the thermal imaging camera and the visible light camera are arranged in parallel.

[0051] In an embodiment of the present application, after the thermal imaging camera and the visible light camera collect the thermal imaging image and the visible light image of the same nozzle or the same group of nozzles, they perform edge detection processing on the thermal imaging image and the visible light image respectively to obtain the position information of each nozzle in the two images, that is, the coordinates of each nozzle in the two images, and the coordinates of the intersection of the spray fans of two adjacent nozzles.

[0052] Step 202: Perform geometric transformation processing on the thermal imaging image and / or the visible light image according to the position information of the thermal imaging image and the visible light image to obtain two candidate images.

[0053] In an embodiment of the present application, after obtaining the position information of each nozzle in the thermal imaging image and the visible light image, a triangle can be determined based on the nozzle coordinates and the coordinates of the two intersection points, and the distance between the nozzle and the two intersection points can also be determined, that is, three distance information, that is, the three side lengths of the triangle.

[0054] Then, based on the relationship between the triangles corresponding to the thermal imaging image and the triangles corresponding to the visible light image, the thermal imaging image and / or the visible light image are scaled and deformed so that the shapes and sizes of the triangles corresponding to the thermal imaging image and the visible light image are exactly the same, that is, the spraying fans in the thermal imaging image and the visible light image are the same, thereby obtaining two candidate images.

[0055] It can be understood that the two candidate images include a thermal imaging image and a visible light image after geometric transformation processing, or include a thermal imaging image and a visible light image after geometric transformation processing, or include a thermal imaging image and a visible light image after geometric transformation processing.

[0056] Step 203: Perform fitting processing on the two candidate images, and update the fitted images as the initial spraying images.

[0057] In an embodiment of the present application, the two candidate images are fitted based on the nozzle coordinates and the spray fan intersection coordinates in the two candidate images, so that the nozzle coordinates and the spray fan intersection coordinates in the two candidate images overlap, thereby obtaining a fitted image, and the fitted image is used as the initial spray image for subsequent segmentation processing.

[0058] It can be understood that the fitted image includes both temperature data and three primary color data.

[0059] In a possible implementation, the fitted image may be cropped to crop the portion of the visible light image that exceeds the thermal imaging image, or to crop the portion of the thermal imaging image that exceeds the visible light image.

[0060] The method provided in the embodiment of the present application can simultaneously collect thermal imaging images and visible light images of the nozzle spraying, and fit the two images based on the position information of the nozzle, and update the fitted image to the initial spraying image, so that the initial spraying image contains multiple data such as temperature and three primary colors at the same time, thereby improving the diversity of image data, effectively reducing the impact of different climate environments, lighting environments, etc. on the image, and improving the accuracy of spraying condition detection.

[0061] The above-mentioned embodiment introduces a scheme for segmenting the initial spraying image. In another embodiment of the present application, the initial spraying image can be segmented in units of nozzles. This embodiment includes the following: Figure 3 Steps shown:

[0062] Step 301: determine the midpoint position between adjacent nozzles and the intersection position of the spraying fan between adjacent nozzles.

[0063] Step 302: Segment the initial spraying image based on the straight lines where the midpoints and intersections are located.

[0064] In an embodiment of the present application, when segmenting the initial spray image, the boundary between adjacent nozzles can be determined first. For example, the midpoint position between the nozzle coordinates of adjacent nozzles and the intersection of the spray fans can be connected, and the straight line where the line segment is located can be used as the boundary of adjacent nozzles, and the initial spray image can be segmented according to the boundary.

[0065] Alternatively, the nozzle coordinates in the initial spraying image may be connected to obtain the line segment where the nozzle is located, and then a perpendicular line to the line segment where the nozzle is located may be made based on the midpoint position between the nozzle coordinates of adjacent nozzles, and the perpendicular line may be used as the boundary between adjacent nozzles, and the initial spraying image may be segmented according to the boundary.

[0066] The coordinates of the intersection points of each spray fan in the initial spray image can also be connected to obtain line segments where multiple intersection points are located, and then perpendicular lines are drawn based on each intersection point, and the perpendicular lines are used as the boundaries of adjacent nozzles, and the initial spray image is divided according to the boundaries.

[0067] The method provided in the embodiment of the present application can segment the initial spray image based on the midpoint position between adjacent nozzles and the intersection position of the spray fan between adjacent nozzles, so that the segmented spray image to be detected can include a relatively complete spray fan of a single nozzle, thereby the spray condition detection result of the nozzle obtained based on the complete spray fan is more accurate.

[0068] In one embodiment, Figure 4 As shown, the training process of the above detection model is provided:

[0069] Step 401: Acquire multiple initial sample spraying images.

[0070] In the embodiment of the present application, a camera may be arranged above the nozzle of the spraying device to collect a spraying image including the spraying condition of the nozzle when the spraying device is working, that is, when the nozzle sprays the inhibitor, as an initial sample spraying image. The initial sample spraying image includes the spraying condition of at least one nozzle.

[0071] The camera may include at least one of a thermal imaging camera and a visible light camera, and correspondingly, the initial sample spray image includes at least one of a thermal imaging sample image and a visible light sample image. It can be understood that the image data of the thermal imaging sample image includes temperature data, and the image data of the visible light sample image includes three primary color data (i.e., RGB), and the brightness data of each pixel of the visible light sample image can be obtained based on the three primary color data.

[0072] In one possible implementation, after the camera captures multiple spraying images, the spraying images can be manually selected, and spraying images with different climate environments, lighting environments, and different spraying conditions can be selected as initial sample images to improve the diversity of samples, thereby improving the accuracy of the trained detection model.

[0073] Step 402: Segment the initial sample spraying image to obtain sample spraying images corresponding to each nozzle.

[0074] In the embodiment of the present application, after the initial sample spraying image is collected, the initial sample spraying image can be segmented according to the boundaries between adjacent nozzles to obtain multiple sample spraying images, wherein the sample spraying image includes the spraying situation corresponding to a single nozzle.

[0075] In a possible implementation, the resolution and data type of the sample spraying images corresponding to each nozzle obtained by segmentation may be corrected and normalized to serve as a standard for the trained detection model to process data.

[0076] Step 403: Determine the label of each sample spray image.

[0077] The labels of the sample spraying image include the spraying uniformity label, the fan angle label and the spraying condition label of the nozzle.

[0078] In the embodiment of the present application, edge detection may be performed on the sample spraying image to obtain the spraying fan angle of the nozzle corresponding to the sample spraying image, that is, to obtain the fan angle label of the sample spraying image.

[0079] If the sample spray image is a thermal image, or a fitted image of a thermal image and a visible light image, the spray uniformity label of the nozzle is determined according to the temperature data of the thermal image. For example, the variance or standard deviation of the temperature data is calculated as the spray uniformity of the nozzle, and the spray uniformity label of the sample spray image is obtained. It can be understood that the temperature data is the temperature data of the area where the spray fan of the nozzle does not overlap with the spray fan of the adjacent nozzle.

[0080] If the sample spray image is a visible light image, the brightness of each pixel is first calculated based on the three primary color data of the visible light image, and then the spray uniformity label of the nozzle is determined based on the brightness data. For example, the variance or standard deviation of the brightness data is calculated as the spray uniformity of the nozzle to obtain the spray uniformity label of the sample spray image. It can be understood that the brightness data is the brightness data of the area where the spray fan of the nozzle does not overlap with the spray fan of the adjacent nozzle.

[0081] The spraying condition label can be manually determined and marked according to the spraying fan angle and spraying uniformity of the nozzle. For example, if the spraying fan angle of the nozzle is within the preset angle range (60° to 120°), and the spraying uniformity is within the preset uniformity range, the spraying condition of the nozzle is marked as qualified; if the spraying fan angle of the nozzle is within the preset angle range, but the spraying uniformity is outside the preset uniformity range, the spraying condition of the nozzle is marked as general; if the spraying fan angle of the nozzle is outside the preset angle range, and the spraying uniformity is outside the preset uniformity range, the spraying condition of the nozzle is marked as unqualified. For example, the spraying condition of the nozzle can also be scored according to the relationship between the spraying fan angle and the preset angle range, and the relationship between the spraying uniformity and the preset uniformity range, to obtain a score between 0 and 1. The closer the score is to 1, the better the spraying effect is, and the closer it is to 0, the worse the spraying effect is.

[0082] Step 404: Input multiple sample spraying images and the label of each sample spraying image into the initial model for feature extraction and feature mapping to obtain the predicted spraying uniformity, predicted fan angle and predicted spraying condition output by the initial model.

[0083] The initial model may be a neural network model including an input layer, a hidden layer, and an output layer. The input layer, the hidden layer, and the output layer may be composed of the same or different numbers of fully connected layers, or may be composed of the same or different numbers of binary convolutional layers.

[0084] The number of fully connected layers or two-dimensional convolutional layers or the number of nodes therein can be determined according to the type of sample spray image. For example, when the sample spray image is a thermal imaging image, the image data only includes temperature data, so the number of fully connected layers or two-dimensional convolutional layers or the number of nodes therein can be relatively small; when the sample spray image is an image fitted by a thermal imaging image and a visible light image, the image data includes temperature data and three primary color data, that is, the amount of image data is large, so the number of fully connected layers or two-dimensional convolutional layers or the number of nodes therein can be relatively large to match more image data and improve the accuracy of the detection model.

[0085] In an embodiment of the present application, after the initial model is set up, multiple sample spray images and corresponding labels can be input into the initial model, and the initial model will perform feature extraction and feature mapping on the sample spray images to obtain the predicted spray uniformity, predicted fan angle and predicted spray condition corresponding to each nozzle.

[0086] Step 405: based on the difference between the predicted spraying uniformity, the predicted fan angle and the predicted spraying condition and the spraying uniformity label, the fan angle label and the spraying condition label of the nozzle, iteratively train the initial model until the difference meets the preset difference condition.

[0087] In an embodiment of the present application, the parameters of the initial model are modified based on the predicted spraying uniformity, predicted fan angle and predicted spraying condition, and the difference between the spraying uniformity label, fan angle label and spraying condition label of the nozzle, and the sample spraying image is again extracted and mapped based on the initial model after the parameter modification, that is, the initial model is iteratively trained until the difference between the output predicted spraying uniformity, predicted fan angle and predicted spraying condition and the spraying uniformity label, fan angle label and spraying condition label of the nozzle meets the preset difference condition, for example, is less than the preset difference threshold, then the training is completed and the detection model is obtained.

[0088] In a possible implementation, an initial model may be pre-set, and the initial model may be iteratively trained based on multiple sample spray images and the label of each sample spray image to obtain a detection model. Alternatively, multiple initial models may be pre-set, and the number of layers or the number of nodes in the fully connected layer or two-dimensional convolutional layer in the multiple initial models may be different. Then, based on multiple sample spray images and the label of each sample spray image, the multiple initial models may be iteratively trained to obtain multiple candidate detection models, and finally the candidate detection model with the highest accuracy may be selected as the final detection model.

[0089] The embodiment of the present application provides a method for training a detection model, that is, first obtaining a plurality of initial sample spray images, segmenting the initial sample spray images, obtaining sample spray images corresponding to each nozzle, and then determining the label of each sample spray image, iteratively training the initial model based on the plurality of sample spray images and the label of each sample spray image, until the difference between the predicted label and the marked label output by the initial model meets the preset difference condition. The embodiment of the present application can obtain a detection model based on the collected sample spray image training, so as to detect the spray image to be detected containing the spraying condition of the nozzle based on the detection model, obtain the spraying condition of each nozzle, avoid manually determining the spraying condition of each nozzle by observation and inspection, and effectively improve the detection efficiency and accuracy of the spraying condition of the nozzle.

[0090] In one embodiment, in order to improve the accuracy of the detection model, the detection model can be updated. This embodiment includes the following Figure 5 Steps shown:

[0091] Step 501: Acquire a target image in which the spraying condition does not meet a preset condition.

[0092] Step 502: Iteratively train the detection model using the target image as a model sample until a preset number of iterations is reached or the model accuracy reaches a preset accuracy.

[0093] Among them, the preset condition can be that the spraying condition is qualified or unqualified in the spraying effect, or the spraying condition is less than 0.3 and greater than 0.7.

[0094] The target image whose spraying condition does not meet the preset conditions is a spraying image with a general spraying effect, or a spraying image whose spraying condition is within the range of 0.3-0.7. This type of spraying image is an image with fuzzy spraying detection results, that is, the detection model cannot clearly determine whether the spraying condition of the nozzle corresponding to the spraying image to be detected is qualified. This indicates that the detection model has poor feature extraction and mapping effects for this type of spraying images, that is, the detection effect for this type of spraying images is poor. Therefore, spraying images whose spraying conditions do not meet the preset conditions can be selected as target images for further training of the detection model.

[0095] In an embodiment of the present application, after the target image is selected, a new model sample consisting of the target image and the label corresponding to the target image is used to iteratively train the detection model until a preset number of iterations is reached or the model accuracy reaches a preset accuracy, thereby obtaining an updated detection model.

[0096] The method provided in the embodiment of the present application can select a spraying image with poor detection effect of the detection model as a new sample image through preset conditions, and iteratively train the detection model again to obtain an updated detection model. The embodiment of the present application updates the detection model based on images with poor feature extraction and mapping effects of the detection model, effectively improving the accuracy of feature extraction and mapping of the detection model, thereby improving the precision of the detection model.

[0097] It should be noted that although the operations of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0098] Further references Figure 6 , which shows an exemplary structural block diagram of a device for spray detection according to an embodiment of the present application.

[0099] In one embodiment, the spray detection device includes: a collection module 601, a segmentation module 602 and a detection module 603, wherein:

[0100] The acquisition module 601 is used to acquire an initial spraying image; the initial spraying image includes the spraying condition of at least one nozzle;

[0101] The segmentation module 602 is used to segment the initial spraying image to obtain the spraying image to be detected corresponding to each nozzle;

[0102] The detection module 603 is used to use the detection model to perform spraying condition detection on the spray image to be detected, and obtain the spraying detection result of the nozzle corresponding to the spray image to be detected; the detection model is trained based on the labeled sample spray image; the spraying detection result includes spraying uniformity, fan angle and spraying condition, and the spraying condition is used to characterize the spraying effect level.

[0103] In one embodiment, the spray detection device also includes a fitting module 604, which is specifically used to perform edge detection processing on the thermal imaging image and the visible light image in the initial spraying image to obtain the position information of each nozzle in the thermal imaging image and the visible light image; the position information includes the coordinates of each nozzle in the thermal imaging image and the visible light image, and the coordinates of the intersection of the spray fan of adjacent nozzles; according to the position information of the thermal imaging image and the visible light image, the thermal imaging image and / or the visible light image is geometrically transformed to obtain two candidate images, the spray fan of the nozzles in the two candidate images is the same, the two candidate images include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image after the geometric transformation processing and the visible light image after the geometric transformation processing; the two candidate images are fitted and the fitted image is updated to the initial spraying image.

[0104] In one embodiment, the segmentation module 602 is specifically used to determine the midpoint position between adjacent nozzles and the intersection position of the spraying fan between adjacent nozzles; and segment the initial spraying image based on the straight line where the midpoint position and the intersection position are located.

[0105] In one embodiment, the spray detection device also includes a model training module 605, which is specifically used to obtain multiple initial sample spray images; the initial sample spray images include the spraying conditions of at least one nozzle; the initial sample spray images are segmented to obtain sample spray images corresponding to each nozzle; the label of each sample spray image is determined; the label of the sample spray image includes the spray uniformity label, fan angle label and spray condition label of the nozzle; the multiple sample spray images and the label of each sample spray image are input into the initial model for feature extraction and feature mapping to obtain the predicted spray uniformity, predicted fan angle and predicted spray condition output by the initial model; based on the difference between the predicted spray uniformity, predicted fan angle and predicted spray condition and the spray uniformity label, fan angle label and spray condition label of the nozzle, the initial model is iteratively trained until the difference meets the preset difference condition.

[0106] In one embodiment, the model training module 605 is also used to determine the spray uniformity label of the nozzle based on the temperature data of the thermal imaging image of the sample spray image, where the temperature data is the temperature data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle; or, determine the spray uniformity label of the nozzle based on the brightness data of the visible light image of the sample spray image; the brightness data is the brightness data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle.

[0107] In one embodiment, the spray detection device also includes a model updating module 606, which is specifically used to obtain a target image whose spraying condition does not meet a preset condition; and iteratively train the detection model using the target image as a model sample until a preset number of iterations is reached or the model accuracy reaches a preset accuracy.

[0108] It should be understood that the units or modules described in the spray detection device are the same as those described in the reference Figure 1 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the spray detection device and the units contained therein, and will not be repeated here. The spray detection device can be pre-implemented in the browser or other security applications of the electronic device, or loaded into the browser or its security application of the electronic device by downloading or the like. The corresponding units in the spray detection device can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0109] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device or server suitable for implementing an embodiment of the present application.

[0110] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0111] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0112] In particular, according to the embodiments of the present disclosure, the above reference Figure 1The described processes may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising a computer program for executing Figure 1 In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 709 and / or installed from the removable medium 711.

[0113] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a different order from the order marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0116] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the computer device described in the above embodiment, or may exist independently without being installed in the computer device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the method described in the present application. For example, it can be executed Figure 1 to Figure 5 The individual steps of the method are shown.

[0117] The present application embodiment provides a computer program product, which includes instructions. When the instructions are executed, the method described in the present application embodiment is executed. For example, Figure 1 to Figure 5 The individual steps of the method are shown.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0119] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

Claims

1. A spray detection method, characterized in that: The method comprises: Collecting an initial spraying image; the initial spraying image includes spraying conditions of multiple nozzles; The initial spraying image is segmented according to the boundaries between adjacent nozzles to obtain a spraying image to be detected corresponding to each nozzle; the spraying image to be detected includes a spraying fan corresponding to a single nozzle; The detection model is used to detect the spraying condition of the spraying image to be detected, and the spraying detection result of the nozzle corresponding to the spraying image to be detected is obtained; the detection model is trained based on the labeled sample spraying image; the spraying detection result includes spraying uniformity, fan angle and spraying condition, and the spraying condition is used to characterize the spraying effect level; The segmenting process of the initial spraying image according to the boundaries between adjacent nozzles includes: Determine the midpoint position between adjacent nozzles and the intersection position of the spray fan between adjacent nozzles; The initial spraying image is segmented based on the straight line where the midpoint position and the intersection position are located.

2. The method according to claim 1, characterized in that The method further comprises: Performing edge detection processing on the thermal imaging image and the visible light image in the initial spraying image to obtain position information of each of the nozzles in the thermal imaging image and the visible light image; the position information includes the coordinates of each of the nozzles in the thermal imaging image and the visible light image, and the coordinates of the intersection points of the spraying fans of adjacent nozzles; According to the position information of the thermal imaging image and the visible light image, geometric transformation processing is performed on the thermal imaging image and / or the visible light image to obtain two candidate images, in which the spray fan of the sprinkler head is the same in the two candidate images, and the two candidate images include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image and the visible light image after the geometric transformation processing, or include the thermal imaging image after the geometric transformation processing and the visible light image after the geometric transformation processing; The two candidate images are fitted, and the fitted images are updated as the initial spraying images.

3. The method according to claim 1, characterized in that The training process of the detection model includes: Acquire a plurality of initial sample spray images; the initial sample spray images include spray conditions of a plurality of nozzles; Segmenting the initial sample spraying image to obtain sample spraying images corresponding to each of the nozzles; Determine a label of each of the sample spray images; the label of the sample spray image includes a spray uniformity label, a fan angle label, and a spray condition label of the nozzle; Inputting a plurality of sample spraying images and a label of each of the sample spraying images into an initial model for feature extraction and feature mapping, and obtaining a predicted spraying uniformity, a predicted fan angle, and a predicted spraying condition output by the initial model; Based on the differences between the predicted spraying uniformity, the predicted fan angle and the predicted spraying condition and the spraying uniformity label, fan angle label and spraying condition label of the nozzle, the initial model is iteratively trained until the differences meet preset difference conditions.

4. The method according to claim 3, characterized in that Determining the label of each sample spray image includes: Determine the spray uniformity label of the nozzle according to the temperature data of the thermal imaging image of the sample spray image, wherein the temperature data is the temperature data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle; or The spraying uniformity label of the nozzle is determined according to the brightness data of the visible light image of the sample spraying image; the brightness data is the brightness data of the area where the nozzle does not overlap with the spray fan of the adjacent nozzle.

5. The method according to claim 3, characterized in that: The method further comprises: Acquire target images when the spraying condition does not meet the preset conditions; The detection model is iteratively trained using the target image as a model sample until a preset number of iterations is reached or the model accuracy reaches a preset accuracy.

6. A spray detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an initial spraying image; the initial spraying image includes the spraying conditions of multiple nozzles; A segmentation module is used to segment the initial spraying image according to the boundaries between adjacent nozzles to obtain a spraying image to be detected corresponding to each nozzle; the spraying image to be detected includes a spraying fan corresponding to a single nozzle; A detection module, used for performing spraying condition detection on the spraying image to be detected by using a detection model, and obtaining a spraying detection result of the nozzle corresponding to the spraying image to be detected; the detection model is trained based on the labeled sample spraying image; the spraying detection result includes spraying uniformity, fan angle and spraying condition, and the spraying condition is used to characterize the spraying effect level; The segmentation module is specifically used to determine the midpoint position between adjacent nozzles and the intersection position of the spraying fan between adjacent nozzles; and segment the initial spraying image based on the straight line where the midpoint position and the intersection position are located.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images